Friday, January 11, 2013
The utility of a crash report
Well, that was fast. I installed the new version of mod_blog, went out
to dinner with Bunny, and came home to over 80 crash reports, reason:
“Address not mapped for object.” Or in other words, “the program tried to
access memory that didn't belong to it.”
Bad news—it wasn't every request, just certain ones. Even worse news—the logging server and the web server have a 3′50″ difference in time (that took me about fifteen minutes to realize). I also realized that even with stack dump and trace, there still wasn't enough contextual information to figure out what was going on.
I decided to dump the command line arguments (a hack—I have to store
the arguments in a global variable and make crashreport(), a
library function, depend upon these global variables, which is bad form) and the environment variables
(less of a hack—the system already has these in a global variable). With
that, I was able to track down the issue.
The issue—some script kiddies are trying to hit the webpage that
creates webpages. This isn't normally an issue, since the POST
method requires authentication, but these script kiddies were trying to
create entries via GET, and the error that was
generating wasn't being properly checked (since that shouldn't
happen—oops). And because of that, some unititialized variables were
dereferenced and boom! A crash report.
The surprising thing about all this is that Apache not once reported the CGI program crashing. To me, that seems fairly obvious, but apparently not. So there's no telling how long this has been happening (yeah, you can tell I check the logs often, can't you?).
I do need to think about how to handle command line arguments in
crashreport(). It's library code and thus, shouldn't rely on
global variables. I have to think about this …
