The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Monday, September 24, 2007

It's really been over ten years since I wrote “An Extended Standard for Robot Exclusion”? Wow …

Around the same time the IETF draft was being discussed, Sean “Captain Napalm” Connor [sic] proposed his own extension to the Robots Exclusion Protocol, which included Allow rules as well as regular expression syntax for rules, and new Robot-version, Visit-time, Request-rate, and Comment rules. Less than 100 of the sites I visited use rules unique to this spec.

Via email from Steve Smith, robots.txt Adventure

[As a small aside, I don't know why people insist on spelling my last name with an “O-R” instead of an “E-R”. It's not like I misppelled my own name on that page. Sigh. —Editor]

That's not the only place “An Extended Standard for Robot Exclusion” has been referenced—it's also mentioned in O'Reilly's HTTP: The Definitive Guide, but until Steve reminded me of it, I basically forgot about it. Understandable since the last time it was edited was November of 2002 (and even then, the previous time it was edited was six years earlier—it's old).

This probably means it's time once again to check the links and make sure they all work.

And maybe clean up the HTML while I'm at it.

Just as soon as I can reproduce that insipid Heisenbug.

Probing further into Alien Abductions

Today, this very day, forty-six years ago, Betty and Barney Hill drove down U.S. 3, right past my house and into history. They were about to become Patient Zero for Alien Abductions with Weird Medical Experiments, Missing Time, and Big-Eyed Extraterrestrials. The first and (we are told) best documented case of Alien Abduction Evah. There was a book. There was a made-for-TV movie. Magazine articles. Mentions in other books. Cl ose Encounters of the Third Kind. X-Files.

So what happened out on Route 3?

Via columbina, Alien Abduction

I'm guessing that this debunking article won't go over very well in Rachel, Nevada. It's also long, but well worth reading since everything about the Betty and Barney Hill abduction is debunked.

I also found it helped to use Google Maps to help follow along on that fateful roadtrip forty-six years ago.

Heisenbugs, II

I spent several days testing the greylist daemaon on the development server, and could not for the life of me reproduce the crash. I cleaned up the code a bit, and again, I couldn't get the program to crash on the development server.

Moved the latest version to the production server, with the checkpoint feature enabled, and after a few hours (about 4½ hours this time) it froze.

Disable the checkpoint feature, and it runs fine on the production server.

I'm giving up on this bug hunt for now. The program saves its state when it stops running—the only way we'll lose the state is if the server it's running on suddenly loses power, and if that's the case, then we have more issues to worry about.

Our wall has been framed!

I suddenly heard a large amount of pounding. I went out to find not The Kids making all that noise, but Spring. She was busy hanging up a metric buttload of pictures in the living room.

[My God!  It's full of frames!]

The whole wall was covered in pictures and paintings (well, one painting). I like the effect—it makes the living room look taller. (For those who are curious, most of the pictures in the brown frames are pictures I took, along with the alligator and the Everglades, which are on either side of the large image, which is an Ansel Adams print. The blue painting was given to The Kids by an aquaintence of ours. At the very far end are pictures of The Kids and a few certificates)

Obligatory Picture

[“I am NOT a number, I am … a Q-CODE!”]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site:, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2024 by Sean Conner. All Rights Reserved.