The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Saturday, July 27, 2002


I'm sorry, I just find this stuff interesting.

Marcus wrote in, asking why Google seems to have abandoned his weblog, although I'm not sure if he was asking about referers from Google, or Google actually spidering his site. Hard to say what is going on, and from doing some research on my own server I did come across some rather interesting figures.

Google spidering vs. Google referers for The Boston Diaries
Date of log Googlebot hits Google referers Comments
October 2001 0 0 blog went live 10/22
November 2001 721 26
December 2001 1,421 348
January 2002 579 733
February 2002 341 1,073 something wrong with Googlebot?
March 2002 854 694
April 2002 1,019 649
May 2002 1,073 675
June 2002 1,228 504
July 2002 994 512 incomplete month

While traffic from Google's spider has gone up, actual traffic from their engine has gone down. I'm not sure if they've tweaked their page ranking algorithm to decrease the their sensativity to blogs, or I'm now fighting with a bazillion other weblogs for Google traffic, or now that I've been live for nine months things have settled down and I can expect a similar level of traffic here from now on. But, for contrast, I decided to scan a static (relatively unchanging) website— (for the same time period as my online journal/weblog):

Google spidering vs. Google referers for Conman Laboratories
Date of log Googlebot hits Google referers Comments
October 2001 185 286 blog went live 10/22
November 2001 275 301
December 2001 435 395
January 2002 279 370
February 2002 74 462 something wrong with Googlebot?
March 2002 226 393
April 2002 226 307
May 2002 382 358
June 2002 540 293
July 2002 215 285 incomplete month

You can see here that the level of traffic from Google searches is pretty constant, although the Googlebot seems to go up and down. It is apparent though, that something was up with Googlebot in February as visits from it dropped dramatically for that month (and I'm only saying for those two months—I haven't actually checked any of the other sites I host).

And after all this checking, I still don't have a difinitive answer for Marcus as to why Google has slacked off his site. My own experience is that Google likes—part of that might be the stableness of the URLs on all parts of (my personal site, used to be hosted elsewhere, but when I first moved it to in October of 98, I placed permanent redirects from the old site to so the engines at that time immediately found my new site) and there's something to be said for stability. It might also like my online journal/weblog (but Google seems to have an afinity for those in general). I'm not sure what to tell Marcus though.

Obligatory Picture

[“I am NOT a number, I am … a Q-CODE!”]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site:, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2024 by Sean Conner. All Rights Reserved.