Consider the hardware: a computer system with close to 400 parallel
processors, 100 terabytes of disk space, hundreds of gigs of RAM,
all for under a half-million dollars. As you'll read in this in
interview, the folks at the Archive have turned clusters of PCs into
a single parallel computer running the biggest database in
existence—and wrote their own operating system, P2, which allows
programmers with no expertise in parallel systems to program the
system.
Via Flutterby
How
The Wayback Machine Works
I find this stuff facinating. Google runs off 8,000 servers, this site
has 100 terabytes of storage, and my friend Kelly works at a place that processes
gigs of log files every day.
He said that before he optimized the processing, it sometimes took about 30
hours to process one day's worth of logs. Now, it can finish (for a real
busy day) in under 22 hours. He works for a really busy site and I
found the inner workings quite interesting.
You have my permission to link freely to any entry here. Go
ahead, I won't bite. I promise.
The dates are the permanent links to that day's entries (or
entry, if there is only one entry). The titles are the permanent
links to that entry only. The format for the links are
simple: Start with the base link for this site: https://boston.conman.org/, then add the date you are
interested in, say 2000/08/01,
so that would make the final URL:
https://boston.conman.org/2000/08/01
You can also specify the entire month by leaving off the day
portion. You can even select an arbitrary portion of time.
You may also note subtle shading of the links and that's
intentional: the “closer” the link is (relative to the
page) the “brighter” it appears. It's an experiment in
using color shading to denote the distance a link is from here. If
you don't notice it, don't worry; it's not all that
important.
It is assumed that every brand name, slogan, corporate name,
symbol, design element, et cetera mentioned in these pages is a
protected and/or trademarked entity, the sole property of its
owner(s), and acknowledgement of this status is implied.
