The Boston Diaries

Monday, January 01, 2007

Happy New Year!

Oh, it does appear that it is some sort of holiday today, something about it being a new year or some such silliness. Of course everyday is the start of a new year! For it was a year ago that it was the same date! Well, except for February 24^th, which comes around once every four years, except on the other three years when it appears yearly.

Yes, there will be a test on this.

In other news, to show just how much of a geek I am, one of the things I did today was telnet to an actual IBM PCjr, complete with ISA network card and custom TCP/IP network stack.

How cool is that?

And no, this will not be on the test.

Maybe.

Tuesday, January 02, 2007

Nice thought, but I don't think it'll work as intended

I've been hearing about the $100 laptop for some time now, and the more I think about it, the more I think it's a bad idea, but I'm having a difficult time expressing why I think it's a bad idea.

I remember back to my own days in school. My first exposure to a computer was in 5^th grade, but I didn't get any access until 7^th. And for me, that lasted all of two classes before I was permanently banned from using the computer (and I was still two years away before getting my own computer).

Sure, some kids will use the computers as expected.

But I suspect more will use the computers in very unexpected ways.

Some of which could lead to trouble (some of the stuff I did in college, had I gotten caught, could have had me expelled—or worse).

But that's even assuming the laptops get to their intended audience. Perhaps I'm being too cynical, but given the fact that the United Nations “Oil for Food” program didn't exactly feed many Iraqis, or that two bit dictatorial leaders kill the educated or intentionally cripple education doesn't lead me to be very optimistic about this endevour (I'm also involved in an online discussion about this topic).

Computers in the classroom

I'm actually very dubious about computers in the classroom, and it's not because of any Luddism I might have (heck, I have trouble getting rid of computers). I'm dubious for multiple reasons, one for the over-reliance on technology aspect. And two, I think it's a huge waste of money.

Years ago, maybe ten or so, I was invited to a middle school in Palm Beach County (forgotten which one it's been so long) to give a talk about the Internet. So one week day I arrive at the middle school and get a tour of their computer lab.

Oh.

My.

God.

A huge room, with about 20 or so high end computers (wouldn't surprise me it was Apple Macintoshes—Apple was always big in the educational system) all networked together in this gorgeous lab. Large windows. No visible wires (all wiring ran in tasteful pipes running up from each desk).

It basically exceeded any computer lab in the Computer Science and Engineering Department at FAU.

And the first thing that ran through my mind was, how much money was wasted on this lab?

Twenty computers. Assume two students buddy up at each computer. Fourty students per class. At six classes per day (the number I had in middle school) that's 240 students that have access to these computers. Out of a school that probaby had close to 2,000 students (it was a huge two story middle school—way larger than the one I attended). That's 12% of the student population had access to this beautiful computer lab.

And I have no idea what they actually taught on the computers (when I attended high school, it was BASIC and Pascal programming, but I doubt that's done anymore). I thought that the money spent on the computer lab could have gone to better use. More teachers. Better text books.

Anything but expensive, fragile computers that the teachers probably couldn't operate.

(Oh, and the students I talked to about the Internet? Totally uninterested in anything I said. Sigh.)

(Oh, and computers in the classroom? Highly distracting. Couple that with networked computers in the classroom, woo boy—just ask my Computer Graphics teacher … on second thought … don't.)

“But it'll be in color!”

From

"Bunny XXXXXXXX" <XXXXXXXXXXXXXXXXXXXXXX>

To

"Sean Conner" <sean@conman.org>

Subject

Wanna talk irony?

Date

Tue, 2 Jan 2007 23:06:43 -0500

I just discovered that the Hallmark Channel is showing M*A*S*H (final episode) which started at 10 p.m. Even if I didn't already have it on tape, I could have taped it for you. You were meant to see it, and I'm pleased to be the one to show it to you.

Guess I'm finally going to see the final M.A.S.H episode afterall.

One funny story about M.A.S.H.—growing up all we had was a small black-and-white TV, so all of my M.A.S.H. viewing was on said set. On the few occasions I've seen M.A.S.H. in color, it just looks … funny. It just … looks weird to me to see M.A.S.H. in color.

And that's the only show that I find weird looking in color. Perhaps because even in color, it's still fairly monochromatic, even if that monochromatic color is green.

Wednesday, January 03, 2007

I've got some good news, and some bad news …

Today felt like one of those Bad News/Good News type of jokes. And it all started out because I needed to modify a client's managed firewall (we manage their firewall).

Bad News: I can't log into the firewall.

Good News: It probably just needs rebooting (which involves walking across the parking lot to another building to the client's office, and flipping a power switch).

Bad News: That didn't work. Perhaps it's the router in the building?

Good News: We have extra ports on that router we can try (which involves making yet another trip to the other building).

Bad News: That still didn't work.

Good News: Which means the router isn't probably at fault.

Bad News: Which means it's the cable that runs from the router to the client's office, or the firewall. Since I have no easy way to test the cable, I'll assume (for now) it's the firewall.

Good News: It's probably a simple configuration setting I forgot to save when setting it up.

Bad News: There's no way to actually log into the firewall at the client's office (doing so requires a terminal or a computer with a serial port and … well … it was probably best not to ask about using a computer with a serial port).

Good News: The firewall itself is pretty small and easy to take back to The Office (which means more schlepping between buildings).

Bad News: When powering up the firewall at my desk, it kernel panics (basically: crashes, and crashes hard) when trying to check the disk.

Good News: Maybe it's just a loose connection.

Bad News: Nope. Still crashes.

Good News: Maybe it just needs some more memory. Here, let's install a 512MB stick of memory.

Bad News: Fails to even do the self test.

Good News: We have plenty of spare Cobalt RaQs (which we use as firewalls) sitting around.

Bad News: Moving the harddrive to another Cobalt RaQ fails to produce any forward results.

Good News: Perhaps if I cannibalize the memory from RaQ I tried to use and put it into the RaQ initially used as the client's firewall.

Good News: There is no more bad news!

Oh wait.

Bad News: Gotta schlep back to the client's office to reinstall the firewall.

Good News: It all now works properly.

“Goodbye, Farewell and Amen”

Bunny made good on her promise, and I saw the final episode of M.A.S.H., almost twenty-four years after it first appeared.

I found it to be excellent (if sad—I mean, it is the end to one of the best series on TV) but a very taxing episode to watch (horrors of war and friends going their own way). I also came to the realization that Charles Emerson Winchester III is one of my favorite characters in the series, trying to maintain his dignity in an undignified location.

Thursday, January 04, 2007

“A student who changes history is probably taking a test.”

Creativity is a weird thing, something I haven't given much thought to to tell you the truth. I've always been creative, and I've managed to surround myself with creative people so it was something that's always there, in the background, something I can pull from, sometimes easily, sometimes not so easily, but I manage to pull something out of the air when the time comes.

And like I said, I've never given it much thought, until I received the following in email from Bunny:

My friend XXX has a son who's been accepted at XXXX XXXXXXX, but needs to do this essay to qualify for a scholarship. I'm really no good at stuff like this. You have any ideas, being creative and all? You have such the imagination! I'm in awe!

Here's what she sent me:

XXXXX has to write a scholarship essay. The prompt is:

A dinner party is a comfortable venue for guests to engage one another in a discussion of ideas. Imagine a dinner party where you could invite any four people, living or dead. Whom would you select and why? What issues or themes would the guests discuss? Answer these questions and develop a portion of the conversation in an essay of no more than 800 words.

He's totally blank. He can think of lots of good people, but can't figure out what they should talk about and how it should be structured. What's the premise, the setup? You can't just jump in and start talking, it wouldn't make sense. He's very uncreative but so am I and I can't think of anything either. He came up with a stinker of an essay that was sort of an allegory tale—World War II Food Fight, where Stalin, Hitler, Churchill and either Mussolini or Hirohito were gathered together and started a food fight that paralleled their battles in the war. But I don't think the prompt wants an allegory, and it wasn't very funny, anyway.

Any ideas?

That forced me to think about creativity, and in the span of maybe twenty minutes or so, I somehow managed to come up with six different groups of people to write about. One of the ideas:

Okay. I might be inclined to do one with just about any four famous people (really, could probably just pick four at random, say, Sean Penn, Karl Marx, Oliver Cromwell and Ayn Rand and just have them complain about the food service, the food itself, and how young kids today just don't know how good they have it—much like the Four Yorkshire Men Sketch from Monty Python).

I'm not sure why exactly I included Sean Penn in the mix (perhaps his self-important goofiness?) but the other three I picked simply because they would hate each other (you'd probably be hard pressed to keep Rand from killing Marx), but then to just have them complain about the service I find terribly funny.

Yes, I do like my humor dry.

Another idea:

Or possibly a dinner conversation between Karl Marx, Vladimir Lenin, Groucho Marx, and John Lennon (with the latter two rubbing the fact they got a stamp together while the other two didn't).

And there is no way that Karl or Lenin would be able to dominate the conversation with Groucho or John in the same room. Oh, to have Lennon tell Lenin to lighten up …

But, again, I certainly can't explain how I came up with these groupings (like the Presidents Andrew Jackson, William McKinley, Franklin Roosevelt and George Bush Jr), other than picking groups of people that would have a complex group dynamic (McKinley and Bush have similar administrations and both got us in unpopular wars, but Bush's cowboy image would certainly not appeal to McKinley, but would appeal to Jackson, but Jackson wouldn't care for Bush or McKinley (or Roosevelt for that matter) because they were city-bred rich folk—dynamics like that). Heck, these groupings almost write themselves.

Perhaps part of it is being well read, and certainly knowing history doesn't hurt matters either.

Anyway, I just find coming up with ideas pretty easy—ooh wait! I know! Three friends grilling George Lucas about Star Wars and what exactly was he smoking when he wrote Episodes 1–3?

See?

I can't exactly say where my creativity comes from, except the few I buy wholesale from Schenectady.

Anyway, some time later Bunny replied to my reply that her friend wrote back ~~with a successful conclusion:~~ saying her son managed to successfully write the essay.

XXX XXXXX XXXXX XXXXX XXXXXX XXXXXXXXXX XXXXXXX XXXX XX XXXX X XXXXXXXX XX XXXX XXX XXXXX XXX XXXXXXXX XXXXXX XXXXXXXX XXXXX XXXXXXX XXXXXXXXXXX XXXXXXXXX XXXXXXXXX XXX XXXXXXXXX XXXXXXX XX X XXXXXX XXXXXX XX XXXXX XXXX XXXX XX XXXXXXXXX XX XXX XXXXX XXXXXX X XXXXX XXXXX XX XXXXX XX XXXX XXX XXXX XXXXXXXX XXX XX XXXX XXXXX XXXXXX XXXXX XXXXX XXXX XX XXXXXXXX XX XXXXXXXX XXXX XXX XXX

XXXXXXXXXXXX XXX XXXX XXXXXXXXXXXX XXX XXXXXXXX XX XXX XXXX XXXXX XXXXXXX X XXXXX XXXXX XXXXXX XXX XXXXX XXX XXXX XXXXX XXX XX XXXXX XX XXXXXX XXXX XXXXXXXXX X XXXXX XX XXXXX XXXX XXXX XXXXXXXXXXXX XXX X XXXX XX XXXXXX XXXX XXXXX XXX XXXX XXX XXXXX XXXXXXXX X XXXXXXX XXXX XXXX XXX XXXXXXX XXXX XX XXXX XXXXXXXX XXXXXX XXX XXXXX XXXX XXXXXXXXXXX XXXXXX XXX XXXXXXX XXXXX XXX XXXXXXXX X XXXXXXXXX XXX X XXXXXXXXX

Update on Friday, January 5^th, 2007

Major redaction as you can tell—Bunny's friend requested I remove this material, and since her friend didn't expect this to happen, I'm respecting her friend's wishes.

Friday, January 05, 2007

Redactions

It's always a risk when writing about other people, so it wasn't too surprising when I got the following:

Looking at it further, could you ask him to strike the whole part about who XXXXX decided to write about? I mean, that would point directly to him. I'd really rather not have anything that could possibly identify him out there like that. Tell him sorry, but I'm weird.

in reference to yesterday's entry about creativity (I also had to remove all references to the college as well). And while I may think that Bunny's friend (who requested I remove the material) is being a too cautious, she didn't ask to have the material placed on a website in the first place (even though I thought I removed enough identifying material).

It's not like this hasn't happened before.

Bunny had written me earlier about removing some information (before she wrote me asking for a whole bunch more stuff to be removed):

Hey, here's something that didn't occur to me. XXX says they check the internet for plagiarism and stuff, and [the college] might google and find your post. Not likely, but do you think you could strike the reference to [the college], just in case? … I know they wouldn't stumble upon it just by the premise for the essay alone. … I can't imagine they have a filter or something set up to ring a chime every time somebody mentions [the college]. But I guess she'd feel better about it. She loved your stories and creativity, though, and wished she'd asked me sooner.

While [the college] itself might not actually check, there are other services that do, and one such service (if not the service) is TurnItIn. So, it is conceivable that [the college] could come across these entries (although I thought that I made it clear that I did not come up with the kid's essay idea, but hey, things I wrote a long time ago have a habit of resurfacing), but whether it will hurt the kid? I doubt it, but hey, like I said, the kid's mother didn't ask for this to be posted, so best to remove it (am I making a mountain out of a molehill? I don't think so—it's just something I can write about, that's all).

Saturday, January 06, 2007

“You know, melting coins for the intrinsic metalic value is now illegal in the United States.”

I was going through my change in the car today when I found this nickel and while the front is typical, it was the back that caught my eye—it looks partially melted. It's not been shot, that would have deformed the coin and this is still quite normally flat. I doubt it's an arc-welder as the front is still fine, and there don't appear to be any scorch marks along the back.

Perhaps acid?

Anyway, it's pretty cool looking.

Sunday, January 07, 2007

Been busy with other stuff, so this is what you get for today.

I spent some time today fixing the mailing list software so it no longer breaks MIME based emails. The software is old, but it has two features I like: the software is straightforward and it's easy to maintain.

I'm not sure why I took as long as I did to fix it—perhaps it's due to my own relunctance towards MIME (“Email is not for file transfers! That's what FTP is for!”) or some form of passive-aggressive attack on my part to show just how bloated HTML email can be (“FTP people! FTP!”). Although once I figured out which headers I needed to preserve (not as many as I feared) it was pretty easy to patch the software.

Monday, January 08, 2007

Jesse has left the planet

I just had a funny thought: There have been numerous sightings of Elvis since his death in 1977, but not one recorded sighting of Elvis' twin brother, Jesse since his death in 1935.

Why is that?

Tuesday, January 09, 2007

Ideas for features for my blogging software

The goal last time was a macro processor. I write a lot of math articles. I get tired of writing 2 every time I want a superscript 2. Even if I bind a function key to that sequence of characters, it's hard to read. But now, with my new Blosxom macro processor, I just insert a line into my article that says:
#define ^2 2
and for the rest of the article, ^2 is expanded to 2.

The world's worst macro preprocessor

This is, without a doubt, an incredible idea and one I wish I had thought of. I already use shortcuts like this, such as “---” for “—” (an em-dash) or “``” for the nice open double quote that are “baked” into the code so to speak, but the ability to add more on a per-entry basis would be very neat indeed.

I'll have to look into this a bit more.

Wednesday, January 10, 2007

Oh, so that's what an ssh scanner looks like

Between metro ethernet woes and a customer's server either attacking other machines, or being attacked by other machines (it was never made clear in what direction the excessive network traffic was travelling), I was made aware that one of our servers was generating a large amount of outgoing ssh traffic.

When I logged in, sure enough, one ps aux code later:

rob      30289  0.0  0.0  8632 2284 pts/1    S    10:53   0:00 ./ssh-scan 100

Only, about four score and seven more copies than the one just listed there. It looks like regular user accounts were compromised (it's a dedicated server to one of our clients so we don't have full control over it). Not much else to do but kill off the offending processes (and finding a second compromised account running an IRC bot), locking out the account and looking at said ssh-scan program.

Interesting stuff—found one file named vuln.txt that seemed to have a list of servers with default accounts and passwords.

Hmmmm …

I tried one system listed in the vuln.txt file and got the following:

-------------------------
Mitel Networks SME Server
-------------------------


Standard user login services have been disabled.


Type "end" and press ENTER to terminate this connection:

I tried another vulnerable system, and was able to actually get a shell:

[spc]shell:~>ssh tester@XXXXXXXXXXXXXX
tester@XXXXXXXXXXXXXX's password: 
-bash-2.05b$

But when I tried to actually use system, it was rather limited. The only commands available were ls, mkdir, mv, pwd, rm, sh, groups, id, ssh and bash and a bunch of builtin shell commands.

Makes it kind of hard to look around, but with discussion with an unnamed friend of mine, we came up with the following to actually view the few files that existed on this system:

(while true ; do read && echo $REPLY ; done) <filename

I'm beginning to think these ssh scans aren't for vulnerable Unix systems, but embedded systems with manufacterer backdoors built in that a certain clientel of user are using to their own nefarious schemes.

Thursday, January 11, 2007

More musings on Star Wars

If we accept all the Star Wars films as the same canon, then a lot that happens in the original films has to be reinterpreted in the light of the prequels. As we now know, the rebel Alliance was founded by Yoda, Obi-Wan Kenobi and Bail Organa. What can readily be deduced is that their first recruit, who soon became their top field agent, was R2-D2.

Via Jason Kottke, A New Sith, or Revenge of the Hope Reconsidering Star Wars IV in the light of I- III

An interesting reinterpretation of Star Wars IV–VI, notwithstanding the fact that George Lucas was making everything up as he went along, especially for episodes I–III but he doesn't care—he's got our money and besides, geeks live for explaining continuity errors in science fiction series (and amusingly enough, nothing that went on in Episodes I–III matched what he wrote about in “Bantha Tracks”—I know this because my friend Hoade used to get this newsletter as a kid).

I just like the quote.

“It's got kind of a slacker appeal, a no-resistance story line.” Animators and children's TV creators around the world must see Scooby and ask themselves: Why can't my crappy show become iconic?

Dog Gone: Scooby-Doo creator Iwao Takamoto died this week, but his legacy lives on.

But it wasn't as if the creators of Scooby-Doo were perfect— far from it and I can prove it with one word (or is it two?):

Scrappy-Doo.

But despite that, Scooby-Doo did reach iconic status.

Somehow.

Sorry, English speakers need not ride

ST. PAUL, Minn. (AP)—A school bus driver let Rachel Armstrong's three children board the bus Monday morning, but he warned them that he wouldn't give them a ride home that afternoon, nor could they ever ride his route again.

The problem: Armstrong's 10-year-old twin girls and 8-year-old son speak English. According to their mother, the driver told them the route had been designated for non-English speakers only.

Via Gates of Vienna, Students kept from bus in St. Paul; told reserved for those learning English

Has desegregation busing gotten such a bum rap that the pendulum has swung around again? Have we gone back to “separate but equal”?

Why didn't I get the memo?

Was there a memo?

I'm horribly confused.

Friday, January 12, 2007

Bug hunt

It wasn't so easy to patch.

Email headers can technically span multiple lines, although the mailing list software I'm using doesn't actually support headers that span multiple lines.

And thus breakage.

The email I sent to the affected mailing list I run went into a bit more detail (more, I'm sure, than the members cared to know):

From

Sean Conner <sean@conman.org>

To

XXXXXXXXXXXXXXXX

Subject

[dss] Mailing list woes

Date

Fri, 12 Jan 2007 16:43:02 -0500 (EST)

Okay, you didn't ask for it, but you got it anyway.

What exactly was going on with the mailing list software, and why were the messages so funky to begin with?

The mailing list software I'm using has a few features that I like:

it's simple.

it's simple to install (this is a very important part).

it works.

I started using it back in the mid-90s but I think it's a few years older than that even. It was written way back before there existed a stardardized method for sending files via email [1], so it never had the ability to handle attachtments, nor, as it turned out, HTML formatted email.

In the mid-90s, a standard was created for the sending of multi-media based emails—MIME. Not only did it allow a way to send attachtments, but it also allowed email messages to contain both a plain text version and a formatted version of an email (say, using HTML or even Microsoft Word Doc). This was done by adding some extentions to the headers of the email, which change the way the email client processes the body of the message.

The mailing list software, did not know about these headers, so it didn't preserve these headers and thus, you got funky looking email. [2]

So all I needed to do was preserve the required headers and bam! The mailing list software would automatically support all this MIME crap [7] and everybody would be happy and be able to actually read the messages.

Only, preserving said lines wasn't as easy as I expected.

You see, the headers in an email message are name/value pairs:

From

Lorie Davis <XXXXXXXXXXXXXXXXXXXXX>

To

The Dragonslayers Society <XXXXXXXXXXXXXXXX>

Subject

Are we done yet?

Date

Fri, 12 Jan 2007 16:43:02 -0500

but the email standard states that a header can actually span multiple lines:

From

Lorie Davis <XXXXXXXXXXXXXXXXXXXXX>

To

The Dragonslayers Society <XXXXXXXXXXXXXXXX>

Subject

Have you finished using me as an example?

Mime-Version

1.0

Content-Type

multipart/alternative;
boundary="—-=_Part_44704_28465339.1168636054704"

Date

Fri, 12 Jan 2007 16:07:34 -0500

(note how the Content-Type header spans multiple lines) Now, not all headers span multiple lines, but it is allowed by the email standard. But the mailing list software didn't allow for multiple lines per header, and for the headers it was originally interested in, were never really split across multiple lines.

I found this out the hard way.

The way the code was originally written made it difficult (read: damn near impossible) to support multiline headers. So it required a bit of rewriting to make this all work.

So, why not just install some modern mailing list software? See rule #2 above? Have you seen what's required to install modern mailing list software? At least a 2GHz machine to start off with. A score of libraries. And now I'm treading into the scripting lauguage du jour as well.

Believe me, it was easier to patch the existing mailing list software than to install something new (which basically, would require updating my entire server, which is something I refuse to do, seeing how it works and is secure as is—and I'll stop here before I start ranting on the whole upgrade culture).

-spc (Who still uses 486s)

EMAIL is NOT FTP!
Sorry.

I was never one for sending files via email, as there are better, more efficient and faster ways of doing that. What no one (and by “no one” I mean those that aren't programmers) seems to realize that when you send a file through email, the file has to be converted to text, least a lot of email software along the way blows up. This conversion process inceases the file size by at least a third (so a file that's 3M now becomes an attachtment 4M in size).

But it's damned convenient [3][5], so everybody does it.

Well, except me. But I'm not your typical user to begin with.

Anyway …

Not that I minded—that's how I receive such messages, even though my the email client I now use understands the MIME headers. Yes, what you were seeing was the raw message body itself, in all its bloated HTML glory that every email client seems to bloody default to these days [3].

Thank you so very much, Microsoft. [4]

May you rot in Hell.

Thank you so very much, University of Washington. [4][6]

http://en.wikipedia.org/wiki/Pine_(e-mail_client)

That's a technical term.

If I come across as a curmudgeon, then yes, I don't like the fact that people send files via email (reguardless of how convenient it is) nor do I like HTML formatted email (if plain text is good enough for me, then gosh darn it's good enough for you!) so quite a bit of this is just me blowing off some steam.

Saturday, January 13, 2007

Notes on some programming postulates

During a lull in the D&D game last night, I was doing the programmer equivalent of doodling, engaging in one of my interests—programming language design. Old interests die hard, and this, along with operating system design, are very old, dating back to high school. Bunny asked what I was doing, and I quickly got self-conscious.

It happens when one is caught doodling.

Afterwards in discussing this, I mentioned some postulates I have about programming; I don't feel it's anything earth shattering, but pondering on them (and I've been pondering on them for a few years now) has lead to some interesting questions.

The first one, which I tend to think of as number zero (which, in programming circles, makes some sense), is: a program is a subroutine writ large.

Now, I'm using some very loose definitions of “program” and “subroutine” here. By “subroutine” I mean any sequence of code that does “something” and can be called one or more times. By “program” I mean any sequence of code that does “something” and can be run by the user. So really, except for the context in which it is used, a “subroutine” and a “program” aren't much different (and in fact, on some platforms like Microsoft Windows, you can treat a program like Microsoft Internet Explorer, for instance, as a subroutine to be used by a program you write).

So, the next three postulates. For each of these, when you see the term “subroutine” you can also substitute the term “program” without any change in meaning.

A subroutine may accept input.
A subroutine may do processing.
A subroutine must produce output.

It was fun coming up with this list. The first version of the first postulate was “a subroutine must have input” which I then had to think if there were any types of subroutines that violated this.

And there were. A program to calculate e to 100,000 digits, the Unix utility yes and any number of utilties to overwrite files with meaningless data. There are a class of programs that can very well deal without any input whatsoever. So input is optional.

And the second one was similar: “a subroutine must do processing.” But the definition of “processing” is a bit thorny—take the Unix utility yes, which just repeatedly spits out lines consisting of a single “y” (you use it to automatically respond “yes” to other Unix utilities, like fsck). I have a hard time with the notion of it doing any “processing”. I mean, the code is little more than:

#include <stdio.h>
#include <stdlib.h>

int main(void)
{
  while(1)
    printf("y\n");
  return (EXIT_SUCCESS); /* not that it ever exits */
}

While technically there is processing going on, i.e. the CPU is processing instructions, it's not actually calculating. Yes, what it does is useful, but is it “processing?”

I wouldn't classify this as “processing” for the types of “processing” I'm thinking of, so postulate number two—a program must do processing, must be wrong. It may do processing.

It's the third one, “a subroutine must have output,” that has given me the most problems.

At first, I had a real hard time coming up with examples of a subroutine that had no output. I mean, why even bother calling a subroutine that had absolutely no output? What exactly, would be the point? I was about to conclude that all subroutines must have output, when I remembered writing just such a subroutine for my first computer, a Tandy Color Computer.

A small digression: The Color Computer technically had a serial port, in that there is a one bit input port and a one bit output port which is driven entirely by software. To send a byte, you drive the output port low for X µseconds, then set the port to match the first bit, wait X µseconds, then set the port to match the second bit, wait X µseconds, and so on, where X is a value dependent upon the baud rate. And I ended up writing code to drive the “serial port” on the Color Computer. And as part of this, I had the following subroutine:

DELAY	PSHS	X,CC
	LDX	#413
DELAY10	LEAX	-1,X
	BNE	DELAY10
	PULS	X,CC,PC

This would delay 3.3 milliseconds, giving me a baud rate of 300bps.[1] The X register, as well as the condition codes, are saved, so the effect of this routine on the CPU state was nil. The only difference between:

NOP		* NO OPERATION

and

BSR	DELAY	* NO OPERATION, BUT A LONG DELAY

was 3,331 µseconds (NOP requiring 2 µseconds to execute).

Well. [Deep subject. —Editor]

Here I have, from my own archive, a subroutine that takes no input, does no real processing, and has no output.

And yet it's a useful subroutine.

Granted, it appears to be an exception to the rule that all subroutines must have output. And really, these days, the majority of programmers don't go about writing delay loops like this. Nope, they make calls to sleep() which … um … causes the program … to … um … delay … for a period of time.

Well … um …

But …

But it seems wrong to conclude that subroutines may have output. Because outside of a few esoteric subroutines like my DELAY routine, or sleep(), a subroutine must have output or else, why bother calling it?

How to reconcile this?

How indeed.

But what if I now ask the question: “What is output?”

In the case of DELAY, I'm calling the subroutine not for any tangible result, but for the side effect of doing nothing for 3,333 µseconds. And in general, output is a type of side effect (as any functional programmer will tell you), since output affects the state of the program (outputing a “y”, followed by a second “y” changes some state somewhere, if only to advance the printing location of subsequent characters).

So, if side effects are considered a type of output, and DELAY has a side effect, then it can be considered to have output, so now I don't have to reconcile anything—subroutines must have output.

Like I said, it was nothing earth shattering here, but even Euclid started from a simple straight line [2] and from there expanded upon geometry.

Actually, this code assumes a 1MHz clock rate on the CPU. The actual clock rate on the Computer Computer was 0.89MHz, but I used 1MHz for this example to make the math easier on me.
Although you start getting odd results when you question the nature of a straight line.

Sunday, January 14, 2007

Some musings on coroutines

Over a year ago, having nothing else better to do (seeing how the power was out), I did some research into co-routines. I figure now is as good a time as any to talk about some conclusions I came to (seeing how all that programming talk yesterday is inspiring).

Subroutines are special cases of more general program components, called “coroutines.” In contrast to the unsymmetric relationship between a main routine and a subroutine, there is complete symmetry between coroutines, which call on each other.

To understand the coroutine concept, let us consider another way of thinking about subroutines. The viewpoint adopted in the previous section was that a subroutine merely was an extension of the computer hardware, introduced to save lines of coding. This may be true, but another point of view is possible: We may consider the main porogram and the subroutine as a team of programs, with each member of the team having a certain job to do. The main program, in the course of doing its job, will activate the subprogram; the subprogram performs its own function and then activates the main program. We might stretch our imagination to believe that, from the subroutine's point of view, when it exits it is calling the main routine; the main routine continues to perform its duty, then “exits” to the subroutine. The subroutine acts, then calls the main routine again.

This somewhat far-fetched philosophy actually takes place with coroutines, when it is impossible to distinguish which is a subroutine of the other. Suppose we have coroutines A and B; when programming A, we may think of B as our subroutine, but when programming B, we may think of A as our subroutine. … It represents teamwork as in a relay race. Whenever a coroutine is activated, it resumes execution of its program at the point where the action was last suspended.

The Art of Computer Programming, § 1.4.2

When researching just about anything dealing with computers, it's probably best to start with the source, Donald Knuth. § 1.4.2 covers the basics of coroutines and gives an example, which would look something like this (when converted from MIX assembly, used for all the examples in the book, to a C-like langauge, which is a bit easier to understand):

int input(void)
{
  int c;

  while(1)
  {
    c = getchar();
    if (isdigit(c))
    {
      int count = c - '0' + 1;
      c = getchar();
      for ( ; count ; count--)
        yield to output(c);
    }
    else
      yield to output(c);
    if (c == '.')
      return;
  }
}

void output(int c)
{
  int count = 0;
  int c;

  while(1)
  {
    yield to c = input();
    putchar(c);
    if (c == '.')
      return;
    count++;
    if (count == 3)
    {
      putchar(' ');
      count = 0;
    }
  }
}

The thing to understand about coroutines is that it isn't a normal subroutine call. Normally, each time input() is called, control is passed to the first line of the routine, but in a coroutine “call,” control is initially passed to the first line, but each subsequent “call” resumes where input() left off (in this case, marked with the keyword yield). If we change the code a bit (remember, this isn't C, but a C-like langauge), this can become clearer:

void input()
{
  while(1)
  {
    c = getchar();
    if (isdigit(c))
    {
      int count = c - '0' + 1;
      c = getchar();
      for ( ; count ; count--)
        send c to output;
    }
    else
      send c to output;
    if (c == '.')
      return;
  }
}

void output()
{
  int count = 0;
  int c;

  while(1)
  {
    receive c from input;
    putchar(c);
    if (c == '.')
      return;

    count++;
    if (count == 3)
    {
      putchar(' ');
      count = 0;
    }
  }
}

Here, it should be a bit clearer what is going on—input() is getting some data, then passing it to the coroutine output(). I've changed the way that coroutines are “called” for two reasons. One, it's a bit clearer what is going on, and two, there's a bit more going on here than just very fancy gotos. In § 1.4.2, Knuth gives the following illustration (recreated here):

[Fig. 22 from § 1.4.2 of “The Art of Computer Programming”]

Passes: (a) a four-pass algorithm, and (b) a one-pass algorithm.

Those familiar with Unix should see a familiar pattern here:

GenericUnixPrompt> pass-a <input >tape.1
GenericUnixPrompt> pass-b <tape.1 >tape.2
GenericUnixPrmpot> pass-c <tape.2 >tape.3
GenericUnixPrompt> pass-d <tape.3 >output

Or, more succinctly:

GenericUnixPrompt> pass-a < input | pass-b | pass-c | pass-d >output

Coroutines are nothing more than routines that communicate via pipes! A type of message passing, if you will.

Not only that, but given the rise of multi-core processors, this appears to be a natural fit for multithreaded programs. A piped Unix command line uses a separate process for each component, so it appears to be a no-brainer that each component in a coroutine chain can be given its own thread. With that, and some syntactic help, we can rewrite Knuth's example and extend it a bit:

void input()
	receive char;
	send    char;
{
  char c;

  while(1)
  {
    receive c;
    if (isdigit(c))
    {
      int count = c - '0' + 1;
      receive c;
      for ( ; count ; count --)
        send c;
    }
    else
      send c;
    
    if (c == '.')
      return;
  }
}

void output()
	receive char;
	send    char;
{
  int  count = 0;
  char c;

  while(1)
  {
    receive c;
    send c;
    if (c == '.')
      return;

    count++;
    if (count == 3)
    {
      send ' ';
      count = 0;
    }
  }
}

void filter(char from,char to)
	receive char;
	send	char;
{
  while(1)
  {
    accept c;
    if (c == from)
      c = to;
    send c;
  }
}

int main(void)
{
  string in;
  string out;

  in = get_input();
  in => input() 
     => filter('a','n') 
     => filter('e','x') 
     => output() => out;
  print(out);
}

Coroutines make the same fringe problem trivial to implement:

void tree_leaves(Tree t)
	send Tree;
{
  if (t == nil)
    send nil;
  if (is_leaf(t))
    send t;
  send tree_leaves(t->left);
  send tree_leaves(t->right);
}

bool same_fringe(Tree t1,Tree t2)
{
  Tree tmp1;
  Tree tmp2;

  while
  (
  	(tree_leaves(t1) => tmp1)
    &&	(tree_leaves(t2) => tmp2)
  )
  {
    if (tmp1 != tmp2)
      return (false);
  }
  return true;
}

There are even more implications here. The producer/consumer problem is one used to teach semaphores, but with coroutines, it becomes rather trivial:

void producer()
	send	datum;
{
  datum data;

  while(1)
  {
    data = generate_datum();
    send data;
  }
}

void consumer()
	receive	datum;
{
  datum data;

  while(1)
  {
    receive data;
    crunch_datum(data);
  }
}

producer() <=> consumer();

Even with multiple consumers, I don't see a problem with this model:

producer() <=> (consumer(), consumer());

And other problems used to teach semaphores, like the multiple-reader, exclusive writer problem, become trivial:

void master_control_program()
	receive	command;
	receive datum;
	send datum;
{
  command cmd;
  datum   data;

  while(1)
  {
    accept cmd;
    if (cmd == read)
      send data;
    else if (cmd == write)
      receive data;
  }
}

void reader(int id)
{
  datum data;

  while(1)
  {
    send read;
    receive data;
    process(data,id);
  }
}

void writer(int id)
{
  datum data;

  while(1)
  {
    sleep(random());
    data = generate_datum(id);
    send write;
    send data;
  }
}

master_control_program() <=> (	reader(1),
				reader(2),
				reader(3),
				writer(1),
				reader(4),
				writer(2),
				writer(3),
				reader(5));

But in retrospect, this is nothing more than a reinvention of communicating sequential processes, and what I've managed to do is basically hide message passing within the framework of coroutines, which is why I suddenly found myself easily solving problems that normally require semaphores.

Granted, there aren't any langauges that do this sort of thing (well, there is Erlang, but it's a functional language, not imperative like C or C++), and if I were to implement a language with coroutine support like this, there are still issues of implementation and syntax to work out, but I'm still playing around with this stuff for now.

Monday, January 15, 2007

A holiday again?

Wlofie asked if I could give him a ride to the bank and the the post office (he needed a stamp to send in a rebate claim). No problem. We head out and arrive a few minutes later at the bank.

You know, these holidays have a way of sneaking up on us.

No bank. No post office.

But hey, for civil rights, that's fine by me.

Some more musings on programming, given the recent spate of entries on the topic

Patterns are signs of weakness in programming languages.

When we identify and document one, that should not be the end of the story. Rather, we should have the long-term goal of trying to understand how to improve the language so that the pattern becomes invisible or unnecessary.

Design patterns of 1972

While I've heard of the Gang of Four and their book, Design Patterns, I've never quite understood all the hoopla over it—the whole concept seemed pretty silly, or too obvious, to warrant a whole book on the subject. And it seems like I'm not the only one to think that as well, and I agree with the author, design patterns are silly.

Tuesday, January 16, 2007

Captain Napalm's Thermonuclear League of Liberty

From

XXXXXXXXXXXXXXXXXXX

To

sean@conman.org

Subject

Captain Napalm's Thermonuclear League of Liberty

Date

Tue, 16 Jan 2007 10:16:38 -0500

So how do I join and get my very own thermonuclear weapons to play with?

About ten years ago, during one of my site redesigns, I embedded the following in an HTML comment on each page:

Join Captain Napalm's Thermonuclear League of Liberty …
write sean@conman.org for details …

That very same comment was even carried over to my blog—go ahead, check the source code and see. I'll wait.

Back?

Today was the first time anyone bothered to write about joining Captain Napalm's Thermonuclear League of Liberty (and it was my old roommate Rob who wrote in).

And it's been so long since I added it, I have no idea what exactly to do.

Other than maybe change the comment and see if it takes less than ten years for anyone else to notice.

Wednesday, January 17, 2007

The lost continent of Africa

Bunny was showing me some brochures for Florida attractions around Lake Okeechobee (most of which I missed seeing on the trip to Bok Tower) when I remembered Africa USA, an attraction, long since gone, was located in Boca Raton.

From the one aerial shot (which is oriented such that west is along the top edge) and some supplementary data, I was able to locate where Africa USA was located (I had originally thought it was a bit further south, at the intersection of 18^th and Old Dixie Highway). It is now, of course, a large development of condos and single family homes.

While looking for the location of Africa USA, I was once again reminded of a long lost theme park I've come across, this time in Pompano Beach—Storyland, another attraction from the the 50s. This one was a bit easier to locate, given both the aerial shot and address. It is now, of course, a large strip mall (and I wonder if this Storyland is at all related to the former Storyland in Pompano Beach).

Thursday, January 18, 2007

Yeah, I know the feeling …

[The task I must undertate is towering over me like a great big monolith. It is too big to contemplate, so I think I will go and have a little look at the Internet.]

(Via The Adventures of Accordion Guy in the 21^st Century)

Friday, January 19, 2007

Don't think this'll happen any time in the near future.

Five years later and the weather hasn't changed as much as it has in 30 years.

Go figure.

A quick note on embedding languages within languages

While I don't fully agree with this article (link via ThoughtStorms, and more on this in a later entry) I do agree that our current methods of programming do fall short (design patterns being silly). This was made plainly apparent today at The Office.

I have this program called ospfquery that not only retrieves OSPF specific SNMP data, but can also dump a routing table from a router.

  Dest           Mask           NextHop         Proto   Metric Age
-------------------------------------------------------------------------------
169.254.0.0     255.255.0.0     XXXXXXXXXX.5    ospf    1     334866
172.16.0.0      255.240.0.0     XXXXXXXXXX.5    ospf    1     334866
192.0.2.0       255.255.255.0   XXXXXXXXXX.5    ospf    1     334866
192.168.0.0     255.255.0.0     XXXXXXXXXX.5    ospf    1     334866
198.18.0.0      255.254.0.0     XXXXXXXXXX.5    ospf    1     334866

But it doesn't quite work with Riverstones:

  Dest           Mask           NextHop         Proto   Metric Age
-------------------------------------------------------------------------------
1.0.0.0         32.161.7.0      26.127.177.69   ???     159791312426295

It's clear that the Riverstone doesn't support the right SNMP MIBs that ospfquery uses to retrieve the routing table. But by dumping the entire SNMP tree, I was able to find equivalents.

Routing table SNMP MIBs for Cisco and Riverstone
Cisco	Riverstone
RFC1213-MIB::ipRouteDest	IP-MIB::ip.24.4.1.1
RFC1213-MIB::ipRouteMask	IP-MIB::ip.24.4.1.2
RFC1213-MIB::ipRouteNextHop	IP-MIB::ip.24.4.1.4
RFC1213-MIB::ipRouteProto	IP-MIB::ip.24.4.1.7
RFC1213-MIB::ipRouteAge	IP-MIB::ip.24.4.1.8
RFC1213-MIB::ipRouteMetric1	IP-MIB::ip.24.4.1.11
RFC1213-MIB::ipRouteType	IP-MIB::ip.24.4.1.6

So, on to the code:

oid objid_ipRouteDest[] = {1, 3, 6, 1, 2, 1, 4, 21, 1, 1};
int length_ipRouteDest = sizeof(objid_ipRouteDest)/sizeof(oid);
oid objid_ipRouteMetric1[] = {1, 3, 6, 1, 2, 1, 4, 21, 1, 3};
int length_ipRouteMetric1 = sizeof(objid_ipRouteMetric1)/sizeof(oid);
oid objid_ipRouteNextHop[] = {1, 3, 6, 1, 2, 1, 4, 21, 1, 7};
int length_ipRouteNextHop = sizeof(objid_ipRouteNextHop)/sizeof(oid);
oid objid_ipRouteType[] = {1, 3, 6, 1, 2, 1, 4, 21, 1, 8};
int length_ipRouteType = sizeof(objid_ipRouteType)/sizeof(oid);
oid objid_ipRouteProto[] = {1, 3, 6, 1, 2, 1, 4, 21, 1, 9};
int length_ipRouteProto = sizeof(objid_ipRouteProto)/sizeof(oid);
oid objid_ipRouteAge[] = {1, 3, 6, 1, 2, 1, 4, 21, 1, 10};
int length_ipRouteAge = sizeof(objid_ipRouteAge)/sizeof(oid);
oid objid_ipRouteMask[] = {1, 3, 6, 1, 2, 1, 4, 21, 1, 11};
int length_ipRouteMask = sizeof(objid_ipRouteMask)/sizeof(oid);


void collect_route_table(ss)
struct snmp_session *ss;
{
  struct ospfRT *cur;
  struct snmp_pdu *pdu, *response;
  struct variable_list *vars;
  int	    good_var, index;
  int	    status, count;

  pdu = snmp_pdu_create(GETNEXT_REQ_MSG);

  snmp_add_null_var(pdu, objid_ipRouteDest, length_ipRouteDest);
  snmp_add_null_var(pdu, objid_ipRouteMetric1, length_ipRouteMetric1);
  snmp_add_null_var(pdu, objid_ipRouteNextHop, length_ipRouteNextHop);
  snmp_add_null_var(pdu, objid_ipRouteType, length_ipRouteType);
  snmp_add_null_var(pdu, objid_ipRouteProto, length_ipRouteProto);
  snmp_add_null_var(pdu, objid_ipRouteAge, length_ipRouteAge);
  snmp_add_null_var(pdu, objid_ipRouteMask, length_ipRouteMask);

  good_var = 7;
  while(good_var == 7){
    good_var = 0;
    status = snmp_synch_response(ss, pdu, &response);
    if (status == STAT_SUCCESS){
      if (response->errstat == SNMP_ERR_NOERROR){
	pdu = snmp_pdu_create(GETNEXT_REQ_MSG);

	index = 0;
	if (rtp == (struct ospfRT *) 0) {
	    rtp = (struct ospfRT *) malloc(sizeof(struct ospfRT));
	    cur = rtp;
	    cur->prev = (struct ospfRT *) 0;
	}
	else {
	    cur->next = (struct ospfRT *) malloc(sizeof(struct ospfRT));
	    cur->next->prev = cur;
	    cur = cur->next;
	}
	cur->next = (struct ospfRT *) 0;
	for(vars = response->variables; vars; vars = vars->next_variable) {
          if (index == 0 && vars->name_length >= length_ipRouteDest &&
              !bcmp((char *)objid_ipRouteDest, (char *)vars->name,
              sizeof(objid_ipRouteDest))){
                cur->ipRouteDest = *vars->val.integer;
                snmp_add_null_var(pdu, vars->name, vars->name_length);
                good_var++;
          } else if (index == 1 && vars->name_length >= length_ipRouteMetric1 &&
                     !bcmp((char *)objid_ipRouteMetric1, (char *)vars->name,
                     sizeof(objid_ipRouteMetric1))){
            cur->ipRouteMetric1 = *vars->val.integer;
            snmp_add_null_var(pdu, vars->name, vars->name_length);
            good_var++;
          } else if (index == 2 && vars->name_length >= length_ipRouteNextHop &&
                     !bcmp((char *)objid_ipRouteNextHop, (char *)vars->name,
                     sizeof(objid_ipRouteNextHop))){
            cur->ipRouteNextHop = *vars->val.integer;
            snmp_add_null_var(pdu, vars->name, vars->name_length);
            good_var++;
          } else if (index == 3 && vars->name_length >= length_ipRouteType &&
                     !bcmp((char *)objid_ipRouteType, (char *)vars->name,
                     sizeof(objid_ipRouteType))){
              cur->ipRouteType = *vars->val.integer;
              snmp_add_null_var(pdu, vars->name, vars->name_length);
              good_var++;
          } else if (index == 4 && vars->name_length >= length_ipRouteProto &&
                     !bcmp((char *)objid_ipRouteProto, (char *)vars->name,
                     sizeof(objid_ipRouteProto))){
            cur->ipRouteProto = *vars->val.integer;
            snmp_add_null_var(pdu, vars->name, vars->name_length);
            good_var++;
          } else if (index == 5 && vars->name_length >= length_ipRouteAge &&
                     !bcmp((char *)objid_ipRouteAge, (char *)vars->name,
                     sizeof(objid_ipRouteAge))){
            cur->ipRouteAge = *vars->val.integer;
            snmp_add_null_var(pdu, vars->name, vars->name_length);
            good_var++;
          } else if (index == 6 && vars->name_length >= length_ipRouteMask &&
                     !bcmp((char *)objid_ipRouteMask, (char *)vars->name,
                     sizeof(objid_ipRouteMask))){
            cur->ipRouteMask = *vars->val.integer;
            snmp_add_null_var(pdu, vars->name, vars->name_length);
            good_var++;
          }
          index++;
        }
      } else {
        printf("Error in packet.\nReason: %s\n", snmp_errstring(response->errstat));
        if (response->errstat == SNMP_ERR_NOSUCHNAME){
          printf("This name doesn't exist: ");
          for(count = 1, vars = response->variables; vars && count != response->errindex;
              vars = vars->next_variable, count++)
			    ;
          if (vars)
            print_objid(vars->name, vars->name_length);
          printf("\n");
        }
      }

    } else if (status == STAT_TIMEOUT){
      printf("No Response from router\n");
      exit(1);
    } else {    /* status == STAT_ERROR */
      printf("An error occurred, Quitting\n");
      exit(2);
    }

    if (response)
      snmp_free_pdu(response);
  }
    /* get rid of last element that contains garbage. */
    /* this loop is ugly and copied from CMU. It needs rewritten */

  if (cur->prev) {
    cur->prev->next = (struct ospfRT *) 0;
    free(cur);
  }
}

Um … yeah … (and that's before adding support for the other set of SNMP MIBs)

Ideally, I'd love do to something like:

OID sys = SNMPv2-MIB::sysObjectID.0;
  
if (sys == SNMPv2-SMI::enterprises.5567.1.1)  /* riverstone */
{
  IpAddress destination[] = IP-MIB::ip.24.4.1.1;
  IpAddress mask[]        = IP-MIB::ip.24.4.1.2;
  IpAddress nexthop[]     = IP-MIB::ip.24.4.1.4;
  int       protocol[]    = IP-MIB::ip.24.4.1.7;
  int       age[]         = IP-MIB::ip.24.4.1.8;
  int       metric[]      = IP-MIB::ip.24.4.1.11;
  int       type[]        = IP-MIB::ip.24.4.1.6;
}
else if (sys == SNMPv2-SMI::enterprises.9.1)  /* cisco */
{
  IpAddress destination[] = RFC1213-MIB::ipRouteDest;
  IpAddress mask[]        = RFC1213-MIB::ipRouteMask;
  IpAddress nexthop[]     = RFC1213-MIB::ipRouteNextHop;
  int       protocol[]    = RFC1213-MIB::ipRouteProto;
  int       age[]         = RFC1213-MIB::ipRouteAge;
  int       metric[]      = RFC1213-MIB::ipRouteMetric1;
  int       type[]        = RFC1213-MIB::ipRouteType;
}

for (i = 0 ; i < destination.length; i++)
{
    print(
        destination[i],
        mask[i],
        nexthop[i],
        snmp.protocol(protocol[i]),
        metric[i],
        age[i]
    );
}

I would love to make SNMP queries like this, but the ability to embed a secondary language within another language is difficult at best. I do recall in college, I seem to dimly recall the ability to embed SQL within C (or was is Pascal? It was on the VAX system). Something like:

int authenticate(char *user,char *password)
{
  dbpass = ~~ SELECT password 
		FROM g_database.users 
		WHERE user="%user%" ~~;

  if (dbpass == NULL)
    return (FALSE);
  rc = (strcmp(dbpass,password) == 0);
  free(dbpass);
  return (rc);
}

Much easier than building SQL strings and hoping you get the quoting right.

But alas, such tools and/or languages don't exist (or have ceased to exist) and I'm stuck patching this program to support an alternative set of SNMP MIBs to pull the routing table from Riverstones.

Upate on Thursday, July 22^nd, 2010

I found a way to do what I wanted …

Saturday, January 20, 2007

Reference materials

Since Bunny was curious about this, here goes.

I generally do the highly technical (of somewhat long) entries from home, where access to reference materials are very close at hand (usually, behind me, although occasionally across the room). This is important because I try to make them as accurate as possible; it's usually the small details that count (like in yesterday's entry where the code sample of how I'd like to read SNMP variables has a bug that would probably prevent it from compiling if indeed, I could compile such code—I'll leave it as an exercise for the reader to find it).

And some posts require a fair bit of reference material.

[This is six inches of reference material right here]

The above photo is the reference material for a post I'm working on, which is related to the one I made about coroutines, but this time, goes into details about various methods of passing data between routines (and in case you can't make it out, that slim black volume in the middle is QNX System Architecture, and yes, the VAX, the Amiga, the Motorola 68K family, and QNX are all related in a James Burkeian way that I hope to explore in a future entry).

Yes.

Parameter passing.

It should be noted that this blog serves a purpose for me, that is, it's a notebook of events, thoughts and ideas for use by my future self. As well as the occasional random person coming through here via the search engines looking for some obscure piece of data, like the SNMP MIBs required to dump a routing table from a Riverstone.

Sunday, January 21, 2007

Africa USA fifty-four years later

This afternoon, Spring sprung a surprise straggle—why not visit Africa USA? Or rather, what's left of Africa USA.

It was fairly easy to find and get to, despite it appearing like a gated community (there were, in fact, no closed gates, and the gate house itself was devoid of anything). The only structure left from the park is a bridge, leading to an overgrown patch of land with a decaying asphalt walkway. The bridge to Monkey Island has long since fallen apart.

Fortunately for us, no velociraptors were left when the park closed.

[Although it's not very well guarded for a gated community]

[It is surrounded by Soviet style apartment buildings]

[Making it a kind of “Soviet Bloc” if you will]

[Welcome to all that's left of Africa USA]

[About the only structure remaining from Africa USA]

[The bridge to Monkey Island is no longer around]

[although with the water level like this, you might not need a bridge to get to Monkey Island]

[Signs of civilization are nearby though]

[The end of the path at Africa USA leads to the Soviet Bloc]

[Some of the vegetation overgrowth at Africa USA]

Monday, January 22, 2007

Yes, I can say “intellectualizing things into the ground.”

You think that's bad? Just look at how my joke about Leni Riefenstahl and Snuggles the Bear has spawned this mega-crossposted thread about the differences between fascism and absolute monarchy.

Can you say, “intellectualizing things into the ground?”

Douglas Lathrop

Via Eric Burns is Visited States Page, where you can get a map highlighting the states you've visited.

[States I've visited, although “wierd” wasn't listed]

The red states represent the ones I've “visited” although I use that term very loosely.

For instance, while I have Texas highlighted, I've only seen the state while sitting in the Dallas-Ft. Worth airport, so to say I “visited” the state is a bit of stretch. Yes, technically I was in Texas, I was looking at plastic seats in an air conditioned terminal that could have been anywhere to tell the truth.

Arizona is also highlighted, but again, it's a technicality—albeit with a better claim of “visited” than Texas. Hoade and I actually saw the state when we parked our car on the Arizona side of Hoover Dam, but it wasn't much different looking than Nevada, and we only really ventured maybe a quarter mile into the state.

I can say with better conviction that I've “vistied” Wisconsin, but that was in Milwaukee, and even then, downtown Milwaukee at the Convention Center (and a few blocks west to the hotel). Actually, I feel better about saying I “visited” Milwaukee than I do about “visiting” Wisconsin. The same about New York—It was New York City I “visited,” not the state at large. Pennsylvania wasn't much better. Sure, I flew into the city, but I actually stayed in New Jersey.

Tennessee, Kentucky and Ohio only get mention because they were “drive through” states—back when I lived in North Carolina, my Grandma would drive down from Michigan to pick me up for the summer vacation. The route followed I-75 through those states.

Maryland was a “sail through” state—one summer I spent with my Dad sailing along the Chesapeake Bay, so was I really in Maryland, or just alongside it?

Really, about the only state I can say that I've “visited” is Florida (and yes, I live here). I've been along both the east and west coast, central and north Florida, and out through the panhandle.

The only other state I've seen significant portions of is North Carolina, and I've yet to actually see the central portion (having lived in both the mountainous western portion, and the coastal eastern portion).

Then again, I'm probably over intellectualizing a silly web meme.

You can't make this stuff up

Yes, the UFCW is outsourcing its picketing to non-union workers, at a sub-standard pay rate with no benefits, in unsafe conditions, with no transportation or means to leave the premises, in order to protest the poor jobs inside Wal★Mart, where workers make twice as much.

Via Right on the Left Coast, The. Best. Union. Story. Ever.

Wlofie and I have had several discussions about unions, how they work here and in Sweden, and I feel he feels that our unions are all smoking crack.

Yes, I would have to agree with him.

Tuesday, January 23, 2007

An even longer entry to scare away the new readers

Now, about that article … (hold on, it's a long one)

I'm talking about the limitations of programming which force the programmer to think like the computer rather than having the computer think more like the programmer. These are serious, deeply-ingrained limitations which will take a lot of effort to overcome. I'm not being pretentious when I say that this will be the next big paradigm shift in programming. We will need to completely redefine the way we write programs.

In this article, I present my view and my current work toward Language Oriented Programming (LOP). First I will show what is wrong with mainstream programming today, then I'll explain the concept of LOP by using the example of my existing implementation, the Meta Programming System (MPS). This article is intended to give you a bird's-eye-view of LOP, to spark interest in the idea, and hopefully to generate feedback and discussion.

Language Oriented Programming: The Next Programming Paradigm

The concept of using the best language suited to the problem at hand dates back to the first computer language devises, Fortran. The very name, Fortran, stands for “Formula Translation” and was a way to express mathematical equations in the more natural notation of

xn1 = ((A * yn) + B) * xn * (1.0 - xn)
yn1 = ((C * xn) + D) * yn * (1.0 - yn)

and have the computer take the work of translating that to

	fld	[A]
	fmul	[yn]
	fadd	[B]
	fmul	[xn]
	fld	[xn]
	fld1
	fsub
	fmul
	fst	[xn1]

	fld	[C]
	fmul	[xn]
	fadd	[D]
	fmul	[yn]
	fld	[yn]
	fld1
	fsub
	fmul
	fst	[yn1]

(the sample above assumes the instruction set supports floating point—if it doesn't, then you have a lot more code to call subroutines to do the work of adding, subtracting and multiplying floating point numbers; it's not pretty)

And even if a single language doesn't perfectly handle the entire problem domain, it has been possible for quite some time to do multilanague compilations, depending up platform. The most common method is to embed some routines written in Assembly to speed things up (since everything is converted to Assembly eventually), but as long as an operating system defines a standardized way to pass data between routines, it's not that hard to mix-n match routines from multiple languages.

I'm looking at the technical information for OS-9 (not to be confused with Mac OS 9), specifically for software modules:

A module need not be a complete program or even 6809 machine language. It may contain BASIC09 “I-code,” constants, single subroutines, and subroutine packages.

The module header contains a value specifying the language type, with values defined for data only, 6809 machine code, BASIC09-I-code, Pascal I-code and COBOL I-code. In theory then, you can construct, say, a pay roll system using a Fortran compiler to generate the math routines into 6809 code, COBOL to generate the business rules, and allow extentions to be written in BASIC. The language type is probably there for two reasons; 1) to know how to run the module, and 2) to figure out what parameter passing conventions to use (if there are any differences between the languages).

AmigaOS has a similar feature—you can create external libraries in any language you care to (say, in Pascal) and call them in programs written in any other language (say, in C—since the operating system itself is a library, any compiler on the Amiga had to support the paramter passing convention used in the system libraries, and while the only compiler I used on the Amiga was a C compiler, it had extentions to make external libraries, so I assume other compilers would have the same).

Even Microsoft Windows and IBM's OS/2 had a similar feature—DLLs. And from what I understand, it's not uncommon to have a “program” using DLLs written in C, C++ and Visual Basic at the same time.

But what Sergey Dmitriev is talking about is not multi-language programs per se but developing a domain specific languages to solve the problem. Or multiple domain specific languages if that's required.

And I have problems not only with that concept, but with his particular way of doing that as well. And no, it's not having to learn a million tiny computer languages (that's actually expected in this industry—when I started, the first language taught at FAU was Fortran, and while I was there, I saw the switch to Pascal and then C; after I left, it went to C++ and then Java).

That's not to say I have anything against DSLs—I think they're great, and I've written one or two in my time. When I worked at FAU, I ended up writing a DSL to generate programs to run Miller's Analogies (I worked as a programmer in the Math Department for a professor of Psychology—no, I don't fully understand how or why a psychologist ended up in the math department) which looked like:

mat 'CROWN' isto 'ROYAL' as
        'prayer'
        'crucifix'
        'priesthood'
        'bible'         [sel]
isto 'RELIGIOUS'
'b'     answer

mat 'SMALL' isto 
        'tiny'
        'petite'
        'little'
        'diminutive'    [sel]
as 'LARGE' isto 'BIG'
'c' answer

mat 'WORM' isto 'BIRD' as 'MOUSE' isto
        'man'
        'snake'
        'rodent'
        'lion'          [sel]
'b' answer

mat
        'artist'
        'description'
        'narration'
        'personality'   [sel]
isto 'CHARACTERIZATION' as 'PICTURE' isto 'PORTRAIT'
'b' answer

mat
        'orate'
        'sing'
        'mumble'
        'speak'         [sel]
isto 'TALK' as 'SCRAWL' isto 'WRITE'
'c' answer

But there was one problem with this, and it's one of the problems I have with DSLs—the time it took me to write the DSL, which generated C code to handle the Miller's Analogies exceeded the time it would have taken me to write the C code initially! How much time? It took me maybe a month or so to write the DSL to the point where it would generate the necessary C code, maybe longer—it's been over ten years.

Now, I was lucky in that my job wasn't exactly fast paced, and I had the luxury of playing around with language design even though that wasn't my job. But as it was, I only did the one program with Miller's Analogies and from a purely economical standpoint, the work I did was wasteful (educational wise, it wasn't, as I was able to later use the work I did for the DSL as part of a class project, but sometimes it can be hard to justify research work in a non-research environment). So basically, unless the time it takes to write the DSL and the solution in said DSL is smaller than the time it would take to write the solution in a pre-existing language, or the DSL can be used more than once, it's not economically viable.

The other problem I have with DSL—what if you have the wrong mental model of the problem?

Going back to my hypothetical SNMP language extentions. The code as I wrote it looked like (bug fixed in this version):

IPAddress destination[];
IPAddress mask[];
IPAddress nexthop[];
int       protocol[];
int       age[];
int       metric[];
int       type[];
OID       sys = SNMPv2-MIB::sysObjectID.0;

if (sys == SNMPv2-SMI::enterprises.5567.1.1)	/* riverstone */
{
  destination = IP-MIB::ip.24.4.1.1;
  mask        = IP-MIB::ip.24.4.1.2;
  nexthop     = IP-MIB::ip.24.4.1.4;
  protocol    = IP-MIB::ip.24.4.1.7;
  age         = IP-MIB::ip.24.4.1.8;
  metric      = IP-MIB::ip.24.4.1.11;
  type        = IP-MIB::ip.24.4.1.6;
}
else if (sys == SNMPv2-SMI::enterprises.9.1)	/* cisco */
{
  destination = RFC1213-MIB::ipRouteDest;
  mask        = RFC1213-MIB::ipRouteMask;
  nexthop     = RFC1213-MIB::ipRouteNextHop;
  protocol    = RFC1213-MIB::ipRouteProto;
  age         = RFC1213-MIB::ipRouteAge;
  metric      = RFC1213-MIB::ipRouteMetric1;
  type        = RFC1213-MIB::ipRouteType;
}

/* skip the printing part */

Nice. Concise. And totally wrong!

Well, not totally—it'll work as is, but it does have a major problem, and it stems from my not fully understanding the SNMP protocol (ah, gotta love those leaky abstractions). SNMP is a network protocol, and each request for data via SNMP requires a round trip across the network. That's fine for single values, like the request for SNMPv2-MIB::sysObjectID.0. But the MIB RFC1213-MIB::ipRouteDest is a “table” (an “array” in SNMP-speak) and each element requires a request (it's a bit more complicated than that, but it's not germane to this discussion). The code above is basically doing the following (and I'll only follow the Cisco branch since it's a lot of code here):

for (
      i = 0 , mib = RFC1213-MIB::ipRouteDest;
      mib != NULL ; 
      destination[i++] = mib.result
    )
{
  send(mib);	/* send the requst */
  /*--------------------------------------------------
  ; due to the nature of SNMP tables, you get back
  ; not only the value of the MIB you asked for, but
  ; the MIB of the next entry in the table.
  ;
  ; Hey, I didn't write SNMP
  ;-------------------------------------------------*/

  mib = receive();
}

for (
      i = 0 , mib = RFC1213-MIB::ipRouteMask;
      mib != NULL;
      mask[i++] = mib.result
    )
{
  send(mib);
  mib = receive();
}

for (
      i = 0 , mib = RFC1213-MIB::ipRouteNextHop;
      mib != NULL;
      nexthop[i++] = mib.result
    )
{
  send(mib);
  mib = receive();
}

for (
      i = 0 , mib = RFC1213-MIB::ipRouteProto;
      mib != NULL;
      protocol[i++] = mib.result
    )
{
  send(mib);
  mib = receive();
}

for (
      i = 0 , mib = RFC1213-MIB::ipRouteAge;
      mib != NULL;
      age[i++] = mib.result
    )
{
  send(mib);
  mib = receive();
}

for (
      i = 0 , mib = RFC1213-MIB::ipRouteMetric1;
      mib != NULL;
      metric[i++] = mib.result
    )
{
  send(mib);
  mib = receive();
}

for (
      i = 0 , mib = RFC1213-MIB::ipRouteType;
      mib != NULL;
      metric[i++] = mib.result
    )
{
  send(mib);
  mib = receive();
}

But the original code in ospfquery is doing the following:

mib.dest     = RFC1213-MIB::ipRouteDest;
mib.mask     = RFC1213-MIB::ipRouteMask;
mib.nexthop  = RFC1213-MIB::ipRouteNextHop;
mib.protocol = RFC1213-MIB::ipRouteProtocol;
mib.age      = RFC1213-MIB::ipRouteAge;
mib.metric   = RFC1213-MIB::ipRouteMetric1;
mib.type     = RFC1213-MIB::ipRouteType;
i            = 0;

while(mib.dest)
{
  send(			/* yes, all this is sent */
	mib.dest,	/* in one packet */
	mib.mask,
	mib.nexthop,
	mib.protocol,
	mib.age,
	mib.metric,
	mib.type
  );
  
  mib.dest,		/* and received in one packet */
  mib.mask,
  mib.nexthop,
  mib.protocol,
  mib.age,
  mib.metric,
  mib.type = receive();

  destination[i] = mib.dest.result;
  mask[i]        = mib.mask.result;
  nexthop[i]     = mib.nexthop.result;
  protocol[i]    = mib.protocol.result;
  age[i]         = mib.protocol.result;
  metric[i]      = mib.metric.result;
  type[i]        = mib.type.result;
  i++;
}

Now, I should mention that send() causes a single SNMP packet to be sent, and receive() receives a single SNMP packet. With that, you can now see that the code as I envisioned it sends a metric buttload of traffic compared to the code as ospfquery implements (about seven times the traffic). The other major difference is that my code requires all the data to be collected before it can be printed, whereas the code in ospfquery can print the results as it gets them (not that it does currently, but it can). So even if the two methods take the same amount of time, the second one seem faster (it's the perception that it's faster that is important in this case).

The distinction is subtle, and on our routers it's mostly academic, what with their two dozen or so routes. But I actually ran ospfquery on the core router, with its 78,500 routes, and the difference is no longer academic (ospfquery does not print routes until it collects all the data, and at first, I thought the program might have crashed; nope, it just took a while).

And the problem stems from my misunderstanding of the problem, and my proposed DSL, while it works, is sub-optimal and in fact, may cause problems as it scales (in this particular case).

I've also seen what happens to code over time as multiple people maintain the code. I shudder to think what would happen to a language over time as multiple people maintain it (hmmm … that might help explain Perl—I'll have to think on this).

Now, getting back to Mr. Dmitiev's article:

When a compiler compiles source code, it parses the text into a tree-like graph structure called an abstract syntax tree. Programmers do essentially the same operation mentally when they read source code. We still have to think about the tree-like structure of the program. That's why we have brackets and braces and parentheses. It's also why we need to format and indent code and follow coding conventions, so that it is easier to read the source.

Why do we resort to text storage? Because currently, the most convenient and universal way to read and edit programs is with a text editor. But we pay a price because text representations of programs have big drawbacks, the most important of which is that text-based programming languages are very difficult to extend. If programs are stored as text, you need an unambiguous grammar to parse the program. As features are added to the language, it becomes increasingly difficult to add new extensions without making the language ambiguous. We would need to invent more types of brackets, operators, keywords, rules of ordering, nesting, etc. Language designers spend enormous amounts of time thinking about text syntax and trying to find new ways to extend it.

If we are going to make creating languages easy, we need to separate the representation and storage of the program from the program itself. We should store programs directly as a structured graph, since this allows us to make any extensions we like to the language. Sometimes, we wouldn't even need to consider text storage at all. A good example of this today is an Excel spreadsheet. Ninety-nine percent of people don't need to deal with the stored format at all, and there are always import and export features when the issue comes up. The only real reason we use text today is because we don't have any better editors than text editors. But we can change this.

Language Oriented Programming: The Next Programming Paradigm

A laudable idea, but not quite so simple as Mr. Dmitriev makes it out to be. Sure, the example I gave before:

xn1 = ((A * yn) + B) * xn * (1.0 - xn)
yn1 = ((C * xn) + D) * yn * (1.0 - yn)

Can be made easily into a graph:

[graph of “xn1 = ((A * yn) + B) * xn * (1.0 - xn)”]

(Okay, so it's a graph of just the first statement)

I even color coded it—red for modified variables, green for variable references, blue for constants, and black for operations. But what about the following bit of code for a Connecton Machine?

(DEFUN PATH-LENGTH (A B G)
  α(SETF (LABEL •G) +INF)		; step 1
   (SETF (LABEL A) 0)			; step 2
   (LOOP UNTIL (< (LABEL B) +INF)	; step 3
         DO α(SETF (LABEL •(REMOVE A G))
                   (1+ (βMIN α(LABEL •(NEIGHBORS •G))))))
   (LABEL B))				; step 4

This algorithm, which finds the length of the shortest path between A and B in graph G is as follows:

Label all vertices with +∞.
Label vertex A with 0.
Label every vertex, except A, with 1 plus the minimum of its neighbor's labels. Repeat this step until the label of vertex B is finite.
Terminate. The label of B is the answer.

The notation is a bit weird I admit, but basically, “α” is similar to Common Lisp's MAPCAR, which maps a function to each element of a list, so the first line could be translated as:

(MAPCAR #'(LAMBDA (V) (SETF (LABEL V) +INF)) G)

The “•” is what we're mapping over, and “β” is a reduce operation—it maps a function over each element of a function; this function accumulates a single value result. For instance:

(REDUCE #'+ 0 '(3 7 2 99 4 3))

will add all the elements in a list and return a sum, whereas:

(REDUCE #'MAX 0 '(3 7 2 99 4 3))

will return the smallest value in a list. “α,” “β” and “•” aren't actually MAPCAR, REDUCE and the target we're iterating over, but it's close enough.

Now, getting back to my argument here. Again, the code itself is pretty easy to turn into a tree (it already is in tree format, but it's hard to see the tree for all the parenthesis) but still, what does that give us? While I think the Connection Machine notation is elegant, C doesn't exactly have the concept of MAPCAR & Co. It can be faked, sure, but it's not an inherent part of the language.

But what it seems Mr. Dmitriev is working on is a type of automated template system, where you define your extensions, then create a templated implementation for your target language, only using a specialized editor. You do this for everything—from simple operations and assignments up through function calls and modules. And you need to define the ways these things all interact (hopefully to avoid some of the darker corners of C++, for instance).

Seems like quite a bit work, even if Mr. Dmitriev states a lot of this is automagically generated (and that's scary in and of itself—automagically generated stuff; I tend to distrust automagically generated stuff unless I understand exactly what's going on).

And from the looks of Mr. Dmitriev's editor (in his system, you don't manipulate textual source code, which is primitive stone age programming) it appears to target Java; I wonder how it would fare in trying to extend C with Lisp like features? Or Lisp with Forth like features? Or even C with my propsed SNMP like features?

Oh, and the reason we still use text to store source code? Because I can still load 40 year old source code into my text editor and modify it. What happens 40 years down the line with code written in Mr. Dmitriev's editor?

Just because.

I guess the part I don't understand is the target audience. Who is so serious about writing that they need a full-screen editor, but so unserious that they don't have a favorite editor already? I've published two full- length books and posted a hell of a lot more than that, and you can pry my text editor from my cold dead hands. I'm not even going to mention which one it is; it doesn't matter. Switching to a new one would be a frustrating and painful experience that would get in the way of my writing for weeks, maybe months.

Via decafbad.com, Mark Pilgrim: wrongroom

It could also be “because I can.”

Or because the software being used sucks and it's easier to start over than try to work on an existing project.

Any number of reasons.

But I can certainly relate to the trauma of switching editors. Heck, it was traumatic switching email clients and I'm still finding flaws in mutt (just last night, I patched it to change the default location for saving attachments, since I hate cluttering up my home directory).

Wednesday, January 24, 2007

The Good Shepard

“The Good Shepard” is about Edward Wilson (played by Matt Damon) and the founding of the CIA. The movie starts with The Bay of Pigs and Wilson's involvement in it. Then through a series of flashbacks we follow the life of Wilson as he attends Yale, is picked for OSS during World War II and later as it evolves into the CIA.

It's a good film, and rather understated for a major Hollywood production, and it didn't feel like the three hour film that it was (a good sign). There are good performances from everybody in the film, but a problem I had was with Matt Damon. While he's good as Edward Wilson, I found it very hard to accept him as middle aged with a son in college (during the Bay of Pigs incident, around which the movie plot revolves). I just found him too young looking to be the middle age secret service beaurocrat—even Angelina Jolie as his wife manged to “age” more believably than Matt Damon.

But that's really my only gripe with this otherwise good film.

Thursday, January 25, 2007

That's not what I wanted, Firefox

That was odd.

I'm working remotely, using a Linux desktop system, and I'm trying to log into the trouble ticket system at The Office. It's a web-based trouble ticket system, so I'm using Firefox as my browser of choice. Now, because I haven't logged into the trouble ticket system from where I'm currently am, it's not bookmarked or in my browser history.

“No problem,” I say to myself. “I'm using X Windows, and since my workstation at The Office also uses X Windows, I can run Firefox on my workstation and have it displayed on the computer I'm currently using.” I mean, that's the whole purpose of X—you can run applications on one computer and have its windows displayed elsewhere. This is so fundamental to X that its been a feature since the late 80s (pcAnywhere is a Johnny-come-lately to remote GUI software).

So I log into my workstation and run Firefox. Yes, there is a lag before the window pops up, but I'm expecting that—I am running a graphical program across the Internet. But the window … it looks … wrong. I mean, it is Firefox, but the window isn't big, and the bookmarks aren't the ones I expect.

In fact, it looks much like the bookmarks on the copy of Firefox I'm running locally. And in fact, the browser history doesn't show the trouble ticket system.

In fact, the Firefox on my workstation at The Office apparently sent a command message to the Firefox on the computer in front of me to open up a new window.

Which, while pretty cool in a “network optimization” way, is not what I was expecting. To get around this mess, I had to shut down the local Firefox before firing up Firefox at The Office.

I know of no other X program that exhibits this behavior. xeyes? It'll run remote and locally. xterm? It'll run remote and locally. Gimp? It'll run remote and locally. Firefox? It apparently goes out of its way to run in only one location—must be a vestige function left over when it was the commercial product Netscape.

There is a way around it though:

It seems the switch you are looking for is -no-remote. And yes, it does not appear with you run firefox -h, because firefox -h gives you the sitches supported by the program firefox.

What's this -no-remote then, you say? Well that switch is implemented by the wrapper script. Pish tosh, a wrapper script is some silly linux distribution script, you would think. But look at the license and the copyright and you'll find this is part of the standard firefox distribution. The method for getting the wrapper script appears to be to open it in vi. At least, that's the method I used. My firefox manpage does mention it too, but that's hardly an excuse.

spc hates software: Okay Firefox, that's … interesting. Now cut it out!

Oddly enough, the man page I have on my workstation at The Office fails to mention -no-remote, and the man page on the computer I'm using fails to mention -no-remote—they do, however, mention - remote, but that appears to be redundant, since that appears to be the default action anyway.

Sigh.

I, for one, welcome our new computer based overlords

I spoke recently with an old friend who is a bandwidth broker. He buys and sells bandwidth on fiber-optic networks around the world. And he told me something that I found not completely surprising, but I certainly hadn't known: Google controls more network fiber than any other organization. This is not to say that Google owns all that fiber, just that they control it through agreements with network operators. I find two very interesting aspects to this story: 1) that Google has acquired—or even needs to acquire—so much bandwidth, and; 2) that they don't own it, since probably the cheapest way to pick up that volume of fiber would be to simply buy out any number of backbone providers like Level 3 Communications.

Wh en Being a Verb is Not Enough: Google wants to be YOUR Internet.

A long staple of science fiction is the sentient computer. From When Harlie was One and The Adolescence of P-1 to The Forbin Project, we've had computers becoming aware and then possibly trying to take over the world (or the moon, in the case of The Moon is a Harsh Mistress) but in all these cases, it was a single computer gaining self-awareness; never was it a cluster of machines (well, maybe the Borg but those aren't exactly computers we're talking about).

But Google?

Google has thousands of computers, all networked. Each computer is simple, but then again, so is a single neuron. But all those simple computers are connected together, again, like neurons. And I think that our conscienceness arises from the pattern of said connections; that you get emergent behavior from billions of such simple connections. And while Google is far from having billions of connections, it certainly has more connections than just about anything else. Unlike, say, my site, where you can point to a single computer (currently in Miami) and say “that's Conman Laboratories,” you just can't point to a single computer and say “that's Google,” much like you can't point to a single skin cell and say “that's Sean Conner.”

Google is more than the sum of its parts.

And if any computer, or cluster of computers, will exhibit emergent behavior, it will be Google (or more technically, the Google cluster).

I'm wondering if that hasn't already happened to some degree.

Or maybe, just maybe, Google is Skynet …

Friday, January 26, 2007

No, I haven't forgotten about it

But I have been neglecting it for the past few months. Partly due to the Holiday Season, and partly because I was (or did) run out of ideas to write about at the Saltmine Chronicles.

And all this is just to inform those that are interested that I just posted—an entry about good and bad web traffic.

A leaky domain specific language

Continuing on about that article, there's yet another reason to be wary of DSLs: The Law of Leaky Abstractions (which itself is an extension of the wrong mental model, to a degree).

Going back to my hypothetical SNMP extensions:

IPAddress destination[];
IPAddress mask[];
IPAddress nexthop[];
int       protocol[];
int       age[];
int       metric[];
int       type[];
OID       sys = SNMPv2-MIB::sysObjectID.0;

if (sys == SNMPv2-SMI::enterprises.5567.1.1)	/* riverstone */
{
  destination = IP-MIB::ip.24.4.1.1;
  mask        = IP-MIB::ip.24.4.1.2;
  nexthop     = IP-MIB::ip.24.4.1.4;
  protocol    = IP-MIB::ip.24.4.1.7;
  age         = IP-MIB::ip.24.4.1.8;
  metric      = IP-MIB::ip.24.4.1.11;
  type        = IP-MIB::ip.24.4.1.6;
}
else if (sys == SNMPv2-SMI::enterprises.9.1)	/* cisco */
{
  destination = RFC1213-MIB::ipRouteDest;
  mask        = RFC1213-MIB::ipRouteMask;
  nexthop     = RFC1213-MIB::ipRouteNextHop;
  protocol    = RFC1213-MIB::ipRouteProto;
  age         = RFC1213-MIB::ipRouteAge;
  metric      = RFC1213-MIB::ipRouteMetric1;
  type        = RFC1213-MIB::ipRouteType;
}

/* skip the printing part */

This time, I'll be concentrating on just querying one SNMP variable:

OID sys = SNMPv2-MIB::sysObjectID.0;

if (sys == SNMPv2-SMI::enterprises.5567.1.1)
  printf("Riverstone\n");
else if (sys == SNMPv2-SMI::enterprises.9.1)
  printf("Cisco\n");
else
  printf("Unknown\n");

I thought it would be instructive to write the minimum amount of code that actually does this, using the net-snmp library. And yes, it was instructive.

/********************************************
* to compile, install net-snmp, then
*
* gcc -o sysid sysid.c -lsnmp -lcrypto
*
*********************************************/

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>

#include <net-snmp/net-snmp-config.h>
#include <net-snmp/utilities.h>
#include <net-snmp/net-snmp-includes.h>

/*************************************************************************/

int main(int argc,char *argv[])
{
  static oid       sysObjectId[] = { 1,3,6,1,2,1,1,2,0};
  static oid       rs[]          = { 1,3,6,1,4,1,5567 };
  static oid       cisco[]       = { 1,3,6,1,4,1,9 };
  netsnmp_session  session;
  netsnmp_session *ss;
  netsnmp_pdu     *pdu;
  netsnmp_pdu     *response;
  int             status;

  snmp_parse_args(argc,argv,&session,"",NULL);

  ss = snmp_open(&session);
  if (ss == NULL) exit(1);

  pdu = snmp_pdu_create(SNMP_MSG_GET);
  snmp_add_null_var(pdu,sysObjectId,9);
  status = snmp_synch_response(ss,pdu,&response);
  
  if ((status == STAT_SUCCESS) && (response->errstat == SNMP_ERR_NOERROR))
  {
    if (memcmp(response->variables->val.objid,rs,sizeof(rs)) == 0)
      printf("Riverstone\n");
    else if (memcmp(response->variables->val.objid,cisco,sizeof(cisco)) == 0)
      printf("Cisco\n");
    else
      printf("Unknown\n");
  }

  snmp_free_pdu(response);
  snmp_close(ss);
  return(EXIT_SUCCESS);
}

So, where are the leaks?

Well, there's creating the actual session, or some other way of specifying which router we're attempting to query and the credientials we need to actually obtain the information (and there are three versions of the SNMP protocol, so we need to specify that as well somewhere). In my code, that's hidden behind the call to snmp_parse_args(), which expects to parse the command line and creates a template “session” for us (and believe me, I tried to see if there was any other way to construct this template “session” and I couldn't find it in the hour or so I looked). This is leak number one.

Then there's the actual call itself. You create the message, fill it with the variable(s) you want, then send the query and get a response back. But, there are several things that can go wrong, which is leak number two.

The first thing is that we never get a response back—the SNMP library simply times out (is the router down? A bad network cable? Who knows?). That's reflected in the return code from snmp_synch_response(). But even if we get a response back, the far side, the device we're querying via SNMP, can return an error—perhaps it doesn't support the MIB we requesting. Or the response itself got corrupted on the way back. And this is still part of leak number two.

The primary line I'm trying to implement:

OID sys = SNMPv2-MIB::sysObjectID.0;

So far, there are three things (at least, possibly more) that could go wrong: I never get a reponse back, the response I get back is an error, or I get the actual data I requested. What now?

Well, my hypothetical langauge extension could just set sys to NULL, indicating an error, but what error? Well, I could just stuff something into errno (if I'm extending C, for example):

string contact  = SNMPv2-MIB::sysContact.0;
string name     = SNMPv2-MIB::sysName.0;
string location = SNMPv2-MIB::sysLocation.0;

if ((contact == NULL) || (name == NULL) || (location == NULL))
{
  if (errno == ETIMEOUT)	/* timeout */
    ...
  else if (errno == EMIBEXIST)	/* MIB not supported */
    ...
  else
    ...
}

But that's error detection, and that's a whole other ball of mud.

And setting aside I still haven't specified which device I'm querying, this has another leak—I should bundle these three queries into a single SNMP query for better (faster, less bandwidth intensive) performance. I suppose the “compiler” could recognize this and do the bundling for us, but that's some sophisticated programming for just a seemingly simple DSL.

Saturday, January 27, 2007

A safari in our own back yard

So while Africa USA might be gone, there's still Lion Country Safari, but unlike Africa USA, where you ride around the park in a train, in Lion Country Safari you drive your own car through the park, gazing at the various animals. And for the most part, they roam around free, except for the lions and giraffes, which are fenced off and the elephants and monkeys, which are kept behind moats.

At the end of the drive, there's a small theme park in place, I guess to keep the kids occupied in case seeing wild animals wasn't enough. And like most theme parks, the food and drinks are vastly overpriced, the gift shop is overly-hawked, and the kids whine to their parents to buy said overpriced food, drink and trinkets.

[This guy just wouldn't get out of the way]

[For some reason they keep the cats locked up]

[Maybe to keep the cats from eating these]

[Those are some large horns. Wonder if he's compensating for anything?]

[Considering their poor eyesight and large weight, yeah, I can see rhinos having the right of way]

[What you don't see is a rhino trying to hump a Volkwagen Beetle—it was most amusing]

[Looks like some horses escaped from a nearby jail]

[And we can't have a theme park in Florida without pink Flamingos, can we?]

[Now this guy … not only did he keep saying “Hello” and ”Goodbye” but he took a fancy to me and kept trying to hog the camera.]

[This guy, however, gave me the cold shoulder]

[The parrots are real, however the girl was animatronic.]

[I think they've confused alligators, which are found in our backyard, with crocodiles, which are found in Africa]

[Still, I don't think I want to argue the point with him]

Sunday, January 28, 2007

A little bit of urban exploration

There's a building under contruction along I-95 just south of Lantana Blvd. that Spring was interested in checking out, although it's still under construction and access to the site is rather restricted.

In order to get around this restriction, we felt it would be easier to walk along the traintracks and approach the site from the east. Once the Tri-Rail trains passed, we proceeded south along the tracks.

Unfortunately, the vegetation along between the tracks and the building in question was too thick to tromp through, and we neglected to bring any form of machette with which to forge a trail. And the one trail we did find had evidence of a homeless homestead and not wanting to provoke any confrontation, we left the area.

We never did learn what the building was being built for.

Monday, January 29, 2007

Thinking with the aliens

My language is not about designing words or even visual symbols for people to interpret. It is about being in a constant conversation with every aspect of my environment, reacting physically to all parts of my surroundings.

Via kisrael, In My Language

It's a video, but you might want to skip to 03:12 where she stops “talking” in her language and offers a translation, or more loosely, an explanation of what she it communicating, which isn't really meant for human consumption.

What makes this interesting to me (at least, the part after the first 3′15″ of her video) is her definition of “language” and “communication.” And since it's not intended for human interaction, it reminds me very much of the concept of varelse, although it's clear she isn't strictly varelse, but possibly ramen (more information). It also reminded me of Dr. Temple Grandin, who has the uncanny ability to think like a cow and can thus, communicate for them to us (and both women suffer from Autism, although to varying degrees).

I write about stuff from the perspective of an autistic person, because that's part of what I am (I'm also a lot of other things, including probably brain damaged). But when I write about the experience of not being a “person” until I learn a foreign language, I'm not writing to give insight into autism specifically. I'm writing because that's true even in accepted languages, that people are more “people” when they speak the dominant language, or the dominant dialect. I'm writing because that's true of people with brain damage, people with Alzheimer's, people with intellectual disabilities, people whose bodies don't let them form speech easily even if their cognition is totally standard, people considered crazy (in all the various ultra-sophisticated medicalizations of that basic concept), etc, and I see it all the time.

This stuff isn't just about autism.

There's also a lot of food for thought in this MetaFilter thread.

Tuesday, January 30, 2007

I thought people worked during the day

I have a few checks to deposit while the bank is still open, I figured, as long as I'm out, I might as well do the grocery shopping. Normally, I would do this around 6:00 pm, but this week's shopping list isn't long, and it would keep the day from being broken up.

And I'm frankly amazed at how crowded Costco is for mid-afternoon on Tuesday. Don't these people have jobs to go to? Normally, I get to park near the store, but today I'm way out in Farlotistan and have to trudge the half mile to the store. Usually the front of the store is crammed full of shopping carts—today, I have to return to Fatlotistan to retreive a cart.

They also seem to have plenty of food tasting stations throughout the store. At the end of every isle is another hot plate stand, manned by a person cooking some type of food-like item. Everything from Burbon Chicken to brownies. And every one of these food tasting stations is blocking half the isle, making it difficult to maneuver through the already crowded store.

So I'm torn. I like the idea of getting the grocery shopping over with while the sun is still up, thus leaving me with larger blocks of time to work. But the store isn't as crowded later in the evening. So, I have to ask myself: is going shopping during the daytime worth the crowds?

Gives a whole new meaning to “bit bucket”

Today it's a chilly 11 degrees outside and, coincidentally, our office Internet connection has been going up and down. The woman walks by me and leans over and tells me that “our Internet connection is bad today because the tubes probably have ice in them.”

It's a Series of Tubes

Nothing much else to add, except for “read the whole thing.”

Wednesday, January 31, 2007

National Gorilla Suit Day

Even though there's no line, there's still a line

All I needed was a single 39¢ stamp.

So on the way to work, I decided to stop by the Post Office because I know they have a machine that sells single 39¢ stamps.

Only the machine is out of order.

Sigh.

I check over in the customer area and amazingly enough there is no line! Suspicious that it might be another holiday I approach, and lo', it's not a holiday! And there are two customers already being helped.

This should only take a minute or two, I think to myself as I get into the non-existant line.

The customer at the far end is talking to the clerk, who makes a dash back into the bowels of the Post Office, only to come back, talk some more with the customer, and dash back into the bowels.

The customer closer to me is patiently waiting while the clerk applies special postal tape to the package, then afixes labels to the package, then fills out said labels, then stamps a whole mess of official looking documents, afixes yet more tape and labels while filling out even more official looking paper work.

Meanwhile, the far customer is still talking to the clerk, who keeps dashing back into the bowels of the Post Office.

A line starts to form behind me. The clerk nearest me finishes wrapping a cacoon around the package, and then I see the customer has two more packages.

So much for it taking a minute, I think, about five minutes later.

Ten minutes after I started waiting, the far customer finally berates the clerk over some small detail, then leaves. Now I can finally buy the single 39¢ stamp.

My alarm clock sounded suspiciously like a phone

Ring.

“Mughuwaha?”

“Sean,” said Smirk. “You awake?”

“Mughuawhaha,” I said.

“Our customer S is complaining about network lag, what's the address of their installation of Cacti?” I gave Smirk the address, and he was able to log in. “Thanks.”

I fall back asleep again.

Ring.

“Hugmugahwah?”

“Sean!” It was Smirk yet again. “Somethings wrong with their installation of Cacti—there's no data for the past two weeks.” Oh great, I thought. “Can you look into it?”

“Yeah,” I said, resigning myself to the fact that yes, I am getting up early today.

Notes on IPTables

The problem this morning was a direct cause of my inability to fully grok iptables. I logged into the customer's firewall (we offer managed firewalls as one of our services), which was also running an instance of Cacti to help monitor their network. Sure enough, the SNMP polling script was failing for some obscure PHP reason.

Poking around the system, I found a few suspicious files, time stamped two weeks ago, named ping, ping.1 and ping.txt. Odd, I thought and when I checked the contents, yup—a script kiddie script, which opens up a connection to a remote computer.

Sigh.

More poking around, and I find rather quickly the IRC bot program the script kiddie was running (all files owned by the webserver).

Okay. Cacti has some … issues … with security, and it's no surprise that the script kiddie … exploited … these issues, to install their nefarious wares. And the network latency the customer was experiencing was due to excessive IRC traffic.

The major problem I had was how the script kiddie got access to the webserver in the first place. Due to Cacti's … issues … with security, I had explicitly blocked access to all network services with iptables (with the exception of traffic from The Office). Only, what I thought I did, and what I actually did were two different things (much like in practice how theory and practice differ). I spent several fruitless hours (including blocking all traffic to the firewall itself but not through the firewall, which made the remote administration … difficult) before buckling down and really reading up on how packets flow through iptables.

Now, I had set this up to match our office setup. The only real difference (and it's a major difference) is our Office Firewall doesn't NAT, but our customer's firewall does. Oh, that, and we don't run any services on our firewall. Two, two major differences between our Office and the customer are our lack of NATing, services, and an understanding of iptables. Our three major differences between … oh, I'm digressing.

About an hour and several hand drawn diagrams later, I finally had a grasp on the flow of packets through iptables:

I had the filtering rules in the wrong place, along the packet forwarding path (right hand side of the diagram) instead of the local interface input path (bottom half of the diagram). Once I solved that little problem, then I could concentrate on removing the IRCbots and fixing Cacti (I'm guessing the exploit causes Cacti to stop functioning properly—easiest fix was to reinstall Cacti and make sure I had the file permissions correct).

Update on Friday, January 5th, 2007

Upate on Thursday, July 22nd, 2010

Obligatory Picture

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

Obligatory AI Disclaimer

Update on Friday, January 5^th, 2007

Upate on Thursday, July 22^nd, 2010