The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Friday, January 12, 2007

Bug hunt

It wasn't so easy to patch.

Email headers can technically span multiple lines, although the mailing list software I'm using doesn't actually support headers that span multiple lines.

And thus breakage.

The email I sent to the affected mailing list I run went into a bit more detail (more, I'm sure, than the members cared to know):

From
Sean Conner <sean@conman.org>
To
XXXXXXXXXXXXXXXX
Subject
[dss] Mailing list woes
Date
Fri, 12 Jan 2007 16:43:02 -0500 (EST)

Okay, you didn't ask for it, but you got it anyway.

What exactly was going on with the mailing list software, and why were the messages so funky to begin with?

The mailing list software I'm using has a few features that I like:

  1. it's simple.
  2. it's simple to install (this is a very important part).
  3. it works.

I started using it back in the mid-90s but I think it's a few years older than that even. It was written way back before there existed a stardardized method for sending files via email [1], so it never had the ability to handle attachtments, nor, as it turned out, HTML formatted email.

In the mid-90s, a standard was created for the sending of multi-media based emails—MIME. Not only did it allow a way to send attachtments, but it also allowed email messages to contain both a plain text version and a formatted version of an email (say, using HTML or even Microsoft Word Doc). This was done by adding some extentions to the headers of the email, which change the way the email client processes the body of the message.

The mailing list software, did not know about these headers, so it didn't preserve these headers and thus, you got funky looking email. [2]

So all I needed to do was preserve the required headers and bam! The mailing list software would automatically support all this MIME crap [7] and everybody would be happy and be able to actually read the messages.

Only, preserving said lines wasn't as easy as I expected.

You see, the headers in an email message are name/value pairs:

From
Lorie Davis <XXXXXXXXXXXXXXXXXXXXX>
To
The Dragonslayers Society <XXXXXXXXXXXXXXXX>
Subject
Are we done yet?
Date
Fri, 12 Jan 2007 16:43:02 -0500

but the email standard states that a header can actually span multiple lines:

From
Lorie Davis <XXXXXXXXXXXXXXXXXXXXX>
To
The Dragonslayers Society <XXXXXXXXXXXXXXXX>
Subject
Have you finished using me as an example?
Mime-Version
1.0
Content-Type
multipart/alternative;
boundary="—-=_Part_44704_28465339.1168636054704"
Date
Fri, 12 Jan 2007 16:07:34 -0500

(note how the Content-Type header spans multiple lines) Now, not all headers span multiple lines, but it is allowed by the email standard. But the mailing list software didn't allow for multiple lines per header, and for the headers it was originally interested in, were never really split across multiple lines.

I found this out the hard way.

The way the code was originally written made it difficult (read: damn near impossible) to support multiline headers. So it required a bit of rewriting to make this all work.

So, why not just install some modern mailing list software? See rule #2 above? Have you seen what's required to install modern mailing list software? At least a 2GHz machine to start off with. A score of libraries. And now I'm treading into the scripting lauguage du jour as well.

Believe me, it was easier to patch the existing mailing list software than to install something new (which basically, would require updating my entire server, which is something I refuse to do, seeing how it works and is secure as is—and I'll stop here before I start ranting on the whole upgrade culture).

-spc (Who still uses 486s)

  1. EMAIL is NOT FTP!

    Sorry.

    I was never one for sending files via email, as there are better, more efficient and faster ways of doing that. What no one (and by “no one” I mean those that aren't programmers) seems to realize that when you send a file through email, the file has to be converted to text, least a lot of email software along the way blows up. This conversion process inceases the file size by at least a third (so a file that's 3M now becomes an attachtment 4M in size).

    But it's damned convenient [3][5], so everybody does it.

    Well, except me. But I'm not your typical user to begin with.

    Anyway …

  2. Not that I minded—that's how I receive such messages, even though my the email client I now use understands the MIME headers. Yes, what you were seeing was the raw message body itself, in all its bloated HTML glory that every email client seems to bloody default to these days [3].

  3. Thank you so very much, Microsoft. [4]

  4. May you rot in Hell.

  5. Thank you so very much, University of Washington. [4][6]

  6. http://en.wikipedia.org/wiki/Pine_(e-mail_client)

  7. That's a technical term.

If I come across as a curmudgeon, then yes, I don't like the fact that people send files via email (reguardless of how convenient it is) nor do I like HTML formatted email (if plain text is good enough for me, then gosh darn it's good enough for you!) so quite a bit of this is just me blowing off some steam.

Obligatory Picture

[It's the most wonderful time of the year!]

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: http://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

http://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2019 by Sean Conner. All Rights Reserved.