The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Wednesday, Debtember 13, 2006

How long until enough admins say “Enough is enough!”?

The Email Situation is getting worse. From a mailing list I'm on:

From
"Jay West" <XXXXXXXXXXXXXXXXXXXX>
To
"General Discussion: On-Topic and Off-Topic Posts" <XXXXXXXXXXXXXXXXXXXXX>
Subject
classiccmp list (sort of) help requested
Date
Wed, 13 Dec 2006 15:51:49 -0600

Listowners perogative to ask a question that is only halfway on topic … ;) I figure some people here may have some good suggestions—offlist please.

There is a SpamAssassin machine(s) filtering spam being sent to the list that sits in front of the classiccmp server (we're also making use of Pyzor, Razor, milter-ahead, and clamav). It's been doing a wonderful job, such that most spam is kept out of the moderators faces. However, over the past few months I've noticed that more and more is getting through (not to the list, but to the moderators eyes who have to kill it all manually). Same goes for many of my customers.

What concerns me is that 99% of the new spam making it through is vaguely sensible english phrases (apparently automatically pulled from online books, or from usenet post archives, etc.). If there was also an advertisement text, Spamassassin could catch that. However, the text is all just english phrases (I've noted them to be targeted phrases, like having to do with computers, sometimes old ones) but … the advertisement is a graphic attachment. Since SpamAssassin can't do OCR on the small gif or jpg attachment that says “buy viagra here” … I am not sure what to do about this. It comes from all over, not just a few servers, etc.

Before you say “just kill all emails with graphic attachments” [the mailing list this appeared on is geared for older computer systems and as such, the general population of the list frown on email attachments, being “old school” and all that; thus this comment from the list owner —Editor] … keep in mind that these spamassassin machines do their job for thousands of domains that I host, not just classiccmp.org. So just killing all emails with graphic attachments is simply not an option. If anyone can give me a few ideas that will work well for ISP/hosting-class environments, I'd love to hear it. Off-list please! Thanks in advance for any advice.

Best regards,

Jay West

I can't see this continuing for much longer before most ISPs and webhost companies simply give up on email entirely (or some people get real serious about solving the spamming problem and we end up with a rash of spammers dying due to excessive rapid lead poisoning).

I wrote the following back to Jay:

From
Sean Conner <sean@conman.org>
To
Jay West <XXXXXXXXXXXXXXXXXXXX>
Subject
Re: classiccmp list (sort of) help requested
Date
Wed, 13 Dec 2006 17:26:24 -0500

I work at a webhosting company, and we're getting swamped. I have a friend who works for Negiyo (huge web hosting company) and they're getting swamped as well. And we both ran out of ideas.

What you might want to start with is disallowing catchalls (all email to a domain going to a single email account). That will probably cut some of the spam down. Another thing you might consider is setting backup MX records to 127.0.0.1. I tried that for my own domain and it cut spam 40% (I don't filter spam to my personal domain, but by the same token, I don't have a catchall for my domain either). You could also try looking into greylisting although it might not scale for a few thousand domains.

Another idea I just had—perhaps do an MD5 hash over the body of the email and store the result. If you get a match (or some number of matches) then it's probably spam and can be deleted (although it may be a mailing list; try applying some heuristics).

Short of that, I don't have many other ideas.

-spc (beginning to think that email will be dead in a few years … )

I wish him luck.

Obligatory Picture

[It's the most wonderful time of the year!]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: http://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

http://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2021 by Sean Conner. All Rights Reserved.