« June 26, 2005 | Main | June 29, 2005 »

Monday, June 27, 2005

Timeout from Spam

Junk mail is increasingly sent by hijacked "zombie" machines infected with virus and worm software and operated by remote control. These zombies connect to a list of victims and blast mail at them as quickly as possible. So quickly, in fact, that they often violate the SMTP standard in various ways. Common violations are failing to wait for the "220" greeting message from the mail transport agent, or sending other parts of the transaction without waiting for the handshake from the last command. Many mail transport agents permit this sloppiness (if they didn't, spammers wouldn't do it), but starting with version 8.13 Sendmail adds options to enforce the standard more strictly and thereby detect and discard junk mail.

The key facility in limiting spam is greet_pause, which allows delaying the greeting message for a specified time after accepting a connection. For details on configuring this feature, and the associated PIPELINING=0, see Jef Poskanzer's mail filtering pages. I tried the following configuration on my server: 

#   Enable access database (required by greet_pause below)
FEATURE(access_db)dnl
#   Delay greeting message by 10 seconds to deter blind spammers
FEATURE(`greet_pause',10000)dnl
#   Sleep one second on all RCPTs after a bad one to deter
#   dictionary spams.  (This should be much longer than one second!)
define(`confBAD_RCPT_THROTTLE', `1')dnl
I added the following to /etc/mail/access to suppress the delay for connections from the local site: 
# Don't delay greeting on mail from localhost or Fourmilab
GreetPause:localhost            0
GreetPause:193.8.230            0
Before making this change, I was averaging about 170 junk mail messages a day detected and discarded by Annoyance Filter; the ten second greeting pause reduced this to about 120 per day--an improvement, but less than I'd hoped for.

I then decided to raise the stakes, and the cost for spammers, by hammering in a 25 second delay between the HELO or EHLO from the sending client and the confirmation message. (I picked 25 seconds off the top of my head--I've not experimented with different settings.) This was done by hammering a sleep(25) call into the source code of srvrsmtp.c as follows (around line 2057 of version 8.13.3): 

   gothello = true;
   sleep(25);
    /* print HELO response message */
    if (c->cmd_code != CMDEHLO)
    {
        message("250 %s Hello %s, %s",
            MyHostName, CurSmtpClient, q);
Note that this delay affects connections from the local site as well--I send so little E-mail that it doesn't bother me, and this is just an experiment, so I decided not to complicate things by getting the access database into the loop at this stage.

So how did it work? Quite well, so far--in the five full days since I've implemented this change, the mean number of junk mail messages as measured by Annoyance Filter has dropped to about 80 per day--a reduction of more than 50%. The reduction in load on the server is much greater than this number suggests (although at this site, spam processing is a minor component of server load). In a single day, introduction of the delays caused a total of more than 2000 SMTP connections from spammers to be rejected or closed without completing a mail transaction; most of these are "dictionary spams", looking for valid E-mail addresses on your site and now frustrated, wasting as much time on the perpetrator's machine as they permit before timing out.

More importantly, introduction of the 25 second "Hello" delay seems to have blocked about 95% of the grossly offensive pornographic spam which coarsens one's online experience simply by watching it scroll down the mail log window, even though it is automatically discarded by Annoyance Filter. Senders of "chinese junk"--the spam which arrives in Asian character sets--appear to be more patient and are not deterred by the delay, but then their blithering is easily filtered by the character set indicator and in any case isn't offensive (well, it might be if I could read it, but I can't).

Some may worry about losing legitimate mail by enforcing standards more strictly. First of all, any mail sending host which complies with the SMTP timeout recommendations in section 5.3.2 of RFC 1123 will have no problems whatsoever with the total 35 second greeting and hello delay introduced here. Secondly, and this represents a change in my personal philosophy regarding E-mail, I couldn't care less. I am tired of having my life coarsened by watching disgusting rejected messages scroll down my screen, and repelled by how having my face rubbed constantly in the bottom of the barrel reduces my estimation of my fellow humans. So I'm changing my approach to E-mail from a "do no harm" approach--no valid message rejected--to "active measures"--no questionable message brought to my attention in any way. This means I will no longer watch the log of arriving messages, spend no more time training Annoyance Filter on new junk mail, and send junk mail detected by Thunderbird's Bayesian filter directly to the trash without logging.

If somebody I don't know wants to send me a message and be sure I get it, they should send a FAX, mail a letter, or use the feedback form. Folks with whom I regularly exchange E-mail are already white-listed and bypass all the filters. Universal E-mail was a utopian idea which, like most utopian schemes, has failed; in my estimation it is no longer worth wasting time trying to patch it up, but rather limit its impact by restricting it by blocking all offensive material, whatever the cost in false positives. That's what I'm doing, anyway.

Posted at 22:54 Permalink