[Spambayes] Ideas for an MSc project please...
cej at intech.com
Mon Feb 9 22:01:39 EST 2004
Ryan Malayter wrote:
>>(We were running a 667 MHz Celeron w/ 128 mb ram.)
>How can you complain about the unreliability of your mail system, when
>you're running it on 1999-era, desktop-class hardware in 2004?
>Especially when you're a *technology* company? Hardware is CHEAP.
>People's time and lost sales are expensive.
Thank you. I did hear that a few times already.
>Yours was a failure in planning, not in hardware or software. Viruses
>and spam have been known quantities for many years now, part of the
>internet landscape. Certainly you figured they might impact your
>infrastructure in some way. Why did you fail to plan accordingly, and
>allocate budget for more robust MTAs?
True, but we have not been hit with any mail virii in the past. The
reason I am here is to handle unplanned situations like this one. I
firmly believe that no amount of planning would have stopped this
particular 48-hour marathon. Certainly no preplanned system can handle
every contingency. The volume of mail was enormous, and the machines I
had at my disposal performed very well.
>Setting up a reliable mail infrastructure is not that hard. Read the
>RFCs. Set up a reliable server on your site, with MX precedence =10.
>Have your ISP's set up their SMTP servers set up to relay to your
>domain, and put them in your DNS as MX=20. No mail bounces, it just gets
>queued at your ISP when you're down. Same hardware costs on your end.
Our ISP is braindead. Not a hell of a lot I can do about it.
>Besides, even if you have only one MX server, well-behaved sending MTA's
>should queue your mail at the sending site for a few days, retrying
>every few hours, before reporting failure to the originating user. Only
>spammers and viruses refuse to retry gracefully.
This became part of the problem. The mailserver was so busy handling
the incoming mail that the spam backlogged on everybody else's servers.
Also, there is something strange about the whole deal, because 90
percent of the incoming mail is all mailer-daemon bounces. It's as if
some spammer decided to spoof our domain (and no, we're not running an
open relay. Already checked and double-checked). While he was at it,
he spoofed from his list of @intech.com addresses. How can I tell you
this? Because the addresses that receive spam are pretty much the same,
month after month. All of a sudden, they went from receiving 50
messages/day to 1000+, all of them mailer-daemon bounces from
mailservers all around the world. What is also strange is that the
whole thing started last Sunday, well after the MyDoom virus made the
rounds. The mailserver had been happily sucking down MyDoom into the
bit bucket for days, and all of a sudden *boom*, the shit hits the fan,
and it's ALL SPAM! It's not MyDoom (MyDoom made up somewhere around
0.0001 percent of incoming mail).
The spambayes filter system I set up is somewhat to blame, since it
greatly limited the mail thoroughput. There was also no cut-out
mechanism to drop the scanning/classifying system when the load started
climbing. Believe me, I will be much more cautious about implementing
these things in the future.
More information about the Spambayes