Had forgot about that SPAM

I had nearly forgotten about email spam and virii for a few reasons: I don’t really get any (at least not until I let my @mac.com address go active again), we’re quite successful at blocking it here and the greater pain has been comment and link spam on websites.

Today it reared it’s ugly head again, forced me to look at the stats and I was reminded of the shock I felt last summer when I saw how much email spam shows up at a place like this.

We’re still in the state where most of our servers have web and email on them (there are geek reasons for this, not just economics, but will be changing seen) and one of them (“bidwell”) was in a state of “CPU load” all day today. It took a bit of looking and then comparing half-a-day of the virus quarantines:

[bidwell]# ls -l /var/virusmails| wc -l
50444
[barclay] #  ls -l /var/virusmails|wc -l
1290
[one] #  ls -l /var/virusmails|wc -l
727

Crazy thing is that they’re almost all zips (which have to be unzipped, inspected, rezipped … it’s a file type that nearly everyone lets through) and then on top of that we were rejecting about 5000-10,000 emails/hour before they hit the mail queue.

So, Ryan made that a bit beefier and we’re looking better.

What still shocks me is that about 90% of the email that comes into our system is certifiably spam, and that are some domains who have a dictionary regularly run against them and there could be 10,000 rejected emails to single domain with a single guy behind it on a single day. What a waste.

Dedicated mail servers with dedicated “spam killing gateways” here we come.

·:· Posted 1 March 2005, 06:39 by Jason Hoffman to Server geek  |  

  1. One company I worked at had the same problems. Opening zips, virus scanning and re-zipping seemed like a good idea at the time, but it hit us really hard once we started emailing our logs around (zipped of course)

    These logs were around a Gig, but compressed down to a few mega or so. Unzipping would take 3-4 mins. Simply seeking through them was an ordeal from a CPU & IO perspective. then recompressing again.

    Of course, we could run out of disk space too …

    — Michael Koziarski    1 March 2005, 08:34    #
  2. Curiously it is one of the core reasons to make dedicated mail machines, precisely as you wrote! :)

    I designed/built a shared system that’s handling +800K mailboxes and so we run in to bad situations quite regularly. Fortunately it does make spotting & dealing with the bad things a little easier. =) For reasons of local web server disk I/O alone it’s a smart move to strip off POP3/IMAP depending on usage.

    — Benjamin    1 March 2005, 19:16    #
  3. I wonder how big a 100Gb file of full of zeros would compress to?

    I have found in the past that any domain that I had at sometime was subjected to mass mailings to every-name-in-the-dictionary@mydomain.com . The more you can do to filter these out the better as I no longer reccomend friends to get their own domain for just this reason.

    Have you considered greylisting?
    http://projects.puremagic.com/greylisting/whitepaper.html

    — Stuart Woodward    1 March 2005, 23:16    #
  4. I was about to suggest greylisting, but I see someone else got there first. It would be very nice if something like that got put in place. See also: http://www.greylisting.org/

    — Bob Aman    3 March 2005, 18:03    #
  5. I had forgotten about spam/viruses/etc as well until this past weekend. I was in Florida visiting my parents and my father told me he thought maybe he needed “that anti-spyware stuff you put on my laptop” for his home pc.

    He had an unpatched Windows ME machine hooked to a cable modem. A quick scan from AVG showed 1400 infected exe files from at least 30 different viruses/malware. After that, IE died during any attempt to connect to windows update. I maunually installed an IE update. Attempting to update a service pack caused the whole thing to fail on reboot. And of course… the people he bought it from didnt give him install discs.

    I told my dad to get a hardware router, return the computer to the people he bought it from and demand at least Win XP next time. He’s not new to computers, he just trusts his OS to not get cracked open within days of getting it. How dare he! Looking back on it, I still can’t figure out why the cable companies dont distribute hardware routers/firewalls with their cable modems or check their networks for heavy emailing activity. It seems like it would reduce their long term support costs.

    — Paul Oswald    7 March 2005, 21:29    #
  6. Do something nice for your father: Get him a Mac.

    — FredB    8 March 2005, 02:58    #