The recent increase in spam

[fanf]

A lot of our users have been asking us about the current spam problem, so I sent the following to our computing support staff mailing list and the ucam.comp.mail newsgroup. I thought it would be worth posting here too.

The volume of spam we are seeing has more than doubled since the summer, from about 15 messages blocked per second to over 35, and the amount of spam that gets past the blocks has increased accordingly. This is really unprecedented: in the preceding two years, the volume of blocked spam increased gradually from about 10 to about 15 per second. For comparison, we're handling about 7 messages per second, which includes internal email (3 per second) as well as non-blocked spam and legitimate email from outside the University.

That is, at least 90% of the 3.5 million messages offered to us each day from outside the University are spam.

It is a coincidence that this increase kicked off at about the start of term: this is a global problem that has been widely noted in the IT press. Unfortunately the current flavours of spam are difficult for our second-level filters (SpamAssassin) to handle because it doesn't have many recognizable features, such as URLs for criminal web sites, etc. We are updating SpamAssassin when new releases come out, which is roughly monthly at the moment.

Comments

[nameandnature]

Of the increasing amount of spam which escapes my filters (SBL+XBL+DCC), most of is the new wave of pump'n'dump stuff, which puts the spam message itself in an image with a randomised background, as well as putting random English text in the message body. I imagine that's what you're referring to?

I've also seen the same program used to send other messages instructing people to type a URL into their browser, although I don't think that's much of a winner for the spammers, because I doubt the people who respond to spam are clever enough to follow the instructions rather than just clicking on a link. Pump'n'dump is ideal for this scheme, though.

I'm struggling to see what Spam Assassin can do about this, but I don't use it myself, so maybe the authors have some cunning plans. I'll probably switch to using the Spamhaus Zen DNSBL when it's ready, as blocking consumer broadband addresses will kill of a lot of the escapees. I might also start insisting that people who want to talk to me at least have some form of rDNS, even if forward and reverse don't match. I suppose these two things are fine for a vanity domain like mine, but might cause complaints if applied to cam.ac.uk :-)

[dwmalone.livejournal.com]

There's a pretty clear trend in the amount of mail filed to my junkmail folder (either manually or by spamassassin) shown on this graph. Spamassassin is catching most of it, but a certain amount is leaking through. I didn't realise that I was supposed to be running sa-update with 3.1.X versions of Spamassassin, but I think that will help will help catch what's getting through.

[oldbloke.livejournal.com]

No really noticeable increase in spam getting through on my 1and1-hosted email addresses. Maybe those addresses just aren't "out there", or maybe 1and1 have a better filter system than everybody else?

spamassassin should get 'em

[jmason.livejournal.com]

'I'm struggling to see what Spam Assassin can do about this, but I don't use it myself, so maybe the authors have some cunning plans.'

hi! SA developer here. ;)

Well, most of those spams have the same poorly-disguised forgery signs in the headers, and we've been writing rules to catch them. (More rule developers are always welcome though.) RCVD_FORGED_WROTE, for example, nails most of one set, I think the SpamThru spammer's output. It's in sa-update.

Running "sa-update" is a very good idea nowadays -- it's easy, and the results are definitely worth it.

Of course, that doesn't solve the other side of the problem with this spam quantity increase -- the CPU load required to process that much mail goes up, too. We have a new subsystem to provide rule short-circuiting in 3.2.0, though, so that may help in future...

BTW, as far as I can tell, this massive upsurge is due to 1 or 2 spammers/spammer organisations. It's amazing how much havoc so few people can cause :(

Re: spamassassin should get 'em

[fanf]

Thanks for the comments, Justin. For some reason I thought that sa-update was a 3.2 feature, so I haven't been using it. I feel dumb now.

For performance I'm fortunately reasonably well-endowed with hardware (as it were) so given the 90% catch rate from the Spamhaus and RBL+ black lists, we're not doing too badly.

I understand the law is after them, but the wheels of justice grind slow.

[alsuren.livejournal.com]

You have to feel sorry for the poor bastards who rely on BT for their email (like my dad's company's @annor.net accounts) who don't have any decent spamassassinating goodness on their servers, and so get about 20 junk emails every day. What do you reckon I should tell him to do? Can't recommend any actually decent email providers can you?

[fanf]

(1) use an MUA with built-in spam filtering, like Thunderbird or Mac OS X Mail

(2) I only use my own email systems so I don't know what the others are like

decent email providers

[http://users.livejournal.com/_duncan/]

I'm very happy with geekISP.com (http://www.geekisp.com/), a small shop in the US which runs spamassassin and, through a squirrelmail plugin that configures maildrop, lets me perform regex filtering on the SA results or on other arbitrary patterns I notice anywhere in a message.

I'm using addresses that are very much "out there" and have been used on websites, USENET and mailing lists for about 15 years. I get about 5000 junk messages a week and see around 25/day. (96+ % catch rate).

Oh, and that 5000 excludes the ones I throw at the BAYES learner. The spam folder is swept every half hour or so for messages older than two weeks and I have close to 10,000 in there.