Password scanning

[fanf]

AJ's idea of scanning email for passwords has provoked a lot of strange suggestions, online and in person. Elaborate password cracking clusters, phishing my own users, hardware acceleration, etc... (When AJ suggested the Idea, my first thought was to use a Bloom filter to find passwords, since you can't recover the contents of a Bloom filter without brute force - but it's very hard to delete items from a Bloom filter, which in this scenario must be done whenever any user changes their password. No, that too is not going to work.)

The whole idea is very borderline: is it worth spending significant effort when the phishing success rate is much less than 1% per incident, and the current fashion for phishing universities is probably short-lived? (This week we got about 2000 messages from the phishers and 5 users replied.) On the other hand it would be very interesting to find out what the detection rate would be. Would there be any false positives? i.e. unintentional password appearances? What is the legitimate positive rate? e.g. senior staff sending their passwords to their PAs? (The latter is against our AUP but it is common practice.) How much password sharing is there outside the anticipated scenarios?

It seems that it's worth making the point that it isn't hard for me to get my users' plaintext passwords: I could just instrument our various SASL implementations. But we (me and my colleagues) don't do that because sysadmins are safer not knowing their users' secrets. This is why we don't log message subjects, and why our accidental-deletion recovery tools don't require seeing any message contents. We don't look at the contents of a user's account in any detail without permission from that user - and even then we'd very much prefer not to know that we're recovering email that was deleted by their jilted lover who obtained their password from a shared browser history.

From my point of view, the interesting thing is that it is feasible to detect when a user is sending their own password in a message, using just a standard Unix encrypted password file and some simple code: crypt every word the user sends and compare with that user's crypted password. This is just a few hundred lines of C, including hooks into our authentication database and email content scanner, and choosing the right words. My prototype code can check 2000 MD5 crypted words per second per CPU, and should be able to skip most words in a message since they are outside our password rules.

There has been a lot of traffic on various mailing lists about these phishing attacks, especially notifications of new reply addresses. But we don't want to be in the business of maintaining blacklists of addresses our users mustn't send to. Password scanning seems like a simple way of avoiding that tar pit, which is I think the main attraction. So why do I think it's absurd?

Comments

HTML mail can post? Users can wrap?

[shae.livejournal.com]

If HTML mail can post/get from the user's client, scanning sent mail won't matter.

If I really needed to send a password to someone, zip or word processor files would get around that.

If you check plaintext, the phishers will likely figure out how to get the user to do this sort of wrapper.

Just a random thought.

Re: HTML mail can post? Users can wrap?

[fanf]

Valid points. However I think it's more likely that phishers will continue to mostly use web forms to capture information, which bypasses any email scanner. This idea isn't (and can't be) watertight.

Re: HTML mail can post? Users can wrap?

[angoel.livejournal.com]

Since you provide the web access, can't you simply stop users sending their password through a web-form too...

Re: HTML mail can post? Users can wrap?

[fanf]

We don't have a web content scanner, and our proxy cache (where you might add one) is about to die.

[bens_dad]

I suppose you could add the password check to prayer, or any other MUA that supports passwords. You would have to do it for every MUA to be sure of catching everything.

But checking in the outgoing MUA could be considered something that was done by the user, whereas doing it in the MTA smells more of institutional nannying.

[gerald_duck]

Since taking silly suggestions seriously seems to be flavour of the month, I'd like to reiterate my suggestion that the correct way to proceed is suspending the account of a user if any e-mail message ever contains their password, whether or not the message was sent or received by them.

If a user's password crops up by chance in a stranger's e-mail correspondence, their password is too weak.

[tamsinj.livejournal.com]

good luck if your password happens to be a common string in MIME encoding then. or if the birthday paradox works on passwords (i don't know how relevant it'd be)

[gerald_duck]

It's only necessary to check delimited strings in the e-mail, though. Lines of MIME would be treated as very long words that could be discounted.

[pjc50.livejournal.com]

This is a "theoretical security is more important than actual ability to do work and user goodwill" argument, isn't it.

[gerald_duck]

Partly. (See "silly suggestions".)

On the other hand, passing a stream of words from e-mails that would be valid passwords in the direction of the server that has the password database might be more secure than letting the mail server peek at a mail sender's password. Also, account suspension only has to happen before the phisher gets around to using the stolen password, not before the e-mail gets sent so that strategy could cause less latency for outbound e-mail.

Also also, someone who's typed their password into an e-mail and tried to send it has very probably left traces of that password in all sorts of other places, too — whether as part of the process of composing the message or incidentally. By the time they've typed their password into an e-mail they've already lost and need to change it.

Also also also, if A sends B an e-mail to say they found C's password by shoulder-surfing, it would be nice if that automatically locked out C's account.

Also also also also, there ought to be a better way of handling the PA problem than senior staff giving more junior staff their passwords.

Yes, there's a balance between security and utility; Tony's already noted that this entire venture might be too much trouble. Although I'm not pretending I have a shred of evidence, and I grant my own intuition is against it, nonetheless it's possible my more draconian idea actually gives a better tradeoff.

[fanf]

more secure than letting the mail server peek at a mail sender's password

How do you think authenticated message submission works then?

less latency for outbound e-mail

0.25 seconds is so long to wait!

left traces of that password in all sorts of other places

Shared browsers are a serious danger.

there ought to be a better way of handling the PA problem than senior staff giving more junior staff their passwords

It's difficult because there isn't an obvious non-personal role address for someone like a professor. There's perhaps a requirement for those kinds of people to have both a private and a public account, where the public account is to be shared with the PA. We support shared accounts on Hermes for backing role addresses, but we don't have a mechanism for people to have more than one account or for CRSID-like shared accounts.

[gerald_duck]

I don't know how authenticated message submission works, or which form(s) you permit. In an ideal world, however, an authentication scheme would be used that could be directly verified by the password server without the mail server seeing passwords in the clear.

What is your aspiration in terms of mail latency? Frequently, I'm on the phone to someone when I send them an e-mail with some supporting document as an attachment, or vice-versa; this means I want, and expect, and generally achieve, latencies of the order of a few seconds. 0.25 seconds isn't a lot, even in those terms, but I don't know how many other checks you run which also take 0.25 seconds each…

Back in the Phoenix days, every mailbox had an ACL. That doesn't solve the entire PA problem — though arguably the senior staff member ought to be using a personal mailbox for personal correspondence — but it at least means the PA could access their boss's mail without knowing the boss's password.

[fanf]

At the moment email authentication (sending and retrieving) is basically done by sending passwords in the clear. There are a few challenge-response protocols, but they're either weak or over-complicated and poorly implemented. They usually require a plaintext password database (in practice if not in theory). In an ideal world we'd have Kerberos, but even then many clients wouldn't be part of the authentication domain and would therfore have to fall back to plaintext auth.

We want latency to be a few seconds typically. At the moment the delay can be quite variable because of the way the batch-mode scanner works, but this summer I'm replacing it with an SMTP-time scanner which I hope should improve that. (Though it has the disadvantage of handling overload much less gracefully... I hope we never see that in action!)

Cyrus (our message store software) supports ACLs and shared folders. However our current partitioning and replication architecture prevents us from using the feature.

This is partly because Cyrus's usual cluster architecture relies on a separate mailbox location server (mapping mailbox names to message store machines), which we don't like because it's a single point of failure. We have our own proxy which maps users to message stores, and is much simpler and more resilient than Cyrus's. However we might be able to adapt the Cyrus proxy to use a more static mailbox->server map so we could avoid the SPOF and enable users to access mailboxes that aren't on their home servers.

The other aspect of the problem is that our original replication protocol had built-in assumptions about how mailboxes relate to accounts, e.g. w.r.t. the database that keeps track of which user has read which messages. The replication had some re-working when it was incorporated into the official Cyrus codebase, so this might have been fixed. However I am not certain about this.

[bens_dad]

... is basically done by sending passwords in the clear.
Fortunately usually over an SSL/TLS tunnel :-)

[reddragdiva]

"Also also also also, there ought to be a better way of handling the PA problem than senior staff giving more junior staff their passwords."

I occasionally encounter a scientist who has a clue about passwords. Most places with scientists they pass their passwords around amongst each other like candy. In such an environment, scanning passwords may as well be security theatre.

[fanf]

Fortunately University members aren't likely to use a misappropriated password for spamming. If something bad happens to their account because of poor password hygiene then we can help by recovering from backups, and if necessary by providing information for disciplinary proceedings.

[bens_dad]

If a user's password crops up by chance in a stranger's e-mail correspondence, their password is too weak.

But this wont happen because weak password will be caught, eg by passwdqc when they are set or changed, or by the regular runs of John the Ripper or similar :-)
Except that implementing all that will result in passwords on post-it notes on monitors :-(

[aldabra]

Why don't you just send five million e-mails back to the address given by the phishers containing either non-passwords or user/password combinations for dummy accounts you've set up specifically to be annoying to phishers?

[fanf]

Ha nice one :-)

[gerald_duck]

Because you have to recognise phishing attacks.

Maybe the system, having recognised that user A@cam.ac.uk has sent their password to B@miscreant.org, should block the message and send five million fake responses.

I'm sure the system wouldn't be abused…

[arnhem.livejournal.com]

If a message part is (say) base64 encoded, would you be processing the encoded or decoded form of it?

[ewx.livejournal.com]

You really ought to be able to base64-decode (or QP-decode) much faster than you can MD5...

[fanf]

Yes, that would be necessary. I'll probably have to do a bit of HTML processing too :-/ Most of the people who were phished this week were not webmail users, so who knows what kind of crap their MUAs put in the message bodies.

[angoel.livejournal.com]

You haven't considered the case when the password contains spaces.

[fanf]

Yes I have :-) My prototype code defines words as whitespace-delimited strings, or strings delimited by "" or ''. (Obviously this needs a lot of work - see the comments about encodings and markup above.) I'm not hugely worried about spaces in passwords on the assumption that they are a sign of a working brain, but then I don't have any idea what proprtion of passwords contain spaces so it could be surprisingly popular. But in any case this just needs to be BALGE not perfect.

Blocking / Replacing the Password won't help ...

[(Anonymous)]

Some commercial "Internet Security" packages offer(ed) a related "security feature", blocking the PIN(s) for online banking in outgoing HTTP traffic. The idea was to prevent exposing the PIN to evil sites, even if the user is dumb as bread (proven by buying that piece of software). Each HTTP request containing the PIN had the four digits replaced with "****". So if some evil phishing site hat tricked the user into entering his pin, it would only see stars.

Solution of the phishers: Use Javascript to iterate over a bigger range and wait for the censoring to happen. Guess the pin in http://evil.example.com/pin-phisher?pin=4709&pin=4710&pin=****&pin=4712&pin=4713 ...

Tux2000

Re: Blocking / Replacing the Password won't help ...

[fanf]

I suppose PIN cracking is feasible over HTTP since there are only 1000 possible PINs. There are over 100,000,000 of the weakest possible passwords on my system.