It appears that one of my servers is undergoing a sophisticated dictionary attack on ssh, in that I am seeing a bunch of usernames being tried on on of my servers alphabetically with password failures.

The unusual things about this are:

There is only one attempt every 2 minutes

every attempt comes from a different IP address

Can you help me understand how this can be done from different IP addresses, can it be effectively blocked? and does the slow persistence of the attack (probably will take several months to go through the alphabet) mean anything specific?

I'd also be interested to know if anyone else is currently seeing similar activity on their public ssh servers (ie. are we a specific target, or is this attacker blanketing thousands of ssh servers with this attack?)

Also, is there any way for me to reveal what passwords (or even hashes) are being tried? I notice that each name is only tried once or twice. I assume they are trying "password" or the user's username - but can i verify this?

5 Answers
5

If you must use passwords, allow them only from subnets you trust, not the whole internet. Also, it helps to run your ssh daemon on some port other than 22. Normally I don't suggest such things, but if it reduces the volume of attacks you see, so much the better.

In answers to your specific concerns:

I've seen these sorts of attacks on any host I run that has a public facing ssh server.

I suspect they're using a dictionary of user names / passwords developed from past experience with what works. For instance, my logs have

18 attempts against user "ftp"

23 attempts against user "forum"

3 for user "bernard"

1 for user "bret"

I've also seen this sort of attack where they used actual site-specific logins, but those were probably culled from that site's wide-open ldap server (universities, gotta love 'em!)

These attacks are probably coming from zombie computers that have some impressive centralized control.

these attacks are coming at a rate of one every 1 to 5 minutes, all day, every day

I just tried trussing sshd to see if it is trivial to collect the password the attacker is trying, and it looks like you'd have to patch the code.

these people did just that and found that the passwords were often permutations of the username or a well-known password such as sql or admin.

Thank you, your answers are a great help. As for your password suggestions, that is not the purpose of my question - but I suppose they are helpful to have in the thread anyhow.
–
Brent Dec 8 '09 at 16:07

The guy who said "ignore the attacks" was spot-on (and I don't know why he was downvoted). if you have a halfway sane password policy, like "don't use your login as a password" these attacks are not going to work.
–
chrisDec 8 '09 at 16:18

Fail2ban is but one tool to improve server security. Of course the first step is to avoid use of passwords and use serverkey authentication instead. In addition limit allowed traffic to certain IPs, disallow IP ranges, use IP tables, use blacklists such as rsync-mirrors.uceprotect.net to update your iptables for known SSH hacker origins and origin networks. For details on above 3 levels of blacklists, see uceprotect.net.

Other items I've seen suggested related to this kind of attempt recommend monitoring for the failed attempts and blocking the source IPs - eventually the same IP will be used for another probe, so by detecting and blacklisting after attempts to log in with a nonexistent account you gradually reduce the threat at the cost of increased firewall rule processing. Also, it might be best to not block after only a single connection attempt - typos do happen.

Fail2ban (the way we have configured it) requires 4 failures within a certain length of time to block an IP address, and I haven't seen a repeated IP address yet - so with attempts occurring at 2 minute intervals, this is not going to happen.
–
Brent Dec 8 '09 at 16:11

I think that banning would have to operate at the same (glacial) speed of the attacks, so your fail2ban timeout would probably need to be set at something like a year. From reading a little bit about how it works, you'd probably need to do your own implementation instead that recorded IPs & timestamps extracted from log files.
–
fencepostDec 8 '09 at 16:31

I think it's pretty likely that a legitimate user would have 4 failed logins in a year - and then what are we going to do? ban them for a further year??
–
Brent Dec 8 '09 at 16:35

That's why I think you'd need to roll your own. Really, the problem divides into two parts: A) identifying connection attempts with usernames that bear no resemblance to existing ones (scoring system?) and B) Blocking access from IPs attempting those connections with scores that are "too low." More energetically, software to do this could be built to exchange results with other servers specified in a configuration. A large-scale hosted list would be too vulnerable to attack, but if you're responsible for 50 servers that's a good base for info gathering.
–
fencepostDec 8 '09 at 16:51

@fencepost: That's a lot of work for no payoff. These attacks are useless against systems with even nominal password policices such as "don't use your login or name for a password"
–
chrisDec 8 '09 at 17:16

Increase the timeout? I'm not sure I understand what you are getting at. Are you referring to the time the server holds on to a bad connection before denying it? The attack is already VERY slow - I can't imagine having a timeout longer than 2 minutes. That would be impractical for legitimate users.
–
Brent Dec 8 '09 at 14:02

How would fail2ban stop random hosts on the internet from attacking you? An example from a box I manage has these attacks: ben from 62.96.29.34, ben from 61.74.75.58, benjamin from 143.107.98.250, benoit from 80.152.177.173, bernard from 41.206.41.90, bernard from 202.37.78.13, bernard from 70.52.195.87, berndt from 78.43.82.153, and so on. And, they're trying at two to five minute intervals. fail2ban is useless against the zombie hordes.
–
chrisDec 8 '09 at 15:21

Chris, I am seeing these exact same names, but different IP addresses. This confirms that it is a large attack not directed specifically at us - which is what I wanted to hear. THANK YOU!
–
Brent Dec 8 '09 at 16:16

I misunderstood the exact nature of the problem, but I suspect that you are seeing more than one hit from a particular IP at some point. At the very least, installing denyhosts or fail2ban won't hurt and can only help the issue. With a distributed attack, you are unlikely to find anything work without setting up a deny all rule and whitelisting select IPs.
–
Aaron BrownDec 8 '09 at 18:36