8 Answers
8

Disable Root Login - Most automated attacks will concentrate on the root account so disallowing logins from that account is a good place to start.

Only Allow Certain Groups/Users - Limit what users and groups can SSH to the system.

Limit by IP or IP Range - One of the most effective ways of securing your system against SSH attacks.

Use Key Based Authentication - As described by Olivier

Use fail2ban - fail2ban will monitor attempted SSH logins and after a determined number of failed attempts it will block the attackers IP address for a set time period. This can be a very effective method for slowing down attackers.

Also, make sure to use any new security features in sshd. Currently this means 'UsePrivilegeSeparation yes' and 'Compression delayed' - both of which help protect against code bugs.
–
Alex HolstFeb 5 '11 at 16:43

And set it up with port knocking (inclusive) or change the SSH port to a different, unused, well-known port (not a high port, or $user can run their own service listening on that port and harvest credentials for more privileged users). And add two-factor (or, at minimum, two channel) authentication.
–
Parthian ShotJul 14 '14 at 20:01

The other answers focus on securing the server -- which is important, but the client side deserves some protection too:

In /etc/ssh/ssh_config (or ~/.ssh/config) set StrictHostKeyChecking to yes or ask. This provides some protection against man-in-the-middle attacks by checking that the server you are connecting to is the one you expect.

If you can add records to your DNS zone, publish an SSHFP record and set VerifyHostKeyDNS to yes or ask. The client will fetch the SSHFP from DNS and verify the fingerprint. (Note that you're still vulnerable if your attacker controls DNS.)

Force protocol version 2. I.e. set Protocol 2 in /etc/ssh/ssh_config -- the default is 2,1 which allows falling back to v1 if v2 is not available.

Prefer keys to passwords for logging in. Use a strong password on your ssh keys. Use keys of adequate length (number of bits).

If you have users providing ssh keys, audit the password strength by running john on the keys.

Use a hardware token to unlock the key instead of typing the password (to avoid keyloggers & shoulder surfers).

Be sure that UseBlacklistedKeys is no (this is the default). This prevents you from using 'blacklisted' keys generated during from weak Debian openssl packages. Check host and user keys with ssh-vulnkeys. On Debian-derived systems (e.g. ubuntu) you can install the openssh-blacklist and openssh-blacklist-extra packages.

Be sure that CheckHostIP is yes (the default). This makes the client check the IP of the host against known_hosts to add protection against DNS spoofing.

Set HashKnownHosts to yes. This prevents leaking information from your known_hosts file.

If you require hardware tokens, you get those benefits. But the link you point at is about software keys, and even leaves open the idea of not having a passphrase on the key - yikes! Without hardware tokens, unprotected keys or keyboard loggers will still get your key or password, as will well-timed shoulder surfers or brute-force attacks.
–
nealmcbFeb 16 '11 at 20:39

For a start you should disallow passwords and only allow authentication using RSA keys. This makes brute force attacks infeasible. People will still try however, so you'll want to limit login attempts and maybe change the port to save your bandwidth. This approach relies on trusting your users to encrypt and secure their private keys.

You should not allow root login remotely. Use su or sudo once logged on.

It may be a good idea to stop people from tunnelling ssh through your gateway server to internal servers. This can stop your internal servers being exposed to the internet.

There is a new option on the server to require passphrases on the client keys. I think this is a very good idea. However, you still don't know the complexity of the passphrase and users could potentially recompile the client to fake this.

It occurs to me that none of the other answers talks about a very important point for securing SSH: educate your users. Whatever you do, if the users do not learn to react sensibly when something fishy happens, then you are doomed. At the very least, users must be aware of the server key handling business (when they first connect to a given server, they must check the key fingerprint with a reliable source; and if SSH warns about a key that has changed, they must not bypass the warning).

Technology can only get you so far; as long as a human is involved in the process, the amount of security awareness of that user is a hard limit on the level of security you may hope to achieve.

As everyone is saying: always disable root login, change the default port number (though this can make using some sftp clients difficult) and only accept Key Based Authentication.

Also, if you change any key sizes in sshd_config or specify a key bit size when you run ssh-keygen, you should review the key sizes you specify from time to time.

Back when ssh-keygen defaulted to generating 1024 bit RSA keys I used to use the -b option to generate 1536 bit keys. Over time the default changed to generate 2048 bit keys and I was still generating 1536 bit keys.