SSH the secure way, with SSH keys

TL;DR

(TL;DR stands for “Too Long; Didn’t Read.”) If you Google this sort of topic looking for the minimum commands needed to get it working, you simply need to generate a good key and copy it to the server. Below are those commands. (Code blocks with strikethough formatting are for illustrative purposes and should not be run.) Do consider reading the whole article, though, to better understand how to secure your connection.
To SSH without a password using SSH keys:
Generate a key pair:

$ ssh-keygen

You know, as long as we’re generating a key pair, let’s give it a more robust key than the default by specifying a byte length of 4096:

$ ssh-keygen -t rsa -b 4096

Copy it to the server:

$ ssh-copy-id you@remote.server

Introduction

In this post, I will walk you through the setup of SSH keys and their basic usage.
(For a simple overview of the technologies involved, visit the Public Key Cryptography page on Simple Wikipedia.)
The goal is to enable you to log in to a remote server without having to type your password each time, while increasing security and improving your ability to automate what you do.

Why not just stick with passwords?

People, when given the opportunity, pick terrible passwords, then reuse them irresponsibly. Even if you pick a decent (8-character NTLM,) password, as of 2012, it was possible to brute-force crack it in ~6 hours, using a fairly affordable computer.
You simply cannot brute-force an SSH key. The NSA might be able to, but any hacker looking to get in to a system is going to look for a different way to get in.
For this, you’ll need a computer running Linux, Windows, or OS X, as well as a server running an SSH server on which you have a login and password.
For simplicity’s sake, we will be doing everything from the command line, so if you’re on Windows, you’ll want to first install Git Bash. If you’re on a Mac, open up Terminal from your Utilities folder. (Yes, many people using Window prefer Putty, but configuring that is outside the scope of this document.)

If everything worked and it generated that weird bit of ASCII art, we’re ready to move on to the next step:

Uploading Your Key

To be of any real use, you need to put your key somewhere. We’re going to upload it to the server. Let’s suppose that you are logging in to the host test.mmdc.net and your user name is jim.
We’ll first log in using a password:

User@x230Len171025 MINGW64 ~
$ssh jim@test.mmdc.net

If this is your first time logging in, you’ll be asked to confirm the server’s identity:

Warning: Permanently added 'test.mmdc.net' (ECDSA) to the list of known hosts.
jim@test.mmdc.net's password:

Type your password and hopefully you are now greeted with your remote prompt:

jim@test.mmdc.net:~$

Very good! Now log out, using Ctrl-d.
Back at your local terminal, we’re going to copy your key to test.mmdc.net:

$ ssh-copy-id jim@test.mmdc.net

This will (hopefully) give you output similar to the following:

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/c/Users/User/.ssh/id_rsa.pub"
The authenticity of host 'test.mmdc.net (23.189.0.8)' can't be established.
ECDSA key fingerprint is SHA256:u6fDHy25AXJD3MWXLi9Up2rmDp2LOjoBdZQOoQCZXBI.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
jim@test.mmdc.net's password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh 'jim@test.mmdc.net'"
and check to make sure that only the key(s) you wanted were added.

Give it a try:

$ ssh jim@test.mmdc.net

If everything went as it should have, you’ll be now logged in to the remote machine, without having been asked for a password. Pretty cool, eh?
How can we make it better?

Customizing your SSH Config

When we created the SSH keys, it stores them by default in a hidden directory in your home directory, called ‘.ssh’ where we can put a special file called ‘config’ that can hold some very useful configuration options.
Let’s edit the config file. (You may need to create it, but when you do, it will be instantly activated, as long as it’s in the correct directory.)
What would we want in this file? Keeping our example above in mind, we might add the following directives for this host:

The first line defines a bunch of aliases for this host, any of which can be used, like:

ssh test

would be the same as typing:

ssh test.mmdc.net -l jim -p 22

(Yes, port 22 is the default, so explicitly specifying it may seem superfluous, but I’ve added it for when you want to specify a non-standard port.)
On my Windows laptop, I never changed from the user called “User,” so if I don’t specify a different user, that’s what it will try to log in with, so I specify “jim” in the config.

Next Steps

If you’ve gotten this far, you are well on your way, but there are things you can do to make your connection even more secure. Let’s get started:

Add a PassPhrase

First off, you need to have a good passphrase for your key. (Wait? What? Wasn’t the whole point of this to get away from passwords?)
The point of having a passphrase to encrypt your private key is to protect your key if your laptop is lost or stolen, or you lose track of an old backup. Without a passphrase, whoever has this key can log in whereever the key has been authorized.
The convenience of a password-free login has a great deal of appeal, which is SSH.com reports that “More than 90% of all SSH keys in most large enterprises are without a passphrase.”
If you have a key that doesn’t have a passphrase, it’s easy to add one:

$ ssh-keygen -p
Enter file in which the key is (/c/Users/User/.ssh/id_rsa):
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.

By default, SSH will ask you for this passphrase every time you invoke the key, something that gets tiresome quite fast. Fortunately, there is a way to minimize the inconvenience:

SSH Agent

SSH Agent is a program that will ask you once at the beginning of your session for your passphrase and cache it for the length of your session
You can run it manually

$ eval $(ssh-agent)

Having to put “eval” in the command has led to a lot of frustration and misunderstanding about ssh-agent. After all, running ssh-agent alone doesn’t seem to raise an error, but doesn’t do what we need:

What’s happening is that ssh-agent is generating some environment variables, but ssh-agent cannot evaluate them for the parent process, (it’s a Unix security issue,) so we must call ‘eval’ explicitly.
Next, we must add our key to the agent:

$ ssh-add ~/.ssh/id_rsa

Believe me, this is *not* something you want to have to do every time you want to get some work done, so let’s script it all to run automatically at login.
On your workstation, open up your .bash_profile (for Git Bash on Windows, create a file called “.bashrc” in your home directory,) and add the following to the end of the file:

.bash_profile is for Bash, of course. For other shells, you’ll need to find its ‘.profile’ equivalent. If, like me, you use Robby Russel’s excellent Oh My Zsh, there’s a plugin for ssh-agent.
When you have that all configured, reboot your workstation if you can, to properly test.
Open a terminal, ssh to your favorite host, enter the passphrase when prompted, hopefully for the only time this session.

Entropy

To be truly secure, you need a good source for entropy. People have gone to great lengths to devise random number generators, but for our purposes, we’re going to look at the two built in to a Unix-like system: /dev/random and /dev/urandom.
There are a lot of differingopinions on which you should use. /dev/urandom is the default and it’s arguably a good choice, but if you’d like to use /dev/random, you can do that by setting an environment variable before generating your key:

$ export SSH_USE_STRONG_RNG=64

As long as we’re being paranoid, let’s now generate a *huge* key, the largest currently allowed:

ssh-keygen -b 16384

That will take some time, probably a minute or two on a typical home computer, but back in 2004, it would have taken days.
So, should you use a huge key? Perhaps, for bragging rights, if you happen to have friends who would understand what you are talking about, but you’d probably be looking at diminishing returns.

What good is any of this, if you cannot know for certain that the server you are talking with is not, in fact, who it says it is?
Remember, above, when we were told “The authenticity of host ‘test.mmdc.net’ can’t be established“? Well, it *is* possible to verify the server’s identity absolutely, but to do so properly, you need to have someone logged *directly* in to the server. Sitting in front of the actual machine, on an attached console or KVM, *not* using SSH to connect.
That person should run the following commands to extract the key signatures: (More on this.)

cd /etc/ssh
for file in *sa_key.pub
do ssh-keygen -lf $file
done

Ideally, this should be done after the SSHD daemon is installed from trusted media, but before the machine is connected to the Internet.
Those signatures should then be printed, signed, notarized, copies made, those signed and the originals placed in an envelope that is sealed with a wax seal and placed in a bank vault deep in the Swiss Alps.
OK, perhaps that last bit is excessive, but how much do you need to be sure that your connection has not been compromised? Having the signatures printed off at install time would be a good idea, as well as having humans involved in the verification process would not be out of line if your organization uses these connections to deal with sensitive or valuable information.

Disable Password Login

Finally, when you are comfortable that you’ve done everything correctly, gotten your keys created, tested and backed up somewhere safe, it’s time to disable password logins completely. Be sure to do this only on a server that you have physical access to, so you can plug in a keyboard and monitor if you screw something up.

$sudo vi /etc/ssh/sshd_config

Find the directives below in that file and change each of them to “no”:

ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

When that’s completed, restart the SSH server:

$ sudo service ssh restart

The great thing about SSH, as you have seen here, is that it will rise to the challenge of verifiably securing your data in transit, while staying fairly simple and unintrusive, after just a bit of configuration.