A pair of encryption keys are
generated which are mathematically
linked to each other: a private key
and a public key. The public key is
used to encrypt a message whereas the
private key is used to decrypt it.
This provides a very nice property
which allows anyone having access to
the public key to encrypt a message
that can only be decrypted by someone
with access to the private key.

The inverse of this property is used
for creating digital signatures in
which a sender uses the private key to
digitally sign a message and a
receiver uses the corresponding public
key (which has been verified via other
channels to belong to the sender) to
verify the signature. The receiver can
then be confident that the message was
indeed from someone with access to the
private key. This forms the basis of
public key authentication.

The SSH protocol uses keys for authentication in two ways:

Authenticating users

This is where the SSH server verifies the identity of the user that is connecting. SSH offers various methods of authenticating users. Password authentication is the most common one but public key authentication is another.
When using public key authentication the SSH server checks if the user trying to authenticate to an account that has an authorized public key is in possession of the corresponding private key. Keys used for this purpose are referred to as SSH user keys.

Authenticating servers

This is where the SSH client verifies the identity of the SSH server. Allows users connecting to an SSH server to verify that this is in fact the same server they connected to the last time or that this is indeed the server that it claims to be (basically to prevent man in the middle attacks). Keys used for this purpose are referred to as SSH host keys.

When referring to SSH keys the speaker could be referring to either of these (user keys or host keys). Most often than not (unless you’re a system administrator) when someone talks about SSH keys they’re generally referring to user keys.

Note on security of keys

While the public key is meant to be shared (in the case of SSH keys copied to the SSH server under a particular location) the private key is, as it’s name implies, meant to be kept private as anyone having access to the private key would be able to decrypt and sign messages on the owner’s behalf. Some general guidelines for protecting a private key are ensuring secure file system permissions (when the key is stored on disk) and generally not copying it from the system where it was generated. It’s also possible (and highly recommended) to encrypt SSH keys with a passphrase. Doing so will prevent someone who managed to get a copy of the private key of using it without having to also know the passphrase.