OpenSSH at CERN FAQ

1. Frequently Asked Questions

Here is a list of problems seen after the introduction of
OpenSSH at CERN. Before contacting ssh.support@cern.ch, we ask you to check here - perhaps
your problem has already been solved before!

1.1. General troubleshooting hints

In order for us to help you, the problem has to be
repeatable.

When reporting problems, always send the output of

$ ssh -v -l <user> <destination>

If you have a root account on the destination host,
please run

# /usr/sue/etc/sshd -d -p 222

or (on Linux)

# /usr/sbin/sshd -d -p 222

as root there and connect using

$ ssh -v -p 222 <user> <destination>

(the sshd server will exit after each connection, and
you may run into trouble with a local firewall that prevents
you from connecting from a different machine. Same-machine
connections to "localhost" should work)

If you do not have root access on the server, you can
generate your own "server" key pair and run on an
unprivileged port:

1.4. SSH-2 protocol, or why it is not the default at CERN

Background: the original version of ssh as written by Tatu
Ylonen was found to have some protocol design flaws, for which
workarounds had to be developed. To overcome these
shortcomings, the protocol between client and server has been
redesigned. This is new protocol then was called SSH-2, and the
previous one has been dubbed SSH-1. Please see this FAQ
from the O'Reilly "snail" book on the topic for
further discussion.

SSH.COM[1] has pushed for
protocol SSH-1 to be declared obsolete. Other bodies like the US
DOE advisory group CIAC
have followed this trend in their recent advisories.
However, please note that the current SSH-1 protocol
shortcomings are all handled, and that both SSH-1
and SSH-2 protocols have had
security-relevant implementation errors.

At CERN, the OpenSSH implementation of both SSH-1 and SSH-2 is
used. This implementation has dealt very quickly in the past
with security holes appearing in their code, and the recommended
versions of ssh at CERN should not be vulnerable to known
exploits.

Unfortunately for CERN, some of the ssh protocol
extensions CERN users depend on heavily (namely AFS and
Kerberos-4 support) are not available in SSH-2. A future move
toward Kerberos-5 could help in this respect since an
implementation of GSSAPI over SSH-2 is available. For the time
being, SSH-1 is still the recommended protocol at CERN.

1.5. log in using RSA keys

Do you really want to do this? Using
RSA for login means you will not get an AFS token, so you cannot
access most of your home directory on the public servers. There
is no way to "translate" between RSA key and AFS tokens.

If you want to give it a try, check the following common
errors:

the UNIX permissions must be correct: 0600 for
~/.ssh/authorized_keys, 0755 for
~/.ssh (and AFS read access for
everybody!), home directory not
writable by anybody but you.

Please make sure that your private key is somewhere
safe (e.g. in ~/private, with a
symlink to ~.ssh), and encrypted
using a good pass phrase.

in ~/.ssh/authorized_keys, there
has to be one key per line (no linebreaks allowed)

The debugging tips at the beginning of this chapter
(running the server in debug mode) should point out the reason
for failure pretty quickly.

1.6. New warning messages

OpenSSH stores both the host name and
the IP number together with the host key. This leads to some new
messages:

Warning: Permanently added 'lxplus001,137.138.161.126' (RSA) to the list of known hosts.
Warning: Permanently added the RSA host key for IP address '137.138.161.126' to the list of known hosts.

If these annoy you, use "CheckHostIP no" in your
$HOME/.ssh/config file.
However, please be aware that you are turning off an intentional
security feature of ssh.

Some warning that may appear while connecting to the PLUS
servers under their common DNS name (e.g. RSPLUS, HPPLUS) is due
to the fact that for load-balancing purposes, these servers' DNS
entry is constantly changing. This is detected and reported by
ssh (as it should be).

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: POSSIBLE DNS SPOOFING DETECTED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
The RSA host key for rsplus has changed,
5 and the key for the according IP address 137.138.246.82
is unknown. This could either mean that
DNS SPOOFING is happening or the IP address for the host
and its host key have changed at the same time
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
10 @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
15 Please contact your system administrator.
Add correct host key in /afs/cern.ch/user/i/iven/.ssh/known_hosts to get rid of this message.
Offending key in /afs/cern.ch/user/i/iven/.ssh/known_hosts:246
Password authentication is disabled to avoid Trojan horses.
Agent forwarding is disabled to avoid Trojan horses.

To avoid these, use qualified hostnames like rsplus01,
hpplus01 etc.. (LXPLUS and SUNDEV are not prone to this problem, since a common
host key is used on all the servers in the cluster)

An alternative is to (manually) insert into
$HOME/.ssh/known_hosts the
PLUS name after each qualified machine name that belongs to this
PLUS service:

To remove the above error message, simply edit the file
~/.ssh/known_hosts (or
~/.ssh/known_hosts2 for the SSH-2 protocol) and
remove the line (which should start with the hostname and/or IP
address). Be careful not to break the long lines, it has to have one
line per host/key. Next time you connect, ssh should ask you whether
you actually want to connect, etc..

1.7. Statistics options for scp

OpenSSH scp does not support a few of
the command line options from ssh-1.2.26. Besides, the
statistics output is different. The environment variables
controlling statistics output (SSH_SCP_STATS,
SSH_NO_SCP_STATS,
SSH_ALL_SCP_STATS,
SSH_NO_ALL_SCP_STATS) are not supported,
either. The changed options are

ssh-1.2.26 option

meaning

OpenSSH option

-a

Turn on statistics display for each file (on by default)

(on by default)

-A

Turn off statistics display for each file. This
appears to be a no-op for ssh-1.2.26

If you actually parse this output in scripts, you would
have to change them.

1.8. Errors on exit regarding X11 applications

Since the ssh client does forwarding for the X11 traffic
from the remote host, it won't exit until the last X11
application has been closed. It appears that this mechanism
sometimes fails, and the ssh program will report errors like
below even if all remote X11 applications are done:

(logout)
Waiting for forwarded connections to terminate...
The following connections are open:
X11 connection from xxxxx.cern.ch port 2352

The session will appear to hang. It can be closed by
typing "~." (without the quotes), and this should return you to
your previous shell. You could use "~&" as well to leave the
current connection as a background process.

If you are sure that there are no X11 windows or icons
from the remote server around, and if you can reproduce the
problem, please contact ssh.support@cern.ch.

A current suspicion is that the regular network scanning
mechanism plays a role in this: by opening a connection to the
remote X11 port, but failing to connect through the forwarded
channel, this could mess up the internal bookkeeping done by
ssh. To be confirmed.

1.9. Other FAQs

Notes

(they may have a vested interest, since they sell ssh
client and servers. The original ssh was freely distributable,
subsequent releases (including the ones that understood SSH-2)
were more and more restricted.)