NX machine, custom ssh configuration and on disk encryption for the faint of heart

Recently I switched an ubuntu test server running Precise Pangolin from x11vnc to No Machine Nx server. I suppose (but haven’t tried) that those indications may also work for FreeNX (whose repository is unreacheable, at the moment)
As usual, it turns out that having a non-standard Ssh configuration or Ubuntu on disk encryption your mileage will vary, as NX machine relies heavily on OpenSSH. Also Ubuntu Precise brings some issues of this own.

I recently detailed the length I had to go to ensure reliable ssh communication with home-directory-encrypting .
However I like to understand what happens under the hood so I decided to go ahead with the set up instead of just rolling back x11vnc.
It’s a test server, so let’s get over this.
Of course, I managed to do everything with the server “near” me. Doing this on a remote server is not a good idea, even having something or someone with an alternative way of access on site. Things can go awry also after a reboot, because of the encrypted home file-system. Checking what happens after a full reboot before declaring ‘OK’ is a good idea.
Versions involved: nxclient,nxnode,nxserver 3.5.0-xx running on Pangolin Ubuntu 12.04 LTS 64 bit.
Premise: I suspect a good deal of problems can be avoided installing and configuring Nx machine with the home directory (e.g the home of the user that is going to connect) mounted. One has only to remember then to cover all bases and modify the relevant directives and copy the relevant files outside the encrypted home. I did it without the home directory mounted.
Read everything before doing anything!

Things to check (aka Checklist)

Check for denyhosts or some other protection mechanism for ssh. Time to whitelist the incoming IP to avoid being throwed out during the setup. Even if you’r not firing up Openssh explicitily, Nx client does everytime you try to access. If denyhost is active before Nx is properly setup, you will be left out in the cold.

Doing a test on a test server is a good idea.

Checking for an AllowUsers directive in sshd_config is a good idea. If it is active, the user “nx” needs to be there. As explained here(en.wikipedia.org), NX creates an ‘nx’ user on the server machine whose shell is executed any time a remote NX user connects to SSH using NX Client.

Issues with encrypted home

The first issue here is simple: NX machine needs to connect the nx user first via openssh. So if the system has encrypted homes, openssh sshd must be reconfigured to search for autorized_keys outside the encrypted home, or the nx user could login just when its home directory is mounted and unencrypted. I went thru this already here. From the private key in /usr/NX/share/keys I extracted the pubkey with ssh-keygen -y and placed it in <whateversshdconfdir>/nx/autorized_keys. “nx” user can now login. /var/log/Syslog shows nx login progress so it may be wise to have a look there.
The encrypted home impacts also the No machine NX ability to create the .nx directory for the logging user. We have to move this directory on a different place, outside the encrypted home. The reason, if you have not read my previous post, is that until the encrypted home is mounted, every configuration file inside it can’t be read, and this affect both OpenSSH and No machine NX ability to login after a reboot, or if the user never logged in. NX solves this with the UserNXDirectoryPath = “whaoever” directive in /usr/NX/etc/node.cfg. Pointing it where NX can write, it will create a directory <whaoever>/<username>/ with the needed files. I used /tmp/ndir as suggested in the file but I now live in fear that something may periodicaly wipe /tmp and leave me outside. It’s a test server, anyway.
It’s not enough. NX uses /usr/bin/xauth and this too, has to access the encrypted home. So let’s mount it, using this directive in node.cfg:

UserScriptBeforeSessionStart = “/usr/bin/ecryptfs-mount-private”

Before going on, I made some other changes.

EnablePasswordDB = “1”

in server.cfg and

CommandStartGnome = “/usr/bin/gnome-session –session=ubuntu-2d”

in node.cfg. This last one is mandatory with Precise and gnome, or nothing happy will happen. I don’t know about KDE.
I also mounted the user home directory manually (remember: cd outside /home/<user> before issuing ecrypt-mount-private. Took me a while to figure out why did not work out at first) and add the existing system user as nx user as specified in the NX admin guide

nxserver –useradd USERNAME [–administrator]
please note: this will create a new keypair for the current user and an authorized_keys2 file in the encrypted user .ssh directory. I guessed and merged this autorized_keys2 with the user autorized_keys. I’m still wondering about possibile shortcomings of this, so think twice about it. Finally restarted the NX server.
Nx machine now works fine also between reboots.