Oracle Blog

Lewis Thompson's weblog

Wednesday Apr 22, 2009

The Problem

Recently one of my customers migrated a legacy SPARC system to a Solaris 8 branded zone on a new T5220 platform. This system provided an anonymous FTP service which no longer worked correctly within the branded Zone. More specifically, files could be uploaded & downloaded but directory listings always returned empty. Non-anonymous users reported no problems with any of the functionality

The Investigation

Initial investigation began by trussing in.ftpd within the Solaris 8 zone to see what was happening. Very quickly it was possible to see that in.ftpd forks and execs /bin/ls to generate the directory listing and this was failing. It's worth pointing out that on Solaris 8 anonymous FTP requires a chroot() environment (as per in.ftpd(1M)) and this was configured correctly

19710/1: execve("/bin/ls", 0x000353F0, 0xFFBFFE0C) Err#2 ENOENT

Very odd as ENOENT indicates 'no such file or directory' and /bin/ls definitely exists. Next step was to look more closely at the execve() syscall... I knocked up a very quick fbt DTrace script that would do this for me before realising that this was Solaris 8! Fortunately we can DTrace from the global zone, so after a few changes I can up with the following to trace the Solaris 8 exec calls:

Running this from the global zone at the same time as issuing an LS in the anonymous FTP client and I could see that ENOENT was being returned from a call to lookupnameat(). The DTrace was updated with the following to print out the full filename of the file that was being looked up:

Another LS from the client and the problem becomes clear -- we are trying to load /.SUNWnative/usr/lib/s8_brand.so.1. Until now I was not familiar with the /.SUNWnative directory but this exists on all Solaris 8 & 9 zones. Within this directory are three lofs mounts that present /lib, /usr and /platform to the local zone from the global:

Those that have kept up will now know why things are failing -- everything here hinges around anonymous FTP using a chroot() environment and those Solaris 8 zone-specific mounts not existing (why would they? in.ftpd(1M) was written years before Zones were around)

The Fix

Despite all of this investigation the real fix is far more obvious: don't use the legacy Solaris 8 FTP daemon when your system is running Solaris 10 (or OpenSolaris)!

The same functionality can be achieved by creating a minimal Solaris 10 local zone & setting up anonymous FTP there. The newer Solaris 10 ftpd provides additional functionality over Solaris 8. The last step in this method is to set up a loopback mount in the global zone to present the relevant directory into the Solaris 8 zone

The Workaround

Not everybody will have the option of adding an additional Solaris 10 zone to deliver "The Fix" so here is a workaround to get Solaris 8 anonymous FTP working. The good news is that it isn't at all hacky, it's just a few extra steps that are missing from in.ftpd(1M)

Armed with the knowledge that we are missing the /.SUNWnative mounts within the chroot() environment we simply add these in and things will begin to work. The updated mount output will now include:

Here my chroot environment exists under /export/home/ftp. And a more permanent way to achieve this is by updating your zone's vfstab with:

/.SUNWnative - /export/home/ftp/.SUNWnative lofs - yes ro

As a final note, in.ftpd(1M) didn't seem to mention that /lib/ld.so.1 should also be included in the chroot() environment. It should, and things won't work unless it is there. Don't forget to ensure the permissions are all set correctly