Meandering thoughts of the Knight household, mostly work and technology related guff.

Tuesday, June 15, 2010

Don’t Use An SMP Kernel When Virtualising FreeBSD 6.4 On VMWare ESX/ESXi

One of my clients has some FreeBSD 6.4 installs on hardware that is slowly dying and asked me to virtualise them onto an HP ProLiant ML350 G5.

The HP ProLiant had been running a Windows Server 2003 VM and a Windows Server 2008 VM on top of a Windows Server 2008 Core install with Hyper-V reliably, but these VMs were no longer in use and surplus to requirements.

I blew away the Windows Server 2008 Core install and installed VMWare ESXi 4.0 with the HP customisations. I then installed FreeBSD 6.4, copied across all the data from the physical install and proceeded to build the required ports.

This is where all the problems started. Random signal 11 crashes started occurring throughout this build process.

So I quickly created a new VM and installed FreeBSD 7.3 to it. Again, installation was no problem. Copied across all the data and successfully built all the ports. Only problem is the servers in question run an old Linux binary. This binary would start up successfully but would not accept any network data. Changing the linux_base port from the old RedHat 7.3 one to the current Fedora Core 4 one made no difference.

At this point I walked away and went to bed. By morning I had worked out what the problem was, so I went back to my FreeBSD 6.4 VM, reduced the number of vCPUs to 1 and swapped the SMP kernel for the GENERIC one. All ports then built successfully. The VM in question is now being stress tested for any problems prior to production use.

Moral of the story? Even though your virtualisation stack may support the Guest OS in question, it doesn’t mean that your Guest OS won’t necessarily have problems when virtualised. Perform burn-in/stress tests prior to production use.

The only reference I could find to related problems was on the FreeBSD-stable and FreeBSD-bugs mailing lists: