ESXi

In a previous post I blogged about creating a vanilla vSphere 5 ESXi USB drive using the VMware .iso file from VMware. This post shows how to create one using the HP version of vSphere ESXi (5.0_Oct_2011_ESXi_HD-USB-SDImgeInstlr_Z7550-00253.iso).

The HP version comes pre-installed with all the HP CIM providers which work very well with HP servers, including the HP MicroServer. Using the HP version gives you the more details in the Hardware Status tab.

I’m going to be using a different method, recommended by Will Rodbard (thanks Will), who is a colleague of mine at VMware, you can see his comments from the previous post. In summary the steps are:

Run the HPUSBFW tool, click on the USB drive, select ‘Fat32′ and click Format

Run UNETBOOTIN, select Diskimage and browse to the ESXi 5 ISO file

Select the USB drive you have just formatted and click OK

If you want to make more USB keys for more servers, then now is the time to create .IMG files using WinImage, then you can basically clone the image of the USB key to more USB keys. Or if you don’t wish to use WinImage then just perform steps 1 to 4 again.

Once completed your USB drive will boot into the ESXi 5 installer. Once booted, install the ESXi 5 Hypervisor to the USB drive (overwriting the installer). This will then leave you with the installed ESXi Hypervisor on the USB.

Note that using this method creates a brand new bootable USB key for use in a new installation of vSphere ESXi. You will have to go through the process of installing ESXi onto the USB key, or another disk or LUN on the target server. If you want a USB key that is already installed with ESXi which saves you from going through the installation wizard, you can use the other method in this post.

[Aside]

I coincidently left an older USB key in my laptop and booted. Here’s a picture of my Macbook Pro running vSphere ESXi, and it all works by the way, including networking!

A very quick post on how to create an image that contains vSphere 5 ESXi Embedded with which you can use to quickly create USB sticks that have the ESXi hypervisor installed. This is not the same as creating a bootable USB key that contains the installation files to install ESXi from the USB stick. For this method please refer to this post.

Use this in your lab environment, I wouldn’t recommend doing this in production environments.

In previous versions of vSphere ESXi, it was relatively straight forward to create a bootable USB key which already contained the ESXi hypervizor. This was done by extracting the files from the ISO and then using ‘dd’ to image the directory structure to the USB stick. With vSphere ESXi 5 however, this technique is no longer possible. There is a workaround however. ESXi is installed and configured in two steps, the installation is done to a disk with a vanilla installation of ESXi without configuration. The server is then rebooted and the configuration of ESXi continues with the creation of the management network vmk0 or vmk1 (depending on your setup), hostname, DNS etc.

For this to work, we do not perform the second part, which is the configuration, but take an image of the USB key directly after the installation of the vanilla installation of ESXi without configuration. This enables us to image this vanilla installation onto as many USB sticks, i.e., servers as we like without clashes in virtual MAC addresses and the like.

Quick steps

Insert the USB stick to your workstation (the same one that runs VMware Workstation).

Boot the VM and connect the USB stick to the VM.

Install ESXi as normal, making sure that you install onto the USB stick, when installation is complete, disconnect the USB stick from the VM and do not reboot the VM, just turn it off. You no longer need this VM.

With the USB stick still connected to your workstation, open up Winimage.

Purpose

This article explains how to use the Operating System Specific Packages (OSPs) to install VMware Tools during the PXEBOOT and Kickstart of a Linux guest OS running on vSphere 4.1 and above.

Background

As of vSphere 4.1, the VMware Tools RPMs are no longer available on the CD image linux.iso. To install you must use the tar.gz package and install it manually.

To quote the vSphere 4.0 U2 release notes:

“The VMware Tools RPM installer, which is available on the VMware Tools ISO image for Linux guest operating systems, has been deprecated and will be removed in a future ESX release. VMware recommends using the tar.gz installer to install VMware Tools on virtual machines with Linux guest operating systems.”

This future ESX release is indeed vSphere 4.1. The reason that the RPM packages are no longer available on the VMware Tools CD image for Linux guest operating systems ISO is to reduce the footprint of ESXi. Therefore the linux.iso no longer contains the RPM installer.

Not only is this a real pain for customers who have always deployed packages onto their Linux guests with RPMs but also removing the VMware Tools RPM also causes issues with the deployment of Linux guests through PXEBOOT and Kickstart methods.

Yes, you could argue that there is the option of using vCenter Templates along with Customization Specifications (CS) for Linux, and I for one love the fact that the CS works for Linux very well. This is not always an option for a customer who has already configured their entire environment to deploy both ESXi servers and virtual machine guests with PXEBOOT and Kickstart.

The only option here is to perform manual installations of VMware Tools using the tar.gz file which is included with vSphere 4.1. This of course poses a few issues.

One: this is a repetitive task which is both time consuming and problematic if the number of new VMs to provision is high.

Two: some security conscious organisations do not allow the gcc Compiler and/or the Linux Kernel sources to be installed on the VM guests. Both the gcc Compiler and the Linux Kernel sources are mandatory for the successful installation of VMware Tools using the tar.gz file.

Three: if the guest VM is using a VMXNET2 or VMXNET3 ethernet adapter, then the guest VM will not have compatible drivers if VMware Tools is not installed.

These are just three of the reasons that I’ve seen in the wild, there are more but I won’t go into much detail here.

The solution here is to use the Operating System Specific Packages (OSPs). VMware Tools OSPs allow you to use your operating system’s native update mechanisms to automatically download, install, and manage VMware Tools for the supported operating systems. For more information regarding OSPs, see http://www.vmware.com/download/packages.html.

Resolution

For this guide, I will be using the Ultimate Deployment Appliance 2.0 (uda20.build17) to deploy a RHEL 5.5 x64 virtual machine.

First we need to prepare to install the operating specific packages (OSPs) for RHEL 5 guest operating systems on the Build Server. We do this by preparing the directory structure for the rpms, and placing these in the correct location on the Build Server for the guest VM to download during a Kickstart installation.

You should now have a vmwaretools folder structure that looks like this

Now copy the entire vmwaretools directory to your Build Server using your favourite SCP application. I’m using the UDA 2.0, so I will be placing vmwaretools in /var/public/www/kickstart/rhel/. “rhel” is the ‘Flavor’ name that I called my RHEL 5 OS configuration.

Perform the following on the Build Server.

On the UDA 2.0 the contents of the /var/public/www/kickstart/rhel/vmwaretools directory should now look like this.

# Create and edit the VMware repository directory and file, note that this points to the Build Server during Kickstart build. Once VMware Tools is installed we shall change the baseurl to point to the VMware OSP URL.

# Add the following contents to the repository file and save

cat > /etc/yum.repos.d/vmware-tools.repo <<\EOF1

[vmware-tools]

name=VMware Tools

baseurl=http://192.168.200.30/kickstart/rhel/vmwaretools

enabled=1

gpgcheck=1

EOF1

# Install VMware Tools by accepting all defaults

yum install -y vmware-tools

# Delete the customised vmware-tools.repo file and reconfigure with the baseurl to point to the VMware OSP URL for RHEL5 64-bit.

I also attempted to use the VMware OSP URL directly in the vmware-tools.repo file without luck either. If you can get this to work then please comment below and I’ll update the post.

You should now be able to deploy a guest Linux VM running on vSphere ESXi 4.1 through PXEBOOT and Kickstart and have the VM automatically install VMware Tools. Plus, all future updates of VMware Tools can just be done by invoking a yum install -y vmware-tools.

Purpose

This guide shows you how to create a new initrd.img with integration for the VMXNET3 driver to allow guest RHEL virtual machines equipped with the VMXNET 3 driver to Kickstart build RHEL using PXEBOOT.

Background

VMware’s VMXNET 3 network adapter supports PXE booting but RHEL 5 does not have a driver that supports network installations using the default initrd.img.

If you tried to perform automated installation using kickstart with the standard initrd.img you will see the following screen:

This is because Anaconda does not recognise the VMXNET 3 device and therefore is not able to load a driver for it.

This guide shows you how to create a new initrd.img with integration for the VMXNET3 driver.

For the impatient few, I’ve made the resulting initrd.img.vmxnet file available for download, it is a clean ramdisk image that was made using the steps below.

It is the PXEBOOT RAMDISK with the VMXNET3 driver for RHEL5 (created from the rhel-server-5.5-x86_64-dvd) [2.6.18-194.el5].

Tested and working to support VMXNET3 in Anaconda. You can download it from here initrd.img.vmxnet and then jump all the way to Step 18 to place it on your Build Server.

Prerequisites

Prepare a Reference Virtual Machine

First create a new reference virtual machine with the following hardware specifications:

Configuration

Value

VM Hardware Version

Hardware Version 7

Network Adapter

VMXNET 3

SCSI Controller

LSI Logic SAS

SYSTEM .vmdk Device

SCSI 0:0 15Gb

Remove Floppy Device

Yes

Install RHEL (rhel-server-5.5-x86_64-dvd) by mounting the ISO to the VM and then perform a manual installation of VMware Tools. This will give you the reference virtual machine with which you will then use to copy the vmxnet.ko and vmxnet3.ko from.

Enable sshd services on the Reference VM by typing:

/etc/init.d/sshd start

This will make it a lot easier to copy files to your Build Server.

Prepare your Build Server

Create your own PXEBOOT and Kickstart installation or use one that you already have. For my example I will be using the Ultimate Deployment Appliance 2.0 (uda20.build17).

Most of the configuration is done on Build Server so by all means enable SSHD to make things a lot easier for you.

My Build Server IP is 192.168.200.30.

Integrating VMXNET 3 into initrd.img

At this point you should have SSH access to both your Build Server and your Reference VM.

Perform the following on the Build Server.

1. Make some working directories to work in

mkdir /tmp/workingdir

mkdir /tmp/workingdir/initrd

mkdir /tmp/workingdir/modules

Perform the following on the Reference VM.

2. Obtain the initial ramdisk initrd.img from the pxeboot directory, this file can be obtained from the rhel-server-5.5-x86_64-dvd ISO file which should still be connected to the Reference VM. The initrd.img file can be found in the images/pxeboot directory.

14. Search for VMware and add the following line under the Abstract SVGA Adapter

07b0 VMware Adapter

The 07b0 number here is whatever was obtained from Step 5 above.

15. Edit the module-info file and add the following entries for the VMXNET and VMXNET 3 Adapters, put it in under ‘v’ to keep it in alphabetical descending order. You should still be in /tmp/workingdir/initrd/modules/

nano /tmp/workingdir/initrd/modules/module-info

vmxnet

eth

“VMware vmxnet Ethernet driver”

vmxnet3

eth

“VMware vmxnet3 Ethernet driver”

16. Import the vmxnet entries from the Reference VM’s module.alias file (now called module.alias.reference) into the Build Server’s module.alias file.