Automating ESXi 4.1 Kickstart Tips & Tricks

While testing the new kickstart functionality in ESXi 4.1, I ran into a few issues trying to convert a classic ESX 4.x deployment to ESXi 4.1. I thought I share some of the tips and tricks I have learned, so others will not encounter the same issues.

If you are going to specify a ks.cfg (kickstart configuration file) in your pxelinux file, make sure that the kickstart entry is appended after the *vmkboot.gz* but before *vmkernel.gz* entry as highlighted in green in the screenshot. If you place it anywhere else in the boot line option, you will receive an error that is not easy to diagnose. Also you want to make sure you add triple dashes (---) after the kickstart line following the required syntax for the boot options as highlighted in orange in the screenshot.

Tip #2

While developing and testing your ks.cfg, you may want to use the new dryrun parameter which parses your kickstart configuration file looking for syntax and formatting errors. In dryrun mode, no installation will be performed but you will be provided with a log of whether your ks.cfg had any errors, warnings or was successful in being validated.

The following screenshot shows a warning where I purposely left out --hostname entry which is generally recommended within the "network" portion of the ks.cfg:

If there are other errors or warnings, they will be displayed within this screen and you can login to the host to view the log for more details (esxi_install.log):

To login to the host, you will press "enter" and you will be prompted for login (press Alt+F1 to go to login screen). The username by default will be "root" and the password is blank, just press enter for the password:

Once logged in, you will want to take a look at the esxi_install.log for more details on how your ks.cfg is being processed and if there are any errors or warnings discovered by the parser:

Tip #3

If you want to enable both local and remote (SSH access) Tech Support Mode on your ESXi host, you now have the ability to do this via host services. You can use the vim-cmd (vimsh) utility to enable these services and both local and remote TSM is disabled by default.

Note: If you want to enable either local and/or remote TSM, you need to make sure you enable and start the service for you to actually be able to SSH into your ESXi host.

Tip #4

With classic ESX, if you needed to transfer additional packages or files to your host, you could easily mount an NFS volume, with ESXi, an NFS client is not available. If need to transfer files for configuration purposes, you can utilize the wget utility.

The syntax for wget is the following:

wget http://webserver/file -O /tmp/file

Tip #5

I have been told by support that you could not configure syslog for your ESXi host without relying on external tools such as vCLI, PowerCLI or vSphere Client. I have found that you actually can configure syslog configurations, though you have to dig a little bit into vim-cmd (vimsh) as it is not available using any of the local esxcfg-* commands. There only three syslog options as provided via vSphere Client in the Advanced Host Configurations: Syslog.Remote.Hostname, Syslog.Remote.Port and Syslog.Local.DatastorePath

Note: Currently you can only configure one syslog server for your ESXi host to forward logs to.

Tip #6

Another new new kickstart parameter introduced with ESXi 4.1 is --level that is used in conjunction with %firstboot stanza. This parameter specifies the specific order in which the kickstart firstboot configurations should run with respect to the other startup scripts when your ESXi host first boots. By default, if you leave this out, VMware will automatically create a script called firstboot_001 and number it 999 which will be the very last script to execute. It is a good idea to move any post configurations to the very end, since most of post configuration may rely on specific VMware CLIs and services which must be started up before executing. You of course can change level, but be careful about moving it too early in the boot process.

Here is an example of changing the level to 998:

Once the host has booted up, you can login to see the script that was created from your %firstboot stanza under /etc/vmware/init/init.d

Note: As you can see, the firstboot script has now changed to 998. You will also notice two other scripts set at level 999 that handles updating the password if you decide to set a root password from the default blank, which you should. These custom scripts are generated after the initial build and upon the next reboot, these will be automatically removed.

Tip #7

Youmay have noticed in Tip #6, we changed the --level to 998, by default all three of these init scripts are set to boot order 999. This was actually done on purpose, the reason being as described earlier, the root password is blank by default. One issue that I found while testing is the inability to enable "Management Traffic" for a VMkernel interface. You can easily enable vMotion and FT Traffic for a VMkernel interface using vim-cmd (vimsh), but you can not for Management Traffic. One way I solved this problem is creating a python script which connects to the local ESXi MOB and enables Management Traffic on a particular VMkernel interface. I have shared this specific script on the on the VMTN communities which can be found here. The script is actually based on modified version that was initially created by Justin Guidroz who blogged about it here.

Here is the snippet that would be included in the %firstboot in which does not require you to expose the root password as it is empty by default:

As you can see, we first create the python script and then we execute it. This allows us to call other utilities within the Busybox console without having to specify the interpreter to be python, we can just use busybox as the interpreter.

If you tried to configure NTP by echoing your NTP servers into /etc/ntpd.conf and restarting ntpd, you will notice that the changes do not take effect. The only way I have been able to get this to work is by issuing another reboot which is specified at the very end of the %firstboot which will then be picked up upon boot up by the host.

If you want to update the default datastore name from "datastore1" to something more useful such as [hostname]-local-storage-1, you can use vim-cmd (vimsh) to do so. Here is the syntax for the command if you want to use the short hostname and append "-local-storage-1" (this should be done in the %firstboot section of your ks.cfg): vim-cmd hostsvc/datastore/rename datastore1 "$(hostname -s)-local-storage-1"

Tip #11

SNMP is another one of those configurations that can not be configured and started up via normal services as you would have done in classic ESX. You can make the appropriate edits to the configuration file and you will need to reboot the host for the changes to take affect just like NTP configurations. You will need to edit /etc/vmware/snmpd.xml and add that to your firstboot section. Here is an example of snmpd.xml file:

1

2

3

4

5

6

7

8

<config>

<snmpsettings>

<communities>public1;private1</communities>

<enable>true</enable>

<port>163</port>

<targets>192.168.1.5public1;192.168.1.6@163private1</targets

</snmpsettings>

</config>

Tip #12

A VMTN user recently posted an issue when applying patches during firstboot, that the init scripts were not being removed and the scripts continue to execute upon every reboot. The problem was that the boot.cfg were not being properly updated under /vmfs/volumes/Hypervisor{1,2}. I did some testing and found that if you created a second customization script and perform the patching as the very last step and issued a reboot, that you would not run into this problem. Here is a small snippet of what your ks.cfg would like look with 2 custom init scripts, one configured at 998 and the other configured at 9999:

Here is a post on Automating Active Directory User Management in ESXi Kickstart As you can see, there are quite a few hacks I had to go through to get an equivalent kickstart build for ESXi 4.1 compared to classic ESX. I am sure there are other gotchas, but these are the ones I had encountered. Overall, ESXi 4.1 has greatly improved in terms of automating an unattended installation and configuration from ESXi 4.0, but there is definitely more work that needs to be done by VMware before users can easily transition from classic ESX to ESXi.

nicType describes the type of VMKernel traffic such as “management”, “vmotion” and “faultTolerance”, this is not the name of a portgroup. Make sure you have a valid vmkernel interface that has already been created and you need to specify the correct device (e.g. vmk0,vmk1,etc). Looking at the error, it sounds like you don’t have a VMKernel interface and that is why you’re getting the error.

One thing maybe you could help with – I’ve got the esxupdate update bundle command working fine but I cannot get the host to exit maintenance mode after. No errors from the script and it works manually too. I’ve tried a separate level just for the vim-cmd /hostsvc/maintenance_mode_exit command as well as rebooting THEN exiting maintenance mode with no luck.

So I am currently looking at deploying multiple ESXi servers to remote sites. This may be a long shot but ultimately what we would want to do is just rack the servers and when we power them on have someone at the site just type the site number into an input prompt and then from there have it completely automated and joined to vCenter with no interaction at all. This is a long shot and may not be so simple to do but would help us not like at Hyper-V or other solutions for virtualizing the sites. If you have any recommendations at all that would be great.

I ran into this issue with ESXi kickstart install process where my system has several nic interfaces and when the VMkernel first comes alive and tries to download the ks.cfg file, it would try to do so using nic0 and fails immediately discontinuing the process. Nic2 is the only active interface and since this is on a chassis we develop there is no wires to move around either to make eth0 working.

Have a look at http://www.ultimatedeployment.org/If you can modify the bootp parameters at your remote sites, or install an uda aplliance per remote site, it’ll be easy to deploy your esxi host.we are very succefully deploying 4.1 vsphere and win7 with UDA.

But what if you don’t *want* to erase the first disk and install there? What if you don’t know _which_ disk you want to install to? ks.cfg requires autopart or it won’t work. All I want is for the server to boot from pxe and load the files, then let me choose which disk I want to use. Is there a boot command to do that, without using ks.cfg?

My problem is, I will be knowing the cluster name where i can deploy the vAPP but as part of My .ovf description i need to select data store in the cluster where i am deploying.i need to select max free size datastore.

Trackbacks

[…] are some minor changes with kickstarting ESXi 5.0 but the majority of your existing ESXi 4.1 kickstart configurations can be re-used with a few modifications. One of my goals during the vSphere 5.0 beta was to […]

Primary Sidebar

Search this website

Author

William Lam is a Staff Solutions Architect working in the VMware Cloud on AWS team within the Cloud Platform Business Unit (CPBU) at VMware. He focuses on Automation, Integration and Operation of the VMware Software Defined Datacenter (SDDC).