How to Persist Configuration Changes in ESXi 4.x/5.x Part 2

Continuing from part 1 of How to Persist Configuration Changes in ESXi 4.x/5.x Part 1, here is another method which I prefer when trying to persist configuration changes in ESXi. When ESXi boots up, it loads it's filesystem into memory which the modules to be loaded up are determined by the configuration found in /bootbank/boot.cfg and /altbootbank/boot.cfg for the two respective partitions (primary / backup).

UPDATE: You can now persist configuration files such as firewall rules and others using the new VIB Author Fling, please take a look at this article for more details.

As we learned from the previous article, the cron'd /sbin/auto-backup.sh generates a local.tgz which is then converted to state.tgz which contains all files automatically backed up by VMware. This file is loaded up along with other modules as part of the boot process. Understanding this, allows us to take advantage of this feature for persisting our own configuration files.

Here is an example use case for creating .ssh directory for SSH keys and persisting a script
(ghettoVCB.sh) in /bin for an ESXi 4.x host:

Step 1 - Re-create the modified directory structure and files in a temporary local path which will then be tarred and gzip. An example would be the following:

Actual change:/.ssh/authorized_keys/bin/ghettoVCB.sh

Temporary local directory structure of change:/tmp/.ssh/authorized_keys/tmp/bin/ghettoVCB.sh

Note: It is very important to ensure that the modified files get the stickybit permission set. As noted in the last article that upon a change, the visorFS will automatically create a special file to denote it for backup but also it allows the file to be writable at some later point for custom files being added.

Step 2 - You will use the tar utility to tar/gzip the contents in a file with extension .tgz. One thing to note, the file name including the extension must not exceed 12 characters. In our example, we made two changes and re-created the local structure under /tmp. We will need to change into /tmp directory and tar up the contents by using the following command:

The next time you reboot the system, you will automatically have your .ssh directory containing your SSH keys and the ghettoVCB script under /bin directory.

Now this is great for an inline modification, but what about creating custom configuration files and including that as part of a default kickstart installation? What about something like custom firewall rules in ESXi 5? In the following example, we'll include a custom firewall rule called "virtuallyGhetto.xml" which will be stored in /etc/vmware/firewall when the contents of the module is extracted.

Step 1 - We of course need to create the XML file containing the firewall rule and create the directory structure in which it will be unloaded to.

Step 3 - This new package will need to be stored on you installation server which will be reachable via http using wget and as part of the %firstboot stanza in your ESXi 5 kickstart. It will download the *.tgz file and append the entry in /bootbank/boot.cfg configuration file. Here are the entries that should go into your kickstart:

Note: If you add custom files that are located under /etc and you have the stickybit enabled on your file, changes made will persist upon the next reboot by either manually running /sbin/auto-backup.sh or letting it run via cron. If you add custom files that are not located under /etc, any change you make must be periodically updated in your custom *.tgz file else the next reboot, the original file will be loaded.

FYI – file name must be in 8.3 format or the loader can’t find it and halts boot.

If that happens don’t panic.

Boot to the installer disk and start like you are doing an “in place” upgrade but STOP at the last F11 to start request.
Instead hit alt+F1 to get a login and then root/. Boot bank and alt boot bank are mounted under /vmfs/volumes. Make your corrections and then reboot.

Primary Sidebar

Search this website

Author

William Lam is a Staff Solutions Architect working in the VMware Cloud on AWS team within the Cloud Platform Business Unit (CPBU) at VMware. He focuses on Automation, Integration and Operation of the VMware Software Defined Datacenter (SDDC).