A colleague and I were recently discussing how he preps his RHEL template VMs for use with Puppet. It inspired me to share how I prepare my Linux VMs to become a template within vSphere 6.5. A point to note is that I don’t prepare my template images for a particular configuration management system like Puppet, but instead bootstrap them once they’re deployed. Why? I use my templates for a variety of things, and sometimes the people who end up with the VMs don’t want my management systems on them. It also means I have to handle manually some of what is done scriptwise via the configuration management system, but that’s just fine. I’d actually rather do it that way because it helps me guarantee the state of the system.

You can perform these steps in full multiuser — runlevel 3 — or in single-user by issuing an “init 1” and waiting for all the processes to stop. I wouldn’t do any of this in runlevel 5, with full X Windows running. In fact, I really don’t suggest installing X Windows at all on Linux VMs unless you really, really need it for some reason… but that’s a whole different topic. I’d also suggest taking a snapshot of your template prior to trying any of this out. As Lenin said, “Trust, but verify.”

This write up assumes you have a fresh RHEL/CentOS install running that has been updated and has your required packages installed. You must also ensure that open-vm-tools is installed before proceeding.

Points to note on the install:

- Don't create a local user (use a root account only)
- Make sure the disk is thin provisioned
- Ensure that the network is enabled and leave it set as DHCP for now

I like to use the --skip-broken flag when running yum update. It's a great feature for skipping packages that have dependency problems or that may introduce problems to the already installed packages.

i.e: yum update --skip-broken -y

12 Steps total

Step 1: Stop the Logging Services

You’re going to all this trouble to create a clean deployable template, so you might as well stop writing new data. Otherwise all your deployed VMs will have a log of you shutting the VM down.

Issue the following commands to stop the logging services:

/sbin/service rsyslog stop
/sbin/service auditd stop

Step 2: Remove any old kernels

You need yum-utils installed to get package-cleanup. This has to go before the yum cleanup in the next step as it needs your channel data. I usually let the post-deployment configuration management take care of this, but this is nice when we create a new template for a intermediate/point release, or just to cover a security hole.

Issue the following command to remove the old kernels (if there are any):

package-cleanup --oldkernels --count=1

Step 3: Clean out Yum

Yum keeps a cache in /var/cache/yum that can grow quite large, especially after applying patches to the template. For example, my lab host currently has 360 MB of stuff in yum’s cache right now, just from a few months of incremental patching. In the interest of keeping my template as small as possible I wipe this.

Issue the following command to clean the cache:

yum clean all

Step 4: Force the logs to rotate and remove old logs we don’t need

Starting fresh with the logs is nice. It means that you don’t have old, irrelevant log data on all your cloned VMs. It also means that your template image is smaller. Change out the “rm” command for one that matches whatever your logrotate renames files as. As an aside, because people usually neglect this step, if you get really, really bored it’s fun to look at the old log data people leave on virtual appliances in cloud templates. Lots of leaked information there ;)

Step 6: Remove the udev persistent device rules

Have you ever noticed that if you clone/deploy a Linux VM that won't bring up it single network interface and renames the interface to something like eth1? Yep, well that's the udev persistent network interface rules coming back to haunt you. This is how I've decided to deal with the problem.

This generally affects CentOS /RHEL 6 and you shouldn't have to do it in v7 but it won't hurt anything.

Issue the following command to remove the udev persistent rules:

/bin/rm -f /etc/udev/rules.d/70*

Step 7: Remove the traces of the template MAC address and UUIDs

This is a corollary to step 5, just removing unique identifiers from the template so the cloned VM gets its own. You can also change the “-i” to “-i.bak” if you want to keep a backup copy of the file.

Step 8: Clean /tmp out

Under normal, non-template circumstances you really don’t ever want to run rm on /tmp like this. Use tmpwatch or any other manner of safer ways to do this, since there are attacks people can use by leaving symlinks and what-not in /tmp that rm might traverse (“whoops, I don’t have an /etc/passwd anymore!”). Plus, users and processes might actually be using /tmp, and it’s impolite to delete their files. However, this is your template image, and if there are people attacking your template you should reconsider how you’re doing business really.

/bin/rm –rf /tmp/*
/bin/rm –rf /var/tmp/*

Step 9: Remove the SSH host keys

If you don’t do this all your VMs will have all the same keys, which has negative security implications. It’s also annoying to fix later when you’ve realised that you’ve deployed a couple of years worth of VMs and forgot to do this in your prep script. Not that I would know anything about that. Nope.

/bin/rm –f /etc/ssh/*key*

Step 10: Remove the root user’s shell history

No sense in keeping this history around, it’s irrelevant to the cloned VM.

/bin/rm -f ~root/.bash_history
unset HISTFILE

Step 11: Remove the root user’s SSH history and other stuff

You might choose to just remove ~root/.ssh/known_hosts if you have SSH keys you want to keep around.

/bin/rm -rf ~root/.ssh/
/bin/rm -f ~root/anaconda-ks.cfg

Step 12: Clear bash history and shutdown for template creation

Time to clear the history of everything you've just done and shutdown the server in a clean state for converting to a VM:

history –c
sys-unconfig

The server will automatically now shutdown.

So that’s my prep routine. It relies heavily on keeping the rest of the VM clean, and only cleans up what we can’t avoid sullying. Once the server shuts down as a result of the sys-unconfig command you can convert to a sanitized template in vSphere and add any customization specs required to the Linux deployments.

3 Comments

Hey Liam,
Awesome How To article! It really helped me out with a project I'm working on! If it's alright, I do have one question about doing the deployments from a template. I'm currently working with SUSE Enterprise Linux 12 and I noticed that when I deploy off a template, the /sda1 and /sda2 UUIDs are the same in every clone. Is this something that I need to change? Or can several servers with the same device UUIDs exist? If I need to change them, how should I go about it?
Thank you in advance for any and all help you may be able to provide!!!