Creating a Windows Server 2016 Vagrant box with Chef and Packer

I've been using Packer
for a bit over a year now to create the Windows 2012 R2 Vagrant
box that I regularly use for testing various server configuration scripts. My packer template has been evolving over time but is composed of some Boxstarter
package setup and a few adhoc Powershell scripts. I have blogged about this processhere. This has been working great, but I'm curious how it would look differently if I used chef
instead of Boxstarter and random powershell.

Chef is a much more mature configuration management platform than Boxstarter (which I would not even label as configuration management). My belief is that breaking up what I have now into chef resources and recipes will make the image configuration more composable and easier to read. Also as an engineer employed by chef, I'd like to be able to walk users through how this would look using chef.

To switch things up further, I'm conducting this experimentation on a whole new OS - Windows Server 2016 TP5. This means I dont have to worry about breaking my other templates, my windows updates will be much smaller (5 updates vs > 220) and I can use DSC resources
for much of the configuring. So this post will guide you through using Chef and Packer together and dealing with the "gotchas" which I ran into. The actual template can be found on github here
.

Preparing for the Chef Provisioner

There are a couple things that need to happen before our chef recipes can run.

Dealing with cookbook dependencies

I've taken most of the scripts that I run in a packer run and have broken them down into various chef recipes encapsulated in a single cookbook I include in my packer template repository. Packer's chef provisioners will copy this cookbook to the image being built but what about other cookbooks it depends on? This cookbook uses the windows cookbook
, the wsus-client cookbook
and dependencies that they have and so on, but packer does not expose any mechanism for discovering those cookbooks and downloading them.

I experimented with three different approaches to fetching these dependencies. The first two really did the same thing: installed git and then cloned those cookbooks onto the image. The first method I tried did this in a simple powershell provisioner and the second method used a chef recipe. The down sides to this approach were:

I had to know upfront what the exact dependency tree was and each git repo url.

I also would either have to solve all the versions myself or just settle for the HEAD of master for all cookbook dependencies.

Well there is a well known tool that solves these problems: Berkshelf
. So my final strategy was to run berks vendor
to discover the correct dependencies and their versions and download them locally to vendor/cookbooks which we ignore from source control:

Configuring WinRM

As we will find as we make our way to a completed vagrant .box file, there are a few key places where we will need to change some machine state outside of chef. The first of these is configuring WinRM. Before you can use either the chef-solo provisioner
or a simple powershell provisioner
, WinRM must be configured correctly. The Go WinRM library cannot authenticate via NTLM and so we must enable Basic Authentication and allow unencrypted traffic. Note that my template removes these settings prior to shutting down the vm before the image is exported since my testing scenarios have NTLM authentication available.

Since we cannot do this from any provisioner, we do this in the vm build step. We add a script to the <FirstLogonCommands> section of our windows answer file
. This is the file that automates the initial install of windows so we are not prompted to enter things like admin password, locale, timezone, etc:

As soon as this runs on our packer build, packer will detect trhat WinRM is accessible and will move on to provisioning.

Choosing a chef provisioner

There are two chef flavored provisioners that come "in the box" with packer. The chef-client provisioner
is ideal if you store your cookbooks on a chef server. Since I am storing the cookbook with the packer-templates to be copied to the image, I am using the chef-solo provisioner
.

Both provisioners will install the chef client on the windows VM and will then converge all recipes included in the runlist specified in the template:

Windows updates and other WinRM unfriendly tasks

The chef provisioners invoke the chef client via WinRM. This means that all of the restrictions of WinRM
apply here. That means no windows updates, no installing .net, no installing SQL server and a few other edge case restrictions.

We can work around these restrictions by isolating these unfriendly commands and running them directly via the powershell provisioner set to run "elevated":

When elevated credentials are used, the powershell script is run via a scheduled task and therefore runs in the context of a local user free from the fetters of WinRM. So we start by converging a chef runlist with just enough configuration to set things up. This includes turning off automatic updates by using the wsus-client::configure recipe so that manually running updates will not interfere with automatic updates kicked off by the vm. The initial runlist also installs the PSWindowsUpdate
module which we will use in the above powershell provisioner.

A couple important things to include when running the chef provisioner more than once is to tell it not to install chef and to reuse the cookbook directories it used on the first run.

For some reason, the chef provisioners will download and install chef regardless of whether or not chef is already installed. Also, on the first chef run, packer copied the cookbooks from your local environment to the vm. When it copies these cookbooks on subsequent attempts, its incredibly slow (several minutes). I'm assuming this is due to file checksum checking logic in the go library. You can avoid this sluggish file copy by just referencing the remote cookbook paths setup by the first run with the remote_cookbook_paths array shown above.

Cleaning up

Once the image configuration is where you want it to be, you might (or not) want to remove the chef client. I try to optimize my packer setup for minimal size and the chef-client is rather large (a few hundred MB). Now you can't remove chef with chef. What kind of sick world would that be? So we use the powershell provisioner again to remove chef:

Testing provisioning recipes with Test-Kitchen

One thing I've found very important is to be able to test packer provisioning scripts outside of an actual packer run. Think of this, even if you pair down your provisioning scripts to almost nothing, a packer run will always have to run through the initial windows install. Thats gonna be several minutes. Then after the packer run, you must wait out the image export and if you are using the vagrant post-provisioner, its gonna be several more minutes while the .box file is compressed. So being able to test your provisioning scripts in an isolated environment that can be spun up relatively quickly can save quite a bit of time.

I have found that working on a packer template includes three stages:

Creating a very basic box with next to no configuration

Testing provisioning scripts in a premade VM

A full Packer run with the provisioning scripts

There may be some permutations of this pattern. For example I might remove windows update until the very end.

Test-Kitchen
comes in real handy in step #2. You can also use the box produced by step #1 in your Test-Kitchen run. Depending on if I'm building Hyper-V or VirtualBox provider I'll go about this differently. Either way, a simple call to kitchen converge
can be much faster than packer build
.

If I'm using ahyperv builder to first create a minimal image, packer puts the build .vhdx file in output-hyperv-iso/virtual hard disks. I can use kitchen-hyperv and point it at that image and it will create a new VM using that vhdx file as the parent of a new differencing disk where I can test my recipes. I can then have test-kitchen run these recipes in just a few minutes or less which is a much tighter feedback loop than packer provides.

Using kitchen-vagrant to test on Virtualbox

If you create a .box file with a minimal packer template, it will output that .box file in the root of the packer-template repo. You can add that box to your local vagrant repo by running:

vagrant box add 2016 .\windows2016min-virtualbox.box

Now you can test against this with a test-kitchen driver config that looks like:

---
driver:
name: vagrant
box: 2016

Check out my talk on creating windows vagrant boxes with packer at Hashiconf!

I'll be talking on this topic
next month (September 2016) at Hashiconf
. You can use my discount code SPKR-MWROCK for 15% off General Admission tickets.