AllGoodBits.org

Chef Cookbook Development

This article aims to be simple distillation of my reading,
experimentation and learning about how best to develop Chef cookbooks
with more than a nod to the principles of Test Driven Development.

Prerequisites

Software tools

First, some software installation so that the example activity/code can
be reproducible. Feel free to use your own processes and substitute
alternative OS/distro, virtualization software choices. The process of
cookbook development is more or less the same across all
Linux/Windows/OSX platforms.

Chef Development Kit.
You'll probably want to ensure that
'/opt/chefdk/bin:/opt/chefdk/embedded/bin' are at the beginning of
your $PATH. Rubocop and Foodcritic are both included.

Tool configuration

Chef

Vagrant

Get a/some boxes, if you like to save time once you are ready to run Test Kitchen (although TK will get them for you, you might not want to wait at that point). Our Test Kitchen setup will by default use an Ubuntu box although Berkshelf's default scaffolding specifies a CentOS box as well.

Getting started

Goal

The intention here is to demonstrate a process for creating a new cookbook with example code/commands. For my example, I'm going to start with a small, simple to configure, network service: mini-httpd, a lesser known http server.

Edit/Verify .kitchen.yml

Change the provisioner to
chef_zero because in constrast to Chef-Solo, Zero means that you don't have to special-case recipes/tests that use search. You'll want to verify that the list of platforms is correct. This is also the file where we specify the chef 'run list' and later might override the cookbook's default attributes for the purposes of testing, but more on that later. For now, we'll just use:

If you want to be able to customize the VM that kitchen/vagrant makes for testing, you can make modifications to the 'driver' section, for example, specifying the hostname and forwarding ports from host to guest looks like:

The cookbook_file resource declared by cookbook_file 'mini-httpd_conf' takes its contents from the COOKBOOK_NAME/files/default sub-directory, the filename is specified by the source attribute. Not really: it searches a series of paths for a filename that matches the source attribute: files/<platform>-<version>/, files/<platform>/, files/default/, for example: files/ubuntu-12.04/, files/ubuntu, files/default. The analogous equivalent is also true for the sources of template resources and integration tests.

In our case, the contents of mini-httpd/files/default/mini-httpd.conf should be:

As we get more sophisticated, using attributes to allow variable configuration, we will remove this resource in favour of a template resource that, by default, will create the same configuration file with the same contents, but allow for alternative values for the mini-httpd directives.

Test Kitchen, first run

kitchen test will result in vagrant instantiating a single ubuntu-12.04 VM, installing chef using the omnibus plugin, provisioning with chef-solo, configuring the VM according to our newly created default recipe from the mini-httpd cookbook and running any test suites we have specified in .kitchen.yml.

But at this stage, our process requires a manual verification step to satisfy ourselves that the package, file and service we attempted to manage are actually managed correctly.

$ kitchen create
$ kitchen converge
$ kitchen login

Now, logged in to your test VM, you can poke around manually to determine whether everything has been configured according to your will.

Later on, when we are running automated tests, we will just use:

$ kitchen test

If you are making lots of experimental changes to your cookbook, you can probably accelerate your test-kitchen workflow by separating the steps:

kitchen create

kitchen converge

kitchen setup

kitchen verify

kitchen login

If you do this, you can edit your cookbook and just run kitchen converge, avoiding the delay of the previous steps, in
particular, the creation/provisioning of a new VM every time.

Extras

Most kitchen subcommands can be run with concurrency greater than zero, specify with -c N

Most kitchen subcommands can be given parameters to limit which instances will be affected, either by a list of instances or a regex: e.g. kitchen converge ubuntu

Static Analysis

Rubocop

rubocop will syntax-check and style-check your code based on the Ruby Style Guide. If you use -a, it will auto-correct where it can.

rubocop --show-cops

will show to which rules our check is subjected.

Foodcritic

Foodcritic is a style-checking tool for Chef cookbooks, hand it one or more paths to cookbook(s). We're hoping for a single blank line of output.

knife

If, from within the cookbook directory, you run:

mini-httpd $ knife cookbook test mini-httpd -o ..

knife will run a ruby syntax check against all files that end in .rb and .erb (respecting .chefignore).

Chefspec

This is (mostly?) for unit testing, so I'll leave this as a stub for later. Initially, we're going to concentrate our effort on functional testing with Serverspec.

Functional Testing

Serverspec

Create a directory for the Serverspec tests, then we can start to make tests in test/integration/default/serverspec/localhost/mini-httpd_spec.rb:

require 'serverspec'
set :backend, :exec
describe 'mini-httpd service' do
it 'should be running and enabled' do
expect(service 'mini-httpd').to be_running
expect(service 'mini-httpd').to be_enabled
end
it 'should be listening on port 8080' do
expect(port(8080)).to be_listening
end
end

For details about what types are available see the serverspec documentation about its resource types.

NOTE Files that contain serverspec tests must be named to match /_spec.rb$/ or test kitchen will not use them.

If you are just modifying your tests, you can accelerate your test-kitchen workflow by separating the steps:

kitchen create

kitchen converge

kitchen setup

kitchen verify

If you do this, you can edit your tests and just run kitchen verify, avoiding the delay of the previous steps.

BATS

Bats, the Bourne-Again shell Test System.

Create a directory for the bats tests and add a file <cookbook_name>/test/integration/default/bats/<cookbook_name>.bats:

This brings up another requirement to ensure that certain packages are available during testing, whether or not they're installed for real. Create a file
<cookbook_name>/test/integration/default/bats/prepare_recipe.rb

%w{ nc nmap }.each { |pkg| package pkg }

Improvement: templating the config files

Instead of using a static file to provide the content for our configuration files, we can use erb templates. In order to do so, we modify recipes/default.rb using the template resource syntax:

Overriding default attributes

One of the major reasons for using wrapper cookbooks is so that we can
easily override attributes set by the cookbook maintainer as defaults to
make them more useful for our purposes.

In this example, I set the array of upstream ntp servers to be
MI[1-3].node[:domain], or otherwise, allow the cookbook default
behaviour, setting default['ntp']['servers'] to various pool.ntp.org
addresses.

Overriding default attributes for testing only

In .kitchen.yml, we can set specific attributes that are
appropriate/required for the test run. For example, preventing the
default behaviour of using a apt cacher/proxy requires us to set
['apt']['cacher_ipaddress']: