Infrastructure Testing With Ansible and Serverspec: Part 1

Ansible allows you to automate the configuration of your IT infrastructure. The administrator enters configuration commands in a set of script files called a “playbook” and then runs them. If the administrator has entered all the commands correctly and nothing unusual happened then it’s likely that the servers were configured successfully. But we are dealing with computers and with computers something always goes wrong. So we always check and validate that our scripts produced the configuration we wanted. Software developers use a variety of automated testing frameworks to do this. We can do the same with infrastructure testing. I will described my method in this and the next few posts.

serverspec

serverspec extends the rspec framework to cover infrastructure testing. In technical terms rspec tests run on the local workstation while serverspec tests run remotely on the server. serverspec runs Unix commands on the remote server over an SSH connection. serverspec and rspec together cover the basic client and server architecture. The nice thing is that you can freely mix the two in your tests.

While these two testing frameworks cover the basic cases, you will need to use some creativity to get them to cover the real world variety of test situations. Often, you need to test a server from a specific perspective. That means you need to run a test from a certain machine on a certain network or subnet. When you test your IT infrastructure always remember that the network is factor too. Just because a machine on subnet A can pass the test doesn’t meant that a machine on subnet B can. serverspec doesn’t support perspective testing natively but there are work-arounds that I will explain in these posts.

Virtualization also adds a new element. With Xen server, for example, the only way that I can validate and test attributes of my VMs is through xe commands sent to the physical host. In other words to test server A (a VM) I have to run the test on server B (the host). Ansible has a cool command to do this called delegate_to. I hope serverspec will add something similar in the near future.

Prerequisites

In order to use serverspec (and rspec) you will need to know basic Ruby programming. Sorry. In the brave new world of automated systems administration we are all programmers now. You can run a few serverspec tests without learning Ruby so you can try it out and see if it is worth the investment in time. To write all the tests you will need, however, you will have to learn Ruby.

On your Linux workstation (I use CentOS for both my servers and workstations), you will need at least the following Ruby components:

Ruby itself (yum install ruby)

The Ruby package manager gem (yum install rubygems)

Make for Ruby: the rake gem (gem install rake)

The rspec gem (gem install rspec)

The serverspec gem (gem install serverspec)

The Bundler utility (gem install bundler)

You should also have git installed. Once you create your tests you should commit them to version control. I will describe my work-flow using a continuous integration (CI) server and git in a later post.

File Layout

In order make my serverspec test mirror my Ansible playbook I use the file layout described here. First create a top level directory to hold your tests. Then create the following layout under it:

The lib/ folder holds custom Ruby code. If you look at my post on Cucumber vs rspec you will see an example of helper code in the dns.rb file. Under spec/ you create a directory named after each of your Ansible roles. In those directories you will create tests unique to that role, for example, tests for the BIND application under the dns_server role.

Also under lib/, my spec_helper.rb file uses the format from the severspec site with only a couple of changes. First I changed user = 'root' because that is how my environment is configured. If you use sudo to manage your servers (as you should unlike me!) then enter the user name you use to login. Other changes for sudo may be required but since I don’t use that I don’t know what they are. Maybe the default spec_helper.rb will work fine for you.

The other change to spec_helper.rb is the added require lines. I have added the standard

require'rubygems'require'bundler/setup'

which you can too. This is also where you put helper code require statement for the files in your lib/ directory. If, for example, you have a file called lib/dns.rb then add require dns to your spec_helper.rb file.

The Rakefile is used to run your tests. You can run all them using the command rake spec or you can run a test on a specific host. You can see all options by entering rake -T. One warning here is that if you run all tests and serverspec finds an error it will not run any further tests.

The Gemfile is a simple list of package dependencies used by Bundler. Mine contains only a few lines:

source 'https://rubygems.org'
gem 'serverspec'

Once you create this file then you can use Bundler to automatically download the packages you need. After creating this file then enter the command bundle install. Bundler will download the packages you need.

The last file I will describe in this post is the .rspec file. It tells rspec how you want to run your tests and how to format the results. I like the verbose, colored text format. I do not like the way rspec randomizes the tests.