Infrastructure Testing With Ansible and Serverspec: Part 3 Jenkins

This is the third and final part of my guide to testing Ansible with serverspec. This part shows how to use the continuous integration server Jenkins to run Ansible and serverspec. I will explain how to install and configure Jenkins on CentOS 6.5 using Ansible (oddly circular isn’t it?). I will describe how to configure a basic Jenkins job to run both Ansible and serverspec from a git repository.

If you read my, um, commentary you may guess that I have a slightly antagonistic relationship with Jenkins. Nevertheless, I know many people want to use it so I will do my best to explain the basics and hopefully save you some headaches.

Prerequisites

I assume here that you already know how to configure and run both Ansible and serverspec. I also assume that you have a git server configured with SSH key or anonymous access. I recommend gitolite for your git server. It has worked well for me.

You will also need your Jenkins CentOS server configured to use the EPEL repository. Ansible is only available from EPEL and not from the default CentOS repository. There are many guides on the Internet explaining how to do this.

Installing Jenkins

My Jenkins playbook is very basic but easy to understand if you are familiar with Ansible. See this playbook when you are ready for a more advanced one. In this guide you will have to take some manual steps, something which I usually disapprove. My poor excuse is that I got too frustrated with Jenkins to spend the time learning the Jenkins CLI.

You will need to generate a key pair to authenticate the Linux jenkins user to the git server using SSH. git needs the public key. If you use gitolite then add jenkins as a read-only user to your Ansible and serverspec repositories along with its public key. Normally, you don’t want to include the private key id_rsa in your files, I do it here for training purposes only.

This will install Jenkins, Ansible, and the required Ruby packages. Note that the Linux jenkins user has /var/lib/jenkins/ as its home directory. All the Jenkins files will be stored there, including your Ansible and serverspec code once Jenkins checks it out from git.

The file jenkins.j2 in the templates directory is the basic jenkins configuration file. I haven’t changed it but keep it here in case I want to chnage it later. I also store files like this as templates so it is easy to add variables.

If the install worked then you should be able to open the Jenkins web interface at http://your.server.com:8080 using the host name of your Jenkins server. If it doesn’t respond then verify that it started correctly and that you have added a rule in iptables to allow incoming connection on TCP port 8080.

Jenkins manual configuration

Once you have confirmed that Jenkins is running you need only make a few configuration changes to run the tests. I will explain only the minimum needed to get Ansible and serverspec to run. If you find Jenkins useful you will want to invest the time to properly configure it; that is, add user accounts, etc.

Plugins

In order to run these tests you will need to install 2 plugins:

Rake

Multiple SCMs

You do this by selecting Manage Jenkins from the home page, then Manage Plugins. From here click the Available tab and enter the names of the two plugins in the Filter field in the upper right corner. Find and install each. Once both are installed restart Jenkins.

Verify tests

Running these tests from Jenkins involves lots of moving parts. Before you create the job described next, I recommend you first do a short test to validate your configuration. You get extra credit if you run these tests from serverspec rather than manually.

First let me warn you that there are some tricky security and permissions issues here. Ansible needs the passwords or keys to log into your servers. serverspec needs the same. In production you will have to incorporate these tools into your overall security framework. In my workflow, Jenkins makes the changes and execute the tests only in the integration testing environment and not to production servers. The Ansible plays need to be thoroughly tested before going into production and this is just a step in that process.

Checkout the source code for both your Ansible playbook and serverspec tests into it (assuming you have gitolite installed): sudo -u jenkins git clone gitolite@[FQDN of jenkins host]:[repo name].git

If this works then you have git configured correctly.

Test Ansible

Now pick a simple play from your playbook and run it as jenkins. For example, sudo -u jenkins ansible -i testing.hosts [myserver] -m setup.

It should run correctly.

Test serverspec

Now do the same for serverspec.

cd into the serverspec directory and type sudo -u jenkins rake spec

Now delete the temp directory you created.

Once you can run these tests manually you are ready to let Jenkins do it for you.

Add a job

In this part you will add a job to automatically run Ansible and serverspec. The trigger and frequency is up to you. You can have git trigger a Jenkins build after a push, you can have Jenkins poll git for changes, or you can simply run it on a schedule.

Create a job:

From the Jenkins home screen click New Item.

Give your job a name, then click the Build a free-style software project radio button, and finally click the OK button.

On the job configuration screen skip down to the section titled Source Code Management. You should see a radio button labeled Multiple SCMs. This is the plugin I asked you to install earlier. Select it.

Add your Ansible playbook repository

Click the Add SCM button and select git. A set of form fields will appear.

Click the Add button labeled Additional Behaviours. Then click Check out to a sub-directory. Enter a name for your playbook code directory.

Click the Add button labeled Additional Behaviours. Then click Custom SCM Name. Enter the same name as in step c .

You will need these sub-directory names for the path statement in steps 7 and 8.

Add the serverspec repository

Follow the same steps as in (4) but use the name of your serverspec repository instead.

Skip down to the section titled Build.

Add Ansible build

Click the button labeled Add Build Step, then Execute shell.

In the Command field enter the Ansible playbook command using full paths. Its a bit tricky but follow this example: ansible-playbook -i /var/lib/jenkins/jobs/[job name]/workspace/[ansible repo name]/[inventory file] /var/lib/jenkins/jobs/[job name]/workspace/[ansible sub-dir name]/site.yml

When you have this working you might add a build step for syntax checking. Use the same command but with --syntax-check at the end. Run this before the actual play.