Bullet proof Jenkins setup

In this
post, I will describe how a neat setup and some discipline will
ensure a Jenkins that can be rolled back and recreated very easily
- a bullet proof Jenkins setup.

I have been working on
configuring our Jenkins instance. This was the first time I had
played around with Jenkins. I am fairly comfortable
with Go from ThoughtWorks Studios. All of
my past teams used Go as their tool for continuous
delivery.

One of the things I found very different from
Go in Jenkins is the absence of the notion of a Pipeline as the
basic entity of build, as proposed
in Continuous Delivery. Although there are
plugins to make this available in Jenkins, we decided to go with
Jenkins' model of Jobs.

Another difference I spotted is
that when a custom task is defined as part of a Job, Jenkins
creates a shell script with all the steps while executing the
Job. In Go, each of the steps will have to be defined as a custom
command.

We wanted to ensure that our Jenkins
configuration is version controlled. While this is a huge win, one
of the ways this situation deteriorates is when a large number of
changes are made to the configuration over a period of time and
these is not checked in. So we decided to take this one step
further and ensure that these changes are automatically checked
in. There are instructions on how to do this, but we had to do
some tweaking to get this working for us.
These are the steps to setup a bullet proof Jenkins setup. This assumes that Jenkins is running on a Linux box.

1. Create a Git repository in Jenkins' base directory - This is generally /var/lib/jenkins2. Create a .gitignore file to exclude Jenkins workspaces. The Jenkins base directory is the home directory for the Jenkins user created to run Jenkins. This means that there will be a number of Linux user specific files like .ssh/ , .gem etc. These files need to be specified in the .gitignore file. A sample .gitignore file is listed below.

3. Setup a Jenkins job to check in the changed configuration files every day at midnight. (Or whatever time interval you choose). Add a custom task with the following steps:

While this ensures that the configuration is more or less tracked well, there are times when somebody makes a massive change in the configuration. This is where the most important piece of the bullet proof configuration comes in - team discipline. The team should ensure that big changes are checked in as soon add possible. This can be easily done by triggering the Jenkins job manually, without having to ssh in to the Jenkins box.

Credits:1. The Jenkins community documentation provided a nice starting point for this.2. The .gitignore file was forked from this gist by sit. I have added some project specific stuff to it.

If you have questions or comments about this blog post, you can get in touch with me on Twitter @sdqali.