Introduction

In this tutorial, we will walk through the process of setting up a local
environment for Clojure development using the Ansible
configuration management tool. Ansible and Clojure are a perfect fit, as
both place an emphasis on simplicity and building with small, focused
components. While we will not cover the Clojure development process
specifically, we will learn how to set up an environment that can be used across
a whole host of Clojure projects.

Additionally, we will see how using a tool like Ansible will help us implement
the Twelve-Factor App methodology. The Twelve-Factor App
is a collection of best practices collected by the designers of the Heroku
platform.

By the end of this tutorial, you will have a virtualized environment for running
a Clojure web application, as well as all supporting software, including MongoDB
and RabbitMQ. You will be able to quickly get up and running on multiple
development machines. You can even use the same Ansible configuration to
provision a production server.

Prerequisites

For the purposes of this tutorial, we will be using Vagrant to spin up several
VMs for our development environment. If you have not used Vagrant before, please
make sure to install the following packages for your OS before continuing:

Finally, this tutorial uses an existing chat application as an example, so
go ahead and clone the repo with this command:

git clone https://github.com/kendru/clojure-chat.git

Unless otherwise specified, the shell commands in the rest of this tutorial
should be run from the directory into which you have cloned the repository.

What Makes Ansible Different

With mature configuration management products such as Puppet and Chef available,
why would you choose Ansible? It is something of a newcomer to the field, but
it has gained a lot of traction since its initial release in 2012. The three
things that set Ansible apart are that it is agentless, it uses data-driven
configuration, and it is also good for task automation.

Agentless

Unlike Puppet and Chef, Ansible does not require any client software to be
installed on the machines that it manages. It only requires Python and SSH,
which are included out of the box on every Linux server.

The agentless model has a couple of advantages. First, it is dead simple. The
only machine that needs anything installed on is is the one that you will run
Ansible from. Additionally, not having any client software installed on the
machines you manage means that there are fewer components in your infrastructure
that you need to worry about failing.

The simplicity of Ansible's agentless model is a good fit in the Clojure
community.

Data-Driven Configuration

Unlike Puppet and Chef, which specify configuration using a programming
language, Ansible keeps all configuration in YAML files. At first, this may
sound like a drawback, but as we will see later, keeping configuration as data
makes for much cleaner, less error-prone codebase.

Once again, Clojure programmers are likely to see the value of using data as
the interface to an application. Data is easy to understand, can be manipulated
programmatically, and working with it does not require learning a new
programming language.

When you adopt Chef, you need to know Ruby. With Puppet, you need to learn the
Puppet configuration language. With Ansible, you just need to know how YAML
works (if you don't already, you can learn the syntax in about 5 minutes).

Task-Based Automation

In addition to system configuration, Ansible excels at automating repetitive
tasks that may need to be run across a number of machines, such as deployments.
An Ansible configuration file (called a playbook) is read and executed from
top to bottom, as a shell script would be. This allows us to describe a
procedure, and then run it on any number of host machines.

For example, you may use something like the following playbook to deploy a
standalone Java application that relies on the OS's process manager, such as
Upstart or systemd.

---# deploy.yml# Deploy a new version of "myapp"## Usage: ansible-playbook deploy.yml --extra-vars "version=1.2.345"-hosts:appserverssudo:yestasks:-name:Download new version from S3s3:bucket=acme-releases object=/myapp/{{ version }}.jar dest=/opt/bin/myapp/{{ version }}.jar mode=get-name:Move symlink to point to new versionfile:src=/opt/bin/myapp/{{ version }}.jar dest=/opt/bin/myapp/deployed.jar state=link force=yesnotify:Restart myapphandlers:-name:Restart myappaction:service name=myapp state=restarted

This example playbook downloads a package from Amazon S3, creates a symlink, and
restarts the system service that runs your application. From this simple
playbook, you could deploy your application to dozens — or even hundreds
— of machines.

Installing Ansible

If you have not already installed Vagrant and Leiningen, please do so now. The
following steps require that both are present on your local machine. We also
assume that you already have Python installed. If you are running any flavor
of Linux or OSX, you should have Python.

Installing Ansible is a straightforward process. Check out the Ansible docs
to see if there is a pre-packaged download available for your OS. Otherwise,
you can install with Python's package manager, pip.

sudo easy_install pip
sudo pip install ansible

Now, let's verify that the install was successful:

$ ansible --version
ansible 1.9.1
configured module search path= None

Provisioning Vagrant with Ansible

Now that all dependencies are installed, it's time to get our Clojure
environment set up.

The master branch for this tutorial's git repository contains a completed
version of all configuration. If you would like to follow along and build
out the playbooks yourself, you can check out the not-provisioned tag:

git checkout -b follow-along tags/not-provisioned

At this point, we want to instruct Vagrant to provision our virtual environment
with Ansible. One of the key concepts in Ansible is that of an inventory,
which contains named groups of host names or IP addresses so that Ansible can
configure these hosts by group name. Thankfully, Vagrant will automatically
generate an inventory for us. We just need to specify how to group the VMs by
adding the following to our Vagrantfile:

This creates 4 groups of servers, each with a single virtual machine. Notice
that both database and broker groups have the same server (infr). This
will cause all configuration for both groups to be applied the the same VM.

While we could start up our Vagrant environment now, Ansible would have nothing
to do. Let's fix that by writing some plays to provision our environment.

Writing Ansible Plays

Before we dig into the plays that we need for our application dependencies,
let's write a simple task to place a message of the day (motd) on each of the
servers that will be displayed when the user logs in. We will be using a
role-based layout for our Ansible configuration, so let's create a common role
and add our config. Your directory structure should look something like the
following:

Vagrantfile
...
config/
└── roles
└── common
├── tasks
└── templates

Next, we'll add a main.yml file to the tasks directory that will define the
motd task.

Briefly, this file defines a single task that uses the template module
built into Ansible to take a file from this role's templates directory and
copy it onto some remote machine, replacing the template variables with data
from Ansible.

Along with this task, we'll create the motd.j2 template.

# config/roles/common/templates/motd.j2
Welcome to {{ ansible_hostname }}
This message was installed with Ansible

When Ansible copies this file to each host, it will replace
{{ ansible_hostname }} with the DNS host name of the machine that it is
installed on. There are quite a few variables that are available to all
templates, and you can additionally define your own on a global, per-host, or
per-group basis. The official documentation
has very complete coverage of the use of variables.

Finally, we need to create the playbook that will apply the task that we just
wrote to each of our servers.

In order for Vagrant to use this playbook, we need to add the following line
to our Vagrant file in the same block as the Ansible group configuration that
we created earlier:

ansible.playbook="config/provision.yml"

We can now provision our machines. If you have not yet run vagrant up,
running that command will download and initialize VirtualBox VMs and provision
them with Ansible (this will take a while on the first run). After we run
vagrant up initially, we can re-provision the machines with:

If all was successful, you should see output similar to the above for each of
the VMs in our environment.

Ansible Play for Application Server

In our infrastructure, the application server will be dedicated to running only
the Clojure application itself. The only dependencies for this server are Java
and Leiningen, the Clojure build tool. On a production machine, we would
probably not install Leiningen, but it will be helpful for us to build and test
our application on the VM.

Let's go ahead and create two separate roles called "java" and "lein".

For the purpose of our application, we would like to install the Oracle Java 8
JDK, which is not available from the standard Ubuntu repositories, so we will
add a repository from WebUpd8 and use debconf to automatically accept the Oracle
Java license, which is normally an interactive process. Thankfully, there are
already Ansible modules for adding apt repositories as well as changing debconf
settings. See why they say that Ansible has "batteries included"?

Next, we'll add the task to install Leiningen. Instead of having Ansible
download Leiningen directly from the internet, we will download a copy and make
it part of our configuration so that we can easily version it:

Ansible Plays for Infrastructure Server

Next up, we'll add the play that will set up our infr server with MongoDB
and RabbitMQ. We'll create roles for each of these applications, and we'll
create plays to apply the mongodb role to servers in the database group, and
the rabbitmq role to the servers in the broker group. If you recall, we only
have the infr VM in each of those groups, so both roles will be applied to
that same server.

We'll set up the role skeletons similar to the way we did with the java and
lein roles.

This role is a little more complicated than what we have seen so far, but it's
still not too bad. We again use built-in Ansible modules to fetch 10Gen's
signing key and add their MongoDB repository. We use the service module to
start the mongod system service, then we use the lineinfile module to edit
a single line in the default MongoDB config file.

You may have noticed that we used several variables in the arguments to a couple
of tasks. There are a couple of places that variables can live, but for this
tutorial, we will declare these variables globally in
config/group_vars/all.yml. The variables in this file will be applied to every
group of servers.

Since there are no surprises in the configuration for RabbitMQ, we will not go
into it here. However, the full config is available in the repo
for reference.

Ansible Plays for the Application

In order to follow the 12-factor app pattern, we would like to have all of our
configuration stored in environment variables. Our app is already set up to
read from HTTP_PORT, MONGODB_URI and RABBITMQ_URI. We'll just write a
simple task to add those variables to the vagrant user's login shell.