Search

Vagrant

How many times you have been hit by unit tests failing because of environment
differences between you and other team members? How easy is it to build
your project and have it ready for development? Vagrant provides a
method for creating repeatable development environments across a range of
operating systems for solving these problems. It is a thin layer that sits
on top of existing technologies that allows people working on a project
to reproduce development environments with a single command: vagrant
up.

Vagrant is open-source software, and it sees lots of contributions
from its community in every release. It was first released in 2010. Its
founder Mitchell Hashimoto founded the company HashiCorp to work on
Vagrant and a number of other DevOps tools in November 2012. HashiCorp
has since released a number of other tools that are worth checking
out, such as Packer, Consul and Serf. These are for different areas of the
development lifecycle.

Vagrant uses virtual machines from various providers. It allows for easy
configuration using a Vagrantfile. This allows development teams to
set up the virtual machine the same way every time in a programmable
fashion. This repeatability encourages developers to tear down and
bring up environments more often. The development environment that all
team members are developing against now can be "stored" in the version
control system in such a way that anyone (new or otherwise) can have an
environment set up after issuing a single command.

Continuous integration
systems can use the same method. Building an environment and setting up
a project in this way allows the developer to be sure that once other
environments are provisioned in the same way, the project will run
as it did in development. Environment-specific unit test failures is a
class of problems completely removed by Vagrant.

Vagrant takes the building and provisioning of your software from tribal
knowledge within your team to a reproducible provisioning script that
is stored alongside your code. Others can read your Vagrantfile and
provisioning scripts and have your project working on their system in
minutes. This helps integrate new developers into your team quickly. It
is easier to enable cross-team development within a larger company as
it's easier to get up and running with a project. It also allows designers
or technical writers to get quick access to your project without having
to know any technical details about how to set up your project.

Vagrant Technologies

Originally, Vagrant supported only VirtualBox as a provider. A provider
is the underlying virtualization technology that Vagrant will interact
with. At the time of this writing, Vagrant supports a wide variety of providers,
VMware, hyper-v and, most recently, Docker. Vagrant only ships with
VirtualBox support by default. Other providers can be added as plugins
using Vagrant's plugin system. For the purpose of this article, I
use VirtualBox as the provider, as it is the default and is likely the
most accessible to everyone. It is not the most performant provider;
VMware in most cases out-performs it, and Docker (although it is not a
drop-in replacement) is a very exciting technology in terms of speed and
definitely is worth checking out.

The next configurable part of Vagrant is provisioner support. After
a virtual machine has been started, Vagrant uses "provisioners" to
provision the box in a repeatable way. This is done as part of bringing
the virtual machine up in most cases. The simplest type of provisioner
is "shell", which runs a defined shell script at boot of the VM as the
root user. More advanced provisioners, such as Salt, Chef and Puppet,
also are available. In my workplace, we use Puppet to provision our virtual
machines so as to better match our development systems with machines
running in the production environments.

Depending on the provider, Vagrant also allows for "Synced
Folders";
this allows very easy sharing of folders between the host and guest
machine. This means every developer on your team can use a different
IDE and tooling that runs on the host, and yet develop on the same
guest machine. Vagrant makes these things incredibly easy to set up as
you will see later in the article.

How to Install Vagrant

Vagrant has installers for all of the main operating systems: Windows,
Linux (deb and rpm) and Mac OS X. This portability across host operating
systems allows developers in a team to get the same local development
experience on any environment.

For this article, I used Ubuntu 12.04 LTS, and I used the deb file from
the downloads section of the Vagrant home page. For Linux distributions
that don't accept deb or rpm package types, there is still the ability to
run older versions of Vagrant, provided that the distribution has support
for Ruby and Ruby gems, but this has been discontinued for newer releases
of Vagrant.

If you have not already installed VirtualBox, you will need to
install it. This can be done through the package manager, although I used
the packages provided on the Oracle Web site.

This article uses Vagrant 1.6 and VirtualBox 4.3.

Using Vagrant

In this article, I run through setting up a project using Vagrant,
including
how to write the Vagrantfile, how to boot the virtual machine using
Vagrant and interacting with that VM. The first thing you need is
the base box on which you are going to build your project. A base box is
a provider-specific bare-bones operating system disk image with some
basic things installed. Vagrant uses SSH to connect to all its boxes,
so an SSH server must be enabled, and a specific user must have SSH
enabled. There are a number of other requirements to making a
Vagrant-compatible box. The process is too involved for this article, but more
specific instructions can be found on the Vagrant or specific providers' Web
sites. Vagrant, by default, expects a vagrant user on the box and the
password (by convention) should be "vagrant".

For the purpose of this article, you will use a publicly available base
box. HashiCorp provide a base box for Ubuntu 12.04 LTS. To add a box
to Vagrant:

$ vagrant box add "hashicorp/precise32"

This command downloads the box from Vagrant cloud. Vagrant cloud
is a service HashiCorp provides, which among other things offers box
discovery. Community base boxes are uploaded and can be used. Vagrant
knows that you are accessing a base box from the Vagrant cloud and will
download it directly from there. Once downloaded, you will not need to
download the base box again to use in other Vagrant projects on the same
host. You can see available boxes by using the list command:

$ vagrant box list
hashicorp/precise32 (virtualbox, 1.0.0)

Now that the box has been added, you will craft a basic
Vagrantfile. A Vagrantfile is a Ruby file that sets up certain variables
for your box. Here's a basic Vagrantfile:

This shows the power of Vagrant—with this simple file, an Ubuntu 12.04
LTS virtual guest machine can be created and started on any system
Vagrant supports. To start the Vagrant VM, simply do the following:

$ vagrant up

It may take some time for the virtual machine to boot, but when
the command returns to the prompt, you successfully have started your
virtual machine. At this time, it might be worth opening the VirtualBox
application so you can see the new virtual machine that has been
created and ensure that it is marked as running.

Figure 1. VirtualBox Screenshot

As I mentioned previously, base boxes for use with Vagrant must be accessible via
SSH. This allows you to attach to your new virtual machine simply by doing
the following in the same directory as the Vagrantfile:

$ vagrant ssh

You will be left in a bash prompt in the virtual machine. In two commands,
you have booted the VM and ssh'd in to the VM. One more thing to note
before continuing on to using Vagrant more powerfully is that Vagrant, by
default, shares the directory that the Vagrantfile is located in under
the /vagrant folder. To demo this capability, while still in the SSH
session, do the following:

Guest machine:

$ cd /vagrant
$ echo "test" > test

Host machine:

$ ls
test Vagrantfile

As should be clear, you created a new file "test" on the guest, and the
file appears in the same directory on the host. Vagrant uses VirtualBox
shared folders to allow you to keep files in sync between the
host and the guest very easily. This is especially useful when developing software
projects using Vagrant when you want to be able to edit files locally
and have those changes automatically appear in your development VM for
testing purposes. It allows developers to run their choice of editor
while developing on the same platform as anybody else in the team.

Provisioning

You've seen how to boot a simple virtual machine using Vagrant, so next,
let's
look at how to provision that VM in a repeatable way. As already
mentioned, Vagrant supports a wide range of provisioners, but for the sake
of this article, bash scripts will be used due to their familiarity. Adding
a provisioning step is a matter of adding a single line inside the
configure block in the Vagrantfile. The bootstrap also can be an inline
script, but for most cases, it makes sense to supply a script file that
contains the provisioning steps.

Add the following bootstrap.sh script to the same directory as the
Vagrantfile:

This bootstrap script makes sure the system sources are up to date
first. It then installs the nginx Web server. The provisioning is added
to the Vagrantfile by adding the following line to the configure block
so that your Vagrantfile now looks like this:

To use the new provisioning, destroy the running VM. This will stop and
remove the virtual machine. To destroy the VM, simply use the command
vagrant destroy. The base box still will be available for use after
running the destroy command. Now to test the new configuration, simply run
vagrant up. When a provisioning step is present in the Vagrantfile,
Vagrant automatically will provision the box once it has booted. You
can turn off the provisioning by using the
--no-provision flag.

When vagrant up is complete, you will be able to
ssh in to the
machine as before. As shown below, let's make sure that the provisioning has been
successful. Note that nginx is now installed on the system and running
on the system:

The virtual machine has been provisioned, and nginx is set up and
started. The Web server can be tested to be working, but it's more
difficult if you have to ssh in to the box to test
this. Next, let's
look at how to configure networking.

The first option to access ports on your virtual machine is port
forwarding. For this example, nginx serves http traffic on port 80. To
forward the guest machine port 80 to port 8080 on the host, add the
following line to the Vagrantfile:

config.vm.network "forwarded_port", guest: 80, host: 8080

As before, destroy the VM and do another vagrant up. Once the
virtual machine has been brought up and provisioned, the Web server
can be accessed on the host port 8080. On your favorite browser, visit
http://localhost:8080/ to be greeted with the nginx home page. This
is one of the easier ways to be able to connect to the virtual machine.

For example, let's modify the bootstrap.sh to link the "html" directory
in the Vagrant directory to the nginx root directory. This would be an
example of doing simple static HTML Web site development. This could be
distributed to anybody to modify the HTML on the host and immediately
see the results on the guest.

You can access the html directory through the nginx Web server using the
forwarded port once the VM has been booted and provisioned by Vagrant. For
example, using curl:

$ curl "http://localhost:8080/html/tester.html"

This example assumes an html directory is in your Vagrant directory that
has a tester.html file in it.

What if you want to provide a way to access the virtual machine as if it
were another machine on the network? Vagrant provides a very easy way to
use VirtualBox networking to set up the virtual machine with a static
IP on the same LAN. Remove the port forwarding line from the Vagrantfile
and replace it with the following:

config.vm.network "private_network", ip: "192.168.56.20"

Once you have re-ran vagrant up, your virtual machine will be accessible
from your host using the IP address specified above. In my own projects,
I use this private networking to assign my static IP a domain by modifying
the /etc/hosts file. This allows testing of complicated http redirect
flows that rely on certain domains.

It also allows multiple Vagrant virtual machines to contact one another,
as it is a shared network between the host and all other virtual machines
that are on the network. For distributed projects, being able to control
the networking access of multiple standalone components all of which
have their own Vagrantfile (or shared) is very powerful.

Again, in my work environment, it's not uncommon to spin up multiple
virtual machines, one for a database, one for an app server and one
for a squid proxy. Running functional tests against these tests a
production-like setup. Vagrant allows you to do this in a simple, easy-to-repeat way, and the private network means there is no hassle in setting
up intercommunication between the various components.

The last part of Vagrant that I want to cover is having
multiple machines in the one Vagrantfile. I already have given
an example of why you might want to have multiple Vagrantfiles for
different components, so that they can be tested/developed on their own
or as a collective. Having multiple machines in the one Vagrantfile
is powerful when a single project can have multiple parts that don't
belong on the one machine. Often in a Web development project, you will
run the app server and database on the same machine for testing but will
have multiple app servers and databases when that system gets into the
production environment. This is the cause of a lot of production bugs
due to untested component configuration.

Vagrant allows you to manage this all through a single Vagrantfile. If
the idea of multiple machines is one that interests you, I also
recommend looking closely at the new Docker container option for Vagrant.
Docker is built on top of Linux containers and allows you to spin up
machines much faster than VirtualBox. Multiple machine support in Vagrant
is all the more powerful if the provider level is much faster to spin
up machines.

For multiple machine support, the Vagrantfile allows you to configure
multiple machines, each having their own configuration. The Vagrantfile
below shows an example of what a MySQL master and slave Vagrantfile
might look like:

Each machine is defined with the first config block as separate blocks.
They support all of the same options as the normal one-machine Vagrant
config blocks. For example, for the master here, you define the shell
provisioning script to be run as well as the static IP address to give
this virtual machine. This allows you to test distributed systems in the
same easy repeatable way as you could with a single machine.

When using a multiple machine configuration like the above, there are a few changes
to the way you have to run Vagrant commands. Running vagrant
up will
bring up both boxes automatically. The master will be provisioned first
and then the slave. You can choose to bring up only one of the machines
by supplying the machine name:

$ vagrant up master

When the boxes have booted and been provisioned, you also have to specify
the machine name when using vagrant ssh.

Conclusion

Let's reflect on
what you have
achieved here: multiple virtual machines booting, being provisioned
automatically and consistent IP addresses using one command—vagrant
up. This is repeatable across the three major operating systems and a
number of different providers including the most popular virtualization
technologies and cloud offerings.

This removes so many of the pain points from working on systems in
local development. There are a lot of advantages to making the switch to
using Vagrant for both local development and your continuous integration
systems. All developers' test machines are running the same code and are
provisioned in the same fashion. Test failures are not tied to a specific
developer's setup and should be reproducable on anyone's machine by running
vagrant up. This article covered only some of the amazing capabilities
of Vagrant, so I encourage you also to read over the brilliant
documentation at http://docs.vagrantup.com.