Sharing a common development environment with everyone on your team is important. It is really hard though to keep the same dependencies, database versions and other systems in sync between different machines.

Vagrant is a great tool that helps with this and manage the lifecycle of a virtual machine. As nice as Vagrant is, provisioning machines with it has always been a pain. A couple of months ago Mitchell Hashimoto, the creator of Vagrant, launched Packer.

Packer lets you build Virtual Machine Images for different providers from one json file. You can use the same file and commands to build an image on AWS, Digital Ocean or for virtualbox and vagrant. This makes it possible to use exactly the same system for development which you then create in production.

In this blog post we will show you how you can use Packer to build your vagrant machines. In a follow up post we will focus on how we use Packer for building all of our Continuous Deployment Infrastructure.

Prerequisites for building Vagrant Machines

You need Virtualbox and Packer installed. Virtualbox provides packages for different Operating systems. Packer is even easier, just download the right zip for your system and unzip it into your PATH

Building your Virtual Machine with Packer

We’ve collected all the files necessary to build a Vagrant Machine with Packer in our Packer Example repository.

Packer uses builders, provisioners and post-processors as the main configuraition attributes. A builder can for example be virtualbox or AWS. A provisioner can be used to run different scripts. Post-processors can be run after the machine image is done. For example converting a Virtualbox image into a suitable image for vagrant is done in a post-processor.

Here is the main packer.json file. You can see the builder, provisioner and post-processor defined:

Run the script and it will create the packer machine and import it into vagrant. Then all you have to do is run

#!javascript
vagrant destroy
vagrant up

And you have your development environment set up.

You can now get into the machine with vagrant ssh and start coding.

Conclusions

Vagrant is an incredibly powerful tool and together with Packer it is easy to build development environments for your whole team.

But this is only the beginning. Packer can go much further than just providing your development environment. We are currently implementing Packer as the tool to build all of our test infrastructure servers. This new set of tools is great for Immutable Infrastructure and Continuous Deployment so you can build more stable, secure and easy to change infrastructure than ever before.

Let us know in the comments how you use Packer and Vagrant. We are excited to hear your thoughts!

Posts you may also find interesting

Subscribe via Email

Over 60,000 people from companies like Netflix, Apple, Spotify and O'Reilly are reading our articles. Subscribe to receive a weekly newsletter with articles around Continuous Integration, Docker, and software development best practices.

We promise that we won't spam you. You can unsubscribe any time.

Join the Discussion

Leave us some comments on what you think about this topic or if you like to add something.

One of the most common situations young developers face is the following:

— New boss on first morning: “Congratulations on your new job! Here’s some really expensive equipment and lots of coffee!”

— New boss on second afternoon: “Also, we need you to upgrade our heavily customized WordPress instance. This app is vital, so we can’t have any downtime. It’s only a dozen updates behind. That’s not a problem, right?”

— New boss on third day (when asked about a testing environment): “We just went with the default EC2 AMI, then tweaked some of the PHP settings — though we don’t remember exactly what we changed. That shouldn’t be difficult to clone, right?”

— New boss on fourth day: “Where’d my developer go?”

(Disclaimer: This is just a general picture of a classic corporate problem, not a dig at any specific employer.)

After more than a year of watching admins wax poetic about how Vagrant/Veewee/etc can help us in this fictional world where everyone sets up reproducible environments *before* deployment, I have a much more practical question for the maintenance programmers among us: Has anyone ever found a reliable script or compiled app that can convert the basics of my company’s *live* EC2 setup into a roughly equivalent local VM? That is to say that my phpInfo() output should look almost identical.

By the time a good answer pops up, I’ll have just taken my chances. But hopefully this question can help somebody else.

The problem you have with getting from a production system to a local vm is that as soon as you want to change the infrastructure you need to run through that process again which is totally not reproducible.

2) Automate the complete setup with a tool like packer and then flip the switch at some point after thoroughly testing it.

Every other step is simply prolonging the pain. Of course you need to do a calculation if it’s worth it, but if you regularly change your system in my opinion it’s definitely worth it.

Don’t think that there is any other way to do this. Not starting from a completely new and clean instance means it’s hard for you to really know your infrastructure well.

Chris O’Neil

How would it even be possible to write a tool that would automatically do that kind of reverse engineering for an arbitrary application? It’s that company’s fault for not automating the setup in the first place, and as a maintenance programmer, it’s your job to pick apart the setup and figure out how to automate it, so that you aren’t in that position again. To be honest, I would take the reverse engineering and automation as an interesting challenge. One of the most fun things about doing automation work is that it really is one of the best ways for you to learn every detail about how something works.

I agree and packer or some other automation tools are perfect for this, as you can slowly build exactly what you really need and everything else you might have needed in the past will just be dropped.

Although if I could choose I’d prefer to just have that automation in the first place :)

dragon788

Easiest answer is to make a full backup of the deployed system and run your tests against that first. If you are already using some kind of virtual infrastructure you simply clone the existing VM(s) and relabel one as cert, and then make another clone and label that as dev. If you are using something deployed internally on bare metal or VMware, there are many tools they have for P2V, V2V, and other types of migration.

pekhee

Thank you Florian. I looked into Packer before and couldn’t see its aim. Your article just helped me understand where can I use it.

So you did remove all Virtualbox machines first? This is not an ssh problem, but Virtualbox can’t start correctly

Mihai Rusan

all virtualbox machines removed.

dragon788

Make sure that Intel VT-x or AMD virtualization option is turned on in your BIOS. Not having those available tends to cause issues with Virtualbox and VMware. You may also need VT-d if its trying to talk directly to a storage controller. Another thing I’ve seen recommended for virtualbox is making the HDD controller types to IDE as it has more compatibility, but was also one of the reasons that we chose to go with VMware to get the speed advantages of SCSI and SATA without compatibility headaches.

jhvaras

I suffer the same issue, launching packer without any VM does not seem to solve, neither increasing ssh_wait_timeout.

How can we debug the boot_option step? I think there is an error at that point but I don’t know what it is. The result is that instead of running all the boot_option script and then runing the other script (root_setup.sh and setup.sh), it boot directly in the live CD givent the choice to install from there. Also, how does the boot_option param change from Ubuntu? 13.10 will not be the same then 12.04.4? I try adding the -debug flag to packer build but the only thing I see is all the opcode sent at the boot_option step. It’s very hard to debug!

the boot_option step with preseed is very hard to debug indeed. I’ve used an existing preseed file there.

You can also use the relatively new virtualbox ovf builder which takes an existing Virtualbox image and starts from there. You can create that base image by hand and take it from there. Not ideal, especially if you want to start from scratch but it is a pretty good option to get started quickly:

Too bad testing the boot_option is soo hard. It’s weird, just playing around with this example here: https://github.com/rokhmanov/packer-teiid and I am able to make a 12.04.4 base box. But I need a 12.04.3 and just changing the ISO and the md5 doesn’t work. My first guest a a boost_option problem but I’m not sure. There is not a lot of information to debug…

Thanks anyway,

Jonathan

dragon788

Just turn off the typically default headless option by adding a provider.gui = True line and you can watch what happens as it boots. Typically the issue is your boot_wait is too long, and the CD has skipped on without you, I’ve found 4-5 seconds to be the best wait time.

Seph

So if you want to develop on a particular os, you’ll have cook up some way of installing/partitioning/configuring that os via a command line option on bootup and write that into boot_command. That is such a hassle. Any idea on how to do that with Centos6.5?

Seph

So you have to use the kickstart configurator to make the kickstart file and then host that file somewhere. Which adds another point of failure for your build process. Minimal Centos has to be installed first and then configured with dhcp or static addressing for networking to work, defeating the whole purpose. Just not seeing the advantage of using this as a workflow.

dragon788

Guest, because you can automate packer with a continuous integration build system and verify that your infrastructure code changes don’t break your tests/applications, you DO write tests don’t you?

You can also use packer + ansible + vagrant. Build machines with packer, provision them with Ansibe, export them to vagrant format and use them to run your development environment with Vagrant.

ULHPC

Is there a way, upon provisioner error, to authorize the connection within the vagrant box being built ? I have to debug provisioner scripts that fail and having to wait 5 minutes for every test attempts is quite annoying… Accessing the logs is useful but it would be more efficient to be able to freeze the packer process (before the cleaning) and somehow connect to the vm before its deletion. Any hint on that?

Thanks for this. Ideally however I want to use packer to build both a vagrant box and a bunch of VMs suitable for things like EC2, vmware etc all from the same source (that is its purpose after all). However I cant work out a way of only installing vagrant stuff for the vagrant box, it doesnt belong anywhere else. I dont want a vagrant user and vagrant packages existing on my production boxes. Do you know if thats possible, without tying the vagrant-only stuff to “virtualbox-iso” or similar, which technically it doesnt belong in (what if I want to build a virtualbox ovf without vagrant as well as a .box file ?)

don’t think that’s possible as vagrant is only a postprocessor for Virtualbox. You might be able Best thing is probably to ask the packer guys directly at either their github or Google Group.

best, Flo

dragon788

You can actually specify that certain provisioners only apply to certain builders, so you could make a vagrant only provisioner to install the stuff you need there, and leave the rest out of everything else.

Hello; I would like to create a virtualbox vm (ubuntu) with zabbix. I think i can use packer + chef solo, right? Any idea how to make it? I cannot find any tutorial about that. Anyone could help me, please?

Demetrius Pampouktsis

Hi,

I am using a modified version of your build. Modified for Mac, and I am not using ubuntu I have a .ova file I am using. I am getting the error below… Even when I run a simple .json build template I am getting this error.. Any help would be awesome.

sad that the content of the article is buried behind some HUGE BLUE AD for a e-book that i dont want. does anyone there actually scroll down on the page and notice that your column formatting is wrong??

5TerisaHaman

Great Article. ideas . I am thankful for the insight ! Does anyone know if I could grab a sample KY 2306 – Boone County version to work with ?

Dorotha Roux

Greetings! my assistant filled out a template a form document with this link http://goo.gl/kxqIJa