Saturday, July 30, 2011

Why machine images don’t work

After investigating the way that AMI's are built and seeing how utterly difficult it was to build one, I've been trying to put to words my feelings about it. This article does a great job of describing it with four simple statements:

Images are too monolithic.

Images are opaque.

Images are too big.

Images are too static.

I have one additional thing that I would like to add:

Images are difficult to upgrade.

I quickly came to the conclusion that building Debian packages that can be quickly installed on any machine is a far better way to go. Not only are they easily to create, but you can integrate them into your continuous integration system so that every time someone commits code, a new package is built and added to a central repository. Updating the code on your machines in all of your environments is as simple as 'aptitude update; aptitude safe-upgrade'.

I also think that using tools like Fabric, Puppet and Chef (FPC) just add another layer of unnecessary complexity which completely fails the KISS principle. You can do everything that FPC can do in a single Debian or break it out into multiple ones depending on how you want to setup the deployment hierarchy. Why install some other complicated piece of software (and all of its dependencies) with its own domain specific language when you can just write relatively simple bash shell scripts?

With my deployments, I like to set things up so that there is a 'base' Debian package which gets installed called project-init. It is responsible for creating users, accepting the JDK license agreement, setting the timezone of the machine and any other low level OS settings that can be used on all machines.

From there, everything gets layered on top of that base package. Some packages will be optional depending on which environment they get put into. For example, you probably don't want or need to install Clarity on your production servers. If you need packages for specific environments (dev, staging, prod), you can use Debian Virtual packages to create 'aliases' for which the system you want installed.

In the end, I know this system works really well. I've done it for one of the most complicated systems one can imagine with 25+ different packages for all of the components that needed to be installed.