If you’re interested in seeing all the recipes available (and there are quite a few), just browse the recipes in the pivotal_workstation repo.

Why Did You Choose That Set of Software?

Early in June, several pivots (Sean Beckett, Matthew Kocher, and David Goudreau, and I) met to decide on the bare minimum set of software and features that our developers would need to function on a new Lion Machine.

This set of chef recipes is the result of that meeting. There have been some changes (we have had great difficulty writing recipes to install firefox addons, so we iceboxed the story; some of our developers contributed recipes for things they wanted, so we added those).

Why Did You Choose Chef?

We chose chef/soloist partly because felt that our previous process had reached the end of its usefulness and were familiar with chef from our work automating server configuration.

Here’s how our previous process worked:

For minor releases (e.g. 10.6.7 → 10.6.8), we would take the previous golden image (a golden image is a snapshot of the disk drive of a machine with the applications, preferences, and settings that we wanted), install it on a workstation, upgrade the OS and possibly upgrade some of the applications. We would then use DeployStudio to take an image of the workstation, and that image would become the new golden image.

For major releases (e.g. 10.5 → 10.6), we would re-create the golden image by hand, manually installing & configuring the individual software. For the Leopard/Snow Leopard transition, my co-worker Kevin Fitzpatrick spent a week painstakingly configuring the new machine. We then took an image using DeployStudio, and that image became the golden image.

This approach had several shortcomings:

It was monolithic: if you were a developer, there was no choice: there was only one image. This wasn’t so bad when we were strictly a ruby shop, but when we expanded into android and iphone development, the monolithic approach began to show some shortcomings.

There was cruft in the image: the golden image had been built up over years.

It wasn’t clear that the Golden Image would make the jump to Lion: Lion introduced some big changes (e.g. no PPC executables).

We were hesitant to approach the developers to ask them what they would like to see on the new image; we worried of re-igniting a Holy War.

We looked for alternatives. We wanted the following features:

We wanted to be able to install all (or almost all) of the features automatically (with minimal user intervention)

We wanted to be able to pick-and-choose which features were installed; the needs of an Android developer were different than those of an iPhone developer.

We want our developers to be happy; sure, they could install the features that they wanted or fix a problem with their workstation, but we want to go a step further: we want to provide the resources they need to write a recipe to minimize the effort the next developer has to go through. [This has been a success: several of our recipes were written in conjunction with developers.]

Testing

Integration tests for the cookbook took several days to set up. We use Faronic’s Deep Freeze on a fairly pristine mac mini to ensure that we have a clean machine when we run our chef scripts. Continuous integration has proven invaluable for collaboration, for we quickly learn if a commit has unintended consequences.

In the more complex chef recipes, we attempt to write tests to test that they [the recipes] have succeeded; sshd_on.rb is a good example of testing that a service (sshd) was correctly started.

Expectations

The chef runs, especially the initial one, are flaky. Our current chef run must download software from over 40 different servers, any one of which being down or having changed the download location can cause a failure. For example, Little CMS, a dependency of ImageMagick, resided on littlecms.com, which was down for a few days. Our integration tests failed during that period.

If you encounter a server being down or a file that has moved, please send us a pull request with an updated download location, or just comment-out the broken recipe.

Target Audience

Our target audience is developers, which is great: they understand errors, and often contribute code fixes. Our goal is for Pivotal Ops to provide a framework for Pivotal Engineers to write the recipes that build the workstations they want.

Acknowledgements

I am grateful to Matthew Kocher, who more than anyone helped me write the bulk of the ruby scripts. Also to Sean Beckett, without whose support this would never have happened. And to the many pivots who offered suggestions & help.

We followed the instructions [firefox addon automation] in the link you mentioned, but we found there were “gotchas” with respect to automated (i.e. command-line driven) installation. For example, sometimes Firefox would rename extensions on the download (e.g. validator@totalvalidator.com.xpi became total_validator-6.13.0-sm+fx.xpi). Also, from my [sketchy notes from two years ago];

“it appears that firefox peeks into the xpi, unwraps the xpi if required, creates the .json patterned after the install.rdf, renames the xpi according to the install rdf”

It became clear that automating this process [the installation of Firefox addons] was requiring more effort than we were willing to put into it. That, coupled with the rising ascendance of the use of the Google Chrome browser among our developers, made the requirement moot.

I'm a systems administrator at Pivotal Labs. I've worked at a slew of startups and with a slew of UNIXes (OS X, Linux, FreeBSD, OpenBSD, HP-UX, AIX, Solaris/SunOS UTS, Xenix, Ultrix, and even the original UNIX). In my spare time I play rugby.