CEO of Stark & Wayne, a consultancy for people who love Cloud Foundry, BOSH and DevOps in general.

One of the greatest devops tools in 2017 will be BOSH - the outer shell for cloud software. I recently summarized why BOSH is still unique in our profession in an article BOSH turns five! If you're intrigued, you might want to know how to get started? Or how to do something useful?

In this article, we will bootstrap BOSH 2.0 on Amazon AWS, and deploy a 5-node Zookeeper cluster. 95% of the instructions are for BOSH (it needs all the AWS credentials/networking info), and 5% of the effort is the Zookeeper cluster. Exciting stuff!

Alternately, you might want to start with BOSH 2.0 on your laptop using Virtualbox. See our previous article Bootstrap BOSH 2.0 with local VirtualBox. The instructions at the end for deploying zookeeper are 100% the same regardless of deploying to AWS or Virtualbox.

The instructions and colourful commentary in this article are based on the BOSH Core team's wonderful new repository bosh-deployment. BOSH is so lovely to use these days.

Terminology

"Bootstrap BOSH" or "Deploy BOSH" refers to bringing up a single VM that runs the BOSH Director and its runtime dependencies (like PostgreSQL). This is the bosh create-env command featured in this article.

"Use BOSH" normally refers to using the bosh in the terminal, which in turn talks to the BOSH Director via its HTTPS API. "Deploy with BOSH" refers to the bosh deploy command which is used at the end of the article to deploy Zookeeper.

Getting started

The bosh CLI can be installed on OS X via homebrew. At the time of writing, you may want to alias it from bosh2 to bosh:

brew install cloudfoundry/tap/bosh-cli
alias bosh=bosh2

The command we will use to bootstrap our BOSH Director VM is bosh create-env. It needs configuration - which infrastructure will it deploy the VM to? Which infrastructure will the resulting BOSH VM then deploy subsequent systems to (probably the same one, but there is a two-step distinction)? what IP/subnet/server size etc will be used?

At the time of writing https://bosh.io/docs/init-aws.html contains lots of screenshots of how to create/find this information with the AWS Console. This documentation page is for the precursor bosh-init command; we're using bosh create-env now. But the AWS information looks good.

If any of the fields above are confusing, let me know in the comments and I'll add extra explanation. Perhaps screencast would be helpful. Let me know.

Waiting

The first time you bosh create-env (this might be your second time if you followed the Bootstrap BOSH 2.0 with local VirtualBox article already) you may have some waiting to do. A few 100Mb of assets will be downloaded and cached to your machine. Avoid slow internet and/or Australia.

One day your output will look like mine - Skipped [Found in local cache]. You will feel the speeeed of not having to download large things to your machine.

Next, bosh create-env will build the "Cloud Provider Interface" (CPI) module for talking to AWS (see bosh_aws_cpi below). At the time of writing the AWS CPI was written in Ruby, so you'll also see it compiling its own copy of Ruby (see ruby_aws_cpi below). Relax, you have nothing to do.

The next stage is quick for AWS: downloading the "stemcell". For AWS this is merely a tiny file referencing all the pre-built AMIs on each AWS region. BOSH uses its own machine images for new VMs so that they have a BOSH agent and other software preinstalled and configured.

NOTE: you might notice external-ip-with-registry-not-recommended.yml has a big not-recommended in it. Let's discuss that another day. For now, we want to be able to access our BOSH VM from our laptop. We need that file included.

Action time! bosh create-env uses the AWS CPI to provision a new VM using the corresponding AMI for the target region.

Waiting for the agent on VM 'i-008b2c9af3c22e220' to be ready... Finished (00:03:17)

I am unsure why it took 3 minutes for the agent on the new VM to be "ready". I should ask about that. Moving on...

Rendering job templates... Finished (00:00:05)

In order to "fail fast", all the inputs you provided are converted into startup scripts, config files, etc. If you missed a property then it will fail here so as to allow you to fix it (and then run the same bosh create-env command again as above).

Target your BOSH

The primary user interaction with your new BOSH VM is with the bosh CLI used above. The bosh CLI can both deploy/update/delete a new BOSH VM; but you will use bosh CLI to deploy actual systems. That is, to do good in the world. bosh deploy is akin to World Peace in Dev Ops.