Get Salted

read

After starting at Rackspace I picked up Chef to work with the automated deployment folks. Chef is Ruby at its core, which makes it very expressive and easy to work with. As a Pythonista though, something kept pulling at me.

Salt Stack is a collection of free and open source infrastructure management software, designed to make deploying services a breeze. It’s also a distributed remote execution system so you can execute arbitrary commands across all your nodes (or an arbitrary selection of them).

Tom Hatch, the original author of Salt, came and gave us a talk a couple months ago but I didn’t have a chance to try it until this weekend. So far, I’m impressed.

Getting Started

The walkthrough for Salt Stack has you run through a bit of manual setup for installation. I imagine later I’ll be able to create infrastructure on demand, provision it with the right resources and get services running all in one command, regardless of the provider. For now, I had to bootstrap the servers.

First I set up the master, which more or less is the commander across all your servers. The servers, in Salt Stack parlance, are called minions. With full on parallel remote execution and terminology like minions and targets, it sounds like you’re building Enterprise Botnet 3.0.

Create at least two boxes (virtualbox, vagrant, Rackspace, AWS, physical boxes, whatever). Create more and you can broadcast your commands across minions. In fact, don’t even bother with the tutorial if you’re not going to launch more than one server. You won’t grasp the raw power of salt without at least two minions.

Bootstrap the Master

The quickest way to get the master started is to use the Salt Stack bootstrapscript. SSH in and run this one liner:

Bootstrap the Minions

The minions use the same script, but without the options above

curl -L http://bootstrap.saltstack.org | sudo sh -s -- git develop

For each of the minions, edit /etc/salt/minion.

Turn this

##### Primary configuration settings ################################################ Per default the minion will automatically include all config files# from minion.d/*.conf (minion.d is a directory in the same directory# as the main minion config file).#default_include: minion.d/*.conf# Set the location of the salt master server, if the master server cannot be# resolved, then the minion will fail to start.#master: salt

Into this

##### Primary configuration settings ################################################ Per default the minion will automatically include all config files# from minion.d/*.conf (minion.d is a directory in the same directory# as the main minion config file).#default_include: minion.d/*.conf# Set the location of the salt master server, if the master server cannot be# resolved, then the minion will fail to start.master:67.207.156.173# Put the IP for your salt master here

Clearly you really only need one line (master: 67.207.156.173). The text above is what gets installed by default using the bootstrap script, you can clean it up like any other Linux config file.

Start the Minions!

On each of the minions, run salt-minion as a daemon.

salt-minion -d

They should each make a callback to the master. Log back on to master and run salt-key -L. Now you’ll see some unaccepted keys.

You can verify that these match the public keys stored at /etc/salt/pki/minion/minion.pub on each of the minions. Then you can simply run salt-key -A to accept all the keys (with confirmation). It worries me a little that I can’t check the public key while going through the confirmation.

Alright Minions, Let’s Play

Now that we’ve got everything configured, let’s do some remote commands on our minions.

If you want to perform traceroutes across lots of servers/minions no matter their location, it’s easy:

Sweet. Of course, no one wants to configure these systems and deploy code by hand. The next stop is salt states. By creating some simple yaml files (that use Jinja templates), we can configure quite a bit about a machine.

The walkthrough has you install nginx but has you do something a bit strange in that it makes you state that the nginx package is installed to run the service. It’s stated twice so it seems redundant. Here’s /srv/salt/nginx/init.sls:

nginx:pkg:-installedservice:-running-require:-pkg:nginx# I feel like I am repeating myself

Looking Forward

The features I’m most interested in are automated deployments with git. In particular, I want to be able to deploy based off of git hooks to production, staging, and development for the IPython Notebook Viewer.