Navigation

Yesterday
we covered the basics to getting started with Chef. You should have
a remote server configured with chef, and have curl installed! Now
lets go ahead and get some useful bits for your Django
application.

So this is going to be based around the way that I set up my
servers, so if this is different than you, I’m sorry. However, I
think it is a pretty solid way of managing them. A lot of the ideas
here are stolen from Travis when he
set up the server for Pypants.

So lets assemble a list of things we’re going to want in order to
get a super basic Django configuration running:

A user to run our code as and who’s home directory we’ll store
the data.

A basic global python ecosystem, including setuptools and pip

A virtualenv to store all the project-specific packages and code
in

A copy of the project that we’ll be running

Let’s get started.

The finished code for today is located on github, with the
tag blog-post-2.
It is a copy of the completed steps, so feel free to peek through
that and come back here for clarification (or to ask questions).

For RTD, I run everything under the
user docs. So we’ll go ahead and set up that user so that we can
get our site set up. We’re going to go ahead and replace our
“default” recipe, because right now it isn’t doing anything much
useful. The relevant part is below:

cookbooks/main/recipes/default.rb

node[:base_packages].each do |pkg|
package pkg do
:upgrade
end
end
node[:users].each_pair do |username, info|
group username do
gid info[:id]
end
user username do
comment info[:full_name]
uid info[:id]
gid info[:id]
shell info[:disabled] ? "/sbin/nologin" : "/bin/bash"
supports :manage_home => true
home "/home/#{username}"
end
directory "/home/#{username}/.ssh" do
owner username
group username
mode 0700
end
file "/home/#{username}/.ssh/authorized_keys" do
owner username
group username
mode 0600
content info[:key]
end
end
node[:groups].each_pair do |name, info|
group name do
gid info[:gid]
members info[:members]
end
end

There’s a lot of stuff going on here, so lets go over it. First you
might notice that there’s this node variable, the node data
structure is the JSON that you have in your node.json file. It is
looping over the keys and values with ruby’s each_pair and pair
functions.

The base_packages bit is a cool example of the power of the chef
configuration. We have a list of packages that we want to install
in our Attributes, and we’re looping over them and setting using
the package Resource.

I realize I skipped over the run_list part yesterday, but it
basically is just a list of recipes to run. Each of the resources
in the default.rb file should be pretty self explanatory. The
Chef Resource Documentation
is really comprehensive, and will probably be the most referenced
document that you use. The main resource’s that we used were
group, user, file, directory, let’s take a look at the
User
declaration in particular.

Everything there should be pretty obvious, as it’s the information
that goes into /etc/passwd for the user. However, the supports
keyword isn’t obvious at first. This is part of the
Common Attributes
that can be set on all Resources. It’s a way of passing along
configuration options to the Resource. manage_home actually just
makes it so that the users home directory is created when the user
is created.

So we’re going to have to go ahead and put some data in there for
it to work with. Our node.json will now look like this:

Here we’re adding some global packages that we need. We’re going to
install setuptools and pip so that we can install further python
packages. python-dev and libpq-dev are so that we have the headers
for libraries that need to compile against postgres and python.
We’ll also be installing virtualenv and mercurial globally so that
we can create our virtualenv and install packages from mercurial.

As you can see, it takes a bunch of arguments, then just wraps up a
bunch of Resource definitions in a nice little package. There is a
little bit of magic with the pip freezing things, but it’s
basically just how we’re checking to make sure that a package isn’t
instead before we install it. We are using only using the
directory and execute Resources here.

Now we’re going to use this virtualenv Definition, and create the
home virtualenv for our site. I like to keep my virtualenv’s in
~/sites/<domain>, so this will go into
/home/docs/sites/readthedocs.org/. Since this is becoming
specific to the site we’re building, it’s going to go into a
readthedocs recipe:

cookbooks/main/recipes/readthedocs.rb

directory "/home/docs/sites/" do
owner "docs"
group "docs"
mode 0775
end
virtualenv "/home/docs/sites/readthedocs.org" do
owner "docs"
group "docs"
mode 0775
end

To get our site set up, we need to pull in the source code, and
make sure our virtualenv has all the requirements. This code is a
little bit hacky, and could probably be abstracted out a bit, but
it will work for now. We’re going to go ahead and add some things
to our readthedocs Recipe.

I like to have my runtime files in the venv/run directory, so
we’ll go ahead and create that directory. Then comes the fun part.

We are checking the Readthedocs source out of github with the
git
Resource. Chef only supports git and svn as far as I can tell, so
luckily I’m using git.

Then we’re going to install from the pip requirements file. This is
using the
script Resource,
which allows you to inline a bash, ruby, python, or more script
inside your Recipe. This is using a hard coded bash script to
install the requirements, which sucks, but will work for now.

Note: Chef appears to buffer output and not show itself as
doing anything when running the script Resource here, so it will
look like your build will hang while it installs your pip
requirements file for the first time.

Alright, this post has gotten long enough, so we’re done for today.
But we’re in a pretty awesome spot, I think. We now have our app
server set up with a runnable version of our code. You can go ssh
in and play around, you should be able to run simple manage.py
commands inside the virtualenv and whatnot (after a syncdb).

Tomorrow we’ll talk about deploying our code with Nginx and
Gunicorn. I’ve been having trouble with Upstart, so we might switch
our deployment to Supervisord, but we’ll see how it goes.

Don’t forget to check out the finished code
on Github
to see the actual running examples.