Provision your laptop with Chef: Part 1

If you have ever tried to follow the Opscode Getting Started Guide for Chef, you'll quickly be overtaken by Chef jargon, confusing instructions, many assumptions, and no clear direction. Even the most experienced developers had a difficult time following the Opscode Wiki. While it serves as a great reference resource, you pretty much have to know Chef before it is of any use.

In Part 1 of this 2-part series, we will walk through setting up the Chef environment on your local machine. In Part 2, we will demonstrate how to provision your personal laptop using Chef.

What is Chef?

One of the best things about Chef is that it is idempotent. Unlike a simple bash script or complex startup script, Chef can be applied to the same machine hundreds of times. Only the "changes" will be executed on each of the clients. This makes deploying and updating thousands of machines as easy as baking a cake!

Understanding Chef & Jargon

One of the most interesting and confusing parts about Chef is its lingo. Everything relates to food. Be sure to grab a snack before diving in any further.

Architecture

One of the most dangerous practices when using Chef is applying prior knowledge. In Chef, everything is named relative to Chef. This means, even though your are provisioning a production server, it's still a Chef client. This can be very confusing at first, but once you start thinking in terms of Chef things will begin to make sense.

(Chef) Client [aka node] - refers to a machine that connects to and is managed by a Chef Server

(Chef) Workstation - refers to a machine where recipes are developed, altered, deleted, and more

This is the most simplistic picture. It's possible for a single machine to exist in all three of these roles simultaneously! There are subsets of Chef, such as Chef-Solo, but those will not be discussed here.

A typical scenario begins at the workstation. A developer creates a cookbook, role, or other artifact on the local machine. When finished, that artifact is uploaded to the Chef Server. The Chef Clients receive these (new) instructions and execute them locally.

Jargon

We've also been throwing around some other terms that we should take a second to define:

environment - provide a mechanism for managing different deployment locations such as production, staging, development, and testing

template - a file (like a config file) to be "rendered" on the server

There are also a few tools:

knife - a command line tool for managing chef and chef recipes

shef - chef console. this is the equivalent of rails console for chef

Before continuing, please make sure you understand that these are very over-simplified definitions and abstractions. Chef is much more powerful that these simplistic definitions allow.

Set up your Opscode Account

For the purposes of this tutorial, we will use Chef Hosted by Opscode (the creators of Chef). The service provides 5 free nodes for use. Go ahead and create your free account by heading over to the Hosted Chef Signup page. You should see something like this:

You'll need to confirm your email, but once that's done, head on over to the Opscode Management Console. It really doesn't matter what you call your organization - I used the same as my username. You should see a screen like this:

You probably only have one organization, but the image should give you a good idea. First thing you'll want to do is Regenerate validation key and Generate knife config. We will talk about this more in detail, but save these files in a handy place for later. These files should always be kept securely.

Also, if you didn't get your private key when registering, you should do that now (for some reason, Opscode does not always stream the key). Go to the Opscode Community Site and login (if you aren't already). Click on your profile and then choose get private key.

At this point you should have the following files:

[your_organization_name]-validator.pem
[your_username].pem
knife.rb

For example, mine would be:

ci-validator.pem
sethvargo.pem
knife.rb

Sometimes Opscode doesn't stream the correct files, so you may need to do some renaming:

This command will install the Chef gem as well as some other dependencies.

A this point, you should have a novice understand of Chef and Chef Jargon, have an account on Opscode Hosted Chef, and have a working version of Ruby, Git, and the Chef gem installed. The rest of this guide will assume you have completed all those steps correctly.

Setup your Workstation

Find a working directory on your local machine where you plan to store your Chef cookbooks. For this tutorial, we will use ~/Development. You'll need a skeleton chef repository. You could make your own or clone the Opscode one:

git clone git@github.com:opscode/chef-repo.git

Take a few minutes to poke around the repository. You definitely won't understand everything, but look at a few READMEs (in subdirectories).

Inside the chef-repo directory (~/Development/chef-repo), we need to create a hidden folder named .chef:

To confirm everything is working, try and list all the clients (it should be empty):

knife client list

If this command succeeds without error, everything is set up correctly!

This concludes setting up your workstation. Now we are ready to create our first cookbook.

Create your first cookbook

Wash your hands, put on your Chef's Hat, and get ready to bake! We are going to create your very first cookbook. The cookbook will be very simple and is more for demonstrating working with knife and chef-client.

Our cookbook will be named "hello world". Let's start by using knife to create our cookbook skeleton:

Open up the project in your favorite text editor and look inside the cookbooks directory. You should see the following:

This cookbook will create a file ~/hello_world.txt that says "Hello World!".

Let's first make the template. A template is like the "view" of MVC. It has access to instance variables and uses embedded ruby. Inside the templates/default directory, create a new file named hello-world.txt.erb and add some content:

The template keyword is a resource like we defined before. It tells the recipe to render the given template source to the given file. We also specify the file permissions using mode.

That's it! Our cookbook is done and ready to be uploaded to Hosted Chef. We will use the knife command to do this:

knife cookbook upload hello_world

You should see output like this:

Uploading hello_world [0.0.1]
Uploaded 1 cookbook.

If you get a message like:

WARNING: No knife configuration file found
ERROR: Your private key could not be loaded from /etc/chef/client.pem
Check your configuration file and ensure that your private key is readable

it means that your key is invalid. Regenerate your personal and/or organization keys and ensure everything is placed in the correct directories.

Our cookbook is now on Hosted Chef and ready to be distributed to our nodes. Chef doesn't automatically tell nodes to update. You can do this with a cron job, running chef-client as a service, or by running chef-client manually on any node.

For simplicity, let's just run this on our local workstation. Let's set up the local workstation as a client. Run the following command from the inside the repository:

Take a look inside your $HOME directory and see if the hello-word.txt file was created...

It doesn't appear the file was created. We must have done something wrong! Actually, we did everything correctly. We just forgot one step - we never told our node to execute the recipe we just wrote. By default, Chef does not execute any of your recipes. You must explicitly require them. There are a variety of ways to do this. We will use a run_list here.

First, we need to figure out what our node is named. Run knife node list and you should now see two nodes:

NODE
[your_organization_name]-validator

Mine looks like:

seth
sethvargo-validator

We obviously want NODE. In my case, it's seth. Now we can edit the run_list for that node:

knife node run_list add NODE hello_world

For me, that command would look like:

knife node run_list add seth hello_world

You should see the following output:

run_list: [recipe[hello_world]]

Let's inspect this node using the show command:

knife node show NODE

You should now see hello_world in the run_list. Run the chef-client command again (remember you might need to use sudo) and you should get output like the following: