Continuous Deployment (CD) is an automated process by which code commits are turned into a trusted executable in the hands of users.

Massive, hardened Continuous Deployment systems have been quietly driving large software companies for years. Amazon's internal Brazil build system, for example, provides a full visualization of the delivery pipeline, and even includes the possibility of human testing as part of the process.

But CD isn't just for the big boys. If you're a software entrepreneur, continuous integration and deployments are more than a nice-to-haves. Of course there are the accepted benefits of reducing repetetive work, codifying the build/release/run stages, and early warning of breakages. However, if you buy into the idea that you should be rapidly learning from customers in order to define how your product evolves, CD is right up your alley; it's a major step toward the tightest possible feedback loop between you and your users.

Having recently come to this conclusion for myself, I spent some time setting up a private CD system using a new project called Drone. What follows is not a complete introduction to Drone, but the steps that I worked through to get my own instance up and running.

Introduction to Drone

Drone is an exciting new Continous Integration platform built on Docker. It comes in two flavors: a hosted version that includes a free tier for public projects, and an open-source release that requires a server and some setup time. This tutorial works with the open-source version.

Like many CI platforms, the idea is that your Drone instance listens to commits on your application's repository and builds your app according to some drone-specific files that you've created. What makes Drone unique is that the build environment is a Docker container; you can choose from one of the provided build images, or use your own. This is a nice little application of Docker. Drone is also helped by a straightforward web-based UI and a growing development community. Because it allows for post-build actions, Drone can work nicely as a full CD solution.

Here's what a build looks like from the Drone UI. All the output from your build process streams in nicely via websockets, as if you were watching it in a terminal:

For each repository Drone watches, there's also an overview screen where you can see result of past commits, and kick off a rebuild:

1. Server Set-up

This tutorial installs Drone on a 2G DigitalOcean droplet with Ubuntu 14.04. This is not a requirement, but is now recommended by Drone as the preferred setup.

I started with the 1G entry-level droplet, but found that I wasn't happy with the speed of my builds until I upped the droplet size to the 2G RAM / 40G SSD / 2 Core instance at $20/month. With a single core entry-level droplet, the Drone UI became unresponsive during a build, and because I had 3 apps being built with fairly large build environments, Docker was getting dangerously close to using all 20G of hard-drive space. $20/month may be steep for some, but since I resized the droplet, Drone has been humming along without a hitch.

At the time of writing, the Drone documentation states that it's tested only on Ubuntu 12.04 and 13.04, but I haven't come across any 14.04-specific issues as of yet. As Docker >= 0.8 is the only requirement, I started my droplet on an image with Docker 1.1.1 pre-installed:

The Drone docs will take you through the simple process of installing Docker if you're not using DigitalOcean.

Because I wanted my Drone server to be publicly accessible, I set up my droplet accordingly using a CNAME to get a subdomain like drone.myserver.com. Assuming you have a domain name assigned to your Droplet like myserver.com, in the DigitalOcean control panel click DNS -> and View your Domain, then add a CNAME record like:

Later we'll use nginx to wire a subdomain to our running Drone instance.

2. Install Drone

Next we'll install Drone:

$ wget http://downloads.drone.io/latest/drone.deb

This will download the Drone Debian package to the current directory. Then install the package:

$ sudo dpkg -i drone.deb

Drone is started automatically, but its installed as an Upstart script, so you can stop/start it like:

$ sudo stop drone
$ sudo start drone

Drone starts on port 80 by default. Verify by running:

$ curl localhost:80

I see:

<a href="/dashboard">See Other</a>.

That should do it!

3. Proxy Drone through nginx

Although Drone's command line tool lets you access a bunch of its functionality, its most handy when we can use the UI from a web browser. We'll use nginx to set up a reverse proxy from port 80 from the outside world to port 8080 within our server. Having a routing layer between the outside world and various apps running on a server is very useful, especially for assigning each app its own domain on port 80.

First we'll have to restart Drone on a different port (we'll use 8080):

$ sudo stop drone
$ sudo start drone -p :8080
$ curl localhost:8080

3.1 Install nginx

Then we'll install nginx. There's a nice DigitalOcean tutorial for this, but it boils down to the following:

$ sudo apt-get update
$ sudo apt-get install nginx

In Ubuntu 14.04, nginx is started automatically, but can be started manually with:

$ service nginx start

The default configuration will have an nginx page served at:

$ curl localhost:80

3.2 Configure nginx

Now let's inform nginx about the Drone instance we want exposed to the world on drone.myserver.com. Nginx has a configuration file structure like:

The /etc/nginx/sites-enabled directory has soft links to actual configuration files defined in /etc/nginx/sites-available. This allows for quick modification of which sites the server exposes. I like to remove default.conf from sites-enabled for hygienic purposes. For Drone, we'll create a file in sites-available called drone.myserver.com with the following contents:

Here we're telling nginx to listen for port 80 connections via drone.myserver.com, and forward them to the locally installed Drone instance on :8080 (also allowing websocket connections). Then we'll set up a soft link to the new configuration file and restart nginx:

If you're DNS changes from step 1 have already propagated, you should see your Drone instance from the outside! Try navigating to http://drone.myserver.com/install in a web browser, and step through the process of registering a user.

More info on configuring Drone—including how to enable SSL connections—is available here.

4. Configure a Repository

This is where we inform the running Drone instance about some code that should undergo an automatic build process. This tutorial uses a private Bitbucket repository, but Drone also supports GitHub and GitLab repos.

4.1 Set up OAuth

First create an OAuth consumer of your version control account. In Bitbucket this is done from the Manage Account -> OAuth screen, by clicking Add consumer:

This will generate a key and secret that act as credentials for the Drone server to access the repository.

4.2 Add the Repository

Now that the repository is accessible, navigate to New Repository in the Drone UI and enter the repository details, including the newly generated key and secret when prompted. If successful, the new repository should be visible in the right column of your Drone dashboard.

4.3 Add .drone Files

The final step is to specify a build and deploy process in a language that Drone understands. Drone only expects a single file called .drone.yml in the repository root. Drone-specific scripts are kept in /.drone by convention:

The .drone.yml file tells Drone how to build and deploy the repository. At the time of writing, there is no one place that describes all the yml options this file takes, but they can be collected from the Drone GitHub README and various pages in the docs. My typical .drone.yml looks something like:

When a commit is detected on the repository, Drone will (1) pull in the latest commit, and (2) run the .drone.yml steps in the correct order, failing the build if any commands along the way don't complete successfully. Simple! Here's an example build.sh script where I create a Docker container:

Note that Drone works on the repository in /var/cache/drone/src/bitbucket.org/team-name/repository-name of the build container. In the above example, my tests are run as part of the docker build process, but they could be run just as well where it says [pass tests here]. Look for another post on using Drone to build and deploy an app as a Docker container (not trivial).

I've been content so far with bash-driven deployments, but Drone supports a healthy number of deploy and publish triggers for e.g. Heroku, Amazon s3, and OpenStack.

5. Fine Tuning

Following are some notes that helped me get Drone humming along smoothly.

5.1 SMTP Setup

An SMTP server is required to send email programmatically for things like build success notifications. (BTW, Drone can do HipChat notifications as well!) Fortunately, one can use their GMail account's SMPT server for this purpose and avoid setting up an email server altogether. From the Drone Sysadmin (gears) page, configure the SMTP server settings:

Where my.gmail.username is your gmail username, and password is your gmail password. The +DRONE part of the From Address is optional, but can be useful to distinguish stuff in your inbox. I don't recommend using the + trick as a way to avoid spam.

Google complained and showed me an unlock captcha before any Drone emails would come through, but hasn't given me trouble since (several emails per week for over a month).

5.2 The vet tool

To avoid lengthy trial-and error cycles while setting up the YML file, its useful to validate the file using the provided command-line utility:

$ drone vet

This will look for and validate a .drone.yml file in the current directory.

5.3 Build Container Caching

Downloading third-party dependencies (like a bunch of JARs or Python packages) can take up significant time, making it desireable to cache such resources between builds. This is not so simple with Drone because a fresh Docker image is used for every new build. Drone solves this problem by allowing the .drone.yml file to specify container directories that should persist between builds, and treats these as mounted volumes:

cache:
- /root/.ivy2/cache

The above persists the ivy cache for a Java-based project, and in this case more than halved the build time.

5.4 Exiting Bash Scripts

When using bash for building and deploying, it's useful to construct them in such a way that any step that fails exits the script with a non-zero code. This is the only way of telling Drone to fail the build. Otherwise, a build that might have failed in actuality could still proceed to a deployment.

5.5 Branch Agnosticism

At the time of writing, Drone kicks off a build any time a new commit comes in. This isn't great if multiple coders are working together on a single feature, or if you work from different machines and like to stash incomplete changes in a remote branch. This is an open issue and I look forward to the merge.

5.6 Bitbucket Teams

To get multiple users administrative access to the same repositories under Drone, its necessary to move those repositories to a Bitbucket team.

Thanks! I'm interested in hearing about people's experiences with Drone, their build and deploy processes, and the issues they've worked through.