Continuous deployment with Mesos, Marathon and Docker

Share

Datacenter automation is one of the strongest features offered by the Mesosphere technology. This post is a follow on from our recent presentation at the Bay Area Infracoders meetup where we demonstrated how an organization can use Mesosphere to easily construct a simple continuous deployment pipeline from source code repository to your datacenter.

Goal

Continuous deployment is the holy grail of datacenter automation. In this post, we demonstrate a simplified example of how we use Mesos, Marathon, TeamCity, and Docker internally to deploy applications automatically to our internal staging environment. The team is automatically notified of new deployments by using Slack.

The following flow chart shows the high level sequence of actions. A check-in to GitHub triggers a build of a Docker image in TeamCity. On a successful build, a special TeamCity Deploy build is triggered, which uses Marathon’s REST API to trigger a new deployment.

Implementation

Our example project is a reveal.js presentation, a developer friendly HTML + JavaScript presentation format that Mesosphere engineers use for talks. We check these into a GitHub repository at github.com/mesosphere/presentations.

The simple Marathon app definition in marathon.json specifies our mesosphere/cd-demo-app DockerHub repository as the source of our Docker image. Note the $tag placeholder, which is required by TeamCity to deploy the latest built tag of our presentation.

We also specify health checks in this definition, which allows Marathon to gauge when the newly deployed container is up and running correctly. The health checks use HTTP requests to a defined endpoint. You can read more about how Marathon health checks work here.

TeamCity Setup

In TeamCity, we have a build that builds and pushes the Docker image for our presentation and another build that deploys the most recently built image to Marathon.

Tag Scheme

Before setting up the build you should develop a scheme to use for generating the Docker image tags.

Here’s the scheme we’re going to use for our demo: %env.BUILD_NUMBER%-%teamcity.build.branch%.%env.BUILD_VCS_NUMBER%.

The components of the tag are as follows:

%env.BUILD_NUMBER%: The build number so that we can always track down the TeamCity build that created the Docker Image

%teamcity.build.branch%: The branch of the Git repo that the Docker Image was built from

%env.BUILD_VCS_NUMBER%: The Git SHA of the code that was added to the Docker Image

This scheme ensures that we have a new tag for every build, meaning that we are creating immutable containers. Also, the scheme ensures that we have visibility into the branch that the code came from and the Git SHA of the commit.

Docker Image Build

The first build in TeamCity has these steps:

docker login: log in to our DockerHub account

docker build: build the specified Dockerfile with a new tag

docker push: push the built image to DockerHub

Report tag into new marathon.json: use jq to write the tag into the Marathon app definition

The first three steps are fairly standard, but the fourth step is the magic that makes this work. Our script looks like this:

After the build completes you can view the generated artifacts on the Artifacts tab for the specific build:

Deploy

After the first build completes successfully and there are new artifacts, TeamCity automatically triggers a second build where we do two things:

PUT to Marathon’s REST API

Send a message to the team on Slack

Marathon’s REST API makes it easy to kick off new deployments of an application. Using HTTP PUT with the newly generated marathon.json artifact creates a new deployment. You can read more about Marathon deployments here. The deployment starts a new instance of the application and stops the old instance only when the new application is deemed “healthy”.

Sending a message on Slack is easy because of their accessible API. The process involves pushing a JSON encoded message to a pre-configured Slack webhook URL (see Slack’s documentation for more details).

Results

Each time a successful Docker image build completes, a deploy build is triggered, the marathon.json is PUT to Marathon, and the app begins rolling out across your cluster. A notification that an application rollout is happening is sent to the team.

Artifact marathon.json

Here you can see the generated marathon.json that is sent to Marathon.

Summary

This post shows how to easily set up automated builds that deploy Dockerized applications automatically to a Mesosphere cluster. We use this same procedure in many of our projects to make life easier for our engineers. Give continuous deployment on Mesosphere a try today!