Multi-host Docker Deployment with Swarm and Compose Using Weave 0.11

In this post I’d like to show how easily one can get up-and-running using Weave with all the latest and greatest Docker tools – Machine, Swarm and Compose. This was made especially simple with two recent release of Weave (v0.10 & v0.11).

This guide builds on what was learned from two previous posts where I showed how one can use Machine with a single VM and Swarm with 3 VMs. In those two posts I used Weave CLI agains remote Docker host(s), leveraging features introduced in v0.10. With proxy being introduced in v0.11, one can use Docker CLI or API (via Compose) directly. Additionally, automatic IP allocation will be also used behind the scenes, lifting the burden of manual IP address assignment, which had been a long awaited feature.

What you will do?

This guide is design to get you started with Docker toolchain and Weave right out of the box, deploy and scale your application across multiple hosts in a very simple and absolutely transparent manner.

Setup a cluster of 3 VMs with Swarm and Weave configured by means of a shell script

Deploy a simple 2-tier web application using Docker Compose

Scale the application from 1 web servers to 3

I will post later on with details on how exactly this kind of setup works, for those who might like to reproduce it in a different environment, perhaps without using Docker Machine and VirtualBox, so you would see how it can work in an existing infrastructure.

If you are using OS X, then you can install these tools with Homebrew, via

brew install docker docker-machine docker-swarm docker-compose

You will need to download and install VirtualBox manually as well, if you haven’t done it yet. Please be sure to install latest version of Machine (v0.2.0), as there are some bugs in the previous release. You also want to use latest boot2docker VM image; you will get it if you haven’t used Docker Machine previously on your computer, otherwise you should delete the cached ISO image located in ~/.docker/machine/cache/boot2docker.iso before you proceed.

If you don’t find a docker-swarm binary for your OS, and have Docker daemon available locally, you can set DOCKER_SWARM_CREATE like shown below before you proceed.

Let’s go!

Now, we’ll provision a cluster of 3 VMs. The following script will make sure all 3 VMs join the Swarm, and sets up the Weave network and WeaveDNS.

If you’ve followed one of my previous guides, you’ll need to clear the VMs you’ve previously created. Check the output of docker-machine ls, and delete them with docker-machine rm -f <vm-name>.

./scripts/setup-cluster.sh

Once the cluster is up, you want to do the following

1. go into app’s directory

cd app/

2. build images on each host

./build.sh (This makes it quicker to scale)

3. setup the environment

eval $(docker-machine env --swarm dev-1)

4. deploy

docker-compose up -d

5. test, scale, test

We have just deployed a standard Compose demo, which consists of a Python Flask app that uses Redis as its database. Our docker-compose.yml file differs slightly from the original, it simply sets hostname: redis.weave.local and hostname: hello.weave.local instead of using Docker links (see diff). These hostnames are picked up by WeaveDNS and can be resolved from any container on the Weave network. WeaveDNS records also survive container restarts, unlike Docker’s built-in links.

You can deploy a different app, if you’d like. You don’t have to reuse all of my scripts for this purpose, although you certainly might like to take a look at how Weave proxy is launched and how Swarm agents and master are setup agains it.

Summary

We have just tested out a full setup with Weave integrated into Docker toolchain. We have first setup 3 VMs locally on VirtualBox, then deployed a very simple 2-tier web application. In the upcoming guide, we will take a look into more details on how exactly this works and how you can reproduce an effectively identical setup on a different infrastructure, using only Swarm and Compose agains Docker hosts you would setup yourself.

Follow @weaveworks, so you don’t miss any new posts. And you can always drop us a few lines at team@weave.works, to let us know what you think about Weave.

About the author

Ilya is a Developer Experience Engineer at Weaveworks, focused on making the adoption of microservices easier. Prior to Weaveworks, Ilya worked at Xively, where he personally experienced the shift to a true DevOps culture. He began to shift focus down the stack, becoming one of the early evangelists of and contributors to open source projects in the emerging Docker/container ecosystem.