As a newbie to the tooling, docker-compose it's great for getting started. To bring up all the service containers with a simple docker-compose up starts everything. However, what if you want to replace an existing container without tearing down the entire suite of containers?

For example: I have a docker-compose project that has the following containers.

Node JS App

CouchDB

Redis Cache

I had a small configuration change within the CouchDB container that I wanted to update and re-start to get going but wasn't sure how to do that.

Here's how I did it with little down time.

I'm hoping there are better ways to go about this (I'm still learning), but the following steps are what I used to replace a running docker container with the latest build.

Make the necessary change to the container (in my case update the couchdb config).

Run docker-compose build couchdb (docker-compose build <service_name> where service_name is the name of the docker container defined in your docker-compose.yml file.)

Once the change has been made and container re-built, we need to get that new container running (without affecting the other containers that were started by docker-compose).

docker-compose stop <service_name> <-- If you want to live on the edge and have the shut-down go faster, try docker-compose kill <service_name>

docker-compose up -d --no-deps <service_name> <-- this brings up the service using the newly built container.

The -d is Detached mode: Run containers in the background, print new container names.

The --no-deps will not start linked services.

That's it... at least for me, it's worked to update my running containers with the latest version without tearing down the entire docker-compose set of services.

Again, if you know of a faster/better way to go about this, I'd love to hear it. Or if you know of any down-sides to this approach, I'd love to hear about it before I have to learn the hard way on a production environment.

UPDATE:

Thanks to Vladimir in the comments below - you can skip several steps above and do it with a single command

docker-composeup-d--no-deps--build <service_name>

I tested this and was able to avoid the build, kill, and up commands with this one-liner.

So, I'm playing around with some personal projects and looking to deploy some simple things with Docker to DigitalOcean. This personal project is a small site, and I'd like to set myself up with a repeatable deployment solution (that may be automated as much as possible) so I don't trip over myself with server hosting as I build out the web application.

I'm not really strong with server infrastructure and some of this is "figure it out as I go", while more of it is asking for help from a good friend @icecreammatt who has been a HUGE help as I stumble through this.

But at the end of this tutorial our goal is to satisfy the following requirements.

High level requirements:

Below are some core requirements this walk-through should help address. There is likely room for improvement, and I'd love to hear any feedback you have along the way to simplify things or make them more secure. But hopefully you find this useful.

I want there to be some semblance of a release process with various deployment environments. Push changes to qa regularly, push semi-frequently to stage and when things are good, ship a version to production.

Have access to environments through various domain names. EX: if prod was my-docker-test-site.com then I would also have qa.my-docker-test-site.com and stage.my-docker-test-site.com

Ability to run multiple "environments": qa, stage, prod in the same system. (prob not that many environments - but you get the picture)

Deploy to various environments without affecting other environments. (Ship updates to qa)

It'd be great if I can figure out a mostly zero-downtime deployment. (Not looking for perfect, but the less downtime the better)

Keep costs low. For a small site - running all environments on say a small DigitalOcean droplet. (Is this even possible? We'll see...)

Build various environments, test them out locally and then deploy them to the cloud ()

While I'd like the ability to run as many environments as I listed above, I will likely use a qa and prod for my small site, but I think the pattern is such that we could easily setup whatever environments we need.

Structure of post:

What we want to do is essentially walk through how I'm thinking about accomplishing the above high level requirements using a simple node js hello world application. This app is a basic node app that renders some environment information just to prove that we can correctly configure and deploy various docker containers for environments such as qa, stage or prod.

In the end, we should end up with something that I like to imagine looks a bit like this diagram:

I'm going to use DigitalOcean as the cloud provider in this case, but I think this pattern could be to other docker hosting environments.

Example Web App Structure

Below is a basic view of the file structure of our site. If you're following along, go ahead and create this structure with empty files, we can fill them in as we go...

.
|____app
||____Dockerfile
||____server.js
|____docker-compose.yml

Let's start with the ./app/* files:

Simple NodeJS Hello World app

This is a simple nodejs server that we can use to show that deployment environment variables are passed through and we are running the correct environment. As well as showing a functioning web server.

The goal of this file is to run a nodejs web server that will return a text document with Hello World! along with the environment variables that the current container are running under such as qa.

This could easily be replaced with a python, php or ruby, or a whatever web server. Just keep in mind the rest of the article may assume it's a node environment (like the Dockerfile up next). So adjust accordingly.

The Dockerfile

Below is pretty basic and says, load up and run our nodejs server.js web app on port 80.

How to get the qa, prod domain mappings

So now that we have a basic application defined, we can't host multiple versions of the app all using port 80 without issue. One approach we can take would be to place an Nginx proxy in front of our containers to allow translation of incoming domain name requests to our various web app containers which we'll use docker to host on different ports.

The power here is we don't have to change the port within the docker container (as shown in the Dockerfile above) but we can use the port mapping feature when starting up the docker container to specify different ports for different environments.

For example I'd like my-docker-test-site.com to map to the production container, qa.my-docker-test-site.com the qa container of my site, etc... I'd rather not access my-docker-test-site.com:7893 or some port for qa, stage, etc...

This project is great, now, as we add new or remove containers they will automatically be added/removed to the proxy and we should be able to access their web servers through a VIRTUAL_HOST. (more on how specifically below)

Networking

Before we get too far into the container environment of our app, we need to consider how the containers will be talking to each other.

We can do this using the docker network commands. So we're going to create a new network and then allow the nginx-proxy to communicate via this network.

First we'll create a new network and give it a name of service-tier:

docker network create service-tier

Next we'll configure our nginx-proxy container to have access to this network:

docker network connect service-tier nginx-proxy

Now when we spin up new containers we need to be sure they are also connected to this network or the proxy will not be able to identify them as they come online. This is done in a docker-compose file as seen below.

Put the two together

Now that we've defined our application with the server.js and Dockerfile and we have a nginx-proxy ready to proxy to our environment-specific docker http servers, we're going to use docker-compose to help build our container and glue the parts together as well as pass environment variables through to create multiple deployment environments.

The list of environment variables are important. The VIRTUAL_HOST and VIRTUAL_PORT are used by the nginx-proxy to know what port to proxy requests for and at what host/domain name. (We'll show an example later) You can see an earlier exploratory post I wrote explaining more about environment vars.

The ports example is also important. We don't want to access the container by going my-docker-test-site.com:8001 or whatever port we're actually running the container on because we want to use the VIRTUAL_HOST feature of nginx-proxy to allow us to say qa.my-docker-test-site.com. This configuration sets it up to only listen on the loopback network so the nginx-proxy can proxy to these containers but they aren't accessible from the inter-webs.

Lastly the networks: we define a default network for the web app to use the service-tier that we setup earlier. This allows the nginx-proxy and our running instances of the web container to correctly talk to each other. (I actually have no idea what I'm saying here - but it is simple enough to setup and I think it's all good - so I'm going with it for now...)

Now what?

So with all of these pieces in place, all we need to do now is run some docker-compose commands to spin up our necessary environments.

Below is an example script that can be used to spin up qa, and prod environments.

This script is setting some environment variables that are then used by the docker-compose command where and we're also setting a unique project name with -p ${VIRTUAL_HOST}.

Project Name

What we said just before this headline is a key part to this. What enables us to run essentially the same project (docker-compose/Dockerfile) but with different environment variables that define things like qa vs prod is when we run docker-compose we're also passing in a -p or --project-name parameter. This allows us to create multiple container instances with different environment variables that all run on different ports and in theory isolate themselves from the other environments.

The thinking here is you could have a single docker-compose.yml file that has multiple server definitions like say a nodejs web, couchdb, and redis database all running isolated within their environment. You can then use the environment variables to drive various items such as feature-toggling newly developed features in a qa environment, but are not necessarily ready to run in a production environment.

Running/testing this out locally

You probably want to play with this idea and test it out locally before trying to push it to a remote system.

One easy way to do this is to modify your /etc/hosts file (on *nix) or follow this on windows to map the specific domain names you have setup for your environments to the actual service running docker. This will allow the nginx-proxy to do it's magic.

I'm currently still using docker-machine to run my docker environment in a VirtualBox VM so my /etc/hosts file looks like this.

### Host Database## localhost is used to configure the loopback interface# when the system is booting. Do not change this entry.##127.0.0.1 localhost
255.255.255.255 broadcasthost
::1 localhost
192.168.99.100 qa.my-docker-test-site.com
192.168.99.100 stage.my-docker-test-site.com
192.168.99.100 my-docker-test-site.com

If you have the docker containers running that we've worked through so far (for all environments) we should be able to visit qa.my-docker-test-site.com in the browser and hopefully get this:

Also try out the production environment at my-docker-test-site.com to verify it is working as expected.

THIS IS AWESOME :) I was actually quite happy to traveled this far in this exploration. But now let's try to take it up a notch and deploy what we just built locally to the DigitalOcean in the cloud ().

Deploy to the Cloud!

Now how do we get this locally running multi-environment system up to a server in the cloud?

Just tonight while researching options I found this simple set of steps to get it going on DigitalOcean. I say simple because you should the original steps I was going to try and use to deploy this... sheesh.

You should now be able to run docker ps or docker-compose ps and see 3 containers running. The nginx-proxy, your qa site and also the prod site.

All that's left is to make sure DNS is configured and pointing to our nginx-proxy front-end...

While playing with this - I kept tearing down droplets and re-building them as I worked through this tutorial and I kept forgetting to adjust my DNS settings. However right in the middle of writing this tutorial DigitalOcean came out with Floating Ip's which wasn't perfect, but definitely made this easier to work with. I didn't have to always update the IP address of my droplet, but instead just update the floating ip to point to the newly created droplet.

Configure our DNS (domain name)

I'm assuming you've already purchased a domain name that you can setup and configure on DigitalOcean. So I don't want to go too far into this process.

I also think DNS is out of scope for this post (as there are many others who can do a better job) but I used some great resources such as these while configuring my DigitalOcean setup.

Environment All Things

If you've made it this far, you hopefully have a DigitalOcean droplet that is now serving qa and prod http requests.

NICE!!!

Now the most important thing - how to seamlessly update an environment with a new build...

Let's make an update to QA.

Now that we've deployed our site to QA and to walk through this a little further, let's make a modification to our qa site and see if we can get it deployed without causing any downtime especially to the prod site, but maybe we can also get an in-place deployment done and have little-to-no down time in qa as well.

I wrote that paragraph above the other night near bedtime and as I'm learning some of this on the fly had no idea if this would be easy enough to accomplish, but to my surprise deploying an update to qa was a piece of cake.

For this test I made a simple change to my node web server code so I could easily see that the change was deployed (or not).

NOTICE: how the -p parameter we added x2 (just to give it a different project name (for being a different version)).

This will bring up another docker container with our updated web application and to my surprise the nginx-proxy automatically chose this new container to send requests to.

So if you docker ps you should see 4 containers running. 1 nginx-proxy, 1 prod container, 2 qa containers (with different names).

You can think about leaving both containers running for the moment while you test out the new release.

One neat thing you can think about with this is if there was something seriously wrong with the new qa release you could just stop the new container and (docker stop <new_container_id>) and the proxy will start redirecting back to the old qa container. (That only works of course if your deployment was immutable - meaning you didn't have the new container run some one way database migration script... but that's not something I want to think about or cover in this post).

Once you're comfortable running the new version you can now bring down and cleanup the older version.

Now let completely remove our test...

You probably don't want to run the sample node script from above forever as you'll be charged some money from DigitalOcean for this and I'd feel bad if you received a bill for this little test beyond a few pennies as you test it out...

The following command will completely remove the droplet from DigitalOcean.

docker-machine rm docker-multi-environment

Wrap Up and What's Next?

I feel like I've done enough learning and sharing in this post. But there is still more to do...

If you want to check out the snippets above combined into a sample github repository I've put it up here.

Future thinking...

I don't know if I'll blog about these, but I definitely want to figure them out. If you find a way to extend my sample above to include the following I'd love to hear about it...

I've been playing with a little toy that uses docker-compose to bring together a web app, couchdb, and redis container into an easy-ier-ish cohesive unit.

While working on it (and to make it a bit more generic), my next step was to find a way to pass the database admin user/pass (and other configuraiton options) into the containers as environment variables which took me way longer to figure out than it should have...

Hopefully this posts helps it click for you a little faster than it (didn't) for me :)

If you land here, you've likely already poured over the different parts of documentation for docker, docker-compose and environment variables.

The docker-compose.yml:

The above file is the part where I got tripped up, and once I added the environment: section it all clicked.

You likely don't want every one of your development or production server's environment variables to show up inside your container. This file acts a bit like the docker run -e ENV_VAR=FOO option and allows you to select specific environment variables that are to be passed into the container.

I like the declarative approach of this file as it makes environment variable dependencies explicit.

The env-file-test/Dockerfile:

Pretty standard Dockerfile, but one thing I learned is you can setup default environment variables using the docker ENV directive. But these will be overriden by the .env file or variables in your terminal's environment.

The env-file-test/docker-entrypoint.sh

#!/bin/shecho"ENV Var Passed in: $DEMO_VAR"

This was just a sample script to print out the environment variable.

Some other things I learned

The docs say you can specify your own env-file or even multiple files, however I could not get that working. It always wanted to choose the .env file.

Also note: that if you have an environment variable specified in your terminal that also exists in your .env file the terminal's environment takes precedence over the .env file.