We're transitioning our infrastructure to the cloud, to reduce our dependence on having a server admin, and to increase our uptime and scalability.

We've decided to build our new infrastructure on top of Docker, for portability and scalability. It also strongly encourages a modular design, which is another goal of ours.

For our production environment we haven't yet decided how we should do deploys, although having our CI setup build and deploy new containers is a likely approach.

For development, however, we wanted to recreate a system we've used until now on our legacy server. That system was very specific and not very portable, so we wanted to build a new, more portable version on top of Docker.

Since our current version is written in Python, and it seems to be a good tool for the job, we chose Python for the new version as well.

The solution

What we ended up building was a docker image that pulls code into a volume that it shares with the web server, based on incoming webhooks from GitHub. When done, it pings the web server to do whatever it needs to do with the new code.

The docker image is even more flexible than that, as it can run any arbitrary terminal commands when it receives the webhook. It can also be configured to serve multiple repositories at once.

2. Write your repos.json configuration file.

This is a configuration file that the webhook listener uses to know which hooks to listen to and how to verify their integrity with a webhook secret. We opted to have it call a separate .sh file, as we needed to add a ssh key, which proved tricky to do through Pythons subprocess. If you know how, let me know in the comments or on twitter.

At this point, you should be ready to receive webhooks. Set up your webhook and update your repository to test it.

If you have the volume shared on the docker host, you should be able to see the git files show up there. If not, you can either try to access the web server container's exposed port, or you can enter the container and look for the files there:

docker exec -it my-rails-server bash

Ideas for where to go next

I'd like to tie all this together with docker composer, so that every container that works together can be launched together. I'd also like to nail down the production version of this.

Another challenge will be to put up a branch system. We have this for our web client in our legacy setup, and I want to bring it over both for the web client and the API. The branch system works by serving each branch from git on a different subdomain, automatically. It calls for some fancy web server work, and I'm not quite sure how to set it up for rails yet.