Using Travis CI to test Docker builds

In last months article we discussed “Dockerizing” this blog. What I left out from that article was how I also used Docker Hub’s automatic builds functionality to automatically create a new image every time changes are made to the GitHub Repository which contains the source for this blog.

The automatic builds are useful because I can simply make changes to the code or articles within the repository and once pushed, those changes trigger Docker Hub to build an image using the Dockerfile we created in the previous article. As an extra benefit the Docker image will also be available via Docker Hub, which means any system with Docker installed can deploy the latest version by simply executing docker run -d madflojo/blog.

The only gotcha is; what happens if those changes break things? What if a change prevents the build from occurring, or worse prevents the static site generator from correctly generating pages. What I need is a way to know if changes are going to cause issues or not before they are merged to the master branch of the repository; deploying those changes to production.

To do this, we can utilize Continuous Integration principles and tools.

What is Continuous Integration

Continuous Integration or CI, is something that has existed in the software development world for a while but it has gained more following in the operations world recently. The idea of CI came up to address the problem of multiple developers creating integration problems within the same code base. Basically, two developers working on the same code creating conflicts and not finding those conflicts until much later.

The basic rule goes, the later you find issues within code the more expensive (time and money) it is to fix those issues. The idea to solve this is for developers to commit their code into source control often, multiple times a day even. With code commits being pushed frequently this reduces the opportunity for code integration problems, and when they do happen it is often a lot easier to fix.

However, code commits multiple times a day by itself doesn’t solve integration issues. There also needs to be a way to ensure the code being committed is quality code and works. This brings us to another concept of CI, where every time code is committed, the code is built and tested automatically.

In the case of this blog, the build would consist of building a Docker image, and testing would consist of some various tests I’ve written to ensure the code that powers this blog is working appropriately. To perform these automated builds and test executions we need a tool that can detect when changes happen, and perform the necessary steps; we need a tool like Travis CI.

Travis CI

Travis CI is a Continuous Integration tool that integrates with GitHub and performs automated build and test actions. It is also free for public GitHub repositories, like this blog for instance.

In this article I am going to walk through configuring Travis CI to automatically build and test the Docker image being generated for this blog. Which, will give you (the reader) the basics of how to use Travis CI to test your own Docker builds.

Automating a Docker build with Travis CI

This post is going to assume that we have already signed up for Travis CI and connected it to our public repository. This process is fairly straight forward, as it is part of Travis CI’s on-boarding flow. If you find yourself needing a good walk through, Travis CI does have a getting started guide.

Since we will be testing our builds and do not wish to impact the main master branch the first thing we are going to do is create a new git branch to work with.

$ git checkout -b building-docker-with-travis

As we make changes to this branch we can push the contents to GitHub under the same branch name and validate the status of Travis CI builds without those changes going into the master branch.

Configuring Travis CI

Within our new branch we will create a .travis.yml file. This file essentially contains configuration and instructions for Travis CI. Within this file we will be able to tell Travis CI what languages and services we need for the build environment as well as the instructions for performing the build.

Defining the build environment

Before starting any build steps we first need to define what the build environment should look like. For example, since the hamerkop application and associated testing scripts are written in Python, we will need Python installed within this build environment.

While we could install Python with a few apt-get commands, since Python is the only language we need within this environment it’s better to define it as the base language using the language: python parameter within the .travis.yml file.

language: python
python:
- 2.7
- 3.5

The above configuration informs Travis CI to set the build environment to a Python environment; specifically for Python versions 2.7 and 3.5 to be installed and supported.

The syntax used above is in YAML format, which is a fairly popular configuration format. In the above we are essentially defining the language parameter as python and setting the python parameter to a list of versions 2.7 and 3.5. If we wanted to add additional versions it is as simple as appending that version to this list; such as in the example below.

language: python
python:
- 2.7
- 3.2
- 3.5

In the above we simply added version 3.2 by adding it to the list.

Required services

As we will be building a Docker image we will also need Docker installed and the Docker service running within our build environment. We can accomplish this by using the services parameter to tell Travis CI to install Docker and start the service.

services:
- docker

Like the python parameter the services parameter is a list of services to be started within our environment. As such that means we can also include additional services by appending to the list. If we needed Docker and Redis for example we can simply append the line after specifying the Docker service.

services:
- docker
- redis-server

In this example we do not require any service other than Docker, however it is useful to know that Travis CI has quite a few services available.

Performing the build

Now that we have defined the build environment we want, we can execute the build steps. Since we wish to validate a Docker build we essentially need to perform two steps, building a Docker container image and starting a container based on that image.

We can perform these steps by simply specifying the same docker commands we used in the previous article.

In the above we can see that the two docker commands are specified under the install parameter. This parameter is actually a defined build step for Travis CI.

Travis CI has multiple predefined steps used during builds which can be called out via the .travis.yml file. In the above we are defining that these two docker commands are the steps necessary to install this application.

Testing the build

Travis CI is not just a simple build tool, it is a Continuous Integration tool which means its primary function is testing. Which means we need to add a test to our build; for now we can simply verify that the Docker container is in running, which can be performed by a simple docker ps command.

script:
- docker ps | grep -q blog

In the above we defined our basic test using the script parameter. This is yet another build step which is used to call test cases. The script step is a required step, if omitted the build will fail.

Pushing to GitHub

With the steps above defined we now have a minimal build that we can send to Travis CI; to accomplish this, we simply push our changes to GitHub.

During the sign up process for Travis CI, you are asked to link your repositories with Travis CI. This allows it to monitor the repository for any changes. When changes occur, Travis CI will automatically pull down those changes and execute the steps defined within the .travis.yml file. Which in this case, means executing our Docker build and verifying it worked.

As we just pushed new changes to our repository, Travis CI should have detected those changes. We can go to Travis CI to verify whether those changes resulted in a successful build or not.

Travis CI, will show a build log for every build, at the end of the log for this specific build we can see that the build was successful.

One important thing to know about Travis CI is that most build steps require commands to execute successfully in order for the build to be marked as successful.

The script and install steps are two examples of this, if any of our commands failed and did not return a 0exit code than the whole build would be marked as failed.

If this happens during the install step, the build will be stopped at the exact step that failed. With the script step however, the build will not be stopped. The idea behind this is that if an install step fails, the build will absolutely not work. However, if a single test case fails only a portion is broken. By showing all testing results users will be able to identify what is broken vs. what is working as expected.

Adding additional tests

While we now have Travis CI able to verify the Docker build is successful, there are still other ways we could inadvertently break this blog. For example, we could make a change that prevents the static site generator from properly generating pages, this would break the site within the container but not necessarily the container itself. To prevent a scenario like this, we can introduce some additional testing.

Within our repository there is a directory called tests, this directory contains three more directories; unit, integration and functional. These directories contain various automated tests for this environment. The first two types of tests unit and integration are designed to specifically test the code within the hamerkop.py application. While useful, these tests are not going to help test the Docker container. However, the last directory functional, contains automated tests that can be used to test the running Docker container.

These tests are designed to connect to the running Docker container and validate the static site’s content.

For example test_broken_links.py will crawl the website being served by the Docker container and check the HTTP status code returned when requesting each page. If the return code is anything but 200 OK the test will fail. The test_content.py test will also crawl the site and validate the content returned matches a certain pattern. If it does not, then again these tests will fail.

What is useful about these tests is that, even though the static site is running within a Docker container we are still able to test the site functionality. If we were to add these tests to the Travis CI configuration, they would also be executed for every code change; providing even more confidence about each change being made.

Installing test requirements in before_script

To run these tests via Travis CI we will simply need to add them to the script section as we did with the docker ps command. However, before they can be executed these tests require several Python libraries to be installed. To install these libraries we can add the installation steps into the before_script build step.

The before_script build step is performed before the script step but after the install step. Making before_script the perfect location for steps that are required for script commands but not part of the overall installation. Since the before_script step is not executing test cases like the install step, it too requires all commands to succeed before moving to the script build step. If a command within the before_script build step fails, the build will be stopped.

Running additional tests

With the required Python libraries installed we can add the test execution to the script build step.

script:
- docker ps | grep -q blog
- python tests.py

These tests can be launched by executing tests.py, which will run all 3 automated tests; unit, integration and functional.

In the above we can see our Travis CI configuration consists of 3 build steps; install, before_script and script. The install step is used to build and start our Docker container. The before_script step is simply used to install required libraries for test scripts and the script step is used to execute our test scripts.

Overall, this setup is pretty simple and something we could test manually outside of Travis CI. The benefit of having Travis CI though is that all of these steps are performed for every change, no matter how minor they are.

Also since we are using GitHub, this means Travis CI will append build status notifications on every pull request as well, like this one for example. With these types of notifications I can merge pull requests into the master branch with the confidence that they will not break production.

Building a Continuous Integration and Deployment pipeline

In last months article we explored using Docker to package and distribute the application running this blog. In this article, we have discussed leveraging Travis CI to automatically build that Docker image as well as performing functional tests against it.

In next months article, we are going to take this setup one step further by automatically deploying these changes to multiple servers using SaltStack. By the end of the next article we will have a full Continuous Integration and Deployment work-flow defined which will allow changes to be tested and deployed to production without human interaction.

About Benjamin

Benjamin is a Infrastructure and Software Engineer. On this blog he writes about Linux, Docker, Programming as well as other Systems topics.

Learn more about Linux

If you liked this article, check out Benjamin's book: Red Hat Enterprise Linux Troubleshooting Guide. Where you can learn a lot more about troubleshooting Linux systems. This book is filled with tips and techniques he has learned over years of managing mission critical systems.