Ufo — Easily Build Docker Images and Deploy Containers to AWS ECS

Quick Introduction

Amazon EC2 Container Service, ECS, is an AWS service that provisions and manages Docker containers on a cluster of EC2 instances. As with most of AWS services, it is great and simply requires a little tooling wrapped around it to create a smooth flow. Ufo is a simple tool that makes building and shipping Docker containers to AWS ECS super easy.

Ufo provides a command called ufo ship that does the following:

builds a docker image.

generates and registers the ECS template definition.

deploys the ECS template definition to the specified service.

Ufo deploys a task definition that is created via a template generator which is fully controllable. We’ll go over a quick example to show what the template looks like and how it works.

Task Definition ERB Template and DSL Generator

The task definition is created from an ERB template in the ufo/templates folder. Here is an example: ufo/templates/main.json.erb.

As you can see above, the task_definitions.rb file has some special variables and helper methods available. These helper methods provide useful contextual information from the project so you don’t have to copy paste and update the code in multiple places. For example, one of the variable provides the exposed port in the Dockerfile of the project. Here is a list of the important ones:

helper.full_image_name — The full docker image name that ufo builds. The “base” portion of the docker image name is defined in ufo/settings.yml. For example, the base portion is “tongueroo/hi” and the full image name is tongueroo/hi:ufo-[timestamp]-[sha]. So the base does not include the Docker tag and the full image name does include the tag.

helper.dockerfile_port — Exposed port extracted from the Dockerfile of the project.

helper.env_file — This method takes an .env file which contains a simple key value list of environment variables and converts the list to the proper task definition json format.

The 2 classes which provide these special helper methods are in ufo/dsl.rb and ufo/dsl/helper.rb. Refer to these classes for the full list of the special variables and methods.

Usage Example

An example will demonstrate how easy it is to use ufo. More details are provided on the project’s official documentation: http://ufoships.com.

This example is a demo rails app that returns the rails welcome page. I have also created bin/worker and bin/clock scripts that just run an infinite loop to mock out worker and clock processes for testing. The full source code for the demo project is available on GitHub: tongueroo/hi.

Test Demo Project Locally in Mac OSX

Let’s setup the app, install the dependencies and start up the web process.

In the above snippet, I ran two ufo commands: ufo init and ufo docker build. Let’s review some of the files that ufo init created:

bin/deploy — Wrapper deploy script that shows how to deploy to 3 common web, worker and clock processes as services at the same time with ufo.

ufo/settings.yml — This is where you set the Docker image name to be built. There is a service_cluster mapping option, which you can use to avoid having to always provide the --cluster option in the CLI. It is quite handy.

ufo/task_definitions.rb — This is where you define the variables to be substituted into the ERB template.

ufo/templates/main.json.erb — This the task definition ERB template. You can modify this and have full control over the task definition template that gets registered to ECS.

.env — A starter .env file is provided if one does not yet exist on your project. You can rename this to .env.prod and .env.stag if that makes more sense for your needs. Remember to update the env_file line in task_definitions.rb if you rename the .env file.

The ufo docker build command created a tongueroo/hi:ufo-2016–11–30T16–25–26-e1d57ce Docker image.

Ship the Docker Image to ECS

Let’s ship the web process as an ECS service. First, create an ECS Cluster called stag that we will use to ship the web service to. You will also need the ELB Target Group associated with the web service so that the app will be accessible from anywhere in the world. I have created a stag cluster, a “hi-elb” and “hi-target-group” for this example. You can grabbed the Target Group ARN from Load Balancing / Target Groups:

You only need to provide the ELB Target Group ARN the first time deploying with ufo since you cannot update the Target Group of the ECS Service afterwards. If you want to change the Target Group, you must create and deploy a new service instead. This is an AWS ECS design decision.

That’s it! The web process for this tongueroo/hi app has been deployed to ECS.

Using bin/deploy

When we use ufo init at the beginning of this example, it generated a bin/deploy script. This script handles deploying common application processes like web, worker and clock all at once. These processes typically use the same codebase, ie: same docker image, but have slightly different run time settings. For example, the docker run command for a web process could be puma and the command for a worker process could be sidekiq. Environment variables are sometimes different also.

Let’s quickly test to make sure that the worker and clock process work locally first. The worker and clock process scripts are actually mocked out and are just simple infinite bash loops. That’s all we need to test things. Here’s a quick local test:

The bin/deploy wrapper script just calls the ufo ships command, which is designed to build 1 docker image and ship it to multiple ECS services. Now let’s ship all 3 processes as services to ECS!

bin/deploy # deploys clock, worker and web!

Summary

The ufo tool automates building the docker image, registering the ECS task definition, and deploying the container to the ECS service. The project page is available on GitHub at tongueroo/ufo. Try it out and let me know what you think!

Thanks for reading this far. If you found this post useful, I’d really appreciate it if you recommend this post (by clicking the clap button) so others can find it too! Also, connect with me on LinkedIn.