Can I use Docker as a hypervisor for OpenStack (Juno)?

There are several possible ways of using Docker with OpenStack. The one described here is based on OpenStack's Nova Docker Driver. That is, it means talking to a container using the Nova Compute API.

Not all Docker features are supported by the Nova Docker Driver.

Features that are either not supported, or have a non-standard implementation are:

Attaching volumes to Docker containers is not supported

Disk usage is limited to 10 GB per container regardless of flavor settings

Compute resources on a host node are distributed according to a CPU share constraint value which depends upon the OpenStack flavor used. Each container will have a CPU share constraint value of 1024 * VCPUs. Thus a 2 VCPU flavor result in a CPU shares value of 2048. More about CPU constraints can be read here: Link26

Note: Mixing a libvirt (KVM/QEMU) hypervisor with a Docker hypervisor on the same host is not recommended.

If libvirt (KVM/QEMU) hypervisors are to be mixed with Docker hypervisors on different hosts, but still as part of the same OpenStack deployment, then it might be necessary to let OpenStack know which compute hosts can support Docker. This can be done, for example, by creating a special flavor and restricting it (via a shared custom extra specs key=value pair) to a particular Host Aggregate, thus aggregating the docker hypervisor hosts. Another way of restricting it would be by setting a special image property and relying on the 'ImagePropertiesFilter' property of Nova Scheduler to schedule the image properly.

Prerequisites

A working deployment of Bright OpenStack 7.1 (Juno) is needed.

Nomenclature

Code snippets used in this article are meant to be executed in the environment indicated by the shell prompts:

master#

Shell on the head node

node001#

Shell on the compute node

docker#

Shell inside of running container

docker-image#

Shell inside of software image chroot

cmsh#

CMSH on the head node

Docker Nova driver installation

1. Preparing software image

First of all, the existing OpenStack software image is cloned for use in Docker:

On the head node /etc/glance/glance-api.conf is edited and docker added to a list of supported images, e.g.:

[DEFAULT] container_formats=ami,ari,aki,bare,ovf,ova,docker

The glance-api service is restarted:

master# service openstack-glance-api restart

Now Docker should be up and running. The Docker hosts should be seen in the OpenStack Dashboard:

3. Testing your docker setup

To test the newly-created setup, some images are first added to Glance. In order to do that, a running machine with Docker, and access to a master node, is needed. The newly-created docker compute hosts (for example node001) can be used for this:

Troubleshooting

Most of time, if the Docker driver fails to boot the container, then the instance will fail to be created with an error "No valid host was found". The real error message is located in /var/log/nova/nova-compute.log on a host node. Below is a list of the most common nova-docker driver errors, along with ways to solve them.

1. Error: the compute node fails to spawn a new docker instance, nova-compute.log contains the following error:

RuntimeError: Cannot find any PID under container "CONTAINER_ID"

Reason: This error can occur for some images (e.g. busybox). It can happen if the container shuts down immediately after being created — this can be checked by having Docker run busybox on a host node.

Solution: Use images that have a running service as an entry point

2. Error: the compute node fails to spawn a new Docker instance. The nova-compute.log contains following error:

Reason: Several Docker operations such as transferring the Docker image from Glance require some time. The default timeout value is 10 seconds, and the nova-docker driver raises a timeout error after that.

Solution: increase the timeout value within /usr/lib/python2.7/site-packages/novadocker/virt/docker/client.py27 (search for the word "timeout")

3. Error: the compute node fails to spawn a new Docker instance. The nova-compute.log contains the error:

APIError: 404 Client Error: Not Found ("No such image: someimage")

Reason: most likely because the Glance image name differs from the Docker image name

Solution: recreate Glance image with the same name as the Docker image name