A Docker Captain's Blog

Docker | Kubernetes | Cloud

New Docker Engine 1.11 integrates runC and containerd

Docker 1.11 is two week old now. Popular for its Software Infrastructure Plumbing(SIP), Docker Inc. first focused on adoption of libnetwork, followed by Notary and Unikernel Projects and now rightly integrated Docker Engine with runC and containerd. What it really means is the management of containers is now split up into a separate piece of infrastructure plumbing called containerd, a daemon for managing runC. With this new release, Docker Engine is now more loosely coupled to the containers.

This new architecture looks promising in the sense that it is expected to bring a significant performance boost when handling a big amount of containers. The new Docker Engine 1.11 execution layer will be entirely relying on well delimited tools that can be used independently, with the drawback that it won’t ship as a single binary anymore. It makes it possible to upgrade the daemon without shutting down all running containers in the future.As stated by Docker Inc., “..this new Engine architecture will use containerd for container supervision. Because containerd ultimately relies on runC and the OCI specification for container execution, this will open the door for the Engine to be able to use any OCI compliant runtime..”

Let’s have a quick look at Docker 1.11 new release components. I had a Ubuntu 14.04.3 system running on AWS cloud instance I tried downloading the latest release and tried looking into the release as shown below:

As shown above, a Linux docker installation is now made of 4 binaries (docker, docker-containerd, docker-containerd-shim and docker-runc). Here is a brief explanation in case you are completely new to runC.

runC:

runC is a lightweight universal runtime container. runC is built on libcontainer, the same container technology powering millions of Docker Engine installations.It is a CLI tool for spawning and running containers according to the OCP specification.The Open Container Project is an open governance structure for the express purpose of creating open industry standards around container formats and runtime. Projects associated to the Open Container Project can be found at https://github.com/opencontainers.

runc integrates well with existing process supervisors to provide a production container runtime environment for applications. It can be used with your existing process monitoring tools and the container will be spawned as a direct child of the process supervisor.

Containers are configured using bundles. A bundle for a container is a directory that includes a specification file named “config.json” and a root filesystem. The root filesystem contains the contents of the container.

Containerd:

Docker Inc. built up containerd as a seperate daemon to move the container supervision out of the core Docker Engine.It is firmly believed that Containerd improves on parallel container start times which means if you need to launch multiple containers as fast as possible you should see improvements with this release.Containerd is claimed to have full Support For starting OCI bundles And managing Their Lifecycle.

Containerd has full support for starting OCI bundles and mananaging their lifecycle. This allows users to replace the runC binary on their system with an alternate runtime and get the benefits of still using Docker’s API. When starting a container most of the time is spent within syscalls and system level operations. It does not make sense to launch all 100 containers concurrently since the majority of the startup time is mostly spent waiting on hardware / kernel to complete the operations. Containerd uses events to schedule container start requests and various other operations lock free. It has a configurable limit to how many containers it will start concurrently, by default we have it set at 10 workers. This allows you to make as many API requests as you want and containerd will start containers as fast as it can without totally overwhelming the system.

Let’s look at how the newly added docker-runc and docker-containerd looks like in this new release.