MesosCon Highlights: Apache Mesos 0.20 and Docker Support

Isabel Jimenez will graduate from her master in Computer Science this month. She spent three months contributing to Apache Mesos with the Twitter team thanks to Gnome’s Outreach Program for Women.

Apache Mesos had its first annual conference in collaboration with LinuxCon North America this year. I attended the conferences because its one of the major open source events of the year and also because I volunteered to be part of MesosCon program committee.

MesosCon started with a keynote from Benjamin Hindman, one of the creators of the project and Mesos team lead. One of the major news items in this keynote was the release of Apache Mesos version 0.20 later that day. This release answered a popular request from Mesos users: Docker integration.

Although it was already possible to launch Docker containers with Mesos, thanks to the external containerizer mechanism, Docker has become a first-class citizen. With Mesos 0.20 it is now possible to use Docker containers without any external installation.

The history of Mesos with containers goes back to the start of the project, in 2011. The Mesos default containerizer used LXC. Later for better integration Mesos decided to use cgroups directly.

With Docker’s popularity growth, Mesos started supporting Docker with some work like mesos-docker, an executor, and later with Deimos, an external containerizer.

But to fully enjoy all of Docker’s features, this year it became fully integrated into Mesos. This means that a specific containerizer was created for Docker inside Mesos.

A new component, ContainerInfo was added as the base protobuf message along with a DockerInfo that helps to lend parameters to Docker container specifics. The ContainerInfo base was added to TaskInfo and ExecutorInfo also to allow users to launch Docker as a task or an executor.

What the Docker containerizer does is simply translate Task/Executor Launch and Destroy calls to the Docker CLI commands.

In my opinion, this announcement is a immense step towards what could be the future of Docker containers orchestration. Other new tools are coming to the surface to answer this huge demand but they are no more than a Docker API smart overlay and still lack lots of improvements.

Supporting Docker Integration

One of the speakers at MesosCon was Victor Vieux, a Docker core maintainer, who talked about how easy it was to use this new integration to build new Mesos frameworks. He demonstrated Volt, a framework I had the chance to work on, that serves as perfect proof-of-concept for Mesos Docker support and its easy and quick application.

Although not every Docker feature is supported yet in Mesos, Volt allows a user to launch tasks inside Docker containers, letting the user benefit at the same time from Mesos resource management.

The Marathon Mesos framework, by mesosphere was also quickly updated to support this new Docker integration.

The community had a quick positive reaction to all these changes and in my opinion, is eager to see the integration fully developed. Especially about letting Docker containers be linked and communicate with each other.

The event was also a great chance to talk with all Mesos users and developers. As part of the organizing committee I had the occasion to participate in the speaker dinner where I witnessed the great energy behind different Mesos use cases.

The hackathon day was a great opportunity to hack on the Mesos ecosystem and implement the newly announced features, for example JenkinsOnMesos, which added Docker support during the hackathon.

It was also an opportunity for me to continue some work on Volt and get firsthand feedback on it.

Of course MesosCon was about much more than this fantastic news. All the talks were rewarding, the open-mindness of the Apache Mesos community lead to very interesting talks on different cluster management approaches and how to make Mesos better. Benjamin Hindman let us know that 1.0 release was as near as some API upgrades and we are all looking forward to MesosCon 2015!