New features in Mesos 1.0, DC/OS, differences to Kubernetes and Docker Swarm

Apache Mesos 1.0 is out! We talked to Michael Hausenblas from Mesosphere about efficient use cases for Apache Mesos, differences between Mesos, Kubernetes and Docker Swarm and changes in version 1.0.

Apache Mesos: Use cases and new features in version 1.0

JAXenter: What does a typical use case for Mesos look like?

Michael Hausenblas: As a cluster resource manager, Apache Mesos is designed pretty low level. It’s similar to the Linux kernel which manages computing resources (RAM, CPU and so forth) for a single machine and facilitates application development by providing abstractions like memory management or processes . Mesos does the same for a whole cluster. Just like the Linux kernel, nobody would use Mesos alone, but in combination with other components like Marathon (distributed init system) or service discovery via Mesos-DNS.

The Mesos Developer Community is diverse and growing.

Mesos is a so-called two-level scheduler, meaning that resource management and process scheduling logic are strictly separated. The latter is realized via Mesos frameworks (cf. http://mesos.apache.org/documentation/latest/frameworks/), which decide how to manage the distributed processes depending on their workload requirements (Long-Running, Batch, …),. This typically includes things like status updates, failover and scaling.

Speaking of use cases, there is a wide range of possible applications in different industries (examples can be found here and here) and different fields of use: stateless services (for example webserver farms) and elastic data pipelines (Spark, Kafka, Cassandra), but also IoT and mobile applications.

JAXenter: Mesos 1.0 has just been published. In your opinion, what are the most important improvements in this release?

Michael Hausenblas: One of the key improvements of Mesos 1.0 is the standard HTTP API that makes developing new Mesos frameworks easier. Another key improvement is the unified containerizer which allows people to use other runtimes in Mesos than Docker containers which have been supported for a while now. In the field of networking, Mesos now supports the Container Networking Interface. Furthermore, there are some improvements in the security department, such as Fine-Grained Auth.

External storages and GPUs are now supported too. In collaboration with Microsoft, we also introduced Windows support for Mesos Agents; this being especially interesting for hybrid workloads. The Mesos Developer Community is diverse and growing, with contributors coming from companies like Apple, IBM, Microsoft and Twitter.

Kubernetes, Docker Swarm, DC/OS

JAXenter: There are other orchestration solutions within the container ecosystem: Kubernetes and Docker Swarm. What are the differences to Mesos?

Michael Hausenblas: Technically speaking, Mesos is just part of an orchestration solution, as I have mentioned before. It isn’t limited to containers though, but able to handle any Linux-64-bit application and a growing number of Windows apps. While Kubernetes and Docker Swarm offer an integrated approach (resource management and scheduling in one logical unit), Mesos is focussed on the pure management of computational resources, while the scheduling is delegated to the Mesos framework. Because Mesos exists for over six years already, we gained plenty of solid experience in the operational use of the system from companies like Twitter, Airbnb, Apple or Netflix.

JAXenter: What will the future of Mesos look like after version 1.0? What is Mesosphere going to do next?

With DC/OS we build something like a Mesos distribution.

Michael Hausenblas: Mesos will continue to grow and make the development of distributed systems easier. You might already know that Mesos poses as the core to another open source project: DC/OS. With DC/OS, we build something like a Mesos distribution: It’s just like Linux – it’s rare that anyone would use the Linux kernel to build their own distribution instead of using existing distributions like Ubuntu or SUSE. DC/OS can be used as a distribution to work with Mesos without caring about the details. DC/OS is offering a distributed package manager, CLI and UI and many more functions like service discovery, load balancing, logging and networking. It allows installations in public cloud and on-premise environments.

JAXenter: Thank you ver much for this interview.

Michael is a Developer and Cloud Advocate with Mesosphere. He helps devops to build and operate scalable & elastic distributed applications. His background is in large-scale data integration, Hadoop & NoSQL, IoT, as well as Web applications and he’s experienced in advocacy and standardization. Michael is contributing to open source software at Apache (Myriad, Drill) and shares his experience with the Datacenter OS and large-scale data processing through blog posts and public speaking engagements.

Hartmut Schlosser is an editor for JAXenter and a specialist in Java Enterprise-Technologies, Eclipse & ALM, Android and Business Technology. Before working at S&S Media he studied Computer Science, Music, Anthropology and French Philology.