2015 Recap: Cloud, Containers and DevOps - Part 1

2015 will be remembered as the year that redefined the way applications are designed, developed, and deployed in the cloud. This year saw a hectic activity within the container community. Containers influenced almost every aspect of cloud computing including infrastructure, platforms, and DevOps. Microservices and Cloud-native computing became the most talked about trends in the industry.

Source: https://flic.kr/p/eP6Z2y

Here is a recap of the major trends of 2015 from the world of cloud, containers, and DevOps.

Container Orchestration gains momentum

Containers gained the industry attention in 2014. This year, the focus has shifted from deploying stand-alone containers to managing a cluster of distributed containers. Kubertenes, Docker Swarm, and Mesosphere became the top container orchestration engines.

Kubernetes has become the preferred tool for managing containerized workloads in production. Given
Google’s commitment to actively maintaining it, Kubernetes is enjoying the popularity within the community. This year saw the release of Kubernetes 1.0 followed by the announcement of 1.1. KubeCon, the inaugural Kubernetes conference, was a huge success that witnessed active participation and contribution from the community.

Docker Swarm, the native orchestration engine has become generally available in November. Currently in its first version, Swarm is backed by Docker, Inc, which is a part of the core Docker platform. It is preferred for its simplicity and compatibility with the Docker API. Swarm supports multi-host networking and persistent storage features that became available with Docker 1.9.

Mesosphere, the commercial implementation of
Apache Mesos is positioned as the data center operating system. Though not exclusively designed for containers, Mesosphere along with Marathon is used for container orchestration. Kubernetes can be run on top of Mesosphere through the Kubernetes-Mesos project.

Going forward, the preferred choice of orchestration engines will be limited to Docker Swarm, Kubernetes, and Mesosphere.

The container war officially ended

CoreOS, the startup that built a container-optimized OS was one of the key allies of Docker, Inc. In 2014, CoreOS launched an alternative container format and runtime called rkt. This widened the gap between the two companies. CoreOS was not convinced with Docker architecture and its intention of bundling additional functionality within the monolithic Docker binary. Rkt came out with its own runtime called AppC that challenged Docker.

During DockerCon 2015, Docker and CoreOS buried the hatchet by agreeing to support an open standard defined by the Open Container Initiative (OCI). Many vendors including Amazon Web Services, Apcera, Cisco, CoreOS, Docker,
EMC,
Fujitsu Limited, Goldman Sachs, Google, HP, Huawei,
IBM, Intel, Joyent, Linux Foundation, Mesosphere, Microsoft, Pivotal, Rancher Labs, Red Hat, and VMware joined OCI as founding members. Formed under the auspices of Linux Foundation, OCI is designed to deliver a standardized container format and runtime that provides a consistent platform for the tool vendors and ecosystem players. Docker donated the draft specification for the base format and runtime along with the code associated with a reference implementation of that specification.

With Microsoft supporting Docker API and format for its Hyper-V Containers and Windows Server Containers, developers can use standard tools to manage and orchestrate containerized workloads deployed on Windows. This compatibility makes it possible to mix and match Linux and Windows clusters running different containers that belong to the same microservices application.

The industry welcomed the Open Container Initiative. It would enable interoperability and integration of current and future implementations of container technologies.

Google was the first to realize the potential of hosted container management service. It exposed Kubernetes through Google Container Engine (GKE), the managed service running on top of Google Compute Engine. Customers can quickly spin up clusters and use the standard Kubernetes CLI to deploy containerized workloads on Google Cloud Platform. Announced in 2014, GKE became generally available this year.

Amazon EC2 Container Service (ECS) was announced at AWS re:Invent in 2014. Amazon decided to build its own orchestration engine without relying on the available open source projects. It lacks a few capabilities such as a discovery service. ECS utilizes many AWS platform capabilities including ELB, EBS, CloudWatch, and Auto Scale. For customers with existing workloads on AWS, ECS is a good choice to run their microservices infrastructure. They can easily integrate existing monolithic applications with the containerized workloads deployed on ECS.

Microsoft has partnered with Mesosphere to build Azure Container Service (ACS). Currently in technical preview, ACS is Microsoft’s answer to GKE and ECS.

CoreOS has launched Tectonic, the commercial version of Kubernetes designed to run within the enterprise data center. Backed by Google, Tectonic targets enterprise microservice workloads that cannot be deployed in the public cloud. Customers get the end-to-end stack that comprises of CoreOS, Quay, and Kubernetes.

Docker is not far behind the game of CaaS. At DockerCon Europe, it launched Universal Control Plane (UCP), the official container hosting platform from Docker, Inc. Boasting of enterprise features such as integrated LDAP/AD integration, high availability clusters, and powerful interface, UCP competes with CoreOS Tectonic. This year Docker acquired Tutum, the container infrastructure company that supports a variety of public cloud platforms. Docker, Inc. might eventually converge UCP and Tutum to deliver unified management platform.

IoT became the key public cloud offering

The hyper-scale capabilities of public cloud are well-suited for running IoT applications. This year saw the launch of AWS IoT and Azure IoT platforms.

Microsoft has been gradually adding IoT capabilities to Azure. The Event Hubs service is designed for ingesting high-velocity and high-volume data generated by sensors and devices. Stream Analytics is used for analyzing stream data in real-time through simple SQL-like queries. DocumentDB, SQL Database, and Azure HDInsight provide storage and processing capabilities. Azure ML was also launched this year that plays a crucial role in IoT through predictive analysis. During the last quarter of 2015, Microsoft launched IoT Hub, the cloud-based gateway for registering and managing IoT devices. Azure IoT Suite connects the dots across a variety of cloud services to deliver a unified IoT solution. It supports scenarios such as predictive maintenance, asset management, and remote monitoring. Microsoft is also investing in Cortana Analytics, which will deliver the processing, analysis, and visualization services. Overall, this year marked the official entry of Microsoft in the IoT segment.

Having acquired 2lemetry, an IoT PaaS startup, AWS launched a managed IoT platform at 2015 re:Invent event. AWS IoT provides the cloud gateway, rules engine, integration with AWS Lambda, and hooks for the integration with Kinesis. The key differentiator of AWS IoT is its Shadow feature that lets developers query the state of devices even when they are offline. The rules engine supports defining the policies based on SQL. Developers can write short snippets hosted in AWS Lambda that can be triggered from AWS IoT. This combination makes AWS IoT platform very powerful. The output from the IoT engine can be passed to Amazon Kinesis for real-time analysis.

Google is yet to make an IoT specific offering available on its cloud platform. Though it is moving fast with Brillo and Weave, there is no managed IoT platform on GCP.