Let us accept the fact – “Managing Docker on different Infrastructure is still difficult and not portable”. While working on Docker for Mac, AWS, GCP & Azure, Docker Team realized the need for a standard way to create and manage infrastructure state that was portable across any type of infrastructure, from different cloud providers to on-prem. One serious challenge is that each vendor has differentiated IP invested in how they handle certain aspects of their cloud infrastructure. It is not enough to just provision n-number of servers;what IT ops teams need is a simple and consistent way to declare the number of servers, what size they should be, and what sort of base software configuration is required. Also, in the case of server failures (especially unplanned), that sudden change needs to be reconciled against the desired state to ensure that any required servers are re-provisioned with the necessary configuration. Docker Team introduced and open sourced “InfraKit” last year to solve these problems and to provide the ability to create a self healing infrastructure for distributed systems.

InfraKit is basically a toolkit for infrastructure orchestration. With an emphasis on immutable infrastructure, it breaks down infrastructure automation and management processes into small, pluggable components. These components work together to actively ensure the infrastructure state matches the user’s specifications. InfraKit therefore provides infrastructure support for higher-level container orchestration systems and can make your infrastructure self-managing and self-healing.

Why the Integration of LinuxKit with Infrakit now??

LinuxKit is gaining momentum in terms of a toolkit for building custom minimal, immutable Linux distributions. Integration of Infrakit with LinuxKit will help users to build and deploy custom OS images to a variety of targets – from a single vm instance on the mac (via xhyve / hyperkit, no virtualbox) to a cluster of them, as well as booting a remote ARM host on Packet.net from the local laptop via a ngrok tunnel.

Under this blog post, I will show you how does InfraKit and LinuxKit work together to build immutable infrastructure. I want to test drive these toolkits on my macOS Sierra 10.12.3 system.

template Render an infrakit template at given url. If url is ‘-‘, read from stdin

util Utilities

version Print build version information

x Experimental features

As you see above, CLI is contextual. It basically discovers the hyperkit plugin running and generates a new command for you to access it

In case hyperkit plugin is not turning up, you can kill the old hyperkit instance process and re-start it.

Verify if the hyperkit plugin is up and running

In case you want to run HyperKit, there is recommended command for it.

bash-3.2$ infrakit playbook linuxkit run-hyperkit

Start HYPERKIT plugin? [no]: yes

Starting HYPERKIT plugin. This must be running on the Mac as a daemon and not as a container

This plugin is listening at localhost:24865

By now, we have everything ready for our LinuxKit SSH playbook

LinuxKit SSHD Example:

Now let me show you how to build a LinuxKit image containing just a simple sshd.The file `sshd.yml` defines the components inside the image. Instead of a standard LinuxKit image yml, it is actually an InfraKit template that is rendered before the moby tool is invoked to build the actual OS image.

sudo infrakit playbook linuxkit demo-sshd

This will show up the detailed information on its usage on your terminal:

Let us first build the “SSHD” YAML file using the below command:

sudo infrakit playbook linuxkit demo-sshd build-image

Verifying the SSH outputs:

Looking into sshd.yml content:

The below content shows us that SSH service has been right created:

The command `build-image` will collect user input such as the public key location and use that to generate the final input to `moby`.

Open up a new terminal to watch our first Hyperkit instance:

[simterm]

watch -d infrakit instance-hyperkit describe

Running the SSH Instance

Using the `hyperkit` subcommand (does not require billing accounts / signup on providers),you can create a single or a cluster of instances after you run the `build-image`.

The command `… hyperkit run-instance` will use hyperkit plugin to create a single guest vm that boots from the image you built with `build-image`.

infrakit playbook linuxkit demo-sshd hyerpkit run-instance

This command brings up the LinuxKitOS instance.

Now you can run the below command to enter into SH shell

docker run --rm -ti -v ~/.ssh:/root/.ssh infrakit/ssh /bin/sh

[ A Special Thanks to David Chung, Docker Team for assisting me understand LinuxKit Playbook thoroughly.]

In the future blog post, I will show you how LinuxKit + Infrakit + GCP works together.

Did you find this blog helpful? Feel free to share your experience. Get in touch @ajeetsraina.

Last week I read a research paper called “Computer Immunology” written by a computer scientist Mark Burgess(Founder of CfEngine). Thanks to Solomon Hykes for referring about it in one of YC (Start-up Incubator Forum). I was heavily inspired by the idea of its writing which,as an analogy to Human Immune System, think about how systems, not just servers but whole systems, could be self-healing and the different scales from the small parts to the large scales. Though the research paper was published around 20 years ago, but interestingly the concept is getting rediscovered in the recent past when we talk about Cloud Immutable Infrastructure Pattern. In one of his recent interview, Mark pointed out an interesting statement –

“..Self-healing of machines was a good strategy at that time. In the future though, if you look at biology, there was also another strategy which is you have sufficient redundancy if you scratch a few skin cells off your arm, you don’t bleed, you don’t die. But in this world of hundreds of servers, if you lose a server it still means a lot.

Today, we’re actually approaching a sort of biological scale which we can argue in this way and this of course is what is happening in cloud with the immutable infrastructure pattern. Don’t repair, simply cast aside and build a new one and replace it.”

Inspired by this research paper, there has been rise of several products and tools for creating and managing declarative and self-healing infrastructure. “Infrakit” is a new toolkit which has recently been open sourced by Docker Inc. in last ContainerCon Europe 2016 held in Berlin, Germany. Last year, “Swarmkit” was introduced as a declarative management toolkit for orchestrating containers which got later integrated into 1.12 as “Swarm Mode”. This year, Docker Team focused on solving another difficult problem – Managing Docker on diverse Cloud Platform like AWS & Azure.

Why Infrakit?

In the recent past, Docker Team came up with various Docker editions like Docker for Mac & Docker for AWS or Azure, trying to make Docker work out-of-the-box and making updates easier, especially Docker for Mac just works great.But infrastructure Management in terms of AWS & Azure is still difficult. Reason – AWS architecture is inherently a distributed system which contains numerous subsystems having lots of moving parts like provisioning VPCs, EC2 instances setup, IAM roles, integration with CloudFormation etc. Before you run Docker containers, you need to setup all these infrastructure management functionalities beforehand in order to provide services to your end-user.

To solve the above problem, Docker Team came up with a solution in the form of toolkit called “Infrakit”. As it name suggests, Infrakit is a toolkit to create and manage infrastructure,that is scalable, self-healing and declarative in nature. Please remember that it is not a standalone tool, it’s a component designed to be embedded in a higher-level tool. What does this mean? One can use Infrakit to provision any kind of infrastructure using a custom “flavor”. This is expected to be integrated into Docker 1.13 for built-in infrastructure management(just like the story of “SwarmKit”)

Defining Infrakit

Infrakit is defined as “Declarative”, “Self-healing” & “Scalable”. Let us spend some time in understanding what it actually means. By declarative, it means that the toolkit rely on JSON schema configuration for the user to describe the infrastructure state. JSON is used for configuration because it is composable and a widely accepted format for many infrastructure SDKs and tools. Since the system is highly component-driven, the JSON format follows simple patterns to support the composition of components.

Talking about “Self-Healing” aspect, it is different from other traditional configuration management tools(like Puppet, CFEngine). Infrakit comprises of whole network sets of active process(called as plugins) which collaborate with each other to analyze and perform action to bring infrastructure in the desired state.These processes are always busy in continuously monitoring the infrastructure to take any kind of state divergence. Based on the state divergence, it decides on the necessary actions. It is continuously monitoring and reconciling the actual state against the specification. This is also valid for rolling updates which is seamless without disturbing the running services.

How does Infrakit Plugins Work?

InfraKit uses Plugins, which are active processes and the only building block to manage infrastructure. Technically, a Plugin is an HTTP server with a well-defined API, listening on a UNIX socket.Currently there are three kind of plugins (also called as Utilities) supported by Infrakit which represents different layer of abstraction representing whole stack of infrastructure, namely – Flavors, Group and Instance.

Group: InfraKit provides primitives to manage Groups plugins which are for managing cluster of machines. These clusters comprises of either cattle (machines with identical configs) or pets(machines with different configs)

Flavors: A Flavor Plugin can be thought of as defining what runs on an Instance. It basically helps in distinguishing members of one group from another by describing how these members should be treated. Flavors allow a group of instances to have different characteristics. In a group of cattle, all members are treated identically and individual members do not have strong identity. In a group of pets, however, the members may require special handling and demand stronger notions of identity and state.

Instance: Instances are members of a group. An instance Plugin manages some physical resource instances. It knows only about individual instances and nothing about Groups. Instance is technically defined by the plugin, and need not be a physical machine at all.For compute, for example, instances can be VM instances of identical spec.

To test drive Infrakit in just 5 minutes, I created a Docker image “ajeetraina/infrakit” which is a Ubuntu 16.04 based image with required GO package and build system ready to use. Let’s use this Docker image to illustrate how Infrakit performs self-healing operation.

Login into one of your Docker host machine and run the container as shown below:

First, we will execute the below command to start the File instance plugin:

Next, we will start a group plugin in the new terminal as shown below:

Then, it’s time to start one of the provided flavor plugins called ‘vanilla’, which will be used to create a group.

This Docker image is already equipped with this file under collabnix directory under Infrakit base directory. Hence, the below commands should work great as expected.

By now, you must see the below command showing the right output:

Let’s execute the below command to keep a close watch over collabnix group.

Under Infrakit-group-plugin, you will find some actions for group “collabnix” where 5 instances of dockercloud/hello-world is ready.

Also, you can inspect “collabnix” group as shown below:

Cool, Isn’t it?

Let’s test the “Self-Healing” aspect now. As we see below, there are number of instances running.

Let us remove few of those instances to see if it actually spun up well:

Wow ! Self-Healing aspect just worked well. Meanwhile, we can check the output at group instance:

This is just one operation with Infrakit. Infrakit is currently under active development and there are lots of further exciting stuffs coming up. In the future post, I am going to see how it integrates under Swarm Mode. Keep Reading !