Menu

When reading & learning about Linux containers (cgroups), Docker containers, and containers in general, it's quick to realise that just like null packets, in concept "they are perfect in their simplicity and they are very true".

Yet, achieving simplicity is hard damn it and at the same time just like "Many experts naively confuse the Null Packet with an Imaginary Packet" many experts appear to confuse kernel level support for multi tenant with containers stuck inside virtual machines.

The goal of the project Crossbow is to different virtual machines share the common NIC in a fair
manner and allow system administrators to set preferential policies where necessary (e.g. the ISP
selling limited B/W on a common pipe) without any performance impact.

https://www.joyent.com/blog/scaling-docker-deployments-from-laptop-to-cloud (July 22, 2015) Summary: Containers on Linux isn't a real solution, but it has a great footprint. Lets abstract the Linux system call table in SmartOS! &: Dockers great, but because its (most commonly?) installed upon Linux it's not possible to have true privilege separation like Solaris Containers can (Linux's cgroups only go so far, so Docker suffers). 'Legacy' cloud providers get round this by wrapping containers in VMs, undoing a lot of the benefits containerisation in the first place (ie multi-tenant, bare metal performance) and the hypervisor then has to visualise the networking between vms adding further performance hits. All these plasters add latency when trying to run docker within a VM. This affects cost legacy providers charge for instances: the VMs you're cloud provider is running in order to achieve privilege separation between containers, if they built upon SmartOS / Solaris / any kernel that supports the seperations needed, there's be no need for the VM.

You're not paying for an application you're paying for instances. Look out for words like 'cluster' and 'instances' and seek clarification. A legacy provider is probably using VMs as the stop-gap for container separation.

tldr; VM-Based container providers leave your instances contained within the VM inside which they run. e.g you might have a simple LAMP stack, you buy a VM with 2 vCPUs, 7.gGB RAM and run four containers within it. The VM is your ceiling:

OK not entirely, because you can replicate your VMs, but this creates wastage: You're still paying for all the VMs you scale to, even if you only needed the extra resources for part of the service/application:

"Everything in Joyents cloud is local disk", they don't use network attached storage because of the performance bottlenecks it creates. Also, because the containers are stored on local disk directly, data persistence is easier because the container instances are not contained within VMs anymore.

From laptop to deploying to the cloud

Things to note:
Joyent does not run the docker binary, it simply implements the Docker API and translates those Docker API calls into its private api calls which provision the containers. When interacting with the Docker API on laptop -> Joyent Cloud your ssh key never gets sent to Triton, it just gets turned into (~used) as a TLS certificate for identification.

https://www.joyent.com/blog/why-container-security-is-critical (October 06, 2015) Summary: The immaturity of Linux containers is a risk, often plastered over by using multiple containers inside a VM. This adds cost and complexity tying the lifecycle of the container to the VM. Being inside a VM makes data persistence (inside containers) a challenge. With container native you get the benefits of bare metal performance, with the security of a multi-tenant kernel with over 10 years maturity.