Featured in AI, ML & Data Engineering

In this article, author shows how to use big data query and processing language U-SQL on Azure Data Lake Analytics platform. U-SQL combines the concepts and constructs both of SQL and C#. It combines the simplicity and declarative nature of SQL with the programmatic power of C# including rich types and expressions.

Featured in Culture & Methods

The book Agile Leadership in Practice - Applying Management 3.0 by Dominik Maximini is an experience report of the agile transformation journey of NovaTec. Maximini shares his experiences from applying principles and practices from Management 3.0, success stories, failure stories, and learnings from experiments.

Featured in DevOps

Yuri Shkuro presents a methodology that uses data mining to learn the typical behavior of the system from massive amounts of distributed traces, compares it with pathological behavior during outages, and uses complexity reduction and intuitive visualizations to guide the user towards actionable insights about the root cause of the outages.

Docker 1.8 Release with Multiple New Tools

Docker inc. have announced the release of Docker 1.8, which brings with it some new and updated tools in addition to new engine features. Docker Toolbox provides a packaged system aiming to be, ‘the fastest way to get up and running with a Docker development environment’, and replaces Boot2Docker. The most significant change to Docker Engine is Docker Content Trust, which provides image signing and verification.

Docker Toolbox is offered for Windows and Mac users and bundles together:

VirtualBox to run a lightweight Linux VM containing the Docker Engine.

Docker Machine, a provisioning tool that installs Docker onto VirtualBox (and that can also install Docker on multiple clouds).

The Mac version also includes the Docker Compose compositing tool (which comes from their acquisition of Orchard Labs for Fig) to help with multi container applications.

The installation instructions provide migration guidance for users already using Boot2Docker, or who already have VirtualBox installed. Using Docker on other Windows and Mac virtualisation environments such as Hyper-V, Parallels or VMware’s Workstation or Fusion platforms isn’t covered (and the potential clash with these isn’t mentioned). Windows 10 isn’t yet supported.

Docker Content Trust is based on the Notary project, which is an ‘opinionated implementation’ of The Update Framework (TUF), a ‘flexible security framework that can be added to software updaters’. It provides secure distribution of images over insecure mediums, by the use of signatures. It also gives freshness guarantees, so that somebody downloading an image knows that they’re not getting an older version that may contain known vulnerabilities. The system makes use of three key types: a root of content trust key that’s intended to be kept offline, a tagging key that’s created per image repository, and timestamp keys. The offline and tagging keys are automatically generated the first time that an image is pushed with Content Trust enabled (which is presently controlled by an environment variable). The system isn’t yet fully integrated into Docker Hub, so automated builds can’t be signed. The library images in Docker Hub (e.g. Ubuntu) are now signed, and it’s also possible for signed user generated images to be pushed to Docker Hub. Docker Inc’s David Lawrence provided a demo of Content Trust during an online meetup that was recorded.

The updated Docker Engine moves support for volume plugins from ‘experimental’ to ‘stable’ meaning that storage can be integrated with offerings from ecosystem suppliers such as Blockbridge, Ceph, ClusterHQ, EMC and Portworx. There are also additions to the logging drivers, with Graylog Extended Log Format (GELF) and Fluentd joining syslogd (which has been supported since 1.6). Due to an issue with pushing images Docker 1.8.1 has already been released to replace Docker 1.8.0.

Since the docker binary works as both a client, and as the daemon, some effort has been expended to separate the two roles and clarify their respective documentation. Most noticeably the ‘-d’ command line switch has been deprecated in favour of ‘daemon’ with the latter also getting its own help. It’s also now possible to use ‘docker cp’ to copy files from the host to a container (in addition to the previously supported copy from container to host).

For Linux users there are new apt repositories for Debian and Ubuntu and yum repositories for RHEL, CentOS and Fedora. In both cases it’s now ‘docker-engine’ that’s installed rather than ‘lxc-docker’, which was considered confusing since Docker dropped LXC as its default execution driver 16 months ago. Switching to the new repos does however require some care, as customisations made to Docker configuration files may be lost, and the ‘docker’ command might not be present when needed to list running containers.

Docker’s orchestration tools have also been upgraded, with Compose moving to version 1.4 and Machine and Swarm both being bumped to 0.4. There are a number of functional improvements in addition to those required to make Toolbox work, some of which will appeal to those running Docker at scale with schedulers such as Mesos. Version 2.1 of the Docker Registry has been released providing faster operation and some new features.

The overall themes for this release appear to be security and ease of use. The former is likely targeted at large enterprise users, whilst the later is aimed at getting Docker into the hands of more developers. Docker will be finding its way to Windows Server 2016, where it's already in technical preview, so Docker Inc are no doubt setting their sights now on the Windows user base.

Community comments

Docker Engine for Windows

Your message is awaiting moderation. Thank you for participating in the discussion.

Another exciting news this week was Microsoft's preview of Docker Engine support for Windows Server 2016, and the launch of WinDocks a third party Docker Engine for Windows Server 2012. These solutions open up the future for combined Windows and Linux clouds, with common tooling and application packaging and APIs. Check out WinDocks at: windocks