Topics

Featured in Development

As part of our core values of sharing knowledge, the InfoQ editors were keen to capture and share our book and article recommendations for 2018, so that others can benefit from this too. In this second part we are sharing the final batch of recommendations

Featured in Architecture & Design

Tanya Reilly discusses her research into how the fire code evolved in New York and draws on some of the parallels she sees in software. Along the way, she discusses what it means to be an SRE, what effective aspects of the role might look like, and her opinions on what we as an industry should be doing to prevent disasters.

Featured in Culture & Methods

Mik Kersten has published a book, Project to Product, in which he describes a framework for delivering products in the age of software. Drawing on research and experience with many organisations across a wide range of industries, he presents the Flow Framework™ as a way for organisations to adapt their product delivery to the speed of the market.

Featured in DevOps

The fact that machine learning development focuses on hyperparameter tuning and data pipelines does not mean that we need to reinvent the wheel or look for a completely new way. According to Thiago de Faria, DevOps lays a strong foundation: culture change to support experimentation, continuous evaluation, sharing, abstraction layers, observability, and working in products and services.

Lessons on Building Continuous Delivery for Infrastructure

Lindsay Holmwood recently recounted his learning experiences on building Continuous Delivery (CD) for infrastructure services. Holmwood offers advice to enable fast feedback loops, notably of Service Level Agreements (SLA) compliance, and to support small, discrete changes. Holmwood is the creator of Flapjack, a monitoring notification routing and event processing system.

Holmwood defines infrastructure services as "a configuration of software and data that is consumed by other software". This includes, for instance, DNS, databases or monitoring services.

Holmwood asserts that to get quality, not just fast, feedback there are five main issues to think about. Start with the CAP (Consistency, Availability, Partition Tolerance) theorem. Decide if the services you're building should go for consistency or availability. Examples of the former may include databases, of the latter, load-balanced servers. Once you've made that choice, it's time to agree with the infrastructure users the SLAs to commit to. For instance, "95 percentile response time for monitoring metric queries in a one hour window is < 1 second". Third, use your CD pipeline to check your SLA's, whenever possible. You must also have clear, well-defined, interfaces between infrastructure services. Those clear interfaces help to verify interactions between parts, both during testing and normal operation. Finally, Holmwood argues for pushing data to one place to enable immutable infrastructure.

On fast feedback, Holmwood and his team found that placing the continuous integration (CI) system near the IaaS provider (be it on-premises or in the cloud) is crucial to reduce latency times. Holmwood cites a project where the build time reduced from twenty to five minutes, just by moving the CI close to the IaaS API.

If your team mostly does its job through the command-line, context switching to and from the CI web interface can be an user experience blocker. To ensure that everyone complies with the rule of pushing every change through the CD pipeline, Holmwood suggests tools such as jenkins-cli. The build process can then be followed from the command-line:

git push && jenkins start $job && jenkins tail $job

You can even graph some data in the terminal with the help of gnuplot.

Fast feedback requires small, discrete changes. This means the tests must execute fast. Holmwood advises to make each test run in under ten seconds. Given we're in an infrastructure context, making tests run fast also means compromises. So Holmwood automates tests that look for unmistakable problems, such as checking for running services and executing simple queries. Monitoring checks complement the CD pipeline tests to verify end-to-end behaviour.

It should be noted that Continuous Delivery for infrastructure is getting more vendor support over time. Hashicorp's Atlas and the forthcoming Chef Delivery are examples of these developments.