Topics

Featured in Development

Peter Alvaro talks about the reasons one should engage in language design and why many of us would (or should) do something so perverse as to design a language that no one will ever use. He shares some of the extreme and sometimes obnoxious opinions that guided his design process.

Featured in AI, ML & Data Engineering

Today on The InfoQ Podcast, Wes talks with Katharine Jarmul about privacy and fairness in machine learning algorithms. Jarul discusses what’s meant by Ethical Machine Learning and some things to consider when working towards achieving fairness. Jarmul is the co-founder at KIProtect a machine learning security and privacy firm based in Germany and is one of the three keynote speakers at QCon.ai.

Featured in Culture & Methods

Organizations struggle to scale their agility. While every organization is different, common patterns explain the major challenges that most organizations face: organizational design, trying to copy others, “one-size-fits-all” scaling, scaling in siloes, and neglecting engineering practices. This article explains why, what to do about it, and how the three leading scaling frameworks compare.

At the recent KubeCon and CloudNativeCon North America events, held in Seattle, USA, a series of updates were provided about the Cloud Native Computing Foundation (CNCF) hosted projects. Highlights included the release of Kubernetes 1.13, an overview of the progress of the Envoy Proxy project (and a discussion of its increasing ubiquity in the cloud native stack), and the inclusion of the Rook storage orchestration and Harbor image registry projects.

The event co-chairs, Liz Rice, technology evangelist at Aqua, and Janet Kuo, software engineer at Google, opened the event with a keynote summarising the latest developments within the cloud native ecosystem. According to the CNCF, cloud native technologies empower organisations to build and run "scalable applications in modern, dynamic environments such as public, private and hybrid clouds".

The CNCF aims to provide "open source components of a full stack cloud native environment", and it does this by hosting a range of projects at various "maturity levels" -- sandbox, incubating and graduated -- that are inspired by Geoffrey Moore's "Crossing the Chasm" (which the InfoQ team also use for curating topics). The CNCF provides facilities such as a neutral home for projects, funding for improving documentation, knowledge share and conference facilitation, marketing, legal services and access to expertise from industry luminaries and end user communities.

Building on the successful launch of Kubernetes 1.12, version 1.13 was released in tandem with the event. New functionality included: simplified Kubernetes cluster management with kubeadm in GA; the Container Storage Interface (CSI) is now GA; CoreDNS is now replacing kube-dns as the default DNS server for Kubernetes; and several other small enhancements.

The introduction of kubeadm in GA will make operational work with Kubernetes easier. kubeadm handles the bootstrapping of production clusters on existing hardware and can configure the core Kubernetes components in a "best-practice-manner" to provide "a secure yet easy joining flow for new nodes and supporting easy upgrades". Further improving cluster management, "kubectl diff" has also graduated to beta, which allows users to run a kubectl command to view the difference between a locally declared object configuration and the current state of a live object.

After being announced as a GA release in Kubernetes 1.11, CoreDNS is now replacing kube-dns as the default DNS server for Kubernetes. CoreDNS is a general-purpose DNS server that provides an extensive (but backwards-compatible) integration with Kubernetes. The release blog post states that CoreDNS has "fewer moving parts than the previous DNS server, since it's a single executable and a single process, and supports flexible use cases by creating custom DNS entries".

Turning attention to other CNCF hosted projects, Rice discussed that the Prometheus monitoring and metric collection platform had become a "graduated" project this year. As reported previously on InfoQ, there is also a lot of associated activity within this space, with Grafana adding log data correlation to time series metrics. Fluentd, the open source data collector for unified logging, also saw additional support for Splunk and Amazon Kinesis added. In regard to distributed tracing, Open Tracing saw Lua support added, and the Jaegar platform saw the addition of a "Jaeger Operator" to reduce the operational overhead of running this on Kubernetes.

In regard to cloud native storage, the Rook project became hosted at the incubating level. Rook is an open source cloud-native storage orchestrator for Kubernetes, providing the "platform, framework, and support for a diverse set of storage solutions to natively integrate with cloud-native environments". Vitess, a database clustering system for horizontal scaling of MySQL through the use of generalized sharding outside of application logic, saw the release of v3, and included functionality such as VReplication, Promethues monitoring integration, and a series of performance enhancements.

In miscellaneous news, the NATS project, a "simple, high performance open source messaging system" for cloud native applications, saw the inclusion of secure multi-tenancy and network topology optimisations over the past year. The Harbor project, "an open source trusted cloud native registry project that stores, signs, and scans content" was also introduced as a incubating project.

Community and commercial interest within the cloud native ecosystems and the CNCF hosted project has clearly grown over the past two years. Approximately 8000 attendees were present at this latest installment of the conference (and related training and workshop days); a figure that has grown significantly since the event was held in Seattle in 2016, which attracted just over 1000 attendees. There are also currently 160+ CNCF meetups being hosted globally, and this year $300,000 in diversity scholarships was raised to enable attendees from diverse and minority background to make the journey to Seattle for the event.