Introducing New Puppet Language Constructs

Breadcrumb

With the release of Puppet Enterprise 2015.3, we’ve added two new mechanisms for managing infrastructure with Puppet: the environment graph and modeling applications. We’ve also introduced the new orchestrator service, which takes advantage of these to help you directly make change to any part of your infrastructure, including your applications.

This post aims to describe these new capabilities, while follow-up posts will demonstrate various ways they can be used to manage infrastructure.

The Environment Graph

This first addition is the foundation for our new capabilities, and is fundamental to so many things we’re working on, some of which were discussed at the two PuppetConf keynotes:

Puppet has a concept of environments, which is useful for a variety of tasks. Many people use environments primarily as a mechanism to isolate groups of Puppet agents for the purpose of testing changes to their Puppet code before implementing a change on every machine they manage. An agent requests instructions from a specific environment on the Puppet Server and receives a catalog containing information about configuration and the relationships between configurations for the individual node.

Now environments have a catalog to call their own, describing the nodes being managed in an environment along with their relationships to other nodes where applicable. We call this the environment graph or catalog.

Taken from David Lutterkort’s Application Orchestration Overview session at PuppetConf 2015 (video), this example demonstrates 18 managed Puppet nodes that all have a relationship to at least one other node, but that are not necessary to represent in the environment catalog. With this information, the new Puppet Orchestrator service knows about every node you might want to deploy a change to, and how a change involving multiple nodes should be coordinated. But how do Puppet and the Orchestrator know about the relationships between nodes?

Application Language and Environment-Level Resources

A collection of additions to the Puppet language enables you to model configurations at a higher level than individual nodes. This higher-level modeling is called application definition. When you define an application, you focus on the functional parts of the application, like its database, app servers or load balancer, without having to think about which nodes each of these application components will ultimately run on. Application definitions therefore serve as a blueprint for stamping out different instances of your application.

There’s more about this in the documentation, but briefly, the application definition is built from lower-level Puppet code that describes the parts you want to manage, like the database. If you’re already familiar with the roles and profiles pattern, these application components are a lot like what you declare in profiles. Here's an example.

In the code below (taken from the documentation), you'll find the lamp::db definition declared inside the application definition. The lamp::db definition creates a database with help from the puppetlabs-mysql module.

This application definition is where distributed infrastructure comes together. In this case, the example declares defined resource types that configure a database, a web server, and a PHP application. The key to being able to manage this application as a piece of infrastructure is exposing relationships between components and the information they exchange to interoperate.

Instead of describing a relationship between component A and component B, relationships are expressed between a component and another new language feature, environment-wide service resources. These environment resources serve double duty. They form relationships between nodes that manage application components and they’re a vehicle for exchanging information between those nodes as necessary. You can learn more about these in the documentation, but environment resources can be very simple, expressing only a type with data parameters.

Puppet Orchestrator

So far, the new additions are all about describing your infrastructure at the environment level in addition to the node level. But we’ve also added a few tools for taking action at this higher level. Primarily, those are the orchestrator service and the puppet job command-line tool.

The orchestrator is a consumer of the environment graph mentioned earlier. With this information, the orchestrator is equipped to work with everything you manage in an environment, and it’s aware of all the relationships you’ve expressed. Paired with the puppet job command-line tool, you can use the orchestrator to directly enforce any configuration change or application deployment you like. It uses the new Puppet Communications Protocol under the hood to manage standard Puppet agent runs, ensuring that runs occur in the appropriate order and complete successfully before downstream hosts are reconfigured.

Our next post will demonstrate the use of all of this to deploy a simple application.