article

OpenStack Heat Template Composition

Heat Orchestration Templates (HOT) for OpenStack's Heat service can quickly grow in length as users need to pile in ever more resources to define their applications. A simple Nova instance requires a small volume at first. Soon it needs a private network, a public network, a software configuration deployment, a load balancer, a deluxe karaoke machine. This means the HOT files get bloated and difficult to read for mere humans. Let's think about why that is.... A template defines the set of resources necessary and sufficient to describe an application. It also describes the dependencies that exist between the resources, if any. That way, Heat can manage the life-cycle of the application without you having to worry about it. Often, the resources in a long template need to be visually grouped together to alert the reader that these things depend on one another and share some common goal (e.g. create a network, subnet, and router). When templates get to this state, we need to start thinking about composition.

The Parent and Child Templates

We'll often refer to a parent template and child template here. The parent template is the user's view into the Heat stack that defines our application. It's the entrypoint. A parent template contains all of the over-arching components of the application. The child on the other hand may describe a logically grouped chunk of that application. For example, one child template creates the Nova instances that launch BIG-IP devices while another child template creates all the network plumbing for those instances.

Now that's still fairly verbose, but its doing some heavy lifting for us. It is creating almost all of the networking needed to launch a set of clusterable BIG-IP devices and a single standalone BIG-IP device. It takes in parameters from the user such as ve_standalone_image and external_network and passes them along to the child that requires them. The child stack then receives those parameters in the same way the template above does, by defining a parameter.

The parent template references the heat_template_test_networks.yaml template directly, expecting the file to be in the same local directory where the parent template is located. This is always created in the type field of a resource. In addition to relative paths, you can also reference another template with an absolute path, or URL.

You can also see the group of responsibilities here. One child stack (heat_template_test_networks.yaml) is building networks, another (cluster_ready_ve_4_nic.yaml) is launching a set of BIG-IP devices ready for clustering and yet another (f5_ve_standalone_4_nic.yaml) is launching a standalone BIG-IP device. Yet the dependencies are apparent in the depends_on property and the intrinsic functions called within that resource (more on that later). You will not successfully launch the standalone device without having the networking in place first, thus the standalone_device resource is dependent upon the networking resource. This means we can easily send data into the networking stack (as a parameter) and now we must get data out to be passed into the standalone_device stack. Let's look at the networking template and see what it defines as its outputs.

You can see the logical grouping here, and this is where template composition shines. This simple template creates a security group, four networks, four subnets, and ties them together with a router. Even though the heat_template_test.yaml parent template uses this to build its networks for defining its application, another user may decide they want the same networking infrastucture, but they want four standalone devices and eight pairs of clusterable devices. Their only modification would be in the parent template, because the heat_template_test_networks.yaml template describes the set of networks those devices need to connect to. It is important to note that the above template is a fully functioning HOT template all by itself. You can launch it and it will build those four networks.

So how does the data get out of the networking template? The outputs section takes care of that. For attaching BIG-IP devices in the parent template to these networks, all we require is the network name, so the networking template kicks those back up to whomever is curious about such things. We saw the get_param function earlier, and now we can see the use of the get_attr function. In the two_bigip_devices resource, the parent template references the networking resource directly and then it accesses the client_data_network_name attribute (as seen below). This operation retrieves the network name for the client_data_network and passes it into the cluster_ready_ve_4_nic.yaml stack.

network_1: { get_attr: [networking, client_data_network_name] }

With that, we've successfully sent information into a child stack, retrieved it, then sent it into another child stack. This is very useful when working in large groups of users because my heat_template_test_networks.yaml template may benefit others. In time, you can build quite a collection of these concise HOT templates then use a parent template to orchestrate them in many complex ways. Keep in mind however, that HOT is declarative, meaning there are no looping constructs or if/else decisions to decide whether to create seven networks or four. For that, you would need to create two separate templates. As seen in the OS::Heat::ResourceGroup resource for two_bigip_devices however, we can toggle the number of instances launched by that resource at stack creation time. We can simply make the count property of that resource a parameter and the user can decide how many clusterable BIG-IP devices should be launched.

New Resource Types in Environment Files

The second way to do template composition is to define a brand new resource type in an environment file. The environment in which a Heat stack is launched affects the behavior of that stack. To learn more about environment files, check out the official documentation. We will use them here to define a new resource type. The cool thing about environments is that the child templates looks exactly the same, and only one small change is needed in the parent template. It is the environment in which those stacks launch that changes. Below we are defining a Heat environment file and defining a new resource type in the attribute_registry section.

The parent template will now reference the new resource types, which are merely mappings to the child templates. This means the parent template has three new resources to use which are not a part of the standard OpenStack Resource Types. The environment makes all this possible.

And here is how we utilize those new resources defined in our environment file. Note that we no longer define all the parameters in the cli call to the Heat client (with the -P flag) because it is set in our environment file.

In my experience Openstack tenant instances that source or receive traffic from/to IPs not tied to a neutron net/port are denied by the default port security. This is the case of traffic from/to VS in LTM. To work around this I usually disable port security when creating the network: