Orchestrates AWS cloud resources using boto and pychef

WARNING! This plugin is in alpha.

I recommend you not use this plugin yet on production instances. Although you likely won't lose ec2 instances or chef data, it's still quite immature. It would be best to try out on dev instances to help test the plugin and generate ideas for new features.

Description

This plugin is meant to fill the coordination gap between AWS and Chef, especially:

Launch an ec2 instance and apply one or more chef roles

Terminate ec2 instances and delete its chef node

There are additional features and quite possibly more to come, but my intention is to keep the scope down to what's needed to orchestrate between AWS and Chef (and not re-implement either the AWS Management Console or the Chef Server webui).

This plugin relies heavily upon ​boto and ​pychef. Special thanks to coderanger for his work and responsiveness on pychef.

Chef configuration

The plugin will use this file for the instance data replacing the role information as needed. As the file contains the validation.pem data, be sure to secure it accordingly. In cases where you need different credentials and/or chefserver url, you can use the following option to specify a base directory which must contain the typical .chef directory and its contents plus (optionally) a validation.pem file within the .chef directory:

[cloud]
..
chef_base_path = /path/to/dir/that/contains/.chef

It's currently assumed that you're using one of Opscode's AMIs or a homegrown AMI derived from it. These AMIs use the instance's id as the node name. However, you may have recipes which for one reason or another change the node's name after the first chef-client run. If so, you need to specify the boto field that maps to the final node name using this config option:

[cloud]
..
boto_field_node_name = private_dns_name

Adding this option will cause the plugin to create a new node of that name upon launching a new ec2 instance and once it checks in will remove the original node. Otherwise you'll have two chef nodes for the same ec2 instance - something you likely don't want. Yeah, this is a bit complicated - an area ripe for improvement.

The plugin is intended to support multiple AWS resource types in addition to instances as needed for orchestration with chef. You specify the default resource to view using this config option:

[cloud]
..
default_resource = instance

Each AWS resource type such as instance above is configured independently. See the examples section below for how to specify these resources.

Whoa! The fields option is a list of those chef attributes (aka fields) where you want to provide either a display name, a format handler, or both. So for example, ec2.instance_type as Instance Type above will use 'Instance Type' wherever the name of the ec2.instance_type field is to be displayed.

Only slightly more complicated are handlers to format fields. So for example, ohai_time_ as Last Check-in < AgoEpochHandler has a display name as we've seen plus the < AgoEpochHandler directs the plugin to use a specific handler to format the ohai_time field's value. In this example, the AgoEpochHandler handler converts an epoch to a string format such as "0:08:36 ago" meaning "8 minutes and 36 seconds ago" as shown in the screenshot above. But wait, what's up with the underscore after the ohai_time_? In most cases when using a handler, you should postfix the field's name with an underscore (_) so that the re-formatted value doesn't get written back to the field's value. [I'm likely to get rid of the underscore business to make it simpler, but that's what it is for now. - rhg] See the handlers.py file for a full list of provided field handlers.

crud_resource - must map to the chef resource name. For ec2 instances, this is nodes.

crud_view - the list of chef attributes to display in order. The field names will use any display names from fields and its cell values will be formatted using any handlers from fields.

crud_new and crud_edit - the list of augmented chef attributes to display for editing in order. The field names will use any display names from fields and its cell values will be formatted using any handlers from fields.

The field names in crud_new and crud_edit are augmented to provide additional metadata so the plugin knows how to properly configure the form. So for example:

The pipe (|) after the field name indicates that the form should use a select/dropdown menu for this field. If the select/dropdown should support multiple selections, then use two pipes instead of one as in run_list_||. The remainder of the definition is a pipe-delimited list of options for this field (much like how custom field options are defined). If there's no list of options, then the plugin will search a chefserver data bag of the same name as the field for the list of options. So for example, ec2.instance_type| will direct the plugin to search for a data bag on the chefserver named ec2_instance_type (periods are invalid and so are replaced with underscores). The individual data bag items should be formatted like this:

The name will be used as the option's displayed name and the value will be used as the option's value. If order fields are provided, then they'll be used to order the options accordingly (much like custom field ordering).

Lastly, if you want to have a field displayed but as read-only, then the field name should be immediately postfixed with an asterisk as in ec2.ami_id* above.