Posts on Cloud,DevOps, Citrix,VMware and others. Also tracking my Continuous learning from Wintel to open source and development.
Words and views are my own and do not reflect on my companies views.
Disclaimer: some of the links on this site are affiliate links, if you click on them and make a purchase, I make a commission.

Saturday, August 4, 2018

Automating Cloud Compliance with AWS OpsWorks for Chef Automate

Chef, InSpec, and Habitat provide workflows for automating everything you manage, from infrastructure configuration to security auditing to application releases. Chef Automate ties these projects together with a powerful dashboard that aggregates data from all of your environments and provides a shared world view of the configuration and security of your estate for everyone within your organization. AWS's OpsWorks for Chef Automate makes it easier than ever to get started with your own Chef Automate server, with push-button installation and AWS-managed backups and updates. Recently, I had the opportunity to host a webinar with Jonathan Weiss, Senior Manager of AWS OpsWorks, to show off just how easy it is to get started quickly with OpsWorks for Chef Automate. Take a look!

Detecting Issues with InSpec Scan Jobs

I split my own demo portions into a few parts. The first (19:40) focused on detecting issues with InSpec. There are a few ways to initiate an InSpec scan, but one of the easiest is via the "scan jobs" feature of Chef Automate which is what we showed off in the webinar (21:55). Once you have a Chef Automate server, all you need to follow along is a target node to scan, and the ability to connect to it over SSH or WinRM. Once you've identified a good target, you can scan it by following these steps:

Install an InSpec Profile: The easiest way to do this is via the Profile Store in Chef Automate. In our webinar, we used the Linux Security Baseline profile, and there is a corresponding Windows profile as well.

Create a login credential: Before we can scan a node, we'll need to tell Chef Automate how to connect to it by creating a credential, providing a username and password or SSH key with which to connect.

Create a Node & Scan Job: Once your profile and credential are in place, you can add nodes to scan and create jobs using your profile(s) via Chef Automate's scanner. In the webinar, we created a one-time scan, but you also have the option to set the scan to recur regularly for continuous evaluation of your security and compliance.

Correcting Issues with Chef

When I scanned my nodes during the webinar (25:45), I found that they had some security issues that needed to be remediated. Just as InSpec can tell us whether our systems are securely configured, Chef can remediate any configuration issues we encounter. The Security Baseline profile we used has a corresponding os-hardening cookbook that can be used to harden our configuration according to the rules laid out in the associated profile. To remediate our servers, I made use of a new feature in Chef Workstation called chef-run. Chef-run allows us to perform ad-hoc configuration tasks by executing local chef resources or recipes against remote targets over SSH or WinRM. This provides us a simple method for quickly configuring single machines, or groups of machines in parallel, with a single command:

That's it! Chef-run will ensure that the chef client is installed, and execute whichever recipes or resources we've provided. In the webinar, however, I also had my results sent to my Chef Automate server, which can be done in a few simple steps:

Find your Data Collection Token: A data collection token is used to authenticate with Chef Automate when sending client data. To find your Automate Server's token, log into your OpsWorks for Chef Automate instance, and run the following command as root:

automate-ctl show-config

This will display a JSON hash of the server's configuration, and the string following token: is what we'll want. Make a note of it for the next step.

Update your config.toml: Chef Workstation has a config.toml file used for any optional configuration paramaters for using chef-run, located in ~/.chef-workstation/config.toml. To configure chef-run to send its data to Chef Automate, add the following lines to that file:

That's it! Now when you run a chef-run command, the results should show up in the "nodes" view within Chef Automate!

Extra Notes: Correcting Issues

As with audits, there are a number of ways to execute Chef on machines. Chef-run is great for ad-hoc tasks, but to configure nodes for regular configuration with Chef, you'll likely want to formally bootstrap them. Again, AWS provides a great guide for bootstrapping your nodes with knife (included in Chef Workstation).

Automatically Bootstrapping Autoscaled Nodes

One of Amazon's most popular cloud features is AWS Auto Scaling, wherein you can define groups of EC2 instances that will automatically add or remove nodes based on metrics like CPU load or network traffic. With OpsWorks for Chef Automate, you can easily define autoscaling groups that will be automatically bootstrapped and managed by Chef. In the webinar (35:00) we did exactly this, making use of the OpsWorks CM API. When you download a starter kit from OpsWorks for Chef Automate, it includes a custom User Data file that will handle this for us with minimal modification. A full overview of the process we used in the demo can be found in the Add Nodes (to Chef Automate) Automatically docs on AWS. This will walk you through everything from IAM profile creation to how to configure your launch configuration for unattended bootstrapping of autoscaled nodes.

Extra Notes: Auto Scaling

One thing that's worth noting about my demo environment was that my configuration targets were running Ubuntu, which requires a few extra things in the autogenerated user data file in your starter kit. Most notably, the stock Ubuntu AMIs don't have zip or python pre-installed, which we'll need to configure things. If you open up userdata.sh in your editor of choice, you'll want to start by adding a function to install these components:

Get a taste for Chef on AWS with these hands-on, guided tutorials. Manage your first node, test Chef cookbooks on temporary cloud instances, and deploy changes to a production-like environment, all on AWS.