Working to keep your digital experiences secure

A First Experiment with Chef

Security professionals are now using cloud solutions to manage large-scale security issues in the production Ops workflow. In order to achieve scale, the security processes are automated through tools such as Chef. However, you can also leverage the automation aspects of the cloud regardless of whether or not you plan to scale it to hundreds of machines. For example, one of my first cloud projects was to leverage Chef to resolve a personal, small scale issue where I might have previously used a VM or separate machine.

Using the cloud for personal DevOps

At Adobe, we’re constantly hiring third party security consultants to test our products. They require environments for building and testing Adobe code, and they need access to tools like Cygwin, Wireshark, SysInternals, WinDbg, etc. For my personal testing, I also require access to machines with a similar set up. Using the cloud, it is possible to quickly spin up and destroy these types of security testing environments as needed.

For this project, I used Adobe’s private cloud infrastructure – which is just an implementation detail – this approach can work on any cloud infrastructure. Our IT department also provides an internal Chef server for hosting cookbooks and managing roles.

In order to connect the Chef server and the cloud environment, I decided to set up a web server which would copy the Chef config files to the cloud instance and launch chef-client. For this, I chose Windows IIS, C# and ASP.NET because I had an unrelated goal of learning more about WMI and PowerShell through a coding project. Vagrant for Windows would be an alternative to this approach but it wasn’t available when I began the project. My personal Linux box was used to write the Chef recipes and upload the cookbook to the Chef server.

The workflow of the set-up process is as follows:

1. Request a new Windows 7 64-bit instance from the cloud and wait for the response saying it is ready.

2. The user provides the domain name and administrator password for the Windows 7 instance to the web page. If I need to set up additional accounts on the machine, I can also collect those here and run command lines over WMI to create them.

3. Using the admin credentials, the web server issues a series of remote WMI calls to create a PowerShell script on the Windows 7 instance. WMI doesn’t allow you to directly create files on remote machines, so I had to use “echo” and redirect the output.

4. By default, Windows doesn’t allow you to run unsigned PowerShell scripts. Although, with admin privileges you can use a command line to disable the signature check before executing the script. Once the script was done, the signature check is re-enabled.

5. The PowerShell script will download the client.rb file and validator.pem key needed to register the Windows 7 instance with the chef server.

6. WMI can then be used to run the chef-client command line which will register the new Windows 7 instance with the chef-server.

7. Since chef requires a privileged user to assign roles to the Win7 node, a separate chef key and a copy of the knife utility are stored locally on the web server. The C# server code will execute knife using the privileged key to assign the appropriate role to the new Windows 7 node in Chef.

8. Lastly, the web server uses WMI to run chef-client on the Windows 7 instance a second time so that the recipes are executed. The last chef recipe in the list will create a finished.txt file on the file system that the web server can verify that the process is completed.

Lessons Learned from Writing the Recipes

Using Chef, I was able to install common tools such as WinDbg, Cygwin (with custom selected utilities), SysInternals, Wireshark, Chrome, etc. Chef can be used to execute VBScript which allowed me to accomplish tasks such as running Windows Update during the setup process. For most recipes, the Opscode Windows cookbook made writing the recipes fairly straightforward.

Some of the installers will require a little legwork. For instance, you may have to search the web for the command line flags necessary for silent install. If you cannot automatically download an installer from the web, then the installers can be stored on the Chef server along with the recipes. For Wireshark, it was necessary to download and install winpcap before downloading and installing Wireshark. One client installer was not Chef-friendly, which lead to the creation of a Chef script that would have Windows Task Scheduler install the software instead.

As a specific example, Cygwin can be one of the more complicated installs because it is a four-step process. There are some public cygwin recipes available for more advanced installs but this is enough to get it installed just using the Opscode Windows cookbook. For a robust deployment, additional code can be added to ensure that it doesn’t get run twice if Cygwin is already installed. The demo workflow below uses windows_package to download the setup executable. The windows_batch commands will then run the executable to install the default configuration, download additional utilities and then install the additional utilities:

There are definitely alternative approaches such as using a baseline VM snapshot or a pre-built AMI with all of these tools installed. They each have their own pros and cons. For instance, local VMs launch faster but the scaling is limited by disk space. I chose Chef recipes because it provides the flexibility to create custom builds and ensures that everything is current. If needed, the system is able to scale quickly. The extra work in writing the server meant that I could make the process available to other team members.

Overall, despite having to be creative with some recipes, the project didn’t require a huge investment of time. A large portion was written while killing time in airports. The fact that I was able to go from a limited amount of knowledge to a working implementation fairly quickly speaks to the ease of Chef. If you are interested in learning more about the cloud, you don’t need a large, massively scalable project in order to get your hands dirty. It is possible to start with simple problems like creating disposable workstations.