While not a regular occurrence, this kind of vulnerability is disclosed from time to time—if not in Drupal core, in some popular contributed module, or in some package you have running on your Internet-connected servers. What's the best way to update your entire infrastructure (all your sites and servers) against a vulnerability like this, and fast? High profile sites could be quickly targeted by criminals, and need to be able to deploy a fix ASAP... and though lower-profile sites may not be immediately targeted, you can bet there will eventually be a malicious bot scanning for vulnerable sites, so these sites need to still apply the fix in a timely manner.

In this blog post, I'll show how I patched all of Midwestern Mac's Drupal 7 sites in less than 5 minutes.

Hotfixing Drupal core - many options

Before we begin, let me start off by saying there are many ways you can apply a security patch, and some are simpler than others. As many have pointed out (e.g. Lullabot, you can simply download the one line patch and apply it to your Drupal codebase using patch -p1.

You could also use Drush to do a Drupal core update (drush up drupal), but you'll still need to do this, manually, on every Drupal installation you manage.

If you have multiple webservers with Drupal (or multiple instances of Drupal 7 on a single server, or spread across multiple servers), then there are simpler ways of either deploying the hotfix, or upgrading Drupal core via drush and/or version control (you are using Git or some other VCS, right?).

Enter Ansible, the Swiss Army Knife for infrastructure

Ansible is a powerful infrastructure management tool. It does Configuration Management (CM), just like Puppet or Chef, but it goes much, much further. One great feature of Ansible is the ability to run ad-hoc commands against a bunch of servers at once.

After installing Ansible, you need to create a hosts file at /etc/ansible/hosts, and tell Ansible about your servers (this is an 'inventory' of servers). Here's a simplified overview of my file:

There are a couple quick things to note: the inventory file follows an ini-style format, so you define groups of servers with [groupname] (then list the servers one by one after the group name, with optional variables in key=value format after the server name), then define groups of groups with [groupname:children] (then list the groups you want to include in this group). We defined a group for each site (currently each group just has one Drupal web server), then defined a drupal7 group to contain all the Drupal 7 servers.

As long as you can connect to the servers using SSH, you're golden. No additional configuration, no software to install on the servers, nada.

Let's go ahead and quickly check if we can connect to all our servers with the ansible command:

This would quickly apply the hotfix on all your servers, using Ansible's shell module (which, conveniently, runs shell commands verbatim, and tells you the output).

Fixing core, and much more

Instead of running one command via ansible, let's make a really simple, short Ansible playbook to fix and verify the vulnerability. I created a file named drupal-fix.yml (that's right, Ansible uses plain old YAML files, just like Drupal 8!), and put in the following contents:

Now, there are again many, many different ways I could've done this. (And to the eagle-eyed, you'll note I haven't included my test for the vulnerability... I'd rather not share how to test for the vulnerability until people have had a chance to update all their sites).

I chose to do the hotfix first, and quickly, since I didn't necessarily have time to update all my Drupal project codebases to Drupal 7.32, then push the updated code to all my repositories. I did do this later in the day, however, and used a playbook similar to the above, replacing the first two tasks with:

Using Ansible's git module, I can tell Ansible to make sure the given directory (dest) has the latest commit to the master branch in the given repo. I could've also used a command and run git pull from the drupal_docroot directory, but I like using Ansible's git module, which provides great reporting and error handling.

Summary

This post basically followed my train of thought after hearing about the vulnerability, and while there are a dozen other ways to patch the vulnerability on multiple sites/servers, this was the way I did it. Though I patched just 9 servers in about 5 minutes (from the time I started writing the playbook (drupal-fix.yml) to the time it was deployed everywhere), I could just as easily have deployed the fix to dozens or hundreds of Drupal servers in the same amount of time; Ansible is fast and uses simple, secure SSH connections to manage servers.

Comments

Great article! Ansible would of save me a bunch of time last night.
One thing worth mentioning is clearing opcode cache after applying the patch or updating Drupal. Maybe include reloading apache in the Ansible playbook example.