How to create encrypted Chef Data-Bags without knife/knife-solo

Hello there. I just wanted to share a short blogpost about how to quickly create encrypted secrets to use in Chef Data Bags.

Prerequisites are a basic understanding of the principles on Chef and a Ruby installation on your Computer. Since you probably already use the Chef Development Kit, wich includes Ruby, this should be no problem.

Thing is, I didn’t reinvent the wheel with this method. There are pretty useful how-tos from the guys at Chef itself (like here) and all over the internet of how to work with data bags and encrypted secrets in Chef, but in my case I found it quite difficult to find a way to help me with my specific situation.

We don’t „really“ use Chef in a „Server – Nodes“ environment. Most of the times we create Chef Cookbooks to deploy our environment to servers via a Chef-Solo runlist and therefore some of the workflows intended for Chef didn’t work on my setup.

So what was the given situation? There are a lot of information that you don’t want to store even in a private Git repository like administrative password and other secrets. Chef offers a function to encrypt this data in json format in so called „data bags“. This means the information is in its encrypted format in the repository and decrypted by Chef at runtime. In order for this to work, the encryption key must be accessible or known on the host server which will be prepared by our Chef cookbook.

The build in tools for creating theses data bags somehow didn’t work on my machine. Knife demanded a certificate I didn’t have and didn’t need and for unknown reasons the installation of the „knife-solo“ rubygem didn’t work properly so I was unable to use this workaround.

When searching for solutions I came across this short but helpful Ruby-Script on this website

So what does it do and how to use it?

Place this Rubyfile in the root folder of your Chef project, where you also should have your „data_bags“ folder and a simple textfile called „chef_secret“ in which you place your keyphrase for en- and decryption. This chef_secret file also has to be on the destination server, otherwise Chef would be unable to decrypt the secret.

One little but important detail: the filename of the json that this script generates (right after File.open in the script) must be the same like the „id“ of the secret, defined in the „data“ part in the script.

The id is used in the Cookbook to identify the correct data and to decrypt the right value at the right place.

For example if you want to encrypt and store a MySQL password in a Chef data bag you would call the „id“ of the secret „mysql“ and the desired password to be encrypted in the „key“ part.

Then you change the destination file to „mysql.json“ and call the script on the terminal with „ruby secret.rb“. When the chef_secret file is in the right place the script then takes the values and creates the myslq.json file in the /data_bags/passwords/ folder which then can be safely committed to Git or Bitbucket.

So there you are, you are now able to deploy a secret value, application password or aws secret to a new server without having to store the password in plaintext somewhere in your Cookbooks. The only thing you must be aware of is not also to commit your chef_secret file to your repository, which would render all the former secrecy obsolete, and that the file has to be placed on the right location on your server.

How to „call“ and decrypt this password within the Chef cookbook will probably be another short blogpost.