Passwords, API keys and confidential data fall into the category of secrets. Storing secrets the secure way is a challenge with limiting access and a true secure storage. Let’s take a look at HashicorpVault and how you can use it to store and access secrets.

How do you store Secrets?

Passwords, API keys, secure Tokens, and confidential data fall into the category of secrets.
That’s data which shouldn’t lie around. It mustn’t be available in plaintext in easy to guess locations. In fact, it must not be stored in plaintext in any location.

Sensitive data can be encrypted by using the Spring Cloud Config Server or TomEE.
Encrypted data is one step better than unencrypted. Encryption imposes on the other side the need for decryption on the user side which requires a decryption key to be distributed. Now, where do you put the key? Is the key protected by a passphrase? Where do you put the passphrase? On how many systems do you distribute your key and the passphrase?

As you see, encryption introduces a chicken-egg problem. Storing a decryption key gives the application the possibility to decrypt data. It also allows an attack vector. Someone who is not authorized could get access to the decryption key by having access to the machine. That person can decrypt data which is decryptable by this key. The key is static so a leaked key requires the change of keys. Data needs to be re-encrypted and credentials need to be changed. It’s not possible to discover such leakage with online measure because data can be decrypted offline once it was obtained.

One approach is putting the key in a hard to guess location before the application starts and wipe the key once it was read to memory. The time in which the key is available is shortened. The attack time-frame is reduced, but still the key was there. Wiping the key works only for one application startup. Containers and microservices in the Cloud are known to be restarted once they crashed. A restart of the application is no longer possible as the key is gone.

Wait, there’s hope!

Doing encryption right is tough, managing secrets is even harder if doing it yourself. Vault addresses exactly these issues. It helps to address the chicken-egg problem and it comes with encryption. Vault is a service to manage secrets. It provides an API that gives access to secrets based on policies. Any user of the API needs to authenticate and only sees the secrets for which he is authorized. Vault encrypts data using 256-bit AES with GCM. It can store data in various backends (files, Amazon DynamoDB, Consul, etcd and much more). The other key aspect is that Vault never stores a key in a persistent location. Starting/restarting Vault always requires one or more operators to unseal Vault. However let’s start with the basics first.

Vault isn’t the answer for all security concern. It’s worth to check the Vault Security Model documentation to get an idea of the threat model.

To bootstrap Vault, you need to download the binary from https://www.vaultproject.io/downloads.html. Vault is written in Go and binaries are available for various platforms.
Unzip the downloaded file and you are ready to use Vault.

Start Vault Server next. You need a configuration file to specify some options.

This config is good for most platforms and to try first steps with Vault. Don’t use it in production.

Start Vault with

$ vault server -config vault.conf

Vault will start as a foreground process.

Congratulations, you started Vault.

Now is a good moment to open a second console to perform administrative tasks with Vault. Vault runs now in plaintext mode because TLS/SSL is disabled. You need to set the VAULT_ADDR environment variable to tell the Vault client to use plaintext:

$ export VAULT_ADDR=http://127.0.0.1:8200

Vault is started. It requires two additional steps before you can actually start using Vault. Vault needs to be initialized and unsealed. Initialization is the process of initial key generation. Unsealing is supplying the keys to Vault so Vault can decrypt encrypted data and start serving clients.

Vault creates upon initialization two things:

The master key and key splits

A root token

Vault allows shared keys using the Shamir Secret Sharing algorithm. Data is usually encrypted with one key. The one, who has access to the key has full control to all data as a single person. Sometimes you don’t want that. Usually you want to distribute the master key amongst multiple people so no one single person is in control of all your encrypted data. Vault allows specifying the number of total key shares and the number of key shares required to unseal Vault during initialization. That setting cannot be changed once Vault is initialized. Initializing Vault from the console will display the full key. Initialization using the API is maybe something you want to adopt with your DevOps tooling by e.g. sending secure messages to your operators who should receive a key share.

Initialize Vault with:

$ vault init -key-shares=5 -key-threshold=2

Vault will display the key shares and the root key. Please note that these values are random
and change upon every initialization. Be careful with that output as you will see it only once. There’s no way to retrieve the keys and the token afterward. Please read the instructions carefully when using Vault with real data otherwise you’ll loose your data.

Then you need to unseal Vault. Vault does not store the key on disk. It’s stored in memory all the time. After initializing and after (re)starting Vault you’re required to unseal Vault with the required number of key shares so Vault can serve secrets. In this case that’s two key shares. Note: There’s also a seal command to make Vault stop serving secrets.

Once Vault is unsealed you can start storing secret data inside of Vault.

Vault requires an authenticated access to proceed from here on. Vault uses tokens as generic authentication on its transport level.

Remember the output from the initialization? The last item after the key shares is the root token. The easiest way for now is using the root token. The easiest way to use the token on the console is storing it in an environment variable:

The generic secret backend allows storage of arbitrary values as a key-value store. A single context can store one or many key-value tuples. Contexts can be organized hierarchically and the used data format is JSON.

Authentication

Vault works primarily with tokens. Each token is assigned to a policy that may constrain the actions and the paths. Policies use path based matching to apply rules. Tokens can get metadata (key-values) and display names assigned which makes administration a bit more ops friendly.

You can create tokens manually and assign them to applications and users. Besides that there are a couple of authentication mechanisms (LDAP, Username/Password, GitHub Token, …) that allow users to login and obtain a token. Tokens and authentication mechanisms can be revoked and that makes it easy to lock out a particular user.

Spring Cloud Vault

We at Pivotal took a look at Vault and considered it a promising tool. That’s why we built Spring Cloud Vault. Spring Cloud Vault is a configuration extension similar to Spring Cloud Config. Spring Cloud Config targets external configuration management backed by data stored in various repositories, such as GitHub, SVN or even Vault.

With Spring Cloud Vault you can access your secrets inside Vault. Secrets are picked up at startup of your application. Spring Cloud Vault uses the data from your application (application name, active contexts) to determine contexts paths in which you stored your secrets.

Make sure to include the Snapshots repository when using SNAPSHOT dependencies.

Setup the configuration

Spring Cloud Vault uses by default application as the default context and the value of spring.application.name as application context. All configuration needs to be specified in the bootstrap configuration. For this example we use bootstrap.yml in src/main/resources:

The spring.cloud.vault.scheme is set to http because we’ve started Vault in plaintext HTTP mode. Don’t do this for production. Plaintext makes the whole secret story useless as all listeners on the network can see your secrets. spring.cloud.vault.scheme defaults to https.

Please note that the token here is taken from the root token. You can create new tokens with: