Why do we hash user passwords in our databases? So that if bad actors somehow gain access to our database, they will have a hard time stealing our secrets. But suppose we need to securely store other sensitive data in our database? Encrypt it. But how do we go about that? Assymetric, public/private key encryption with PGP would undoubtedly be the best way to go, but in the case at hand, it’s too impractical. Faced with this situation, I reasoned that symmetrical encryption was the next best option: use one key for both encryption and decryption. But how to store the encryption key securely? As a plain text file sitting on the same server as the database username and password? Which, by the way, are also sitting there in plain text, if not in plain sight?

Disclosures, Disclaimers, Excuses

Now that I have your attention, a quick digression. I am not a security expert — I just try to pay attention to those who are. So please, use your own judgment at your own risk yadda yadda yadda. Moreover, I’m not even officially a web application developer. My job title has nothing to do with that. Long story short: I stumbled into coding some 20 years ago because in my workplace we could find no software, commerical or otherwise, that met our needs in managing a busy federal court interpreters office. So we (OK, I) rolled our own, and have been rolling ever since. Now I’m working on the next iteration of our great project, using Zend Framework 3 and Doctrine, and that’s the context in which this security problem arises.

Wandering back to our main topic: we need to store some sensitive data of contract court interpreters in a database, we want to encrypt it symmetrically, and we need to keep the secret encryption key secret.

Hashicorp Vault to the rescue

The very clever people at Hashicorp generously provide, among other things, a secret-management tool aptly named Vault. As their website puts it,

Vault is a tool for securely accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, and more. Vault provides a unified interface to any secret, while providing tight access control and recording a detailed audit log.

The use of Vault is a complex subject, and the following discussion is no substitute for the documentation. We should just point out that secrets (yes, that’s the technical term) can be stored in several different types of storage backends, and authentication can happen by way of a number of different authentication backends. These things are identified and accessed by way of paths similar to a filesystem. And that unified interface mentioned above is an HTTP API. You talk to Vault by sending HTTP requests with an authentication token in your request headers, and get JSON responses. Of course, this makes it a snap for applications to communicate with Vault with the language of your choosing.

The challenging part is devising a good security model, and there is a vast — if not to say baffling — array of choices and decisions to be made. I’m still not convinced my current solution is the best, so it’s subject to change. But let’s step through it as it is at the moment. Of course, you’ll first need to install and configure and run Vault, and they make that pretty damn easy. (Though you do have to decide what machine(s) to put it on. There’s replication and high availability and all that sexy stuff for large-scale deployments. Ours happens to be a modest configuration, involving a small set of users behind our organization’s firewall.)

Creating Policies

In Vault, you create authentication tokens that have policies attached to them, determining what the bearer of the token is allowed to do. I want our authenticated application not to be authorized to read our ultimate secret, but to grant an auth token bound to a policy that does allow it. So we need a policy for reading the secret. We also need a policy for creating the token that’s attached to the policy for reading the secret. Sound complicated? Don’t worry: it is! But the underlying principle is a basic one: grant access that is as permissive as necessary, and no more. So I took out my quill and wrote policies, thus:

read-secret.hcl

path "secret/data" {
policy = "read"
}

create-token.hcl

path "auth/token/create/read-secret" {
policy = "write"
}

Incidentally, that .hcl extension you see refers to Hashicorp Configuration Language. It’s a lot like JSON and you’ll pick it up immediately. Then, after logging into Vault with sufficient privileges, we wrote our policies to Vault via its CLI:

vault policy-write read-secret read-secret.hcl

and

vault policy-write create-token create-token.hcl

and finally, the rather more esoteric one:

vault write auth/token/roles/read-secret allowed_policies=read-secret

which magically allows the “role” to bestow access to something that the role itself cannot access.

then with the HTTP API using our good friend curl. By the way, before thinking too much about integration with ZF, I followed the same procedure with all this token stuff until I was able to go from zero to reading the secret (encryption key) from the command line. I predicted (correctly!) that the heavy lifing was setting up the Vault stuff, and getting it to play well with ZF would be more like fun than work.

Getting a token for getting the secret

After the previous command, now that we’re authenticated via TLS, we can go

and that “cipher” you see is the encryption key for encrypting/decrypting our sensitive database fields, a step we will perform in our PHP application.

In all honesty, it’s a bit more complicated than this because we are going to be using response wrapping. When a client sends the header so indicating, the response is not the thing requested, but another auth token with which you can get the thing requested. And that token is a single-use token: use it once and then it’s revoked. You can also limit the time-to-live on this token, so that whether it gets used or expires, it is not going to be hanging around for long as a valid token. This exemplifies the Vault (and general security) principle of limiting the exposure of a secret.

Plugging it into ZF

The front end

At last, the fun part. (Not to say that Vault isn’t loads of fun — it absolutely is! But after dealing and struggling with something unfamiliar, it’s comforting like a warm bath to return to the familiar, isn’t it?) I decided that this application would have its Vault capabilities in a separate module to make it easy to enable or disable. If it’s disabled, they simply don’t get to work with sensitive data. Let’s work from the front end to the back. The form has a good number of fields, but the ones we’re concerned with look like this:

because there’s no reason to expose this data if they don’t ask. The encrypted values are tucked away as hidden fields. When they click those lock thingies, we display a modal dialog inviting the already-authenticated user to re-authenticate (in case they load the form, wander off to the bathroom, and just then an identity thief comes up to their workstation). A couple of Javascript xhr calls later, if all goes well, the dialog goes away and these fields are re-populated with the decrypted values.

The Controller

And the above-mentioned xhr sends a POST request containing the encrypted values to /vault/decrypt, a route mapped to this action method in our module/Vault/src/Controller/VaultController.php:

The part involving Vault gets our encryption key, and it’s a one-liner. The next bit simply makes use of zend-crypt to take care of decryption.

You notice $this->vaultService. That’s a dependency injected via the constructor, so there’s a factory which pulls it from the container. Boring, so we’ll skip that part. Let’s have a look at the module’s module.config.php (ooh, exciting!)

Configuration

<?phpnamespace SDNY\Vault;use Zend\Router\Http\Literal;return['vault'=>[// override these with a local configuration'vault_address'=>'https://vault.example.org:8200','sslcafile'=>'/usr/share/ca-certificates/ca-chain.cert.pem',// these settings must match the configuration set in Vault'ssl_key'=>'/path/to/your-private-key.key.pem','ssl_cert'=>'/path/to/your-cert.pem','path_to_secret'=>'/path/to/your/secret',// including leading slash// but do not change this adapter'adapter'=>'Zend\Http\Client\Adapter\Curl',],'service_manager'=>['factories'=>[
Service\Vault::class=> Service\Factory\VaultServiceFactory::class,]],'controllers'=>['factories'=>[
Controller\VaultController::class=> Controller\Factory\VaultControllerFactory::class,]],];

So the module’s config sets some default/dummy values, with the expectation that they will be overridden by a local configuration file called something like config/autoload/vault.local.php, consistent with the ZF convention. No reason to put this in a public repository.

The Vault service

We have a VaultServiceFactory that injects all that config for us when it instantiates our VaultService, which is what does the talking to Vault.

This Vault service extends Zend\Http\Client because it basically is a specialized http client. The general pattern is that most of these methods attempt to get the token we need, store it in $this->token, and return $this so the lazy coder using the class can save some keystrokes.

We make use of the handy Zend\EventManager\EventManagerAwareInterface and Zend\EventManager\EventManagerAwareTrait and do a fair amount of $this->getEventManager()->trigger(). It’s most definitely a work in progress, but by the time this project is done (or pretty close) there are going to be event listeners responsible for logging things and generally paying attention. On the Vault side, I plan to (figure out a way to) monitor the audit log with the ultimate goal of trying to detect a possible breach and hit the panic button.

So we’ve demonstrated decryption, how about encryption? Coming soon! But as you can see, it will be straightforward.

Further thoughts on Vault and PHP Applications

Remember we started out with a reference to the practice of storing database credentials in plain text? Vault provides a database secret backend so you can avoid doing that. Vault also has a very straightforward user/password authentication backend that makes it tempting to consider implementing a Zend\Authentication\Adapter\AdapterInterface that uses it.

All of this is to say that in terms of security, I think Vault offers us ways to take Zend Framework applications, and PHP applications generally, to the next level.

Conclusion

As noted at the beginning, this solution to my challenge of keeping our encryption key secure may not be the best possible. But it’s a start. The first objective was to avoid leaving our secret lying around in plain text, and this accomplishes that. My holy grail is to make it so that even in a worst case scenario, where an attacker gains root access to the machine where this application is installed, my precious social security numbers and dates of birth would still not be compromised. Like anything, this model is not guaranteed bullet-proof against a breach that severe, but again, it’s a start.

Hopefully this demonstrates that once you have Vault going, it’s quite easy to integrate with your ZF application. As a personal note, I would add that this is actually my debut blogging about Zend Framework, and I do feel a little self-conscious about presuming to address an audience with so many badasses in it. Stage fright notwithstanding, I am truly interested in hearing your comments, so please feel free.