Step Certificates

By Mike MaloneDecember 11, 2018

An open source solution for secure automated certificate management

At smallstep we’ve been focused, lately, on building technology that makes it easier for you to access your stuff. As things stand today, access is really hard. It’s really hard for developers to access internal services in production and pre-production environments (e.g., to debug using curl). It’s really hard for your stuff running in AWS to access your stuff running in GCP and vice-versa. It’s really hard for employees to access enterprise IT applications from their iPhones while they’re on the bus heading into work. Supporting these use cases, and dozens more like them, is so hard that it’s often not done. Access that would make people more productive, make systems better, reduce costs, improve performance, and generally improve software and employee well-being is simply denied. That sucks.

There’s a simple and universal answer that makes access easy. Fundamentally, if you’re able to authenticate the identities of two things that are trying to communicate, and if that communication is encrypted, nothing else matters. Instead of relying on trusted networks, access can be authorized based on identity, and security can be guaranteed cryptographically. Even better, you can ditch VPNs and whatever operational nightmare powers your network policy rube goldberg machine. For code and devices it’s actually really easy to do this using TLS. But there’s a catch: to use TLS you need certificates — a sort of credential that TLS uses for authentication.

Certificates are the best way to identify code and devices. The problem is that certificate management is really hard. So hard, and so abstruse, that many people assume it’s not for them… or at least not worth their while. That’s wrong. It is. But it’s true that there’s no good way to automate the secure delivery of certificates to devices and workloads, and to manage certificate lifecycle (renewal and whatnot). That’s why we built step certificates (GitHub). It’s an open source project that makes secure automated certificate management easy, so you can use TLS and easily access anything, running anywhere, from everywhere.

Before we start, you might want to install step so you can follow along.

On macOS using brew:

$ brew install smallstep/smallstep/step

Binary tarballs are also available for Linux. For instructions on building from source see GitHub.

More than a Certificate Authority

At the core of step certificates is an online certificate authority (CA): a piece of infrastructure that most production environments don’t have (but should). A CA signs certificates. Other components verify certificates by checking signatures. The step CA is lightweight, fast, and easy to operate. It scales horizontally, and multiple instances can be deployed for high availability.

But step certificates is more than a certificate authority. It provides all the missing bits you need to run your own internal public key infrastructure (PKI). It exposes APIs and extends the step CLI (blog, GitHub) with subcommands to fill all the gaps that keep internal PKI out of reach for most teams and organizations. It’s all the stuff you need to issue certificates everywhere, and to securely access your stuff from anywhere using TLS.

To use TLS your code and devices need to generate keys and obtain certificates from the CA. The root certificate that belongs to the CA also needs to be distributed everywhere (it’s used to verify certificate signatures). Certificates expire and stop working, so they need to be renewed before that happens. These processes need to be secure. They also need to be automated. Step certificates does all this and more.

Let’s look at some example workflows to see how.

Generating keys and getting certificates

Once you have a CA up and running, step certificates makes generating keys and obtaining a certificate trivial. You need just one simple command:

The positional arguments in the step ca certificate command indicate the name we’d like bound in the certificate (e.g., the DNS host name of a service) and the filenames to write the certificate and private key out to.

Using certificates with TLS

As mentioned, clients and servers need to be configured to use the CA’s root certificate. step can grab this root certificate from the CA and verify it using a fingerprint (which can be produced using step certificate fingerprint and embedded in scripts):

Setting up services to use these certificates is generally not that hard (though it helps to have a good understanding of how certificates and PKI work). One nice thing about TLS is that you have the option of baking it into your applications for complete end-to-end encryption or deploying a sidecar proxy to terminate TLS with no code change required. Let’s look quickly at both scenarios.

Baking TLS into an application might sound hard, but once you have certificates it’s really not that bad. You’ll have to check out the docs for your language or framework for particulars but, generally, you’ll just need to parameterize some TLS API with your root certificate, leaf certificate, and private key. Here’s a working example in golang:

There’s still some work to do because this certificate will eventually expire. We’ll need to renew and rotate it somehow before that happens. If you’re doing end-to-end TLS like this, the only thing that ever needs the private key is the application itself. So it’s unfortunate that we’re generating the private key outside the application and putting it on disk. Fixing all this is a bit more work. To make all of this easy, step certificates includes a golang SDK that generates keys and obtains a certificate for you and will automatically renew it before it expires. There’s an example in the step certificates repository. Another example provides a more thorough demonstration of everything the SDK can do.

We’d like to provide SDKs for other languages. Right now it’s golang only. This is a relatively easy and high value area to contribute, if anyone’s interested!

If you’d rather not change any code, pretty much everyproxycan do [m]TLS with a bit of light configuration. For this sort of deployment you can throw step ca renew in a cron job with the --expires-in and --force flags to keep your certificates fresh. Some proxies require a kill -HUP to pick up the new certificates, so consult your docs.

So far we’ve used step ca certificate to generate a key pair, construct a certificate signing request (CSR), and make an authenticated request to the CA to obtain a certificate with a single command. To authenticate we’ve been prompted for a password to decrypt a provisioner key. A provisioner is simply a thing that’s authorized by the CA to assign names and enroll people and things in your PKI (i.e., get certificates). We’ll discuss how all this works in more detail in the next section.

Provisioners are specified simply in the CA’s configuration file, located by default at $(step path)/config/ca.json. You can use step to manage this file, or edit it directly. A single provisioner is added automatically when the CA is first initialized, but you can add more by creating a key pair and registering it with the CA (i.e., adding the provisioner’s name and key to the CA configuration file). If you do add multiple provisioners, you’ll be prompted to select which to use when you create certificates.

Interactive workflows and passwords make sense for people, but they’re no good for automation. Let’s take a closer look at the bootstrapping protocol and, more broadly, at how step certificates makes it easy to automate certificate management.

Automated Certificate Management

Provisioners are instrumental in facilitating automation. Name assignments are made via signed attestations that are generated by the provisioner, passed to the thing being provisioned, then used to authenticate certificate requests submitted to the CA — to obtain a certificate.

The hardest problem here is securing this first interaction between a thing that needs a certificate and the CA. Authentication is required, but we can’t use certificates because a thing that needs a certificate obviously doesn’t have one yet.

What’s needed is some trusted component that can measure and attest to the identity of a thing that needs a certificate. For systems under operational automation, the right tool for this job is whatever’s already being used to provision systems and deploy code: Puppet, Chef, Ansible, Kubernetes, Terraform, etc. You already trust this stuff to deploy the right code to the right places. To do so these tools must have some notion of identity. So they’re trusted and capable of assigning names and signing attestations. The stuff you’re already using for provisioning should be your step certificates “provisioner” (hence the name). Step certificates makes using these tools for this purpose super easy. That’s how we automate certificate management.

Summarizing, provisioners help authenticate the initial request for a certificate. They do this by putting the name of the thing being provisioned in an attestation that the CA can verify: a short-lived one-time token called a bootstrap token. Bootstrap tokens are signed by the provisioner’s private key, and passed to whatever’s being provisioned to authenticate with the CA.

The unified step ca certificate command we used above is actually a wrapper around this more elaborate process. For automation we’ll break this process into its constituent parts. The bootstrap token will be generated in one place and used in another.

If you’re confused, an example should clear things up. Let’s start by generating a bootstrap token for svc.example.com and use it to obtain a certificate:

This opaque looking token is actually a JWT: an RFC standard, JSON-based, base64-encoded signed assertion with good library support in pretty much every programming language. For fun, let’s take a peek at the token contents (using step crypto jwt inspect):

The best way to generate bootstrap tokens and get them where they’re needed depends on which tools you’re using and the particulars of your environment. Stay tuned for more guidance here as we document best practices for various tools.

For now, in general, the easiest thing to do is to place the private key on a machine that’s been locked down and is only accessible by your configuration management or orchestration system. With a provisioner’s unencrypted private key available, a bootstrap token can be generated non-interactively to support automation:

Starting the CA

To wrap up our demo let’s initialize a CA from scratch and start it running.

Like provisioner management, step certificates provides a command line workflow to simplify initial configuration. It asks you a couple questions then writes a working configuration to the default location:

If you have an existing PKI, step certificates can also federate or operate as a subordinate to your existing root. See the documentation forstep ca init for more information.

That’s it. Once you’ve run this initialization process you’re ready to go. The online certificate authority itself ships as a separate binary called step-ca, so it’s not necessary to have step installed to run the CA (though it doesn’t hurt and is useful to have available).

Roadmap

Everything documented here is ready to go, but we’ve got a long roadmap of features we’d like to add to step certificates in the future. We’re committed to open source — we believe everyone deserves world class PKI — and we’re also working on an enterprise offering to better integrate in enterprise environments and address enterprise-specific use cases. We’d love your input on where to draw the line between these two offerings.

Anyways, there are a bunch of advanced features to add like revocation (CRL and OCSP), HSM support, and certificate transparency (CT). We’d also like to add a UI to supplement the command line tool and to help with certificate management (e.g., present a searchable certificate inventory built off of CT). The bootstrap protocol is great, but we’d like to support alternative authentication mechanisms like mTLS and OAuth OIDC for agent-based certificate provisioning and to allow people to easily obtain certificates for themselves, respectively. Bootstrapping was designed to be simple and universal for MVP, but we’d like to harden the protocol with support for additional factors like instance identity documents in cloud environments and MFA for people. As mentioned, we’d also like to provide SDKs in other languages.

Try it out!

We’d love to hear from you if there are any other missing features, bugs, or oversights, and to help prioritize these existing ideas. If you have thoughts, please shoot us a tweet or open an issue on GitHub.