This post looks at several different ways to automate cert renewal. I tried to cater to everyone by including cron and systemd options. If you don't have your server set up to send emails, you might want to do that first.

Code

Note

I'm testing out some new tooling. This will be wotw-highlighter's shakedown run. Let me know what you think about the syntax highlighting! I'm pretty excited because (no whammies) it should work well in AMP and normally.

I wrote the majority of the Apache examples with httpd in mind, i.e. from a RHEL perspective. If you instead use apache2, most of the stuff should still work, albeit maybe just a little bit different.

Automating Renewals

Now that everything's installed and in place, we've got to think about keeping the cert current. Let's Encrypt certs have a 90-day lifetime, which is substantially shorter than a typical commercial cert. certbot is built to handle automated renewals and can update everything in place without any intervention on your part.

If you try running certbot renew right now, you'll probably get something like this:

-------------------------------------------------------------------------------Processing /etc/letsencrypt/renewal/example.com.conf-------------------------------------------------------------------------------Cert not yet due for renewal

The following certs are not due for renewal yet: /etc/letsencrypt/live/example.com.pro/fullchain.pem (skipped)No renewals were attempted.-------------------------------------------------------------------------------

While the cert isn't due for renewal, we can actually test the renewal process like this:

-------------------------------------------------------------------------------** DRY RUN: simulating 'certbot renew' close to cert expiry** (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed: /etc/letsencrypt/live/example.com/fullchain.pem (success)** DRY RUN: simulating 'certbot renew' close to cert expiry** (The test certificates above have not been saved.)-------------------------------------------------------------------------------

This is useful to make sure everything's in place for automation.

Hooks

You might have noticed the Dry run: skipping deploy hook command line in the output. certbot can run commands or scripts at several stages in its lifecycle. You can either add hooks via flags every time you renew, or you can offload them to executable scripts in /etc/letsencrypt/renewal-hooks.

For this example, all I'd like to do is restart the server process following successful renewals. Assuming we've got a local script capable of that called server-reboot, this should add it to certbot's pipeline.

Nginx

Apache

/var/letsencrypt/renewal-hooks/deploy/server-reboot

1
2
3

#!/bin/bash

apachectl -t && systemctl restart apachectl

Scripting a Renewal

The official documentation suggests running an automated renewal task at least twice a day (e.g. the CentOS instructions; scroll down). certbot also asks that you run it at a random minute. To make things easier later, let's isolate our renew command:

Adding extra flags is straightforward. renew only exits with a nonzero code if the renewal failed (paragraph right above the link), meaning the skipped renewals we saw earlier don't generate any traffic. They do, however, send many things to STDOUT, which is enough to trigger cron's mail action. The quiet flag suppresses STDOUT, so you won't get multiple emails a day letting you know certbot did nothing. If you're into that you don't have to use it.

Most of the solutions I've seen for the randomness do some cool stuff with advanced PRNG and then pass the result to sleep. There's nothing wrong with sleep if you're pausing tasks that don't actually need to run. Anything that kills the thread kills the task.

at

The at utility shall read commands from standard input and group them together as an at-job, to be executed at a later time.

In other words, at is a single-execution cron. It manages an at queue, most likely accessible via atq, which means random power failure or accidentally nuking a remote session won't kill the delayed task. Of course that means some setup is required:

# Pull out the PRNG into a functionfunction openssl_prng {MAX=$1# Create 7 pseudorandom bytes, output as hexPRN_HEX=$(openssl rand -hex 7)# The hex is converted to base 10PRN_TEN=$((16#$PRN_HEX))# Finally, PRN_TEN is taken mod MAX to fit the domainPRN_MIN=$(($PRN_TEN%$MAX))return$PRN_MIN}

# Path to renew taskTASK_FILE=/sbin/certbot-renew-everything

# This assumes you've got some control over the machine's at queuesSCRIPT_QUEUE="y"TASK_QUEUE="z"

# A hard cap on run count to account for unpleasant randomnessABSOLUTE_RUN_COUNT_MAX=10

# The number of minutes in 24 hoursMINUTES_IN_TWENTY_FOUR_HOURS=$((24*60))

# When to schedule the next renew runTASK_SLEEP_MINS=$( openssl_prng $MINUTES_IN_TWENTY_FOUR_HOURS)# Delay scheduling the next self run by an arbitrary amountSCRIPT_SLEEP_MINS=$(($TASK_SLEEP_MINS+30))

Scheduling the Renewal

With or without at, you've got to ensure the task is actually being run.

cron

$ sudo crontab -e

crontab -e

1
2
3
4
5
6
7
8

# Add a `MAILTO` that looks like this:MAILTO=your@email.address# Add one of the following, depending on how you set it up:00,12 * * * /full/path/to/certbot renew --quiet# or00,12 * * * /sbin/certbot-renew-everything# or00,12 * * * /sbin/at-random-renewal

If you're not changing the time in the script itself, you probably don't want to use 0 0,12. This launches the task at 00:00 and 12:00 every day. If launching means at assigns a random time, or checks to see if it's running, those times aren't a problem. If you're actually hitting Let's Encrypt every day at that time, that's not a great idea.

systemd

(Note: my systemd knowledge is still pretty rudimentary. I'm using to userspace cron. If you see anything I can improve, I'd love to know about it!)

Dec 17 15:03:21 wizardsoftheweb1 systemd[1]: Started Run certbot-renew.service every day at 00:00 and 12:00.Dec 17 15:03:21 wizardsoftheweb1 systemd[1]: Starting Run certbot-renew.service every day at 00:00 and 12:00.

Before You Go

Let's Encrypt is a fantastic service. If you like what they do, i.e. appreciate how accessible they've made secure web traffic, please donate. EFF's certbot is what powers my site (and basically anything I work on these days); consider buying them a beer (it's really just a donate link but you catch my drift).

Legal Stuff

I'm still pretty new to the whole CYA legal thing. I really like everything I've covered here, and I've done my best to respect individual legal policies. If I screwed something up, please send me an email ASAP so I can fix it.

The Electronic Frontier Foundation and certbot are covered by EFF's generous copyright. As far as I know, it's all under CC BY 3.0 US. I made a few minor tweaks to build the banner image but tried to respect the trademark. I don't know who the certbot logo artist is but I really wish I did because it's a fantastic piece of art.

Let's Encrypt is trademarked. Its logo uses CC BY-NC 4.0. I made a few minor tweaks to build the banner image but tried to respect the trademark.

I didn't find anything definitive (other than EULAs) covering Nginx, which doesn't mean it doesn't exist. Assets were taken from its press page.