Red Hat Nordics SA Demo lab

We are Red Hat Solution Architects, in this blog we are sharing content that we have used to create our own demos and labs. Some of it has proven to be useful starting points for our customers and partners. You are free to use it as is. If you use it, and it breaks your stuff, you get to keep both pieces ;-). Ansible, Tower, CloudForms, Satellite, RHV, IdM, RHEL, Gluster, Ceph. Disclaimer, This is not a Red Hat official blog, nor a service and we will not provide support on this.

Use Ansible and Let's Encrypt to get (OpenShift) proper SSL certs

Written by Ilkka Tengvall

The Challenge

For those often rebuilding OpenShift
environments, it is ugly to have invalid SSL certificates for the site. Or any
web site, for that matter. It requires you to set all tools and browsers to
ignore invalid SSL certs. It might even break your testing. You test your
OpenShift upgrades and changes in separate test cluster instead of production,
right ;) ?

I describe here how I use Ansible, Let’s Encrypt and Route53 DNS to set up my
test/demo OpenShift cluster with proper certificates. The method would apply to
pretty much any web service you have, not just OpenShift. Check out
Let’s Encrypt website if you are not familiar with
the wonderful free certificate service.

The Environment

This write-up is about test environments, so I start by describing mine. I have
rented a server which comes with CentOS. I use Ansible to install all OpenShift
and virtualization requirements into it. Then I create 5 VMs on top of it,
and install OCP in there. I also install Cockpit
for having graphical panel to control and get visibility into environment. You
probably have proper private cloud or virtual environment for this, I don’t.

As an application user, I want web services on top of OpenShift properly
secured

In short, the cluster looks ugly with confirmations about invalid certs, and
tools and automation on top of OCP likely breaks without valid certs.

Pre-Requisites

Get domain. This is the hardest part. You need to think of cool name here :) I
ended using my “container school” domain, konttikoulu.fi. To get domain, follow
AWS
instructions,
out of scope of this blog.

You need a way to prove you own the domain you request the certificate for. Like
mentioned, I use Route53 from AWS to control my domain name. It’s nice that
Let’s Encrypt -service’s certbot -tool supports it out of box. There are variety
of other DNS provides supported by Let’s Encrypt tooling, see e.g.
certbot and
acme.sh documentation.

There are at least two different ways of Let’s Encrypt verifying your domain:

Let’s encrypt’s server talks to either:

Your DNS vendor to verify you own domain

Your web site (OpenShift router) and verifies that FQDN leads to your web
page (see linked blog at bottom for this method)

Let’s get to business. Let’s look at what needs to be done to have proper certs:

Install let’s encrypt tools to bastion

Get certificates for

Cockpit on host

OpenShift API (same as web GUI)

OpenShift Router cert (* -cert)

Modify OpenShift installer inventory to use FQDN

Modify OpenShift installer inventory to use certs

run the installer

I have my OpenShift install automation in git. I did this all in one commit
which you can see in git
diff.
Well, actually two, as I got sloppy in first commit, I fixed some variable names
in second
commit

Let's go through the changes step by step

Setup epel repo for bastion to get certbot tool

Note that I also disable it by default, so that I don’t accidentally install
non-supported versions of software, which might break installation.

Keep it optional

This is for myself only. I wanted to keep an option to use the same ansibles
without using certbot. If I don’t define AWS keys, it won’t do certs. This is
work in progress still… I forgot to make it conditional in inventory template.

when:
# we do let's encrypt if AWS keys are provided, and we do it only once
- aws_access_key_id is defined
- aws_secret_access_key is defined
- cert_exists.stat.islnk is not defined

Host’s Cockpit Certificates

There is similar logic for host server’s Cockpit. See the commit in git,
I don’t copy paste it here as it’s almost identical to above steps.

You could do the same for Cockpit at OpenShift master. I didn’t bother, as I use
the Cockpit from host, and tell that to connect to OpenShift master’s Cockpit.
So my browser never hit’s it directly. Did you know you can have several host’s
Cockpits used from one Cockpit web GUI?

Note, if you copy paste the code in previous link, there is an erroneous
redirect. Change >> to > in there, to avoid stacking the certs into one
file. It’s fixed in later commits.

That's it, folks, we are done!

So now, with those few ansible lines, and running installer, you could turn your
OpenShift and Cockpit to use proper certs. I believe they are pretty re-useable
to any SSL service you might have. Happy safe internetting for you and your
OpenShift users!

Links for related docs and blogs

Here are links to official documentation, and another blog that has a bit
different way of setting the router address. It’s definitely better for router.
It is based on the non DNS way of ensuring web site, and will generate
application specific certs instead of wildcard cert. It’s a good reference
for the next step for you to take.