In this post, we’re going to look at how to get the CoreOS Clair container security tool running from the command line, with a view to integrating it into a CI/CD workflow using the Klar CLI tool.

The way most people are using Docker is as a “better VM” in that they’re taking Ubuntu LTS releases and deploying on top of them. This is as opposed to using a “stripped down”/minimal image like Alpine Linux. The advantage of the minimal images is that they often remove a whole bunch of software that’s not needed in a container, reducing the potential attack surface and making the images smaller and easier to work with.

A few of the thorny issues which are not always considered in this scenario is how to upgrade/patch the “OS” (libraries etc…) running in Docker and how to check whether a Docker image is missing patches.

To solve this, CoreOS released a tool called Clair which allows you to “scan” Docker images in order to ensure that all patches/upgrades have been applied.

tldr;

Install CoreOS Clair onto Minikube using the Helm chart. Ignore the PostgeSQL part of the chart as the PV doesn’t work with Minikube. Ensure it can talk to DB.

Expose Clair to the cluster. Download and install Klar, run it, pointing at the Clair service and pass it an image to scan (needs docker locally?).

Ignore v1 API error message from Clair logs

Installing CoreOS Clair onto Minikube

Installing Clair onto Minikube is fairly straight forward thanks to Clair providing a Helm chart in GitHub, which you can find here. Personally, I found an issue with the “PostgreSQL” container within this helm chart, which caused it to fail to start, due to something to do with PhysicalVolumes:

The output currently will likely be “Found 0 vulnerabilities” which is a bit of an issue (it should be > 0). The reason for this false report is that the Clair “updaters” take a while to download the full list of vulnerabilities and lists from the Ubuntu/Debian/Alpine etc… databases. The solution is to wait a few minutes for the updater to get the full list of vulnerabilities (can tail the logs for the “vuln db updated” or whatever message)

Integrating into Continuous Integration workflow

So, at this point, you can automate the above to fail our build if the newly built/pushed image has an unacceptable level of vulnerabilities. The Klar command exit value is determined by whether the vulnerabilities exceeded the configured thresholds (0 = all good, 1 = fail). Therefore, it’s trivial to integrate into the build workflow.

You do need to sit down and work out where to set the thresholds at (i.e. what constitutes an unacceptable level of risk).

Conclusion and Further work

It would have been great if Klar supported scanning images locally (before they were pushed to a registry) as it would somewhat further reduce the risk that an “insecure” image might be deployed.

Also, we didn’t cover any alerting or notifications when Clair does find security issues in an image (i.e. who should be alerted? what should they do?).

Finally, there was no discussion of scanning already deployed images or any kind of periodic “background” scanning of a deployed environment.

For other options for Clair integration instead of Klar, have a look at the Clair integrations page.