The solution to a general problem is simpler than a solution to a specific problem, the solution of many problem is simpler than the solution to one problem.

Saturday, October 21, 2017

Cleaning Openshift Origin Images Registry

When using and tending an Openshift Origin cluster (for example, Origin version 3.7), it is normal to start the storage allocation in small sizes. However soon we find that storage for registry get filled up quickly with images from each build process. This post will show how to clean them up.

Preparation before pruning

First you need oc (origin client) binary and a user account with cluster administration capability.

If the openshift docker registry is installed inside the cluster without external access, then you also going to need OS access to one of the hosts inside the cluster.

First step is to login to the cluster from your client or inside one of the hosts:

oc login

Prune steps

Reading the documentation (https://docs.openshift.com/enterprise/3.0/admin_guide/pruning_resources.html) we find that the pruning starts at deployment, then builds, and last images.

Pruning Deployment

Run this to preview which deployment are going to be pruned:

oc adm prune deployments

Then execute the pruning :

oc adm prune deployments --confirm

We could use the CLI utility oadm (origin adm) or oc adm command, depending on availability of the executable.

Pruning Builds

Run this to preview the builds :

oc adm prune builds

or

oadm prune builds

Then execute the pruning :

oc adm prune builds --confirm

Pruning Registry Images

And finally, we prune images. Images could not be pruned if there are some deployments are referring to the image, thus the prune deployment and build steps above are need to be done first.

oadm prune images

Confirm the pruning with additional --confirm flag:

oadm prune images --confirm

If we the registry is not accessible, we get this message :error: error communicating with registry: Get http://IPredacted:5000/healthz: dial tcp IPredacted:5000: getsockopt: operation timed out
Such error means we need to ssh into one of the hosts in to be able to prune images.