Maintaining the Registry IP Address

OpenShift Container Platform refers to the integrated registry by its service IP address,
so if you decide to delete and recreate the docker-registry service,
you can ensure a completely transparent transition by arranging to
re-use the old IP address in the new service.
If a new IP address cannot be avoided, you can minimize cluster
disruption by rebooting only the masters.

Re-using the Address

To re-use the IP address, you must save the IP address of the old docker-registry
service prior to deleting it, and arrange to replace the newly assigned IP address
with the saved one in the new docker-registry service.

Make a note of the clusterIP for the service:

$ oc get svc/docker-registry -o yaml | grep clusterIP:

Delete the service:

$ oc delete svc/docker-registry dc/docker-registry

Create the registry definition in registry.yaml, replacing <options>
with, for example, those used in step 3 of the instructions in the
Non-Production Use section:

$ oc adm registry <options> -o yaml > registry.yaml

Edit registry.yaml, find the Service there,
and change its clusterIP to the address noted in step 1.

Create the registry using the modified registry.yaml:

$ oc create -f registry.yaml

Rebooting the Masters

If you are unable to re-use the IP address, any operation that uses a pull specification
that includes the old IP address will fail.
To minimize cluster disruption, you must reboot the masters:

This ensures that the old registry URL, which includes the old IP address,
is cleared from the cache.

We recommend against rebooting the entire cluster because that incurs
unnecessary downtime for pods and does not actually clear the cache.

Whitelisting Docker Registries

You can specify a whitelist of docker registries, allowing you to curate a set
of images and templates that are available for download by OpenShift Container Platform
users. This curated set can be placed in one or more docker registries, and then
added to the whitelist. When using a whitelist, only the specified registries
are accessible within OpenShift Container Platform, and all other registries are denied
access by default.

Allows the registry to act as a proxy for remote blobs. See
below for more details.

4

Allows the registry cache blobs to be served from remote registries for fast
access later. The mirroring starts when the blob is accessed for the first time.
The option has no effect if the
pullthrough is disabled.

5

Prevents blob uploads exceeding the size limit, which are defined in the
targeted project.

6

An expiration timeout for limits cached in the registry. The lower the
value, the less time it takes for the limit changes to propagate to the
registry. However, the registry will query limits from the server more
frequently and, as a consequence, pushes will be slower.

7

An expiration timeout for remembered associations between blob and
repository. The higher the value, the higher probability of fast lookup and
more efficient registry operation. On the other hand, memory usage will raise
as well as a risk of serving image layer to user, who is no longer authorized
to access it.

CloudFront Middleware

The CloudFront
middleware extension can be added to support AWS, CloudFront CDN storage
provider. CloudFront middleware speeds up distribution of image content
internationally. The blobs are distributed to several edge locations around the
world. The client is always directed to the edge with the lowest latency.

The CloudFront
middleware extension can be only used with
S3 storage.
It is utilized only during blob serving. Therefore, only blob downloads can be
speeded up, not uploads.

The following is an example of minimal configuration of S3 storage driver with a
CloudFront middleware:

The S3 storage must be configured the same way regardless of CloudFront
middleware.

2

The CloudFront storage middleware needs to be listed before OpenShift
middleware.

3

The CloudFront base URL. In the AWS management console, this is listed as
Domain Name of CloudFront distribution.

4

The location of your AWS private key on the filesystem. This must be not
confused with Amazon EC2 key pair. See the
AWS
documentation on creating CloudFront key pairs for your trusted signers. The
file needs to be mounted as a secret into the registry
pod.

5

The ID of your Cloudfront key pair.

Overriding Middleware Configuration Options

The middleware section cannot be overridden using environment variables.
There are a few exceptions, however. For example:

A configuration option that can be overridden by the boolean environment
variable REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_ACCEPTSCHEMA2, which
allows for the ability to accept manifest schema v2 on manifest put requests.
Recognized values are true and false (which applies to all the other
boolean variables below).

2

A configuration option that can be overridden by the boolean environment
variable REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_PULLTHROUGH, which
enables a proxy mode for remote repositories.

3

A configuration option that can be overridden by the boolean environment
variable REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_MIRRORPULLTHROUGH, which
instructs registry to mirror blobs locally if serving remote blobs.

4

A configuration option that can be overridden by the boolean environment
variable REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_ENFORCEQUOTA, which
allows the ability to turn quota enforcement on or off. By default, quota
enforcement is off.

5

A configuration option that can be overridden by the environment variable
REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_PROJECTCACHETTL, specifying an
eviction timeout for project quota objects. It takes a valid time duration
string (for example, 2m). If empty, you get the default timeout. If zero
(0m), caching is disabled.

6

A configuration option that can be overridden by the environment variable
REGISTRY_MIDDLEWARE_REPOSITORY_OPENSHIFT_BLOBREPOSITORYCACHETTL, specifying
an eviction timeout for associations between blob and containing repository.
The format of the value is the same as in projectcachettl case.

Image Pullthrough

If enabled, the registry will attempt to fetch requested blob from a remote
registry unless the blob exists locally. The remote candidates are calculated
from DockerImage entries stored in status of the
image
stream, a client pulls from. All the unique remote registry references in
such entries will be tried in turn until the blob is found.

Pullthrough will only occur if an image stream tag exists for the image being
pulled. For example, if the image being pulled is
docker-registry.default.svc:5000/yourproject/yourimage:prod then the
registry will look for an image stream tag named yourimage:prod in the
project yourproject. If it finds one, it will attempt to pull the image
using the dockerImageReference associated with that image stream tag.

When performing pullthrough, the registry will use pull credentials found in the
project associated with the image stream tag that is being referenced. This
capability also makes it possible for you to pull images that reside on a
registry they do not have credentials to access, as long as you have access
to the image stream tag that references the image.

You must ensure that your registry has appropriate certificates to trust
any external registries you do a pullthrough against. The certificates need to be placed
in the /etc/pki/tls/certs directory on the pod. You can mount the certificates
using a configuration map
or secret. Note that the entire /etc/pki/tls/certs directory
must be replaced. You must include the new certificates and replace the system certificates
in your secret or configuration map that you mount.

Note that by default image stream tags use a reference policy type of Source
which means that when the image stream reference is resolved to an image pull
specification, the specification used will point to the source of the image.
For images hosted on external registries, this will be the external registry and
as a result the resource will reference and pull the image by the external
registry. For example, registry.access.redhat.com/openshift3/jenkins-2-rhel7 and
pullthrough will not apply. To ensure that resources referencing image streams
use a pull specification that points to the internal registry, the image stream
tag should use a reference policy type of Local. More information is available on
Reference Policy.

By default, all the remote blobs served this way are stored locally for
subsequent faster access unless mirrorpullthrough is disabled. The downside
of this mirroring feature is an increased storage usage.

The mirroring starts when a client tries to fetch at least a single byte of the
blob. To pre-fetch a particular image into integrated registry before it is
actually needed, you can run the following command:

Manifest Schema v2 Support

Each image has a manifest describing its blobs, instructions for running it
and additional metadata. The manifest is versioned, with each version having different
structure and fields as it evolves over time. The same image can be represented
by multiple manifest versions. Each version will have different digest though.

You should be wary of compatibility issues with various Docker clients:

Docker clients of version 1.9 or older support only schema1. Any manifest
this client pulls or pushes will be of this legacy schema.

Docker clients of version 1.10 support both schema1 and schema2. And by default, it will
push the latter to the registry if it supports newer schema.

The registry, storing an image with schema1 will always return it unchanged
to the client. Schema2 will be transferred unchanged only to newer Docker
client. For the older one, it will be converted on-the-fly to schema1.

This has significant consequences. For example an image pushed to the registry
by a newer Docker client cannot be pulled by the older Docker by its digest.
That’s because the stored image’s manifest is of schema2 and its digest can
be used to pull only this version of manifest.

For this reason, the registry is configured by default not to store schema2.
This ensures that any docker client will be able to pull from the registry any
image pushed there regardless of client’s version.

Once you’re confident that all the registry clients support schema2, you’ll
be safe to enable its support in the registry. See the
middleware
configuration reference above for particular option.

OpenShift

This section reviews the configuration of global settings for features specific
to OpenShift Container Platform. In a future release, openshift-related settings in the
Middleware section will
be obsoleted.

Currently, this section allows you to configure registry metrics collection: