Dynamic iSCSI Provisioning with Target

Introduction

Often in the early stages of implementing OpenShift with our customers, we find they don’t have high quality distributed storage available to OpenShift. It isn’t because there isn’t distributed storage in the organization, most of the time it’s just that the storage team has not been involved (yet) in the OpenShift initiative.

The OpenShift Ansible installer takes into account this scenario and allows you to create an NFS server as part of the installation. This option creates a non-HA NFS server which is good enough for non-production environments and provides distributed file storage.

The registry can use this storage, but logging and metrics (and potentially some customer’s applications) require block storage.

When reasoning on this problem, a colleague of mine (Clark Hale) and myself noticed that it would be relatively straightforward to create an iSCSI server using target.

This would enable a non-HA block storage solution with a similar purpose as the NFS server installed by the Ansible Playbook to be used in non-production environment until an HA storage solution is implemented.

We also noticed that the target exposes a dynamic provisioning API via the targetd RPM package. So we built an out-of-tree iscsi/targetd dynamic provisioner.

We merged our initial project with the external-storage section of the kubernetes-incubator repo. The remainder of the article explains how to setup the iscsi server and configure dynamic provisioning.

Configuring Dynamic Provisioning

Pointing Logging and Metrics to an iSCSI Volume

As we said in the introduction, OpenShift logging and metrics require block storage.

Assuming you have installed logging and metrics with the OpenShift Ansible Playbook, the following instructions will refactor them to use iSCSI (notice that by running them you will lose the current content of the Cassandra and Elasticsearch database).

Conclusions

The Ansible Playbook and the dynamic provisioner for iSCSI target are meant to facilitate the implementation and configuration of iSCSI block storage when a more enterprise alternative is not available. Normally the iSCSI target is not configured to be HA (and I doubt it can be configured to be so) therefore I don’t recommend using it in production environments.