Cloud Services for your Kubernetes Applications

I’ve been playing around with the Kubernetes Service Catalog and in this blog post, I’d like to share some of the fun. First things first: what is this Service Catalog thing? Here’s what the documentation has to say about it:

Service Catalog is an extension API that enables applications running in Kubernetes clusters to easily use externally managed software offerings, such as a datastore service offered by a cloud provider.

Quite, obvious, no? Well, not really. Actually, we need to grasp several concepts in order to understand how Service Catalog makes Kubernetes ever more powerful. Let’s go briefly through them.

What is a Cloud Service?

A large portion of the code powering most modern cloud native applications is written by someone else. Even in enterprise environments, where trust in the origin of your code is typically of crucial importance, adoption of third party software is an established practice.

Some of this software is so popular that several companies developed a business model that revolves around productizing, managing and offering it as commoditized products. These products are often called Services.

Through public APIs, companies like Amazon, Google and Microsoft offer message queues-as-a-service, databases-as-a-service, storage-as-a-service, logging-as-a-service, and the list goes on. All of these APIs make adding advanced capabilities to any application a breeze.

What is a Service Broker?

The explosion of third party service offerings and the simplification given by the APIs to manage them is certainly making our application more interesting and reliable. In fact, managing several services for several applications rapidly becomes an operational nightmare: how do you handle credentials to the third party service providers? How do you track used and unused services? How do you inject endpoints and credentials for the services in your application?

The answer to these questions is the Service Broker. A Service Broker is a piece of software that implements the Open Service Broker API that manages the lifecycle of Services. Its job is to create new Service Instances and handle their associations to one or more applications by generating Service Bindings. Typically, a Service Binding will also provide credentials to the application that makes use of the Service.

What is the Service Catalog?

Each Service Broker exposes a definition of the Service it offers and the relative Plans associated with it. For example, a Service Broker could offer a memory cache service with different memory sizes, each one associated with a subscription plan with different costs involved. Here is an example:

The collection of these service definitions exposed by the Service Brokers it is effectively a Service Catalog, available on your cloud platform for your application to use.

The Open Service Broker API

All of these concepts are generic enough to be encapsulated in a well defined API and that is exactly what the Open Service Broker API is. It was launched in 2016 by the Cloud Foundry Foundation with the goal of creating a standardized, structured way to connect any service to any application on any cloud platform.

We can map the concepts introduced by Open Service Broker API to the API Resources provided by the
servicecatalog.k8s.io API in Kubernetes:

Shell

1

2

3

4

5

6

7

8

Open Service Broker API Kubernetes servicecatalog.k8s.io Kinds

Platform Kubernetes

Service Broker ClusterServiceBroker

Service ClusterServiceClass

Plan ClusterServicePlan

Service Instance ServiceInstance

Service Binding ServiceBinding

Show me the money

I wanted to see the Kubernetes Service Catalog in action, so I decided to give it a shot. At one of our clients, we set up a CI/CD pipeline in which each feature branch was deployed in a separate namespace and a bunch of smoke tests were run against it. Each feature deploys the entire platform, including backend services like databases.

This seemed like the perfect scenario to test some of these ideas. I wanted to be able to provision a database for each feature branch without having to worry about credentials and their lifecycle. To make things a little spicier, I am going to host all the Services, Service Provider and Service Broker on the same Kubernetes cluster!

Here’s a diagram of what we want to achieve:

To keep things simple and reproducible, I used minikube 0.25.2 with RBAC and CSR siging enabled.

Services

So we want to provide our Applications with SQL Databases services. To do this I decided to use one of my favorite open source project, CockroachDB. Luckily, there is a Helm chart that makes deploying it very easy:

Service Broker

Since the Service Catalog implements the Open Service Broker API, we can reuse any Service Broker that respects the same API. For this hands-on example, I am going to use a CockroachDB Service Broker I found on github.

To make it run on Kubernetes all I needed to do is fork it, dockerize it and write a few manifests (and fix a couple of bugs :)). The Service Broker runs as any other application. So it has a deployment and a service. Typically, a third party provider will expose a Service Broker on their own infrastructure, but as I mentioned above, I decided to run it on my own same Kubernetes cluster.

The Service Broker reads its configuration from environment variables which are defined in the env section of the deployment manifest I created:

The Service Catalog will extract this information and will make it available in the Kubernetes cluster. To make the Service Broker known to the Service Catalog we need to create a ClusterServiceBroker object:

YAML

1

2

3

4

5

6

apiVersion: servicecatalog.k8s.io/v1beta1

kind: ClusterServiceBroker

metadata:

name: cockroachdb-service-broker

spec:

url: http://user:pass@crdb-service-broker.default

Let’s install the Service Broker and check the resulting entity created in the Service Catalog:

pnembfjkbagknfdfbkpekmihebkllnkf@cockroachdb-public.default.svc.cluster.local:26257/cf_nnmpifmgiloklfaofkpdjlmjbaieacke> SHOW TABLES;

+---------+

| Table |

+---------+

| mytable |

+---------+

(1 row)

Time: 14.004174ms

Under the hood

When we create a Service Instance the Service Broker creates a new database (as in CREATE DATABASE) in the CockroachDB cluster with a new user which own that database and an associated password. As a result, we can see this by connecting as root to the CockroachDB cluster:

warning: no current database set. Use SET database = <dbname> to change, CREATE DATABASE to make a new database.

root@cockroachdb-public:26257/> show databases;

+-------------------------------------+

| Database |

+-------------------------------------+

| cf_nnmpifmgiloklfaofkpdjlmjbaieacke |

| system |

+-------------------------------------+

(2 rows)

Time: 9.212959ms

When unbinding this service (i.e. kubectl delete serviceinstance) the database is removed from the CockroachDB cluster.

Conclusions

The Service Catalog offers powerful abstractions to make services available in a Kubernetes cluster. These services are typically third-party managed cloud offerings but there is nothing preventing us from offering self-hosted services as demonstrated. On the one hand, this allows Kubernetes operators to integrate third party services and offer fine tuned services hosted on the cluster itself. On the other hand, developers can focus on the applications without the need for managing complex services deployment.