Categories

Cloud Foundry is a wonderful on-premise PaaS that makes it very easy to build, deploy while providing scalability and high availability to your stateless applications. But Cloud Foundry is really a Application Platform Service and does not provide high availability and scalability for your data. Fortunately, there is Amazon RDS, which excels in providing this as a service.

In this blog I will show you how easy it is to build, install and use a Cloud Foundry Service Broker for Amazon RDS. The broker was developed in Node.JS using the Restify framework and can be deployed as a normal Cloud Foundry application. Finally, I will point you to a skeleton service broker which you can use as the basis for your own.

Cloud Foundry Service Broker Domain

Before I race of into the details of the implementation, I would like to introduce you into the Cloud Foundry lingo. If you are aware of the lingo, just skip to the paragraph ‘AWS RDS Service Broker operations’.

Service – an external resource that can be used by an application. It can be a database, a messaging system or an external application. Commonly provided services are mysql, postgres, redis and memcached.

Service Plan – a plan specify the quality of the service and governs the amount memory, disk space, nodes etc. provided with the service.

Service Catalog – a document containing all services and service plans of a service broker.

Service Broker – a program that is capable of creating services and providing the necessary information to applications to connect to the service.

Now a service broker can provide the following operations:

Describe Services – Show me all the services this broker can provide.

Create Service – Creating an instance of a service matching a specified plan. When the service is a database, it depends on the broker what this means: It may create an entire database server, or just a new database instance, or even just a database schema. Cloud Foundry calls this ‘provisioning a service instance’.

Binding a Service – providing a specific application with the necessary information to connect to an existing service. When the service is a database, it provides the hostname, portname, database name, username and password. Depending on the service broker, the broker may even create specific credentials for each bind request/application. The Cloud Controller will store the returned credentials in a JSON document stored as an UNIX environment variable (VCAP_SERVICES).

Unbind service – depending on the service broker, undo what what done on the bind.

Destroy Service – Easy, just deleting what was created. Cloud Foundry calls this ‘deprovisioning a service instance’.

In the next paragraph I will map these operations to Amazon AWS RDS services.

AWS RDS Service Broker operations

Any Service Broker has to implement a REST API of the Cloud Foundry specification. To create the Amazon AWS RDS service broker, I had to implement four out of five methods:

describe services – returns available services and service plans

create service – call the createDBInstance operation and store generated credentials as tags in with the instance.

Design decisions

That does not look all too difficult does it? Here are some of my design decisions:

Where do I store the credentials?

I store the credentials as tags on the instance. I wanted to create service broker that was completely stateless so that I could deploy it in Cloud Foundry itself. I did not want to be dependent on a complete database for a little bit of information. The tags seemed to fit the purpose.

Why does bind return the same credentials for every bind?

I wanted the bind service to be as simple as possible. I did not want to generate new user accounts and passwords, because if I did, I had even more state to maintain. Even more, I found that if I bind two applications to the same MySQL service, they could see each others data. So why bother creating users for binds? Finally, making the bind service simple, kept the unbind service even simpler because there is nothing to undo.

How to implement different service plans?

The createDBInstance operation of AWS RDS API operation, takes a JSON object as input parameter that is basically the equivalent of a plan. I just had to add an appropriate JSON record to the configuration file for each plan. See the description of the params parameter of the createDBInstance operation.

How do I create a AWS RDS service within 60 seconds?

Well, I don’t. The service broker API states that you have to create a service within the timeout of the cloud controller (which is 60 seconds), but RDS takes a whee bit more time. So the create request is initiated within seconds, but before you can bind an application to it may take a few minutes. Nothing I can do about that.

Why store the service broker credentials in environment variables?

I want the service broker to be configured upon deployment time. When the credentials are in the config file, you need to change the files of the application on each deployment.

Installation

In these instructions, I presume you have access to an AWS account and you have an installation of Cloud Foundry. I used Stackato which is a Cloud Foundy implementation by ActiveState. These instructions assume you are too!

Create a AWS IAM user
First create a AWS IAM user (cf-aws-service-broker) with at least the folllowing privileges

Assign privileges to execute AWS RDS operations
The newly created IAM user needs the privileges to create RDS databases. I used the following permissions:

Install the service broker
This script is a cunning implementation which create the service broker in Cloud Foundry and makes all the plans publicly available. In stackato we use the curl commands to achieve this. This script requires you to have installed jq, the wonderful JSON command line processor by Stephen Dolan.

[bash]
stackato bind-service mysql-844b1 paas-monitor
Binding mysql-844b1 to paas-monitor … Error 10001: Service broker error: No endpoint set on the instance ‘cfdb-3529e5764’. The instance is in state ‘creating’. please retry a few minutes later (500)
[/bash]