It's been an interesting journey watching the different components of Cloud Foundry evolve since its first release back in 2011. The Data Services are just one of the major components to receive a fresh overhaul in v2 of the Cloud Foundry project.

Why Node.js?

Here at ActiveState we think that having a wide choice of data services allows for greater innovation and optimization of your apps and adding new ones to Cloud Foundry should be a simple process.

We also enjoy using Node.js and taking advantage of the vigorous growth in the community supporting it. With 160+ new modules being added to NPM every day, there is a lot of good code available that's useful for writing REST servers.

At its core Node.js provides everything you need to get started including:

The npm module can be used via 'require' or as a standalone broker using the CLI.

Provides service record persistence via a LevelDB backing store

Old CFv1 Services

The v1 service architecture generally worked well, but the separation of the service into two distinct components meant that either of these were potential single-points-of-failure. With the v2 services you can set up as many brokers as you need and these can all interface to the same data service to scale horizontally.

Writing v1 services was a slightly awkward affair as you were generally limited to the Ruby implementation which involved building on top of two large Ruby repositories. If the Cloud Controller was updated, then these needed to be updated to ensure compatibility. You also had to house the service processes in a location that could be reached by both a NATS message bus and the Cloud Controller API. This generally restricted services to the same network as the Cloud Foundry cluster itself.

CFv1 Gateways were designed to advertise any number of services to the Cloud Controller that operated under its domain. Its responsibility was to run an asynchronous REST server that solicited bidirectional communication with the cloud controller to advertise the service catalog. It would also maintain synchronization with the service nodes over the local NATS message bus meaning the gateway and node were tightly coupled.

CFv1 Nodes were primarily responsible for managing specific service plans. It was the the primary orchestrator between the actual data service and the gateway and had to maintain the correct set of services dictated by the gateway. Most of the CFv1 core suite of services stored particular provisions of a service within a local SQLite database to ensure consistency.

Detailed information regarding the legacy v1 services implementation can be found here.

New CFv2 Services

The v2 services API brought about a few welcome changes, such as more detailed plan attributes for a more granular billing API, authenticating all requests via HTTP basic auth (previously this was using shared tokens) and a more visible change to how external services are managed with a Cloud Foundry installation. All catalogued services and plans must now have a unique ID assigned to them, and their legacy v1 "version" and "provider" counterparts are deprecated in v2.

Under the hood, the v2 services API introduced a slightly different mode in which you would implement a service. The concepts of the gateway and node as two separate entities no longer exists. Now, there is only a "broker" which is responsible for:

Implementing a REST server to interface with the Cloud Controller

Authenticating requests using HTTP basic auth

Providing an interface to the data service itself for all provision/unprovision & bind/unbind events

Maintaining a catalog of all available services and associated service plans

Maintaining a consistent record of the provisioned service instances and bindings to ensure services are persistent between broker restarts

Service MarketPlace

Another welcome change was the introduction of a centralized "MarketPlace" system, which allows the CF admin to install services and make them available on a per-organization basis. To enable the marketplace to further support externalized services, the brokers now only require unidirectional communication. This in turn means that you no longer need to expose the Cloud Controller HTTP port over a public or untrusted network to the service broker.

Registering v2 Services

Unlike v1 services, with v2 services you will need to register the broker with a separate service API call. For example, using the cf client:

Broker Implementation

We've kept it minimal to allow greater flexibility with your own broker implementations, with the central idea being that you simply supply it with a JSON configuration file detailing the services and associated plans and then simply listen for the events as the cloud controller initiates them:

So far, so simple. Note that your own broker must subscribe to all the events emitted above.

Broker Configuration

With our Node.js implementation of the service broker, all configuration for a broker is encapsulated in a JSON config file.

Here is the configuration file that the above broker uses.

increment-services.json:

{"apiVersion":"2.1.0","authUser":"<changeme>","authPassword":"<changeme>","defaultVersion":"1.0","name":"Increment Service","port":5006,"services":[{"id":"47c6bff8-c653-4812-89f8-ee9a9b6c3d55","bindable":true,"name":"Increment Service","version":"1.0","description":"Increment Service - increment","metadata":{"providerDisplayName":"Increment Service Ltd.","tags":["increment","demo"]},"plans":[{"name":"default","id":"6b6fdd28-eb91-4eb7-9936-bf82ec1ec2d5","description":"This is the first plan","public":true,"free":true},{"name":"secondary","id":"a3d1fb32-ba06-4122-9013-d42ef84d8b75","description":"This is the secondary plan","public":true,"free":false}]}],"database":{"enabled":true,"allowOrphanedBindings":false,"databaseFile":"./leveldb","encryptionKey":"!!changeme!!"}}

Hopefully, the config above is fairly self explanatory. Most importantly, don't forget to change "authUser", "authPassword" and if using the built in database, "encryptionKey".

If you set the "allowOrphanedBindings" to true, the broker will return a 500 error if it tries to delete a service instance that has bindings. You may want to automatically delete all bindings yourself when a service instance is deleted so this is set to false by default.

We hope this enables you to start effortlessly writing data services for Stackato 3 and Cloud Foundry v2. If you encounter any issues or want to provide feedback, then please open a Github issue or join the #stackato channel on irc.freenode.net.

Share this post:

I've been a DevOps engineer at ActiveState for over three years, with a passion for the new league of cloud platforms, virtualization technologies, and software networking with a drive to make Stackato the leader in its field. I've worked with big companies such as HP and Mozilla to bring PaaS to their repertoire of products. I can usually be found tinkering with CI systems, debugging deep inside all layers of the TCP/UDP networking stack and making software work better for everyone.