CEO of Stark & Wayne, a consultancy for people who love Cloud Foundry, BOSH and DevOps in general.

Perhaps the best way to feel confident using Cloud Foundry is to know how it works. And perhaps the best way to learn how it works is to rebuilt it from the ground up. In the DIY PaaS articles, we will re-build Cloud Foundry from the ground up piece-by-piece. This is article number 2, and we're going to take a specific application, stage it into a generic droplet, and run it.

TL;DR

At the core of Cloud Foundry, or your own DIY PaaS, is a way to run arbitrary applications - Ruby, Java, PHP. The DEA, which actually runs applications, doesn't know anything about types of applications. Something else needs to create the droplets for the DEAs which know about Ruby language and about Ruby on Rails framework, for example.

It's a stager that creates droplets.

Skip the tutorial, just run something quickly for me!

You're busy. You want to see something shiny. Here, run this and you'll see a Stager and a DEA running, and being told to stage and then run a Ruby/Sinatra application via a message bus:

Now, if you'd like to learn more about what just happened, then let's get started!

Preparation

Everything in this tutorial can be done on your local computer. The wonders of cloud computing are for another day. I already have Ruby 1.9.3 installed on my laptop and available in my $PATH.

As I go along, I'll clone/submodule the Cloud Foundry repositories that I need and show the bare minimum configuration files. The final product is available in a git repository as a demonstration of the minimum parts of Cloud Foundry required to run an application.

Feeding the DEA with droplets

Each time you "add an instance" (1) of a running application, the DEA takes a packaged version of an application (a droplet), unpacks it, and runs a single startup script. The DEA knows nothing about Ruby or Java or PHP; it only knows about the startup script within the package. How does the package get created?

The package that includes an executable startup script and the original application is known to the DEA as a "droplet". It is created during a staging step. It is the staging step that knows about Ruby and Java and PHP applications and how to run them. This article investigates how Cloud Foundry stages an application and creates the package for DEAs.

The stager is nicely isolated from the rest of Cloud Foundry. Its only external dependencies are:

NATS server

HTTP endpoint for getting a zipped version of the application

HTTP endpoint for posting the droplet package

As I go along, I'll reference the projects/repositories that I need and show configuration files and helper scripts.

This token is replaced by the DEA when it unpacks the staged package to use the runtime selected for the application. In this example, this might be the ruby executable for either the ruby18 or ruby19 runtimes.

Staging as a service

This staging code is not run within the DEA, nor is it run within the public API (called the Cloud Controller). It is its own service within a running Cloud Foundry installation.

Above we are running one stager. You can run multiple stagers within Cloud Foundry. Each of them watches a NATS queue staging (configuration queues above) and pops the requests off into an internal thread pool (configuration max_active_tasks sets the thread pool size, if you're into optimizations based on the RAM & CPU of server running the stager).

That is, we can tell a stager to stage an application in preparation for deploying it to a DEA by publishing a message to NATS on a queue 'staging'.

The properties document is equivalent to the cfg_filename contents from the vcap-staging example above.

See below for a fleshed out example. But first we need to look at download_uri and upload_uri.

What makes playing with the stager in isolation tricky is that you need to host the original source code via an HTTP URI (download_uri above). And even more tricky, you need to provide an HTTP endpoint for uploading the resulting staged package, (upload_uri above), which will then be used by DEAs.

In Cloud Foundry, the download/upload endpoints are in the Cloud Controller.

Summary

Cloud Foundry has a standalone service, stager, that listens for requests on NATS to stage a zipped application into a droplet - a package that contains the original application together with startup and stop scripts.

Next, combining the stager and the DEA

In the next DIY PaaS article, we will take the next logical step: create a droplet and make a DEA download it and run it

Follow @starkandwayne for the release of the next article and other blog posts from the wonderful world Cloud Foundry.

Footnotes

(1) I don't agree with this terminology in Cloud Foundry. When you add "instances" to a running application, you are actually adding running processes. If you've used Heroku, this is akin to adding dynos. To me, "instances" means virtual machines or servers. In Cloud Foundry, it means "processes of your app". In future Cloud Foundry, it will be a Linux container running an application process.