Cloud Foundry Tutorial : Taking a PaaS on the hard stuff

Unless youve been living under a rock for the last five years, youve probably heard a lot of buzz surrounding clouds, and  unless somebodys given you a clear, specific introduction, youre no doubt leery of the term. Andy Piper and Josh Long dig deeper into Cloud Foundry

Unless you’ve been
living under a rock for the last five years, you’ve probably heard
a lot of buzz surrounding “clouds,” and – unless somebody’s given
you a clear, specific introduction, you’re no doubt leery of the
term. Broadly, “clouds” are being used to describe applications and
services that are delivered over the internet through a
self-service consumption model: need e-mail but don’t want to learn
how to set up Microsoft Exchange or Sendmail? Just sign up for an
e-mail account with GMail. Need presentation software or backup
software? Sign up for SlideRocket or Mozy. And, if you need a place
to run your next application, but don’t want to invest in setting
up a datacenter and purchasing or requisitioning hardware, then you
can use a compute cloud. This last segment is what we’re going to
look at in this article.

IaaS, or Infrastructure-as-a-Service, describes
the ability to dynamically set up operating systems (and
operating-system resources like RAM, hard disk space, etc.). Common
examples of IaaS solutions include Amazon Web Services, and
VMware’s vCloud Director. These solutions let you treat a large
array of computing resources – hard disks, memory – as a pool from
which individual instances may be drawn and used. If your problem
is, “I want to set up a new Linux (or Windows or Solaris) machine
quickly,” then IaaS is probably what you’re looking for. For a long
time, thiswaswhat most people thought they wanted.

After all, once you can set up new servers with
the literal push of a button, then how “hard could it be to get a
working application up and running?” we collectively balked. The
trouble is that for most application of any significance, it can
bequitedaunting. After all, suppose your application is/.’d(or “Oprah”’d or
Digg’d), will standing-up Linux machines be enough to meet the
demand? Will you be able to get from a Linux machine to a
load-absorbing, redundant application instance in time to make a
difference?

If you’re
on-ramping new developers, will those developers be able to deploy
your code base and iterate with it quickly? Or, will they spend the
day (week?!) figuring out the unique permutations of drivers,
operating system revisions, directory structure combinations and
networking settings to get the application even running?
Platform-as-a-Service is the answer to these questions.

There is one other dimension to this discussion:
choice. It’s important to point out what “cloud” doesnotimply. “Cloud”
doesnot imply
publicly-hosted, and it doesnotimply lock-in. AgoodPaaS technology will run both on-premise
or in somebody else’s datacenter. AgoodPaaS will be open source, and
applications written against it portable and easily moved from one
environment to another as the requirements dictate.

Enter
Cloud Foundry

CloudFoundry.com is a publicly hosted cloud environment (sign up
for an account at www.cloudfoundry.comthat is free while the hosted
account is in beta), as well as a solution that you can install
entirely behind your firewall. You can use BOSH, the Cloud Foundry
orchestration engine, to install Cloud Foundry entirely behind your
firewall on vSphere, or on
Amazon Web Services. Naturally, because it’s open source, there
are still other ways to install it in your environment, including
Chef recipes and an Ubuntu package.

Cloud Foundry is licensed under the Apache 2 license and has
enjoyed very successful ecosystem growth. People have taken the
code, forked it, and created their own, more specialized versions
of Cloud Foundry. There are, let’s say, more hobby-ist centric
ports of Cloud Foundry – supporting
Haskelland Erlang – as well
as more robust, productized implementations. ActiveState provides a
private PaaS called Stackato that supports
Python and Perl, in addition to the supported runtimes and
languages included in the base release. ActiveState, as many will
know, is the leading maker of IDEs and tools for languages like
Python, Perl, and TcL. AppFog
(formerly PHPFog) provides a PaaS based on Cloud Foundry that
supports PHP.

This is all important because it gives you freedom: You can run
the application on a publicly hosted environment if you like, to
simplify getting started, and – if scale, QoS demands, supported
services, or regulatory restrictions require – feel no hesitation
in moving the cloud (and your application) in-house, confident that
the same command line tools, APIs, and services are supported
across environments.

Deploying an
application

There are a few
ways to get started. The easiest, perhaps, is to sign up on the
public site and create an account there. As development becomes
more involved, you may decide you’d like to iterate quicker and
work with a cloud that’s running locally. For this very specific
use case, consider the Micro Cloud Foundry (MCF). MCF is a virtual
machine image that runs in VMware Player on Linux and Windows, and
VMware Fusion on OSX (Player is free, but Fusion has a nominal
price attached to it.) You can sign up for an account and
download the image for free from the MCF website
then simply follow the
setup guide. For our purposes, let’s simply work with the
online site.

There are a few tools we need to effectively work
with any Cloud Foundry instance: the vmc command line tool, and – if you’re a developer
working with the JVM (Spring, Grails, Scala, regular applications,
etc.) – the Eclipse support for Cloud Foundry.

There are many ways to interact with Cloud
Foundry. Cloud Foundry’s public facing endpoint, called thecloud controller, is a
RESTful API through which all commands pass and eventually effect
some sort of response from one of the components in the cloud’s
architecture itself. While you manipulate Cloud Foundry using this
RESTful API itself, it’s easier to use language and client-specific
integrations. The vmc command line tool provides a great way to interact with
Cloud Foundry for those more comfortable banging around in the
shell; the Eclipse support provides a WTP-connector view of Cloud
Foundry; both the Grails and Roo RAD frameworks provide add-ons,
and there is support for deploying applications with a Maven
plug-in.

Signing up for a
CloudFoundry.com account

Working with
vmc

The instructions
are presented here, but they may change in time, so be sure to
consult
the official documentation section for the latest and greatest,
if anything is in doubt. The vmc command line tool is written in
Ruby. So, first, you need to install Ruby on your operating system.
At the moment, in the second quarter of 2012, Ruby versions 1.9.2
(or later) and 1.8.7 (or later) are supported. On Linux, you can
use your operating system’s package manager to install Ruby (if
it’s not already installed). If you’re on Windows, you can use the
Ruby installer from
RubyInstaller.org. On OSX, things are less rosy. The built-in
Ruby is fragile. It’s easier to install something like the Ruby
Version Manager (rvm) which lets you maintain multiple concurrent
installations of Ruby on a given machine. The setup for Ruby, with
rvm, looks like this:

Once you have Ruby installed, getting the vmc command line
tool’s a snap! Ruby comes with a package management tool called
gem, which it uses to manage dependencies. It’s akin to a
Ruby-specific version of the yum, deb, or apt package management
systems on various Linux flavors. To install vmc, you will use the
gem command line tool, like this:

$ sudo gem
install vmc

If you’re running on Windows, then remove the wordsudo, whichis used on Unix variants like OSX and Linux to
obtain elevated privileges when invoking a command, and is
unnecessary on Windows. Once vmc is installed, open a new shell to
test some of the basics out. When working with Cloud Foundry
instances, you need to follow some basic workflow steps.

First, you need totargetthe correct instance, since Cloud Foundry
is not one cloud by one vendor, but a stack that anybody can run.
This is similar to the way you work with Git and Subversion: You
wouldn’t install theGitcommand and
expect it to automatically know which host toclonethe source code from, would
you? As a good rule of thumb, targeting involves specifying
theapiURL for
the Cloud Foundry instance. This is pretty consistent of the
patternapi.foo.com, wherefoo.comis the host of your Cloud Foundry instance. Here’s
my console session targeting CloudFoundry.com:

$ vmc
target api.cloudfoundry.com

Successfully
targeted to [http://api.cloudfoundry.com]

Then,
you need to login. You only need to login once for a given
session.

$ vmc
login

Attempting login
to [http://api.cloudfoundry.com]

Email:
jolong@vmware.com

Password:
*******

Successfully
logged into [http://api.cloudfoundry.com]

Working with Cloud Foundry

If everything succeeds, you can now start to ask questions and
interact with Cloud Foundry. Here’s my output of the vmc info
command.

$ vmc
info

VMware’s Cloud
Application Platform

For support visit
http://support.cloudfoundry.com

Target:
http://api.cloudfoundry.com (v0.999)

Client:
v0.3.16.beta.5

User:
jolong@vmware.com

Usage: Memory
(1.6G of 2.0G total)

Services (6 of 16
total)

Apps (3 of 20
total)

Another handy command is thevmc appscommand, which tells you which applications you’ve got
running. I get the following output when I run the vmc apps
command:

$ vmc
apps

No
Applications

Is there anything more depressing? We need to deploy an
application!

I’ve created a simple Spring MVC application that runs on
both your local machine (working with an in-memory database, called
H2, and working with Cloud Foundry-managed instances of PostgreSQL, a powerful relational
database and Redis, the lightning
fast data-structure service. To get the code, simply clone it from
thecloudfoundry-samples
GitHub project.

The application is a Java project, and uses Maven for its build. To compile the
code, you simply enter the root of the project and then run the mvn
install commands, like this:

$ mvn
install

Now that we’ve got the platform, and an application, let’s
deploy it using thevmctool. I can
imagine your thoughts: “C’mon, man! We can’t deploy this thing yet!
It’ll blow chunks – youjustfinished telling us it needs PostgreSQL and
Redis!” And, that’s a good point! But, don’t worry! We’ll take care
of deploying the application and configuring those services in one
stroke. You see, the vmc toolhasto rank among the friendliest of command
line tools out there. Let’s walk through the push
interaction:

$ vmc –path
target/springmvc31-1.0.0 push

This tells vmc to push the compiled code in the directory
specified in the–pathargument. If you don’t specify–path, it looks for
the code to deploy in the current directory. If you’re deploying a
project in a language that doesn’t have a compiled form, then
specify the root directory of that project instead.

Application Name:
springoncloudfoundryforjax

Specify a name for the
application. The name you specify must be different than the one
I’ve specified here (otherwise, there would be a collision!)

vmc has detected the type of the application (“a Java
SpringSource Spring Application”). We have instead selected option
5, “JavaWeb,” because our application doesn’t need to beauto reconfigured.
Auto reconfiguration is a service that Cloud Foundry provides when
applications are uploaded: It redefines the definition of services
in certain types of applications and automatically binds them to
the appropriate definition for the data source managed by Cloud
Foundry. This is useful if you have only one data source, or one
Redis declaration, and would simply like to have that automatically
rewired to work on Cloud Foundry. The alternative is to dynamically
look up the connection information from a well-known context, like
theVCAP_SERVICESenvironment variable that is set in the Cloud Foundry
environment. There is also a Java
API, called thecloudfoundry-runtime which provides a
type-safe Java API to do the same work. The application already
looks up the definition using thecloudfoundry-runtime,
and does not need auto-reconfiguration.

Memory
reservation (128M, 256M, 512M, 1G, 2G) [512M]:

How many
instances? [1]:

Next, vmc asks us to specify some of the runtime characteristics
of the application: How much memory does it need? How many
instances should be started? For now, we’ll just accept the
defaults.

Bind existing
services to ‘springoncloudfoundryforjax’? [yN]: n

Create services
to bind to ‘springoncloudfoundryforjax’? [yN]: y

1:
mongodb

2:
mysql

3:
postgresql

4:
rabbitmq

5:
redis

What kind of service?: 3

Specify the name of the service
[postgresql-63c39]:

Create another? [yN]: y

1: mongodb

2: mysql

3: postgresql

4: rabbitmq

5: redis

What kind of service?: 5

Specify the name of the service
[redis-48adb]:

Create another? [yN]: n

Finally, the services: the Redis and PostgreSQL instances,
are set up. Cloud Foundry is nice enough to ask us if we need
services for our application and prompts us to specify as many
services as we need. Services in Cloud Foundry are not coupled from
the applications that consume them: You can specify as many
services as you like. They must beboundto the applications that use
them. You may share services across applications, and may use as
many services as you like for any application. Our application is
written to lookup references bytype, not by name, so it does not matter how
we name the services.

Would you like to save this configuration?
[yN]: y

Manifest written to
manifest.yml.

Creating Application: OK

That wastedious! Sure, it did deploy our application and
completely configure the services we need, but I’d sure hate to
answer all those questions every time I deploy or update an
application. No, no. Leave that to somebody else, I say. This is
what the Cloud Foundry manifests provide: a manifest of everything
Cloud Foundry needs to deploy the application without prompting
you. This makes it easy to share your configuration with others.
Cloud Foundry’s even willing to write themanifest.ymlfile for
us!

Creating Service [postgresql-63c39]:
OK

Binding Service [postgresql-63c39]:
OK

Creating Service [redis-48adb]:
OK

Binding Service [redis-48adb]:
OK

Uploading Application:

Checking for available resources:
OK

Processing resources: OK

Packing application: OK

Uploading (12K): OK

Push Status: OK

Staging Application
‘springoncloudfoundryforjax’: OK

Starting Application
‘springoncloudfoundryforjax’: OK

At this point, the application’s available and running! You
can confirm this by checking the output ofvmc apps
(Figure 1).

Figure 1: Output of vmc
apps

That’smuchbetter! Bring up the application by opening the URL
specified in the output of vmc apps in your browser. This
application has a simple UI to let manipulate records of customer
data. There is also a RESTful endpoint that you can hit at
/crm/customers. The result is a JSON structure with individual
records. When you first deploy the application, there are no
records! Let’s fix that.

The light
(of data) at the end of the tunnel

We’ve had an easy time of deploying an application thus
far, but the operations and DBAs reading this are probably
nervously tapping their fingers at this point: “Where is my
database console? My backup scripts?” Indeed, the operations people
in your organization are the ones who have to wear the beeper when
they go to bed, and they want to know that they can get access to
the data with ease, or run their routine diagnostic and backup
jobs, just like before. Cloud Foundry providescaldecott(named for
the caldecott tunnel in San Francisco, in case you’re
wondering…), which is a tunnel. Caldecott lets you poke a hole in
the cloud, connecting you to your services running inside the cloud
from the outside. Let’s connect to our PostgreSQL instance and
install some data using the vmc tunnel command.

$ vmc tunnel

1: postgresql-63c39

2: redis-48adb

Which service to tunnel to?:
2

The command will list the services that we have created, and to
which we can establish a tunnel. Specify the number for the
PostgreSQL instance in your account, in my case, #2.

Deploying tunnel application
‘caldecott’.

Uploading Application:

Checking for available resources:
OK

Packing application: OK

Uploading (1K): OK

Push Status: OK

Binding Service [postgresql-63c39]:
OK

Staging Application ‘caldecott’:
OK

Starting Application ‘caldecott’:
OK

Getting tunnel connection info:
OK

Service connection info:

username :
u63a3924f90c6446f9737390675f1a9e0

password :
p9ccfd8a6b4d04e308396a872a9711155

name :
d040a2dbbc977418f9eb599a2245fa96d

By default, the command outputs the information you’d need
to establish your own connection from any client, command-line or
not: the host, port, username, password andname, which basically just meansschema, for
most databases.

If that weren’t enough, vmc will even start up many
well-known, local command line tools likepsqlfor you! I let it, and it dumps me into the standard
psql shell, where I can execute
my SQL statements. The project provides some sample data in the
filesrc/main/resources/import_psql.sql. Copy and paste that into the shell, and
then revisit the/crm/customersJSON dump page. You should see your new data
reflected in the results. The tunnel is a powerful tool, and
provides everything you need for ad-hoc queries, as well as
operational features like nightly dumps of the data.

Closing Thoughts

We’ve looked at how to deploy a
simple Spring application to Cloud Foundry entirely from the
command line. We did not look at how to use the Eclipse WTP
support, which is ably covered on the Cloud Foundry website and in
the
SpringSource blog. We also have not looked at the development
model support for Spring, Ruby, Node.js, Scala, etc. Cloud Foundry
is, after all, a polyglot PaaS!

If you’re a Spring developer, then you have a lot of clean
integrations to help you along; it provides a rich set of APIs that
maps nicely to the supported services on Cloud Foundry: Spring AMQP
and Spring Integration make RabbitMQ-based projects very easy.
Spring Core and Spring Batch make batch workloads simply with MySQL
and PostgreSQL. Spring Data Redis and Spring Data MongoDB both
provide rich support for Redis and MongoDB. If you’re a Java
developer, and want to build applications on top of Cloud Foundry,
you’re in luck!

At this point you’ve got all you needed to get started and
plenty of information on where to go from here. If you have any
questions, don’t hesitate to reach out to me atjosh.long@pringsource.com, or reach out at one of the numerous public
forums on the Cloud Foundry website. So, sit back, relax, and enjoy
the flight to the clouds!

——-

Author Bios:

Josh Long (@starbuxman, and josh.long@springsource.com)
is the Spring developer advocate. Josh is the lead author on
Apress’ Spring Recipes, 2nd Edition, O’Reilly’s “Getting Started
with Spring Roo,” Manning’s “Cloud Foundry in Action.” and a
SpringSource committer and contributor. When he’s not hacking on
code (for Spring Integration, Spring Batch, Spring MVC, Activiti,
and much more), he can be found at the local Java User Group or at
the local coffee shop. He blogs at springsource.org, blog.springsource.org or
joshlong.com.

Andy Piper is part of the VMware Developer Relations
team, and works as Developer Advocate for Cloud Foundry, the Open
Source Platform-as-a-Service. He is probably best known online as a
“social bridgebuilder” spanning a number of areas of technology and
interest. He was previously with IBM Software Group for more than
10 years, as a consultant, strategist, and WebSphere Messaging
Community Lead.