Sunday, July 20, 2008

Defining Cloud Computing

Generally, when a new fad [1] comes along in the software industry there is a fantastic initial rush of enthusiasm, buoyed by the initial success and enthusiasm of the innovators...

...This is generally followed by a series of attempts to define just what the fad is. We've seen this in recent history with topics such as Service Orientated Architecture (SOA) or Web 2.0 - or even things that seem very specific at the outset, like AJAX. There are countless other examples going back further.

This is now the case with the term Cloud Computing. Cloud Computing is difficult to define, but it's very generally about the commoditisation of computing resource. Instead of buying a server, with a specific processing and storage capacity - you simply buy an allocation of capacity from a vendor, a capacity that you can then increase or decrease as necessary. Instead of having a number of discrete resources (e.g. Ten disks on ten different servers), you get access to storage as one, virtual service.

It seems a simple proposition from many perspectives, but in reality the distinction is pretty subtle. After all, buying a physical server in principle achieves the same result - you pay and get more capacity. However, the real difference is in the definition of the service you get, and the mechanics for accessing that service - which is where the definition gets complicated.

So Cloud Computing is not unique in that is lacks an explicit boundary. Definitions in the SYS-CON Article [2] varied quite a bit. Some of the commentary included:

..the broad concept of using the internet to allow people to access technology-enabled services

Most computer savvy folks actually have a pretty good idea of what the term "cloud computing" means: outsourced, pay-as-you-go, on-demand, somewhere in the Internet, etc.

Clouds are the new Web 2.0. Nice marketing shine on top of existing technology. Remember back when every company threw some ajax on their site and said "Ta da! We’re a web 2.0 company now!"? Same story, new buzz word..

One of the definitions I particularly liked was by Yan Pritzker [3]. This definition makes three key points (refer to the article for the full explanation):

Clouds are vast resource pools with on-demand resource allocation.

Clouds are virtualized.

Clouds tend to be priced like utilities.

All of these definitions are generally from respected pundits and enthusiasts in the area. In the formative (storming) stage of a technology these are often very insightful and useful. However, as the audience and participants grow, this is diluted very quickly. In my experience, the definition of a fad is (eventually) corrupted by these definitions - In particular, as software and hardware vendors inevitably take control of the spotlight.

It's very common in my experience to see vendors come in with the following line - (a) here is this great fad and how it's changing the world, (b) it's obviously complex, so here is a definition of it that you can digest - and (c) here is our our products... Coincidentally, our products fit that definition exactly.

This is perhaps cynical. It's perfectly natural for a vendor to define their strategy in line with industry thinking and then produce their product suite to fit. Arguably, IBM know quite a bit about Cloud Computing. IBM has been pushing the "On Demand" brand across their various offerings since 2002 [4]. This is not to say that IBM's On Demand necessarily fits some of the definitions of Cloud Computing [5] - but it's certain that IBM will fold the buzz Cloud Computing into it's strategy around On Demand [6].

However, the cynicism around the action of vendors in this space is not unfounded. There are a lot of products that suddenly become SOA or BPM or AJAX once the term becomes a hot property in the industry. Or you simply take an existing product suite, add another product or extension that fits the mould, and rebrand the lot. This also highlights one good strategy for a startup - Find an existing, established vendor, spot some kind of technology gap or inconsistency, develop against the gap and get acquired.

There is an underlying reason that this definition exercise is possible. To an outside party it may be clear that a new approach or technology has potential. This is driven by the outstanding success of the innovators and early adopters. This success often generates a justified amount of buzz in the industry. So there is potential, and a new way of doing things - and generally organisations have problems to solve. So the question inevitably arises - "This sounds good, but what can it do for me?".

Whilst this generally results in a bewildering number of answers, usually designed to push product, the fact is that often the cornerstones underlying these fads are relatively straightforward. SOA really has it's roots in Web Services - and although the two are theoretically independent, they have walked hand in hand through the hype cycle. Every presentation you will see will belie this fact, in fact they will often state the exact opposite "SOA is not Web Services" [7]. This is somewhat true, but this often leads to a definition that is a tautology.

These snippets usually get entangled for good reason. The fact is, just adopting Web Services (or whatever equivalent) simply isn't the end of the story. In fact, it's often just the first step in thousands. You have Services now, so you need to lifecycle manage them, instrument them, align them to Business Processes to get "true business benefit and reusabily&quot, etc, etc. Most of these aren't new problems at all, but the componentisation around the original Service approach (the Web Service) has changed the unit of work, and changed how a lot of moving parts are now expected to interconnect - software, hardware, processes and people. So something like SOA moves in to fill these gaps, which clearly can be nebulous in the extreme.

So what you have is a shift in technology that quickly becomes a bigger beast. Now that you have this technology, you need the consummate shifts in the methods, tools and outlooks that support this technology. Sometimes these shifts are on the periphery, sometimes they are orders of magnitude greater than the original technology driver. This is where the definitions become nebulous. e.g. "Now that I'm building Services, which ones should I build and how do I maintain them?"

In that sense, Cloud Computing falls into a similar trap. Accepting the fundamental technology premise around Cloud Computing opens up a lot of questions. Cloud Computing may be quite simple technically, but there are a whole raft of other implications that you need to deal with - many of which are old ground. I'm sure Mainframe pundits look at concepts such as SOA and Cloud Computing and say "but we've already solved that". In fact many cynics I speak to in the industry label these initiatives as the industry's desire to get back to the Mainframe - arguable, but you can usually see their point. CORBA pundits are probably part of a similar chorus.

Irrespective, something like Cloud Computing puts these combinations together in unique ways. So whilst you're treading old ground, some of the solutions might not necessarily be the same. For example, one of the keys in Cloud Computing could be achieving a cost basis that drives economies of scale.

So, to contradict myself and provide a definition - here are some of the technology keys that I see as driving Cloud Computing:

Even thought it might be transparent to an end user, Cloud Computing is run on "Commodity Hardware". This is a loose definition in itself, but generally means off-the-shelf x86 servers, often running Linux.

Further to this, instead of a handful of Big Iron servers, Cloud Computing tends to rely on a large cluster of these commodity machines.

Cloud Computing relies on increases in bandwidth to allow access to processing and storage resources via the Internet.

What this generally means is that:

Your computing services are cheaper because they are using cheaper hardware.

Your computing services are cheaper because utilisation is higher - if you're not using your capacity, someone else probably is.

It's easier to scale, as you are tapping the bigger resource pool that a single vendor can offer [8].

The Quality of Service (e.g. Reliability) is tuned by the inherent capacity and redundancy of the cloud. The systems aren't specified to be ultra-reliable, the resilience of a cloud is driven by having redundant nodes to take over the in the result of failure.

You were reliant on external parties before (e.g. ISP, Web Host), but you're increasingly reliant on them "higher up the stack".

Cloud Computing relies less on references that refer to physical assets (e.g. a fileshare on a specific server), but instead virtualises these for the customer. You just need a reference to the file, and a place to ask for it - you don't care where it actually resides.

Whilst most of these technology shifts are all about "Business Benefit" of some sort, it's often useful to take a concept back to it's absolute fundamentals to put it in perspective. Much of the surrounding definition often points to a philosophy or method - underlying all of this is often a series of technical shifts that is prompting this.

Going back to the technology definition can obscure some of the more conceptual benefits, but it also gives perspective - many of the pitfalls with a new technology have already been encountered before. The SYS-CON [2] links to a Paul Wallis' definition [9] highlights this fact further.

...And perhaps most of all, the key technologies are simply the easiest part to nail down.