The Next Five Years: Real Choice In The Cloud

Below is a full transcript of Rackspace President Lew Moorman’s June 20, 2012 talk at the GigaOm Structure Conference[1] in San Francisco. Video of the full presentation is available here[2].

It’s amazing that Om [Malik, GigaOm Founder] has been gathering us here for five years to talk about the cloud and its impact on technology. It’s been an amazing five years. What I want to talk about today for a few minutes is the next five years, which I think is going to be radically different than the first five years. But, let’s take a minute to celebrate the innovation that’s been driven by the cloud.

Here’s an example my mom can relate to, which has been very hard to find over the last few years. But just a few years ago, five years ago, when we started getting together, if she wanted to rent a movie or buy a movie, she had to get in the car and go down to Blockbuster. Or ask Netflix to send a DVD through the mail — took a couple days. Today, five short years later, there are many services where you can go buy or rent movies and do it on demand, instantly. This is an incredible advance.

This is what the promise of the cloud has been; speed and choice. And it’s changing every aspect of our lives. There are thousands and thousands of consumer services and many being planned and built today. Nothing is going to be left untouched.

Of course, what we talk about here at Structure are the underlying cloud platforms that power these consumer services, and there have been incredible advances there as well. In fact, we’ve spent the last five years talking about how instant computing has changed everything in the IT world. The ability to spin up and spin down, to scale and demand has changed everything. Choice, that has not been nearly as robust. And look, let’s give Amazon their due. They’ve been incredible pioneers in the space and they’ve built a dominant computing platform.

Let’s go back to our movie services example to show it. Netflix has been an incredible early adopter and evangelist for the cloud and what it can do for a business. But it says a lot about the state of choice when they’re evangelizing one of their most ruthless competitors.

So they’re a smart company. [Netflix Cloud Architect]Adrian Cockcroft, who I can see in the audience, loves the power of the cloud, wants to continue to bet the company on the cloud. And it’s the right thing to do. But he also wants choice. In a blog post that he posted this is what he asked of us as a community: “Please try to build AWS clones that scale.”

Well, I’m here to tell you the choice is coming but this is not how it is going to come. And the reason why it’s not going to happen is because there’s one main difference between consumer cloud services and the underlying platforms. It’s the fact that they’re programmatically accessible. And this is actually the root of the revolution of the cloud.

If you think about the pre‑cloud era, applications were isolated from the infrastructure. It’s like a rider on a horse. If you needed more power, more features, it was nothing more than a data transfer problem to get a new platform — very simple. The cloud is very different. This is a metaphor I like, remember the banshee in the movie “Avatar?” Here, the rider fused for life to the animal and their brain controlled the actions of the flying beast. This is very much how the cloud has changed application development. Modern applications control the infrastructure, they interact with the infrastructure. They rely on its behaviors and its specific features. And this is why small teams have been able to build incredibly robust applications that scale and do things that were not possible without huge operations teams in the past. Absolutely revolutionary.

Now, many people would say, “This is not a problem with the cloud. There is no lock in.” And I think part of the reason why is that it happens slowly over time as you start to work with it and use the power of the cloud. What I think this really says is that in the cloud era hardware has become software. I love the old metaphor and the old saying that is updated for today which is that “You date your hardware provider, you marry your cloud.”

And this is how it happens. It happens really in three stages. When you start to use the cloud, it does feel like traditional infrastructure. You can get Linux or Windows boxes on demand and it feels like everything is like traditional infrastructure; it’s just much faster and much more responsive. And at this point migrating off the cloud or to another cloud provider is fairly easy and it looks very much like traditional infrastructure.

But the second phase is where things get more complicated. And this is when the fusing begins. When you start to programmatically access the controls of the cloud; where your application stack reaches down into the cloud and they become one.

The last phase is once you’re integrated into a cloud provider it’s very tempting to use proprietary features of those clouds. And Amazon’s made great advances here and launched a lot of great services like SimpleDB and DynamoDB. But once you use these applications and once you use these components, you are truly fused to that cloud provider.

So this is what the community has asked for: APIs. Let’s just clone the APIs. This is a simple thing to do. There’s an interface. If programmatic access is the issue, let’s just make it one common language. This is what Netflix is asking for. And this is why our industry has been obsessed with API standards. And what I want to say today is that cloning APIs in this way is not possible. This is not how it’s going to happen. And this is hard to hear. This is going to be frustrating for people to hear, but let me explain to you why I feel this is absolutely the case.

First of all, there’s the basic fact that APIs are nothing more than an interface to real technology. They’re a bridge, a protocol. But the truth of the matter is the cloud is not a protocol. The cloud is a set of incredibly complex technologies that does incredibly complex things. So if your strategy is to clone the APIs, clone the whole experience of the cloud, you need to build a clone with that technology. But everything behind the API is not exposed in proprietary clouds and so the experience you build won’t be the same. It just is not possible.

Eucalyptus is a company that’s devoted themselves to doing this strategy, to cloning the cloud. And they’re doing good work. But if you look at one example of a service that they’re trying to replicate, their S3 service — so it copies the first Amazon service and one of the most basic in terms of features. And they’ve done a great job, about two‑thirds of the feature set is there but it’s certainly not complete.

Furthermore, if you look at their own documentation, the scaling characteristics are very different than S3. And this is not because they’re building bad technology, it’s because they’re building different technology. And the truth of the matter is that this is the best case scenario. While they have API coverage over a set of services, there’s a huge number of services they have zero API coverage across. This is the problem with the cloning strategy. I call it the OpenOffice problem.

How many of us, especially this group in Silicon Valley, haven’t said at one point or another “I don’t need Office. I hardly use any of the features. Why do I want to pay the tax?” Well, when one of 20 documents you open looks like this [a jumbled and unformatted document], you suddenly realize you need it. This is exactly true with the cloud as well. What is the power of a model that is always behind and not quite the same? I think that this is something that goes unspoken in our industry. I think it’s known, but we don’t talk about it.

Here’s a blog post that Thorsten [von Eicken], CTO of RightScale, wrote … he uses fancy words here but the effect of his language is, if you want to copy the experience of the cloud beyond anything that is basic, you’ve got to have the whole cloud, APIs are not enough. So who cares? Does this matter? If your cloud provider’s doing great things, continuing to add features, do you need to have a choice? Netflix says they want choice. I think others want choice, I think we all inherently believe choice is going to be good.

But what are some of the other reasons to have a choice? Well, what if you want to be in a region where your provider isn’t? What if there’s features that you need that could really advance your service and make things much easier for your users and much more performant. And then there’s the cost issues. Being beholden to one provider doesn’t typically end up with the best cost arrangement in terms of leverage in the long run.

But even more importantly, the ability to customize your cloud can make enormous differences to the underlying cost structure. When you have an application that is running at a massive scale or in a very specific use case, the ability to customize the underlying architecture has huge implications to the economics. This is why private clouds and specialty clouds are going to be around for a long, long time. And the truth of the matter is that Netflix is already proving this. In the last couple of weeks they announced their own CDN. Why? The ability to control the experience and the cost structure is an important element of their long-term future. To have that flexibility really matters and they’re running at such a scale that it makes a big difference.

How long until they think this way about computing? Now, I have to give them credit. The truth of the matter is there hasn’t been a real alternative in the first five years. There hasn’t been a good choice for them and they’ve done the right things for their business. What is the answer? A true end‑to‑end open and scalable cloud. What do we mean by end‑to‑end open? The ability to truly move the cloud anywhere you need to go; true portability. When you have access to the entire code base from APIs down to the actual source of the features, you can run it anywhere: public, private, hybrid. You choose the features, a true open source model that allows you to contribute to the evolution of the project and is openly extendable to any technology.

And finally, all this flexibility allows you to optimize your cost structure to be as competitive as possible for your business. And of course, this is what we started OpenStack for. This is why OpenStack exists. And OpenStack is here. Now we’ve talked about it for the last two years and it’s shown great promise. It is now time for the delivery. Starting with the core components of the technology, OpenStack is ready. The number of private deployments is going to start exploding. It is exploding. And I’m happy to announce that on August 1, Rackspace will officially make OpenStack the engine of its public cloud. And from there, the innovation is going to really start to advance. Rackspace and developers from all over the world are building features around the code base to have it continue to grow and get the features of all the proprietary vendors. And because it’s openly extendable, open source projects and enterprise vendors can also plug in and make seamless offers.

This is the power of an open model and in the long run open models create more innovation faster than any single vendor can. This is why over 180 companies have committed to the OpenStack project. Not just with press releases but with dollars, with developers and with a commitment to create real solutions. And the start of them is going to be coming every day. You’re going to start seeing more and more news around OpenStack. This world knows that we need choice. We are talking about the platform of all modern computing. There must be an open alternative. And there must be choice. Why? We need choice on how you can deploy your cloud, choice on where you can deploy it, anywhere you need to deploy it, on what features you want to deploy it with and what partners you want to go to market with. This is the power of choice. It’s imperative for our industry.

I’m going to leave you with an example of choice and what it might look like in this world. Wikipedia was an early proponent of the OpenStack code base and has run the Object Storage Swift code on premise. And they’ve had good success with it. They’ve been at the OpenStack conferences talking about it. The great thing about them is they now have choices. Of course, if they want to use a public cloud vendor, we hope that they’ll use our Cloud Files service, but the truth of the matter is that there are other providers out there offering similar services, SoftLayer, HP and many others. And they can do this with seamless interoperability because everything up to the APIs down to source is the same — extremely powerful. But that’s not all. The truth of the matter is that the same basic lowest common denominator interoperability exists with the proprietary stacks as well. And the reason why is because it’s an open model and anyone who wants to build API extensions into those proprietary stacks can do it and will continue to do it as long as there’s demand for that. That is the power of choice.

The first five years have been incredibly exciting. I think the next five years will be even bigger. Everything in technology is going to change. But now is the time to have a true open alternative. One that gives real choice and allows unconstrained innovation. We at Rackspace are really thrilled to be part of it. We are more excited about the future than ever. We hope you will join the era of the Open Cloud.