Disney Builds Private Cloud for Videogame Empire

Cloud computing isn’t just a new way of hosting and running applications. It’s also a new way to think about how applications are designed in the first place, how they store and retrieve data.

Joe Arnold was the chief technology officer at Engine Yard, a San Francisco company that offers a cloud service where developers can house their applications, and over the last half decade, he witnessed firsthand the shift towards a new way of building software that runs across tens, hundreds, or even thousands of servers. “We were watching people build more mobile and web applications,” he says. “And how they used storage was a lot different from how it started when we first started doing infrastructure.”

That was because Engine Yard plugged into a cloud storage service from Amazon known as Simple Storage System, or S3. By stripping away many characteristics of traditional storage systems — such as the hierarchical “file folder” style of organization — S3 and systems like it can provide more speed and scalability while remaining extremely reliable. It’s an approach called “object storage,” and thanks to Amazon, it has become a popular way for websites and web applications to serve static content, such as images, videos, and other media.

But not everyone is ready to trust a cloud provider like Amazon with their data. Some companies want the power and flexibility of cloud services while still running everything in their own data centers, and these companies are embracing open source software that mimics the S3 way.

The latest is Disney Interactive, the division of Disney that manages the company’s video game and web properties. On Tuesday, at a conference in San Francisco, Disney revealed that the company has built a private cloud on open source technologies, including a clone of S3 called Swift, one of several open source alternatives to S3.

Swift began life at Rackspace as Cloud Files in 2008. Then, in 2010, Rackspace teamed up with NASA to create OpenStack, an open source alternative to Amazon Web Services that anyone can use to build a private cloud. Rackspace contributed its Cloud Files code to the project under the name Swift.

But Disney isn’t using the rest of OpenStack to build its cloud. Much like S3, which can be used independently of the rest of Amazon’s cloud services, Swift can be used with other cloud systems. In this case, Disney is using another open source Amazon alternative called CloudStack.

It’s important to understand that both CloudStack and OpenStack are actually collections of different technologies, not monolithic applications. OpenStack, for example, is divided into several core technologies, including a processing system called Nova and a networking system called Neutron as well as the Swift object storage system. CloudStack includes a processing layer, but unlike OpenStack, it doesn’t include a storage layer.

To integrate Swift with CloudStack and the rest of its enterprise systems, Disney called on Arnold’s company, SwiftStack, a two year old startup dedicated to developing, managing, and supporting Swift. “I started working with OpenStack during the early days,” Arnold says. “I got to know the Rackspace guys really well. Then we started talking about building a product out of Swift.”

SwiftStack hired several key players from the OpenStack community, such as the Swift project lead John Dickinson. Together, the new team worked to make Swift into a stand-alone storage system with enterprise-grade features, such as LDAP and Active Directory integration. Arnold Swift’s LDAP support as one of the major reasons that Disney chose the product.

Although Arnold is bullish on OpenStack, the CloudStack integration for Disney illustrates Swift’s potential to transcend its parent project. And he thinks more companies will end up choosing to host their own private object storage clouds instead of choosing Amazon or other hosting providers for a simple reason: money.

“What we’ve found is that anyone sitting down with a spreadsheet will realize [S3] is an expensive service,” he says. “That may make sense if you need elasticity, but our customers have determined that it doesn’t make sense to run storage in a public cloud. Data doesn’t really burst out, it grows forever.”

Here’s The Thing With Ad Blockers

We get it: Ads aren’t what you’re here for. But ads help us keep the lights on. So, add us to your ad blocker’s whitelist or pay $1 per week for an ad-free version of WIRED. Either way, you are supporting our journalism. We’d really appreciate it.