9/28/2009 @ 6:00AM

Supercomputing In The Enterprise

Virtualization and cloud computing sound logical enough on paper, but exactly how do you get there from here and what sorts of problems can you encounter along the way?

These questions have already been answered by the supercomputing world, where a select few have been wrestling with these problems for years–usually with huge government grants and budgets that dwarf the resources of even the best-equipped corporate data centers. In an ironic twist, that approach to computing is now being used to save money.

Forbes caught up with Michael Jackson, president and COO of Adaptive Computing, to discuss what’s changed and what to watch out for in the commercial data center.

Forbes: Some of the same techniques that were needed in the supercomputer space are now filtering into the commercial world. When did that start?

Jackson: We saw that crossover beginning in 2007 at the leading-edge sites. It may be a little ahead of the rest of the industry, but virtualization was preparing data centers to be able to change the location of resources and also do some of the adaptation of that environment. If you look at Gartner’s research, the No. 1 IT spend trend for 2009 is IT consolidation and virtualization.

What are the problems CIOs need to be aware of?

As soon as you go from all these silos into a shared environment, you run into contention. When these computers were in their silos, they were protected and there weren’t multiple users trying to access it. And there weren’t multiple workloads contending for the resource. When you bring it all together with virtualization, then you’re trying to solve this class of problem that supercomputers deal with.

So what’s the upside?

You can get three to five times the utilization of servers with virtualization that many companies currently have. You also can get flexibility to respond to customer trends. Where we’re seeing interest in this technology first is in the business-to-consumer space, because the customer piece creates unpredictability. It’s affected by buying trends or a surge in the financial markets or perception-based decisions.

Are there applications available to take advantage of multicore machines?

We’ve hit a barrier that is slowing the ability to improve the speed of individual cores, so now we have a proliferation of cores instead. Application providers are facing a dilemma with this. Are they going to be valuable only in a virtualized environment, where they’re forced down to only a single core, or are they going to make sure the application is multithreaded so it can scale out to at least two cores, and maybe out to four or eight cores?

But we haven’t seen much change so far.

The market pressure on the providers of enterprise applications will force them to make changes to their code. That will also make their applications relevant to the high-performance computing environment. The other thing supporting this trend is Microsoft coming to the plate in high-performance computing. They’re going to try to build into their development tools a choice between building for a traditional environment or a high-performance parallel environment. There is going to be some delay in getting there, but there are multiple pressures and demand to get there that will support the mainstream adoption.

If you can parallelize an application, it’s easy to see how the resources in a chip can be used. How do you do that in a cloud or virtual environment?

For an application, it’s not simply the consumption of a CPU. It’s also the bus, storage, memory. The same applies with transactional workloads. It’s not just how many core hours you need. You need to apply multiple levels of the resources. Even in high-performance computing, some have GPUs, memory and multiple cores; you have to be able to track that. You also need to characterize the workload because it has to be scheduled. Sometimes you don’t know what to characterize, so you have to historically look at the applications. But it’s only so many factors to test where it’s performing better and when it’s performing better. Maybe some servers have less reliability and they’re down more often. It’s application profiling, resource profiling coupled with autonomic learning.

What does this mean to the CIO vs. the large cloud providers?

We are seeing substantially more interest now in the private world. They want to transform from a rigid environment to a flexible environment, where they also still have full control over security and usage. What it means is they’re going from 10% to 15% utilization [of servers] to something three to eight times higher. That’s a substantial hardware savings and a substantial management savings because they can reduce the sprawl. The second thing that comes into this is energy management. The cost of energy is projected to be between half or equal to the price of the hardware. Even if you’re not reducing your hardware and adding more computing, you can pack resources tighter and power down resources. That saves huge energy costs.

How big?

One company we work with was able to achieve a 50% power reduction, although we’re estimating ultimately it will be 25%. They paid for their investment in less than a month. They also can reduce their hardware costs. But if it’s a company like Boeing, that kind of flexibility also allows them to make a decision to prioritize resources for a project in one day. That time-to-market advantage may make a $100 million or $1 billion difference to their business. In the financial sector, it will have equal if not more substantial impact. CIOs will see new flexibility in their ability to focus their resources. They can accomplish a much higher return on their hardware and software investments.

Is this moving downstream from very large enterprises?

There are large enterprise companies trying to solve very complex problems and consolidate many groups, as well as those trying to solve Sarbanes-Oxley requirements or other single-purpose issues. The smaller organizations are looking at the public cloud and virtualization where they are dealing with things more manually. Some businesses might only have 16 servers; they can deal with this using an appliance.

In the past, the biggest hurdle for this kind of flexibility was cost. Has that changed?

Yes. We’re at the point where the number of moving parts has been reduced significantly and you can utilize your existing investments to a very high degree. If your ROI is multiples over the silo-based environment, and the cost of getting there is only a fraction of that, it makes a lot of sense. Right now it’s more an awareness issue than anything else, and companies like IBM and HP are out in the market working on that.