Windows CCS and the end of *nix in HPC

There is an article over at SearchDataCenter.com on Windows CCS adoption and trends in the HPC industry. The article is useful as an overview of who’s adopting CCS and what they’re doing with it.

The article plays up the apparent conflict between the adoption of CCS by users and vendors and the Windows HPC naysayers, primarily large scale HPC users from the scientific side of HPC. As an example, pro:

Professor Saifur Rahman of the Advanced Research Institute at Virginia Tech in Arlington, Va., uses Windows on the institute’s HPC cluster to run and analyze medical data.

…Rahman chose Windows rather than traditional Linux because students are familiar with Microsoft software.

And con

Don Becker — who is the founder and chief technical officer of Scyld Software and the co-founder of the original Beowulf project, which is the cornerstone of commodity-based, high-performance cluster computing — was surprised to hear such positive feedback about Windows on clusters.

…the idea of using Windows in the HPC space is outlandish to Becker, who believes that Linux is the ideal operating system for clusters; the OS, he said, “grew up in that space and has good high-performance communications.”

Anthony Gold, vice president and general manager of open source business solutions for Unisys, had a reasonable and (to my mind) correct point of view

“Windows environments are widely used by clients, so from our perspective, moving users off Windows on to Linux isn’t always the best choice,” Gold said. “Conversely, there are many people in the Unix space who will never move off Linux; it’s like a religion for them, and they won’t change.”

People in old-line HPC who blithely dismiss Windows CCS as a marketing stunt are missing the major shift in our industry; a shift that will shape the next decade of HPC systems development.

HPC is pushing down into the enterprise. Enterprises that run Windows. Enterprises with bottom line profit motivators and a religious devotion only to getting their work done. There is no element of computer research or advancing the state of the art in their use of HPC. They’re looking for a least pain, highest payback path for their HPC investment.

For many of these companies at the start of HPC adoption, Windows CCS will be just what they need, and that will create an immediate huge commercial driver for hardware vendors to develop solutions that support CCS. This will skew the HPC market even more severely than it’s already skewed toward commodity components, and may be what kills the old line independent HPC hardware manufacturers.

In the long run enterprises will gain more experience with HPC, and they may find a business need to run very large jobs on HPTC-class systems, a market currently served exclusively by *nix. But if Windows CCS can grow as its customers’ needs grow, there won’t be a reason for enterprises to ever leave CCS. Widespread adoption of large HPC in the enterprise will create a market that is at least an order of magnitude larger than the HPTC market will ever be, and that will spell the end of *nix in HPC. A $10B customer wins over a $1B customer.

Now, can Microsoft screw this up? You bet. But right now, enterprise HPC is theirs to lose, and traditional HPC needs to be pragmatic as it draws up plans for the next 5 to 10 years.

In my opinion any HPC company (software or hardware) without at least a plan to support Windows CCS is taking an enormous risk. And any serious HPTC customer that isn’t talking to Microsoft, and trying to get them to understand our requirements, is putting themselves in a position of having to settle for whatever operating system enterprise customers demand.

Comments

I agree that Windows CCS will likely become important in HPC, especially in desktop-up migration scenarios. My bigger question is, what do you mean by “the enterprise”? This term is used by people in unclear ways. Do you mean in HPC existing industrial markets, such as automotive, aerospace and life sciences; or HPC proximity markets, i.e., technical computing in financial services, movie-making, etc.; or do you mean in non-technical computing business applications, such as HR, payroll, customer relationship management? I’m just seeking terminological clarity. Thanks.

Steve: good point. For the most part I look at HPC in existing industrial segments (aerospace, automotive, energy) as HPTC of the same class that we do in the US federal laboratories. In many cases they are solving the same problems with the same codes as we do on the federal side.

“Enterprise HPC” in the context of this web site usually means applications in commercial settings that are applying big(ger) compute to core business applications (data search and management, ERP, logistics…) as well as industries that will use HPC in non-HPTC ways (like entertainment, for example).

Resource Links:

Latest Video

Industry Perspectives

In this special guest post, Axel Huebl looks at the TOP500 and HPCG with an eye on power efficiency trends to watch on the road to Exascale. "This post will focus one efficiency, in terms of performance per Watt, simply because system power envelope is a major constrain for upcoming Exascale systems. With the great numbers from TOP500, we try to extend theoretical estimates from theoretical Flop/Ws of individual compute hardware to system scale." [Read More...]

White Papers

While some domains that rely on computing systems are satisfied with the performance gains that are continuously delivered from processor manufacturers, others require performance that can only be attained through massive parallelism. High-performance computing is for those that need the highest performance in order to solve many of today’s most difficult scientific challenges.