Supporting The HPC Hero

I have been reviewing HPC Trends for 2010: On The Rebound from Addison Snell of InterSect360 (Formally Tabor Research). You can find the power-point presentation link at InsideHPC. Addison gave this presentation at the 24th Annual Newport HPC Conference. I was not able to attend, but any conference that has been meeting for 24 years has to be doing something right.

The presentation has some great marketing data and I suggest that if you have 15-minutes, take a look at it. Between IDC and InterSect360 it is possible to get some really good market data on the state of HPC. The good news is, according to InterSect360, the worst is over. Overall, according to Addison HPC took a 20% hit in 2009 going from $19.0 to $15.2 Billion. In 2010 things should be bouncing back.

Overall the presentation covered quite a bit of ground including Workstations, SMPs – virtual and physical, Windows, Cloud/SaaS/Utility, InfiniBand, Accelerators, and File Systems. In my opinion, Addison made some excellent points at the end of this presentation, but missed with the Windows assessment. Of course, this being Linux Magazine, I’m not going to spend much time discussing Microsoft’s HPC value proposition, but there are real reasons why Windows, and Mac OS X for that matter, don’t have a big foothold in HPC. I covered this five years ago when I wrote Why Linux On Clusters? and what was true then is true today. I’ll save you the detailed reading. Clusters are about building machines around problem sets. To achieve an efficient design you need flexibility and choice. Open source and the Linux OS provide the best flexibility an choice.

My main point of interest in the slide deck is about HPC community (side 34). Communities are about conversations and open HPC conversations are what helped create the current market. The Internet was also a key factor in brining people together. One shinning example is The Beowulf Mailing List hosted by Penguin Computing. I have been reading this list since it began and it is still a great source of discussion of best practices, old and new technologies, problem solving, and help for new users. Some of the early contributors are not as vocal as they used to be, but I am sure they still follow the conversations on the list. By the way, aside from some occasional flareups, the list provides rather civil conversations about HPC and is spam free (thanks to Don Becker’s diligence).

One of the aspects that makes the “Beowulf List” successful are the “open discussions about open HPC plumbing.” Once again we come back to the “Open” idea, which is why I think closed source plumbing in this space will have limited utility. Notice that I used the word “plumbing,” that is, the software infrastructure that links all the hardware together. Some may find it surprising that most cluster users employ closed source commercial compilers on top of the open plumbing. While this may seem contradictory, it is actually quite telling. In my opinion, the HPC crowd is a pragmatic bunch. There is a large amount of utility in “open plumbing,” but not so much in having an open compiler, although the GNU C and Fortran compilers are also used. The same can be said for many of the commercial applications such as resource schedulers and application programs. In the HPC arena, open source has been applied to places where it provides the most utility and choice. Where closed source solutions make sense, HPC users are not reluctant to spend money. (i.e. If you can make it go faster or work better, we will gladly pay you.)

Some may argue that the wide acceptance of open source is because it allows for no cost scaling of the OS in clusters. While this is certainly true, most production systems are paying OS support fees that are based on the number of nodes, i.e. there is a scaling cost. The real power of open software is the ability of users to try before they buy.

Coming back to Addison’s slides, I wish to call your attention to his Conclusions and Free Marketing Advice that I have reproduced below.

HPC, at all levels, is about leadership

Do not think of entry-level users as small: To them this is the biggest system theyâ€™ve ever bought

A supercomputer without an application is an expensive space heater

An application without data is only a nice theory

How do your users (not only supercomputing level) exemplify leadership?

I could not agree more with the leadership point. The HPC people I have encountered are often pioneers within their own organization. Supporting these “hero’s” is one way the HPC community can grow and flourish outside of big universities and government labs. And yes, I believe there are also some Top500 space heaters siting out there.

In closing, I want to thank Addison for sharing this information. In a market that is based on sharing ideas, solutions, and opinions, it helps make us all stronger when we get good information. I am sure InterSect360 can compile much more information and insights should your company need such assistance.

Comments on "Supporting The HPC Hero"

Helpful info. Fortunate me I discovered your website by chance, and I am shocked why this coincidence didn’t took place in advance! I bookmarked it.

It’s a shame you don’t have a donate button! I’d without a doubt donate to this fantastic blog! I suppose for now i’ll settle for bookmarking and adding your RSS feed to my Google account. I look forward to brand new updates and will talk about this site with my Facebook group. Talk soon!

What i don’t understood is actually how you’re not actually much more neatly-favored than you might be right now. You’re very intelligent. You already know therefore considerably in relation to this topic, produced me personally imagine it from so many numerous angles. Its like men and women don’t seem to be fascinated until it¦s something to do with Girl gaga! Your personal stuffs nice. At all times care for it up!