Stay connected, up-to-date, and informed on all things parallel development via Go Parallel, where you'll find viewpoints, how-to's, software tools, and educational information to help your software development work shine. http://goparallel.sourceforge.net/

Geeknet: I was reading through some recent presentations of yours – in particular the presentation you did for the Texas Advanced Computing Center, back in April. You started questioning the readiness of tools for making best use of really advanced coprocessors like Xeon Phi. Do you see us approaching a watershed moment in tools development, now that we have things like the MIC architecture to work with?

James Reinders: Yeah, I do. And there’s always this push/pull thing that occurs in order for any technology to evolve. Some people like to say the hardware always is either behind or ahead of the software – it’s really funny, because people will take either view. And that really says to me that – you usually get a capability in one or the other first, and then look at how the other one is going to change to take advantage of that. And I think we’re really in that world. And I think we’re ready to use large levels of parallelism, but not as ready as we will be, ten years from now. Meaning that I think we have a lot of room to grow.

For instance, if you look at something very simplistic – say a dual-core when that first came out, almost no software was optimized to use dual-core in a single application – not software that most of us would use. But we all felt that dual-core machines were so much faster. One person simplified it as: “I’ll give one core to the OS; the other one’s for me.” And then, after a while, some people said “Oh, but I’m running Outlook too, so I’m on a quadcore – I can give two cores to Outlook, one to the OS and one to me.”

But that sort of parallelism sits on multitasking lines, and we do that and it sustains us for a while. What’s interesting is that it got us to value dualcore and quadcore. Now we’re getting them on cell phones. I remember the first times I started telling people, and they’d say “Well, you said multicore was going to take over all processor designs. But you didn’t mean mobile, did you?” And I said “Yeah?” And now, people are fighting over who’s got the first quadcore tablet and phone and things like that and you know what? It won’t be enough.

Geeknet: But they’re also fighting now about whether Android makes efficient use of dualcore.

James Reinders: Absolutely, that’s a great example. If the hardware does this, has the software caught up yet? It will. And if it doesn’t, then another operating system is more appropriate and may jump in and take over. Same thing with applications. If you have a favorite photo editor, once some of the photo editors start taking advantage of multicore, all of them need to, or they don’t compete. Sometimes we call that a ‘software spiral’ at Intel, where hardware gives you a capability, and we use it in sort of a lame way, or less-capable way, like running multiple applications on your quadcore. But then people start showing up with applications that alone can use multiple cores, and they show extra value in that hardware. That’s where we’re at.

But yeah, we’re kicking this into high gear. Xeon Phi? Coprocessors with more than fifty cores? Oh my gosh, I was just talking about four – what are we going to do with fifty? And there’s a domain of customers – there’s a domain of applications that can definitely take advantage of that. But there will be a larger number of applications that can take advantage of that in ten years than there are now, because, by making this readily available, applications will show up.

And hardware will evolve a little too. The hardware right now is … Well, give us ten years and we’ll be able to have many more times transistors, and so we’ll be able to build more complex designs, and we’ll be able to help the software more. So the hardware will make it easier to take advantage of that over time because of its natural growth, and the number of applications that can take advantage of that will grow because of that too.

But today, the number of people who have an application that can really take advantage of a 50-core device is pretty darn limited. There are some really fascinating things; there are things that may change our lives. Someone may discover a drug that cures a cancer that you haven’t quite gotten yet, and you may someday say I’m glad that my life was saved because someone had fifty cores. So I’ve learned never to say gee, there aren’t very many applications – there are enough out there that are important.

And I probably say this too often, but I worked on ASCI Red, which was the first TeraFlop machine on the planet. And we built that in the late 80s, mid-80s. And we turned it on in ‘96, and we got more than two TeraFlops of performance. And there were very few programs in the world that could take advantage of the TeraFlops. And given that there was only one machine and it was at Sandia, that’s not very surprising. Now, there are probably thousands of computers with TeraFlop capability. I would propose that people aren’t buying machines with that much performance and not using them. I think there’s a lot more applications that can use that level of performance. So I’m not worried about 50 cores being too far out there. It has to exist a while for the number of applications to grow.

Posted on November 26, 2012 by Jeff Cogswell and John Jainschigg, Geeknet Contributing Editors

Use this MPI library for better application performance on Intel® architecture-based clusters by implementing the high-performance MPI-2 specification on multiple fabrics. Quickly deliver maximum end-user performance even with new interconnects—without requiring major changes to the software or operating environment.