Mitrionics Looks Beyond FPGAs

By Nicole Hemsoth

November 18, 2009

At SC09 this week, Mitrionics announced it has started to work on an experimental compiler that aims to make parallel programming architecture-agnostic. The goal of the work is to extend the Mitrion-C platform for FPGAs to multicore CPUs, cluster architectures, and eventually even GPGPUs. We asked Stefan Möhl, Mitrionics’ chief science officer and co-founder, to explain what’s behind the new technology and what prompted the decision to add support for other parallel architectures.

HPCwire: Can you tell us about the new programming capabilities of the Mitrionics platform that you announced here at the Supercomputing Conference?

Stefan Möhl: Well, we haven’t added new programming capabilities to the Mitrionics’ Accelerated Computing Platform yet. We are still in the proof-of-concept stage with this new compiler, but things look very promising. For this proof-of-concept compiler, the news is that existing Mitrion-C code, originally written for the MVP on FPGAs, will now also run on multicores and clusters. This initial proof-of-concept was made only to prove that the basic principles work, so there are limits to what code we can currently run. A production version of a portable programming language will require changes to Mitrion-C to make it less focused on what is needed for FPGA acceleration.

HPCwire: How does it work?

Möhl: The main challenge when porting between parallel architectures is that the level of granularity of the parallelism differs. For example, to parallelize code for vector processors, you would have to parallelize inner-most loops. To parallelize code for clusters, you would have to parallelize the outer-most loops. Doing general automatic parallelization (parallelization without re-writing the code) has not been solved, even after decades of research. Nor is there a general automatic way to transform one kind of parallelism into another.

Mitrion-C was originally developed as a programming language for the Mitrion Virtual Processor (MVP). The MVP is a hardware design for a compute engine specifically developed for high-performance execution in FPGAs. As such, it is full MIMD ((Multiple Instruction stream, Multiple Data stream) at the individual instruction level, so it potentially executes every single instruction of the program in parallel. This can be thought of as a limit-case for parallelism. Mitrion-C is a C-family language that supports and aids the programmer in specifying the kind of parallelism that the MVP requires. It is roughly as similar to ANSI-C as Java or C# are, so it isn’t too unusual to use.

The trick that makes Mitrion-C work for parallel portability comes from an important asymmetry in parallelization. Though automatic parallelization without code re-writes is very hard to achieve, general automatic sequentialization is much, much easier. Trivially, operating systems have run multiple programs in parallel on sequential processors for many years. For efficient execution, there are of course many optimization considerations, but it is still much easier than automatic parallelization. This property is what we use to port Mitrion-C between platforms. Since the code is fully parallel from the start, we never parallelize at all, we only sequentialize. So for a cluster, instead of parallelizing outer-most loops, we sequentialize everything except the outer-most loops. And for a vector processor, we sequentialize everything except for the inner-most loops.

HPCwire: So if you don’t have the parallelization problem, how can you handle the various memory architectures of multicore CPUs, GPGPUs and clusters, and so on?

Möhl: Our FPGA background has required us to consider these issues carefully from the start. FPGAs are usually connected to the system on data buses designed for devices with an order of magnitude less performance than FPGAs. So Mitrion-C was designed from the start to allow programmers to manage both memory latency and raw memory bandwidth in an effective manner. This issue will become increasingly important also for multicores and manycores, since increasing core counts without increasing clock-frequencies of data buses will put them in the same situation FPGAs have always been in.

Another important aspect comes from the diversity of FPGA cards. There are almost no two FPGA cards with the same memory sub-system, so we had to design Mitrion-C to have a memory model that addresses this from the start.

In Mitrion-C, there is no assumption of a single monolithic memory space. Instead, each collection may have its own address space, and different ones for different memory size and bandwidth requirements. This allows programmers to manually stage data from few, large and slow memories to many, small and fast memories in any number of levels. There are also several different built-in types for multi-dimensional data collections that let programmers specify what kind of access patterns a collection should permit. This helps the programmer in making correct and efficient programs, and also lets the compiler know what types of memory to place the data collection in. Of course, you can still write a program that requires more, larger or faster memories than a particular system has, but Mitrion-C will at least make you aware of what you demand of the system.

HPCwire: Can the exact same Mitrion-C source code be compiled to any of target architectures?

Möhl: Yes. You will need to parameterize for the number of cores you want to run on in a cluster or multicore, or how much unroll you want for loops in an FPGA, but other than that, the same code works without changes. However, not all algorithms will be efficient on all architectures, so the programmer will in some cases need to consider what platform to run the algorithms in, or change the algorithms to suit the available platform.

HPCwire: Mitrionics has focused on FPGA software development since its inception in 2001. What prompted the decision to target other architectures?

Möhl: Well, we are actually still focused on FPGAs. What prompted this is a customer interest in running Mitrion-C on standard processors and not only the Mitrion Virtual Processor. Customers want to be able to write an algorithm once and make efficient use of it on systems with and without FPGAs. They would also like to avoid having their code “locked in” to FPGAs. So we set up an experiment at Mitrionics to see what can be done with Mitrion-C on other platforms. And, as it turns out, very much can be done!

HPCwire: There are already a number of programming environments and languages that target multicore CPUs and GPGPUs and clusters. What does Mitrionics brings to the table?

Möhl: Three main things. First, Mitrion-C is a single, coherent language that maintains the same style of programming regardless of what platform you run it on. Programming languages like MPI, OpenMP, OpenCL and CUDA are really several different languages mixed together. There is the base-line C-code which is purely sequential, then there are added parts for clusters (in the case of MPI), multicores (in the case of OpenMP), or GPUs (in the case of OpenCL and CUDA). Often, you even have to combine them, such as with MPI+OpenMP. The additions introduce completely different ways of doing things than what the sequential C code does. They are not just added syntax in the sequential C paradigm. That means that you are really writing in several different languages at the same time, and need to learn them all to be able to do it properly. It also complicates the code dramatically.

Second, the fact that you have separate syntax for each architecture means that you need to re-write your code to move it between architectures. With our solution, software developers can make a single investment in writing code, and then use it on any architecture depending on what is optimal under current circumstances. With a universal programming language that can be used to target any architecture without changing syntax, it also becomes possible to explore the benefits and possibilities of different architectures much faster, in the end resulting in more efficient code.

Finally, and perhaps most tantalizing, is that the portability is not limited to the architectures that are popular today. History has seen a wide range of architectures — from the old scalar processors, vector processors, Thinking Machines, MasPar and SIMD, the Multi-Threaded Architecture, large shared memory machines, MPPs, and clusters to today’s FPGAs, GPUs, Cell, multicores and several others. Each new generation has required code re-writes. Though this is not yet proven, there is good hope that Mitrion-C would be efficient without re-writes on most of the historical popular parallel architectures. If that is the case, it bodes well for parallel architectures of the future too. Though we probably won’t be able to say “Never again!” to re-writes for all eternity, Mitrion-C holds the promise of dramatically reduce the number of re-writes we will need to do in the future.

Seeking to reign in the tediousness of manual software testing, Pfizer HPC Engineer Shahzeb Siddiqui is developing an open source software tool called buildtest, aimed at automating software stack testing by providing the community with a central repository of tests for common HPC apps and the ability to automate execution of testing. Read more…

By Tiffany Trader

In just a few months time, Senegal will be operating the second largest HPC system in sub-Saharan Africa. The Minister of Higher Education, Research and Innovation Mary Teuw Niane made the announcement on Monday (Jan. 14 Read more…

By Tiffany Trader

If it's Nvidia GPUs you're after to power your AI/HPC/visualization workload, Google Cloud has them, now claiming "broadest GPU availability." Each of the three big public cloud vendors has by turn touted the latest and Read more…

Previous:

STAC (Securities Technology Analysis Center) recently released an ‘exploratory’ benchmark for machine learning which it hopes will evolve into a firm benchmark or suite of benchmarking tools to compare the performanc Read more…

By James Reinders

Quantum computing has lived so long in the future it’s taken on a futuristic life of its own, with a Gartner-style hype cycle that includes triggers of innovation, inflated expectations and – though a useful quantum system is still years away – anticipatory troughs of disillusionment. Read more…

By John Russell

Anyone who has checked a forecast to decide whether or not to pack an umbrella knows that weather prediction can be a mercurial endeavor. It is a Herculean task: the constant modeling of incredibly complex systems to a high degree of accuracy at a local level within very short spans of time. Read more…

By John Russell

Cray revealed today the details of its next-gen supercomputing architecture, Shasta, selected to be the next flagship system at NERSC. We've known of the code-name "Shasta" since the Argonne slice of the CORAL project was announced in 2015 and although the details of that plan have changed considerably, Cray didn't slow down its timeline for Shasta. Read more…

By Tiffany Trader

It’s been a good two weeks, AMD’s Gary Silcott and Andy Parma told me on the last day of SC18 in Dallas at the restaurant where we met to discuss their show news and recent successes. Heck, it’s been a good year. Read more…

By Tiffany Trader

For nearly two hours on Monday at SC18, Jensen Huang, CEO of Nvidia, presented his expansive view of the future of HPC (and computing in general) as only he can do. Animated. Backstopped by a stream of data charts, product photos, and even a beautiful image of supernovae... Read more…

By John Russell

Riding healthy U.S. and global economies, strong demand for AI-capable hardware and other tailwind trends, the high performance computing server market jumped 28 percent in the second quarter 2018 to $3.7 billion, up from $2.9 billion for the same period last year, according to industry analyst firm Hyperion Research. Read more…

By John Russell

As part of the run-up to SC18, taking place in Dallas next week (Nov. 11-16), Intel is doling out info on its next-gen Cascade Lake family of Xeon processors, specifically the “Advanced Processor” version (Cascade Lake-AP), architected for high-performance computing, artificial intelligence and infrastructure-as-a-service workloads. Read more…

By Tiffany Trader

Networking equipment powerhouse Mellanox could be an acquisition target by Microsoft, according to a published report in an Israeli financial publication. Microsoft has reportedly gone so far as to engage Goldman Sachs to handle negotiations with Mellanox. Read more…