Startup Cracks MultiCore, Thread Programming Problem

The toughest problem in computing today -- easily creating software that takes full advantage of multi-cores/processors -- has been cracked.

SVIRAL, a young Silicon Valley company working on programming and computing issues, appears to have cracked one of the toughest problems in computing today -- that of creating software programs that can effectively use multiple processing cores and threads without requiring programmers to have PhDs in computer science.

Meet the problem
Here's the problem in a nutshell. Most processors follow the Von Neumann or Harvard-type architectures. These architectures are ideal for executing traditional software programs in which the instructions are processed sequentially (one after another).

For many years, processor manufacturers boosted performance by increasing the system clock speed (yes, of course this is a simplification, they use all sorts of other tricks, but clock speed is a major contributor). This strategy started to stall circa the beginning of the 2000s. For a variety of reasons, including power consumption and associated thermal issues, system clocks -- which were by then running at microwave frequencies -- simply could not be ramped up any higher. The solution was to deploy multiple processor cores on a chip and/or multiple multi-core processor chips in the system. The end result is a plethora of processors. For example, some of today's mobile devices may boast 70+ processor cores, and this number is increasing as we speak.

The problem comes when we try to write software that can use these processors effectively. One technique is to partition the software into multiple threads, where threads are sequential processes that share memory. Now, it's true to say that there are a few "embarrassingly parallel" applications that can take full advantage of multiple threads running on multiple processors, but this is largely because these applications can be divided into sub-tasks that can be performed in isolation. The vast majority of programs do not fall into this category. (See The Problem With Threads by Edward Lee for a more in-depth discussion of this concept.)

In the case of the majority of programs, the use of threads ends up being something of a nightmare. A very common problem, for example, is non-determinism. Suppose that every time you run a particular application, a message from one thread to another arrives at a fractionally different time relative to some other occurrence. And, of course, the application can be riddled with cases like this. The end result is that the application can behave very differently on each run (this is not considered to be a desirable attribute). Debugging something like this can bring the strongest of programmers to their metaphorical knees.

Another problem is the inefficiencies caused by using threads. When you have multiple threads all clamoring for attention and all waiting on each other to complete various actions, you can end up with a Gordian Knot of threads in which the system spends more time servicing the inter-thread communications than it does performing the main task. As I write these words, I'm looking at screenshots of CPU usage associated with a classic example -- a task that was first run in single-threaded mode in which 61% of the CPU's cycles were used to do real work, 7% were consumed by system-level/background tasks, and 32% simply could not be used at all, thereby leaving the CPU idle during these times. This resulted in an overall application performance of 3.4 giga-operations per second.

When the same task was run in a multi-threaded mode, only 45% of the CPU's cycles ended up being used to perform real work, while a whopping 55% ended up being used for system-level tasks (predominantly inter-thread communication), and there were 0% idle CPU cycles. This resulted in an overall application performance of only 2.5 giga-operations per second. Thus, the end result was that the multi-threaded version saw a 26% decrease in performance while consuming more power as compared to its single-threaded counterpart. You donít have to be a software genius to recognize that this is not a good thing.

If I were them, I'd look for a problem that responds well to parallelism, and offer to solve it for a lower price than the present technique. Keep undercutting the competition, and pretty soon they'll notice you.

Sounds really promising and I would hope that folks make the effort to at least conduct a trial of the approach. How has it been productized? Do they sell a tool, a service, license updated libraries, or what?

We've all heard reports about "latest and greatest" techniques to crack the multi-core programming problem, and we've all gotten tired of being disappointed, but I truly believe that this is "the real deal" -- this could change so many things...