Abstract

In the past half century, parallel computers, parallel computation,
and scientific research have grown up together. Scientists
and researchers’ insatiable need to perform more and larger
computations has long exceeded the capabilities of conventional
computers. The only approach that has met this need is
parallelism—computing more than one operation simultaneously.
At one level, parallelism is simple and easy to put into practice.
Building a parallel computer by replicating key operating components
such as the arithmetic units or even complete processors is
not difficult. But it is far more challenging to build a well-balanced
machine that is not stymied by internal bottlenecks. In the end,
the principal problem has been software, not hardware. Parallel
programs are far more difficult to design, write, debug, and tune
than sequential software—which itself is still not a mature, reproducible
artifact.