Code controls communication
to boost computer performance

Posted August 27, 2009

As high-performance computers bring more power to programmers,
communication often limits an operation’s overall speed.

Even a seemingly simple computing challenge like building a
histogram
can reveal the need for new approaches to orchestrating parallel
interactions between hardware and software. That’s because datasets
too large to fit on a single processor’s memory must be spread over
multiple processors.

“Randomly reading or writing a distributed data structure like a
histogram can be very expensive on large-scale parallel machines,”
says Katherine Yelick, professor in the computer science division at the
University of California at Berkeley and NERSC (National Energy Research
Scientific Computing Center) division director at Lawrence Berkeley
National Laboratory.

Other factors, such as the increasing number of processor cores per
chip and the difficulty of
random-access
communication
across all of
a machine’s cores, further push the limits of today’s parallel
computing approaches. Such challenges stimulated the development of Unified
Parallel C, better known simply as the UPC language.

Message Passing Interface
(MPI)
has long been the primary way processors
communicate in high-performance computers (HPC), “but people are
finding some limitations in it,” says Paul H. Hargrove
of the HPC research department at Lawrence Berkeley National Laboratory.

For instance, if data must be used by multiple processors, MPI often
makes copies of that data. In the histogram example, MPI might replicate
and later combine instances of the histogram. Even when using nearest-neighbor
communication, MPI often replicates some of the data to reduce some forms
of communication. As a result, MPI uses up some memory just for copies of
data.

UPC tries to resolve some of MPI’s shortcomings, Hargrove says. In
particular, UPC takes a new approach to communicating between processors.
As a result, a range of experiments show UPC often outperforms
MPI – usually by a large margin.

Taking sides

MPI uses so-called two-sided message passing. When passing data between
processors, a programmer must use a “send” and a
“receive” command; one processor uses “send”
to alert another that data are coming, and the target processor uses
“receive” to say that it’s ready. The “send”
provides the data destination, the length of the information being sent,
and so on. An MPI “receive” requires matching information.