Bolt, Baranek, and Newman (BBN)

BBN of Cambridge, Massachusetts was a venerable high-tech institution,
pioneers in acoustics and electronics, at the heart of the development
of both sonar and the internet. Their supercomputing venture was a
failed sideline, the demise of which was not fatal to the parent.
Nevertheless, the company no longer exists independently, having been
bought out by GTE.

The following is pieced together from anecdotes and various references
on the web and in the literature. Any BBN project veterans are kindly
requested to write up their recollections and send them to kevink@acm.org.

Architecture

The BBN Butterfly was so named for the "butterfly"
multi-stage switching network around which it was built. Up to 512
CPUs, each with local memory, could be connected to allow every CPU
access to every other CPUs memory, albeit with a substantially greater
latency (roughly 15:1) than for its own. The CPUs were commodity
microprocessors. The earlier, GP-1000 models used Motorola 68020's and
scaled to 256 CPUs. The later, TC-2000 models used Motorola 88100's,
and scaled to 512 CPUs.

Technology

The Butterfly did not employ particularly exotic technology in electronics or
packaging. This reduced the development and manufacturing costs, but may have
contributed to reports of the system running hot and having reliability issues.

Software

The Butterfly began life, poetically enough, under a proprietary OS
called Chrysalis, but moved to a MACH-based operating system in 1989.
While the memory access time was distinctly non-uniform, the machine
still had SMP memory semantics, and could be operated as a symmetric
multiprocessor.

Strong Points

TotalView[tm], the parallel program debugger developed for the
Butterfly outlived the platform, to be ported to a number of other
massively parallel machines.

Weak Points

These machines were physically large and very expensive for the
performance they delivered. The commodity microprocessors of the period
did not support multiple outstanding memory requests, so while it was
technically possible to run SMP parallel programs on the machine, the
remote memory access penalty killed performance unless good data
distribution was achieved. And if one has solved the data distribution
problem, why pay for global memory access when a message-passing
machine will deliver similar parallelism at a lower cost. Case in
point: Oracle did some of their early massively-parallel database
server experiments on the TC-2000, but abandoned it in favor of the
Ncube-2.

Lessons Learned

A spiffy interconnect by itself will not really allow the construction
of a supercomputer out of common household appliances.