On Jan 27, 10:41 pm, "Nick" <Ibeam2...@gmail.com> wrote:>> What ever happened to massively parallel and array architectures, or> more accurately, why doesn't any of this ever reach the mainstream?

Because it is a complicated programming model. People haven't even
mastered simpler programming models yet. People have been doing dual
CPU - and nowadays dual core CPU - programming badly for quite a
number of years. This is merely at the level of coarse grained thread
programming with different task allocations, and people haven't been
getting it right. There is no market incentive for them to do so: the
CPUs keep getting faster every year, so they can just keep writing
their code to 1 CPU. Moore's Law really hasn't abated, despite the
gloom and doom.

Only now are starting to see 8-core CPUs such as the Cell Broadband
Engine. Only now are such architectures being commodified, such as in
the Playstation3. You really can't get intelligence on such problems
until a lot of people are pursuing them, for everyday commercial
reasons. 8-core is a different focus than 2-core, especially when the
SPEs of a Cell are designd for intensive single precision floating
point 4-vector tasks, not general purpose processing. It shifts the
rational focus of a problem away from coarse threads implementing
different tasks, and towards finer threads partitioning the same
task. Without industrial commodification, there will be no wisdom on
such tasks. We're in very early days for what the cheap HW can
actually support, so I say all your gloom and doom for nothing. We
have to pass through necessary stages of evolution where we get the
basics of parallel computing models correct, and then scale up. 2
cores is not 8 cores is not 80 cores. Nor are multiple cores on a
CPU, the same as multiple CPUs on a board, the same as multiple
computers in a network. We have a lot of memory access models to
consider also.

> Compilation is the economic necessity of underperforming hardware.

Some application domains - 3D graphics, AI - always underperform.
It's not an accident that the Cell BE is being commodified in a game
console.