> OK, let's look at some numbers and compare. 100 Mb/s EtherNet is now a> commidity item: Gb/s is still experimental. So we have networks with> 10 MB/s bandwidth and 2 millisecond latencies.

Hogwash! I'm the Network Eng. at the county offices in Martin CountyFlorida. We not only have gigabit ethernet in our (sun) servers, but useit for connecting remote sites!

Yes, the workstations are still connected via switched 100Mb/s.. Butgigabit ethernet works and will be commidity someday soon (5 years soundsgood).. However, switched 100Mb/s has a bandwidth of 20MB/s (full duplex).

Btw- 100Mb/s has better latency then 2ms! I imagine that if you had a MPIimplimentation that banged the hardware correctly (or a speical networkAPI for doing zero copy for MPI and such) you could get 1/4 or less thatlatency.

> Now let's look at the bandwidth of a modern, commidity computer, say> PII. With 100 MHz SDRAM, a 4-1-1-1 burst gives you 8 bytes per step;> with 4 steps that's 32 bytes for the burst. It takes 7 cycles. Let's> say 8 to make the maths simpler. So 8 cycles gives 32 bytes. That's 4> bytes per cycle: 400 MBytes/s. The latency is going to be well under> 200 nanoseconds.> > So: we have a factor of 40 in bandwidth and 10000 in latency. And this> comparison is only done with the boring 64 bit bus in your average PC.> It gets worse if you look at a 256 bit bus: bandwidth goes to 1.6 GB/s,> which is 160 times better than current EtherNet.

Okay, cant argue there.

> Right now, there's no competition: networks are so much slower than> what you find in a computer.

[snip]> Networking has not kept up with Moore's Law. It seems computers have,> however. So the logical progression is that in the future, we're going> to see more of the same. It's going to take years before gigabit> EtherNet is affordable, and by then computers will be even further> ahead.

So? You have failed to show that slow networking makes clusteringworthless.

Consider, the Avalon which is interconnected with switched 100Mb/sachieved enough performance (after a memory upgrade) to reach the 50thfastest computer in the world at a total price of around what? $280K?

Even on REAL (and network smashing) code (spasm and grav tree code) theygot better price/performance then out of real supercomputers.

The nodes were 21164a's with more memory bandwidth then your 100MHzexample (66Mhz SDRAM/ 128Bit bus). Yes, the network was slow, and eventhough it slaughtered performance (1 node performance * n nodes, wassomething like 10 times better for some codes), clustering was still worthwhile.

> > Heck even if this view is off and skewed to heck, *it is possible*.> > And things are getting faster, it's only a matter of time before> > networking is fast enough (or integrated enough) that clustering> > such as this would be possible.> > Let me make a bottom-line statement: networking will *never* be as> fast as internal computer speeds. Technology trends point to computers> increasing at a greater rate. And then there's physics. Unless you're> planning on breaking the speed of light, it's always going to take a> lot longer to send information across a room than across a chip.

And then there is reality: Faster chips and 'real supercomputers' willalways cost so much more then commodity supercomputers that clusteringwill be worthwhile long into the future.

> [...]> > Think.> > Count.

Think different.

-To unsubscribe from this list: send the line "unsubscribe linux-kernel" inthe body of a message to majordomo@vger.rutgers.eduPlease read the FAQ at http://www.tux.org/lkml/