What's the best way to build networks?

Speakers at the annual Hot Interconnects conference compared communications channels to two tin cans and a string, which are being stretched to the breaking point. Also, they said these channels are sometimes victims of poor engineering decisions.

Silicon photonics is on the near horizon to help fill a few of the gaps, they said. But it brings its own challenges.

Fittingly, the event was held on the headquarters campus of Google, one of many cloud computing giants whose 100 megawatt data centres are stretching networks to the breaking point. "The constant topic is: What's the most cost effective, high performance way to build these networks?" said Andy Bechtolsheim, chair of Arista Networks and serial entrepreneur.

The requirements in requests-for-quotes from data centres have leapt a hundredfold over the last eight years, he said (see above). That's because many data centres have taken an approach to building flat, fast networks so applications can be blissfully unaware of the details of the network.

Arista specialises in building such nets based on the latest, fastest merchant switch chips it can get, mainly from Broadcom. The approach currently can put nearly a million servers on a single network using 10Gbit/s and 40Gbit/s connections (see below).

The job is software intensive. Ninety per cent of Arista's 600 engineers are writing software, Bechtolsheim said. It's only going to get worse with companies such as Google calling for ways to get within milliseconds the precious data about traffic patterns, currently inaccessible on those chips.

Facebook decided it couldn't wait for companies like Arista to come out with new switches, so it will build its own. The Wedge switch (above), already being tested in production networks, will become a design Facebook will contribute to its Open Compute Project, an open-source hardware initiative.