Datacenters Drive 100G Race

SAN JOSE, Calif. — About 20 companies led by Arista Networks and Intel proposed a standard for 100 Gbit/s links for datacenter switches -- the third such proposal in as many weeks. The efforts indicate an accelerating race to ship 100G systems this year, responding to a rising data tsunami in datacenters.

The CRL4 100G Alliance aims to specify a 100G coarse wavelength division multiplexing (CWDM) transceiver in a QSFP package that measures 8.5 x 18 x 72 mm. It will dissipate 3.5 W max and support distances up to 2 km using two singlemode optical fibers. The group will take industry feedback on a preliminary spec, aiming for a final spec in May.

The proposal comes one day after companies including Avago, Finisar, JDSU, and Oclaro proposed a similar spec called CWDM4. At the OFC event in March, Mellanox announced with partners its Open Optics specification based on dense WDM technology.

The three specs all aim to serve datacenter switches hungry for bandwidth, filling a gap between short- and long-range 100G specs today. Separately, Cisco and Avago have developed a proprietary 40G link some expect to scale to 100G.

"I think the real story is how broken the optics industry must appear from the outside with everyone tripping over each other to claim they have the 100G duplex standard of choice," said the chief executive of one optics vendor who asked not to be named. "I suspect the plethora of choice of different ways to do the same thing will lead to user confusion and a market delay."

Time to market is a top priority, said veteran server and switch entrepreneur Andy Bechtolsheim in a press conference announcing the CLR4 effort.

"We need those optics now. We can't wait anymore," said Bechtolsheim, noting his company, Arista, announced a 100G linecard for its switches last week. "People want to deploy in volume this calendar year; we are simply out of time to fuss around -- we've got to get the optics nailed as soon as possible."

Just nine months ago an IEEE committee looking into the issue failed to get a vote for any proposal on the kind of midrange optics datacenters need, Bechtolsheim said, noting that it was "predictable" vendors would respond with a variety of competing ad hoc proposals known in telecom as multisource agreements.

The Mellanox approach is based on the DWDM technology it acquired with startup Kotura and said it will ship late this year or early next year. CWDM "makes the most sense to me, given broad industry support and likely the one to market most quickly," he said.

As for physical interconnects, Bechtolsheim complained about "at least six different form factors in the market shipping not even 10,000 units per quarter -- that's more form factors than 10G ever had. It led to confusion in the industry over what to build and to invest in."

Today's high-speed optical modules come in at least six different flavors.

The CLR4 group supports QSFP, which Bechtolsheim says has market traction. The CWDM4 spec is "form factor agnostic... however, we expect QSFP28 to be the mainstream form factor, though alternate form factors such as CFP4 and CFP2 could also be developed if required by the market," said a spokesperson.

Bechtolsheim has been promoting silicon optics as a solution for 100G datacenter switches since an October 2012 keynote. Intel announced it was sampling its 100G silicon photonics components in January 2013 at a Facebook server design event where it got support from Bechtolsheim.

The industry is now developing 100G per lane wavelength optics. For a good update consider the all day Ethernet Alliance and OIDA Workshop on this topic in June. The industry is also defining 50G per lane electrical technology in the OIF. 100G per electrical lane is some years off, but expect people to start looking at it in 3 to 5 years.

The general purpose (i.e. used for a variety of MMF and SMF optics) 10G form factors in chronological and density order are:

300-pin MSA, XENPAK, X2, XPAK, XFP, SFP+, QSFP+ (quad channel 10G).

So that's a total of 7 general purpose 10G form factors, counting the similar X2 and XPAK separately.

There was also a high density SFF-8470 InfiniBand form factor which supported 2.5G lanes, which could have been used for XAUI 10G interfaces but was not.

The general purpose 100G form factors in chronological and density order are:

CFP, CFP2, CPAK, CFP4, QSFP28

That's a total of 5 general purpose 100G form factors, counting the similar CFP2 and CPAK separately.

There is also a high density CXP InfiniBand form factor, which is restricted to DAC and MMF interfaces, for both multi-channel 10G and single-channel 100G. If CXP is included in the general purpose form factor counts, the totals are 8 for 10G and 6 for 100G. So today there are more 10G form factors than 100G form factors.

Because SFP+ is so dominant, we tend to forget that it is not the first 10G form factor. The first 10G module (300-pin MSA) was introduced in the late 90s. By today's standards, it is incredibly clunky because it's ~20x the size of the SFP+. However, when introduced it was a breakthrough product because it replaced full size cards.

100G skipped 300-pin MSA as first form factor. We should also not expect QSFP28 to be the end of the road for 100G. A possible next form factor is SFP100, utlizing future 50G or 100G per lane technoogy. This is normal evolution. As optics and IC integration improves, the size shrinks and density increases.