Solutions

Recent data centers trends

Data center power consumption continues to draw a lot of attention. According to a recent IDC report, power accounts for 40% of the operating expenses of a data center and the cost of this power has doubled in the last five years. By 2012, IDC estimates data center power costs will equal server capital cost. Additionally, the contribution of data centers as a fraction of total energy consumed is growing as well. As per the 2007 EPA report on data center efficiency, data centers contributed to 1.5% of all the power consumed in the US, with the number expected to double by 2011. These trends are driving the emphasis towards data center power efficiency.

A key element in the next generation energy efficient data centers is server virtualization. Virtualization enables multiple virtual servers to run a single machine, enabling companies to consolidate their data center infrastructure into fewer, high density, high efficiency facilities. To efficiently use server virtualization, it is essential that available I/O bandwidth can support the data storage and retrieval requirements of multiple virtual servers running on a physical machine. Without sufficient I/O bandwidth, the available processing power in today's multi-core processor-based servers is not fully utilized. This I/O bandwidth requirement is driving the transition of data center interconnect speeds from the current 1 Gbps to 10 Gbps.

As 10 Gbps interconnects become ubiquitous in servers, providers are looking for a low cost, low power, and space efficient interconnect solution for the short reach (5-7m) links that dominate the data center environment. A length of seven meters covers all connections between server cards and switches, typically mounted on a single data center rack, and a vast majority of inter-rack connections.

The SFP+ Direct Attach Copper is the 10G interconnect technology that matches these requirements, playing a vital role in enabling the next generation power and cost efficient data centers.

What is 10G SFP+ and its advantages?

Small-form-factor pluggable (SFP) is the industry standard for data rates up to 4 Gbps. In the recently standardized SFP+ MSA, the small form factor optical module for 10 Gbps is identical in dimensions to the SFP. To achieve the SFP+ form factor reduction, only the optical-to-electrical and electrical-to-optical conversion functionalities occur inside the optical module. The key advantage of SFP+ over the existing 10 G optical interconnects is the higher port densities enabled by its dimensions, and lower power consumption.

SFP+ Direct Attach Copper

To further reduce cost and power in interconnect distances of less than 7-10 meters, which is sufficient to link server cards and switches, the SFP+ Direct Attach Copper replaces the optical modules and fiber with a passive copper cable with connectors identical to an SFP+ optical module. The reduction in cost and power are significant since the price of two optical modules required to support a full duplex link is approximately 10 times the cost of a fully connectorized SFP+ Direct Attach cable. In addition, the optical SFP+ modules consume around 1W each, adding 2W per port to the overall system power budget and cooling requirements.

Data is transmitted as 10 G serial NRZ (Non Return to Zero) symbols with transmit pre-emphasis and receive equalization compensating for Inter Symbol Interference (ISI) caused by the board trace and the copper cable. This choice of serial NRZ transmission over a passive medium makes SFP+ copper both a low power and low latency solution compared to its alternatives.

How does SFP+ Direct Attach compare?

The other short reach interconnect alternatives to SFP+ Direct Attach are 10GBASE-CX4 and 10GBASE-T. SFP+ Direct Attach has significant power, cost, and performance advantages over these.

10GBase-CX4 achieves the aggregate rate of 10 Gbps by transmitting over four independent cables at 3.125 Gbps. The disadvantage of this solution is the bulkiness of the cables that have eight twin-ax cables within them for a duplex link. And the bulkiness of the cables makes cable management more difficult. This becomes a significant limitation as the port densities and inter-port connection densities increase. Additionally, the number of conductors make these cables expensive compared to SFP+ cables.

10GBase-T enables 10 Gbps transmissions over Cat 6 and higher quality cables using complex signal processing and channel coding. The potential advantage of this technology is its extended reach. The extended reach and ability to enable structured cabling are not required for the short reach interconnects between servers and switches collocated in a rack. The disadvantages are its high power consumption and latency. The high latency in particular is a key limitation in latency sensitive data center and storage applications.

Conclusion

As more emphasis is placed on energy efficient data centers and higher bandwidth applications, the need for a small form factor, low power, low latency and low cost interconnect makes SFP+ Direct Attach the optimal solution for short reach 10 G interconnects. Through aggregation, SFP+ Direct Attach also provides a low cost upgrade path for higher speed (40 G and 100 G) interconnects.