HP Networking – discover the power of networking simplicity.
Get a variety of perspectives on HP Networking innovations, products and solutions from some of the leading experts in enterprise business networking. HP Networking has been a leader in SDN since 2007 and provides leading campus, data center and network virtualization solutions.

Data networking and storage networking technologies have evolved on parallel, but separate paths. Ethernet has emerged as the technology of choice for enterprise data networks, while Fibre Channel (FC) became the dominant choice for enterprise shared Storage Area networks (SANs). Ethernet and Fibre Channel continue to evolve on their way to higher speeds and new capabilities.

Most large organizations have invested in both technologies for data center needs. Ethernet provides the “front-end” Local Area Networks (LANs) linking users to enterprise servers while Fibre Channel provides the “back-end” SANs links between server and storage. Maintaining separate data and storage networks adds to the complexity of managing and maintaining data centers. As enterprises embrace virtualization to increase operational efficiencies, the next logical step appears to be converging server I/O traffic and links onto a common lossless Ethernet transport.

Migrating to a lossless and faster 10/40GbE transport opens the door for data center managers to converge LAN, FC SAN and IP storage traffic over one shared link. With that comes the need to understand where each of these storage technologies fit in future data centers. This document examines some aspects of deploying iSCSI, FCoE and FC in data centers with converged networking infrastructure.

Data Center Storage Technologies

When discussing convergence in the data center, many talk about the various storage technologies as replacement for one another. That is a narrow viewpoint, because technologies, while they may overlap in certain aspects, always find scenarios where they fit best. FCoE, iSCSI and FC are no different for each has its own unique capabilities and can benefit data center operations.

The Fibre Channel over Ethernet (FCoE) protocol aims at enabling the transport of FC storage traffic over a new lossless Ethernet medium; Data Center Bridging (DCB). FCoE runs over Ethernet and does not utilize TCP/IP; thus avoiding the pitfalls of relying on TCP flow control. The most visible benefit of FCoE is in the area of server I/O consolidation. Today, servers use separate adapters to handle storage and networking traffic; Host Bus Adapters (HBA) carry storage traffic, while Network Interface Cards (NICs) handle LAN traffic. With FCoE, a single Converged Network Adapter (CNA) will be used to handle both storage and networking traffic. Using server CNAs will reduce the number of server adapters needed, which in turn reduces the number of I/O cables, expensive transceivers, and the number of switch ports used. Reducing hardware resources simplifies severs I/O configurations and lowers cost of acquisition (CapEx) along with the associated operating costs (OpEx).

Before the introduction of FCoE, there was iSCSI, another encapsulation protocol that enabled transporting of SCSI storage block traffic over 1GbE TCP/IP networks. iSCSI delivered lower costs and enabled routing storage data over IP to remote data centers for backup. The price structure of iSCSI benefited from the fact that it utilized software drivers, lower priced NICs, transceivers, and Ethernet switches. Despite the lower costs, iSCSI saw deployments in small to medium enterprises with limited success in Fortune 1000 enterprises. The limitations were due, in most part, to the lossy nature of Ethernet, its higher latency, and the need for storage management tools in a networking environment. With the move to 10/40GbE and the addition of lossless features to Ethernet as part of the FCoE enablement effort, iSCSI will benefit and the potential for its deployment will improve. The most notable benefit will come from Priority Flow Control (PFC) that will help reduce the iSCSI flow control overhead associated with TCP/IP. Other areas affecting iSCSI performance, such as context switching and data copies will not benefit from the DCB features. The migration to 10/40GbE is a double edged sword for it increases the cost of deploying iSCSI, while giving it a faster transport.

Fibre Channel remains the dominant storage technology in large enterprise data centers. Storage Areas networks (SANs) continue to rely on the high performance and reliability of FC to run demanding database and transaction heavy applications. An increased ramp in 8/16Gbps FC is already underway, and an even faster FC technologies would follow soon. The higher protocol speeds will provide an easier route for data centers to deploy higher performance SANs without disruption. Higher FC performance will ensure that highly demanding data intensive applications continue to have a simple migration path.

Both iSCSI and FCoE are encapsulation protocols and are not FC replacement technologies. In fact, FCoE builds on the success of FC in the data center and utilizes FC along with new lossless Ethernet, DCB, to solve servers I/O challenges facing IT professionals in data centers. Similarly, iSCSI is taking advantage of the idea of SAN that was popularized by FC success in data centers and will benefit from a wider adoption of 10/40GbE adoption driven by FCoE.

Figure 1: Storage technologies stacks showing FCoE and iSCSI encapsulation layers as well as iSCSI use of TCP/IP

Customer Segments

It is widely accepted that iSCSI found success in small and medium size enterprise, while FC is the technology of choice for large enterprise data centers. The advent of FCoE on a DCB transport created new dynamics for customers.

While lossless 10/40GbE benefits iSCSI by reducing TCP/IP flow control overhead, it shrinks the price advantage of iSCSI, because FCoE uses the same 10/40GbE components. In other words, iSCSI will run better and faster on DCB, but the performance improvement will not be free as it will be accompanies by a move up in prices due to 10/40GbE levels. While iSCSI performance will improve benefiting from Priority Flow Control (PFC), it will remain below that of native FC protocol, because 10/40GbE does not remove other areas affecting iSCSI performance such as data copies, context switching and higher Ethernet latency. Customers should compare overall systems performances and not get distracted by results of carefully structured laboratory tests that don’t reflect real life deployments or measure relevant applications performance. The overall price performance curve for iSCSI will get better with DCB and will improve the prospects for wider iSCSI deployments.

It is safe to say that small businesses will continue to deploy iSCSI. On the other hand, medium sized businesses looking for new server and storage deployments will have a choice between iSCSI and FCoE. Those with existing iSCSI installed base are likely to opt for iSCSI, while ones with FC installed base will find FCoE a better fit for their needs.

Figure 2: FCoE, iSCSI and FC overlap, but serve different customers

Storage management will play an important role in the SAN installation decision process. It is logical to select new storage resources that utilize existing management applications, because that reduces acquisition costs and running costs associated with training and IT operations. FCoE was designed to maintain FC upper constructs allowing it to take advantage of FC management. That in turn makes it possible to seamlessly integrate FCoE into existing FC SAN environments. Role-based management will ease IT transition to converged server I/O links. With role-based management, networking administrators can continue to manage the LAN resources, while storage administrators continue to manage shared storage SAN resources.

Large enterprises with existing FC SANs are likely to continue adding to their FC environments taking advantage of higher speed FC. When moving to converged infrastructures, large enterprises will lean towards FCoE for it can integrate seamlessly into existing FC installed base and can be managed by existing applications.

In short, small businesses will continue to use iSCSI and large enterprises are most likely to continue investing in FC. Medium size customers will have a choice between iSCSI and FCoE deployments.

Deployment in the Data Center

An easy way to appreciate where the different storage technologies may be deployed in enterprise environments is to look at three tiered data center.

As encapsulation protocols, FCoE and iSCSI will perform their functions with some overhead above that of native FC protocol. As a result, data centers with genuine need for high performing FC may question the benefits of sharing a 10/40GbE link with LAN traffic. It is expected that FCoE and 10/40GbE iSCSI will most likely get deployed in environments that are currently using 4Gbps FC and 1GbE links. Customers using the lower speed links will see great benefits from migrating to 10/40GbE links. Benefits will come in the form of reduced I/O links and higher server utilization in virtualized environments.

iSCSI will continue to be deployed in tier three applications and may slowly migrate to a few tier two applications. It is best utilized in areas where it provides the most benefit, such as remote data backup.

FCoE is a technology that enterprises will deploy for applications where some risk can be tolerated before they expand the deployment to other areas. In the near term, FCoE will find a home in new server deployments in Windows and Linux environments with virtualized tier three and some tier two applications.

When looking at data center traffic, tier three servers providing web access generate traffic primarily made of TCP/IP data. Both 10GbE NICs and CNAs can easily service this class of servers and their related traffic.

Tier two applications servers with business logic applications tend to host application of greater value to enterprises. These servers are normally connected to LANs and SANs, because their traffic is split between storage and TCP/IP. Some tier two applications, such as email, are good candidates for FCoE and iSCSI and would benefit from server I/O consolidation.

On the other hand, tier one servers tend to host data base applications that support enterprises mission critical applications that are the backbone of business. It is natural for businesses to deploy mature and reliable technologies for tier one servers and applications. In addition to that, tier one applications have a genuine need for performance and processing power that make customers look to higher performance and reliable I/O technologies such as Fibre Channel. It is unlikely that FCoE or iSCSI will find their way in tier one environments in the near term even with 10/40GbE transport, for speed is not the only factor in tier one environments.

Figure 3: Model tiered data center and sample applications

Summary

While Ethernet and Fibre Channel continue to evolve on separate parallel tracks, the time is right to converge data and storage traffic onto shared links. Server I/O consolidation is the area where the benefits of FCoE convergence are most pronounced. FCoE and iSCSI will coexist with FC SANs, but will grow at higher rates than FC as 10/40/100GbE reduce the performance advantages of native FC.

As IT managers grow more comfortable with the maturity of iSCSI and FCoE management tools, we will see the two technologies handing more business applications in enterprise data centers.