Lenovo is open and ready for business with System xIBM x86 products are now products of Lenovo in the U.S. and other countries. IBM will continue to host x86-related content on ibm.com until migrated to Lenovo.Learn more about the acquisition

Abstract

The IBM Flex System™ FC5022 16Gb SAN Scalable Switch is a Gen 5 Fibre Channel (FC) high-density, 48-port switch that operates at 16 and 8 Gbps FC speeds and is used in the IBM Flex System chassis. The switch provides 28 internal ports to compute nodes by way of the Flex midplane and 20 external SFP+ ports, as shown in Figure 1. The FC5022 SAN switch module delivers a scalable embedded option for IBM Flex System users connecting to 8 or 16 Gbps FC storage, deploying a new storage area network (SAN) or connecting to an existing SAN in their enterprise. The switch offers end-to-end 16 Gb with backwards compatibility to 8 Gb and 4 Gb to the SAN connectivity. Monitoring and management are simplified using end-to-end Brocade tools, such as Web Tools and the Brocade Network Advisor (BNA).

Contents

The IBM Flex System™ FC5022 16Gb SAN Scalable Switch is a Gen 5 Fibre Channel (FC) high-density, 48-port switch that operates at 16 and 8 Gbps FC speeds and is used in the IBM Flex System chassis. The switch provides 28 internal ports to compute nodes by way of the Flex midplane and 20 external SFP+ ports, as shown in Figure 1. The FC5022 SAN switch module delivers a scalable embedded option for IBM Flex System users connecting to 8 or 16 Gbps FC storage, deploying a new storage area network (SAN) or connecting to an existing SAN in their enterprise. The switch offers end-to-end 16 Gb with backwards compatibility to 8 Gb and 4 Gb to the SAN connectivity. Monitoring and management are simplified using end-to-end Brocade tools, such as Web Tools and the Brocade Network Advisor (BNA).

Figure 1 shows the switch module.

Figure 1. IBM Flex System FC5022 16 Gb SAN Scalable Switch

Did you know?

Only the FC5022 provides up to 48 FC ports in an embedded form factor that provides the Flex System a unique differentiation in the market including:

Increased server density. The 28 internal FC ports allow up to 28 Information ITEs in a single chassis, which lowers the average server cost of the solution.

Greater reliability. When used with 4-port FC mezzanine adapters, the 28 internal ports enable greater path redundancy and increased solution availability with no single point of failure.

Multi-chassis Scalability. The 20 external SFP+ ports allow for scaling up to 18 chassis without the use of external FC switches, such as FC top-of-rack (ToR) switches, thereby greatly reducing the overall cost of the Flex Solution.

Part number information

The part numbers to order the switch and optional upgrades are shown in Table 1.

The part numbers for the Features on Demand upgrades include the following items:

Features on Demand Activation Flyer

Upgrade authorization key

The switch does not include a serial management cable, however, IBM Flex System Management Serial Access Cable, part number 90Y9338, is supported and contains two cables, a mini-USB-to-RJ45 serial cable and a mini-USB-to-DB9 serial cable, either of which can be used to connect to the switch locally for configuration tasks and firmware updates.

The FC5022 12-port and 24-port ESB SAN switches come without SFP+ media, which must be ordered separately to provide outside connectivity. The FC5022 24-port SAN switch (part number 00Y3324) comes standard with two Brocade 16 Gb SFP+ transceivers. Additional SFP+ can be ordered, if required. Table 2 lists the supported SFP+ options.

* With Fabric OS 7.2 and above, the Brocade Fabric Vision functionality is included with the 24-port 16 GB ESB SAN switch, but may not be displayed as an individual license key. The functionality may instead be displayed along with the Fabric Watch and Advanced Performance Monitoring license descriptions with the added verbiage "Fabric Vision Capable."
Table 4 provides compatibility information for the switch modules and Features on Demand upgrades for them.

With Brocade's Dynamic Ports on Demand (DPOD) feature, ports are licensed in one of two methods: as they come online or as selected by the administrator. With the 16 Gb SAN switch, the first 12 ports reporting (on a first-come, first-served basis) on boot-up are assigned licenses. These 12 ports can be any combination of external or internal Fibre Channel (FC) ports. After all port licenses have been assigned, you can manually move those licenses from one port to another. Because this is dynamic, no defined ports are reserved except ports 0 and 29. The 24-port 16 Gb SAN switches have the same behavior; the only difference is the number of ports, which is 24.

Table 5 shows the total number of port licenses on the switch after applying compatible port upgrades.

Note: A maximum of one upgrade of each type (one Upgrade 1 and one Upgrade 2) is supported per one switch module. Upgrade 1 and Upgrade 2 can be applied independently of each other.

Table 5. Ports on Demand upgrades

Ports on Demand upgrade

Total number of port licenses

24-port 16 Gb ESB SAN switch

24-port 16 Gb SAN switch

12-port 16 Gb SAN switch

Included with base

24

24

12

Upgrade 1, 88Y6382 only (adds 12 port licenses)

Not supported

Not supported

24

Upgrade 2, 88Y6386 only (adds 24 port licenses)

48*

48*

36

Both Upgrade 1, 88Y6382 and Upgrade 2, 88Y6386 (add 36 port licenses)

Not supported

Not supported

48*

* With the dual-port FC adapters installed in compute nodes, up to 34 ports (14 internal, 20 external) can be utilized. The remaining 14 internal ports require use of the 4-port FC adapters in the server's ITE.

Benefits

The switches offer the following key benefits:

Exceptional price/performance for growing SAN workloads

The FC5022 16 Gb Switch delivers exceptional price/performance for growing SAN workloads through a combination of market-leading 1600 MB/sec throughput per port and an affordable high-density, 12-port to 48-port form factor.

Enhanced server performance while using 16 Gb Gen 5 FC technology

IBM testing demonstrates that using 16 Gb FC adapters on the server and the FC5022 modules can help achieve the following server enhancements, even if 8 Gb FC storage is used:

Up to 30-50% better virtual machine density and higher number of concurrent users due to increased storage bandwidth

Up to 20-40% fewer servers required to support the workload specified

Twice lower number and higher speed of inter-switch links required in scalable SANs

Higher reliability and availability of services due to fewer number of components used to build the solution

Twice faster access to the business critical data

Lower acquisition costs due to fewer number of systems and components

Shorten ROI time frame and decrease overall TCO with the efficient utilization of server resources and lower power, cooling, and management costs

Superior scalability both for internal and external ports that reduces costs and increases availability

The FC5022 16 Gb switch is the only switch that provides up to 28 internal server facing FC ports, which allows the Flex Chassis to scale to 28 server ITEs in a single chassis (when using double density half-width ITEs) that lowers the overall per-server cost of the solution. These 28 internal ports also enable greater path redundancy for ITEs using the 4-port FC mezzanine adapters. Additionally, the FC5022 offers an industry leading 20 external SFP+ ports that allows for multi-chassis scalability up to 18 chassis without the use of external FC switches, thereby greatly reducing the overall cost of the Flex Solution.

The FC5022 16 Gb Switch supports multi-tenancy in cloud environments through VM-aware end-to-end visibility and monitoring, Quality of Service (QoS), and fabric-based advanced zoning features. The FC5022 enables secure long distance extensions to virtual private or hybrid clouds with dark fiber support, as well as in-flight encryption and data compression.

All models of the FC5022 16 Gb Switch can be deployed as a full-fabric switch or as an Access Gateway, which simplifies fabric topologies and heterogeneous fabric connectivity. Access Gateway mode utilizes N_Port ID Virtualization (NPIV) switch standards to present physical and virtual servers directly to the core of SAN fabrics. This makes it transparent to the SAN fabric, greatly reducing management of the network edge. The IBM SAN Scalable Switch in Access Gateway mode can connect servers to NPIV-enabled SAN fabrics, including IBM, Brocade B-Series, other vendor Brocade SAN fabrics, or Cisco SAN fabrics.

Maximizing investments

To help optimize technology investments, IBM offers a single point of serviceability backed by industry-renowned education, support, and training. In addition, the IBM 16/8 Gbps SAN Scalable Switch is in the ServerProven® program, ensuring compatibility among a variety of IBM and partner products. IBM recognizes that customers deserve the most innovative, expert integrated systems solutions.

Features and specifications

The FC5022 16 Gb Switches have the following features and specifications:

One RS-232 serial port (mini-USB connector) that provides an additional means to configure the switch module

Access Gateway mode (N_Port ID Virtualization - NPIV) support

Power-on self-test diagnostics and status reporting

Inter-Switch Link (ISL) Trunking (licensable or included with the 24-port ESB model), which allows up to eight ports (at 16, 8, or 4 Gbps speeds) to combine to form a single, logical ISL with a speed of up to 128 Gbps (256 Gbps full duplex) for optimal bandwidth utilization, automatic path failover, and load balancing

Fabric OS (FOS) version 7.0.x or higher, which delivers distributed intelligence throughout the network and enables a wide range of value-added applications, such as Advanced Web Tools and Advanced Fabric Services (on certain models)

Advanced Zoning: Segments a SAN into virtual private SANs to increase security and availability

Enhanced Group Management: Enables centralized and simplified management of SAN fabrics through IBM Network Advisor

Server Application Optimization: Allows VM QoS, authentication, and monitoring from SAN through adapter to VM and associated virtualized application

The 24-port ESB model (part number 90Y9356) is a fully enabled switch with a complete set of licensed features that maximizes performance, ensures availability, and simplifies management for the most demanding applications and expanding virtualization environments. The factory-installed licenses include:

ISL Trunking: Consolidates ISLs into fault-tolerant and load-balanced trunks with bandwidth of up to 128 Gbps for greater performance and simplified management

The switches are installed in switch bays in the rear of the IBM Flex System Enterprise Chassis, as shown in Figure 2. Switches are normally installed in pairs since I/O adapter cards installed in the compute nodes route to two switch bays for redundancy and performance.

Figure 2. Location of the switch bays in the IBM Flex System Enterprise Chassis

The connections between the 2-port adapters that are installed in the compute nodes and the switch bays in the chassis are shown in Figure 3. Figure 3 shows both half-wide servers, such as the x240 with two adapters, and full-wide servers, such as the x440 with four adapters.

Figure 3. Logical layout of the interconnects between the 2-port FC adapters and I/O modules

The connections between the 4-port FC adapters that are installed in the compute nodes and the switch bays in the chassis are shown in Figure 4.

Figure 4. Logical layout of the interconnects between the FC5054 4-port adapters and I/O modules

The IBM Flex System FC5022 16Gb SAN Scalable Switches can be installed in bays 1, 2, 3, and 4 of the Enterprise chassis. A supported adapter card must be installed in a corresponding slot of the compute node (slot A1 when switches are installed in bays 1 and 2 or slot A2 when switches are in bays 3 and 4).

With compute nodes that have an integrated dual-port 10 GbE network interface controller (NIC) these switches can only be installed in bays 3 and 4 because integrated NICs ports are routed to bays 1 and 2 with a specialized periscope connector and the FC adapter card cannot be installed in slot A1. When needed, however, the periscope connector can be replaced with an FC adapter card, in which case the integrated NIC will be disabled and FC switches can be used in bays 1 and 2.

Table 6 shows the connections between dual-port and 4-port FC adapters installed in the compute nodes other than x222 to the switch bays in the chassis with IBM Flex System FC5022 16 Gb SAN Scalable Switches.

Table 6. Adapter to I/O bay correspondence

I/O adapter slot in the server

Port on the adapter

Corresponding I/O module bay in the chassis

Slot 1

Port 1

Module bay 1

Port 2

Module bay 2

Port 3

Module bay 1

Port 4

Module bay 2

Slot 2

Port 1

Module bay 3

Port 2

Module bay 4

Port 3

Module bay 3

Port 4

Module bay 4

Slot 3(full-wide compute nodes only)

Port 1

Module bay 1

Port 2

Module bay 2

Port 3

Module bay 1

Port 4

Module bay 2

Slot 4(full-wide compute nodes only)

Port 1

Module bay 3

Port 2

Module bay 4

Port 3

Module bay 3

Port 4

Module bay 4

Table 7 shows the connections between 4-port adapters installed in the x222 compute nodes to the switch bays in the chassis with IBM Flex System FC5022 16 Gb SAN Scalable Switches.

Table 7. x222 adapter to I/O bay correspondence

Server

FC5024D 4-port16Gb FC Adapter

Corresponding I/O modulebay in the chassis

Upper server

Port 1

Module bay 3

Port 2

Module bay 4

Lower server

Port 1

Module bay 3

Port 2

Module bay 4

With the x222, each node bay consumes four ports on the switch, therefore, additional port licenses might be needed depending on your configuration (see Table 1 for FoD port upgrades).

The OK LED indicates that the switch module has passed the power-on self-test (POST) with no critical faults and is operational.

The blue Identify LED can be used to identify the switch physically, by illuminating via the management software.

The Error LED (switch module error) indicates that the switch module has failed the POST or detected an operational fault.

One mini-USB RS-232 console port that provides an additional means to configure the switch module. This mini-USB-style connector enables connection of a special serial cable (cable is optional and it is not included with the switch; see the part number information section for details).

Figure 7 shows use of the IBM Flex System FC5022 24-port 16Gb ESB SAN Scalable Switch to route two ports on the adapter card installed in slot 2 of the compute node. Each compute node has two 16 Gb FC ports, and two FC5022 ESB SAN switches are installed in bays 3 and 4 of the Enterprise Chassis. The connections between the adapter card and the switches are internal to the chassis. No cabling is needed.

Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.