Configuring Fibre Channel Interfaces

Licensing Requirements for Fibre Channel

On
Cisco Nexus devices, Fibre Channel capability is included in the Storage Protocol Services license.

Ensure that you have the correct license installed (N5010SS or N5020SS) before using Fibre Channel interfaces and capabilities.

Note

You can configure virtual Fibre Channel interfaces without a Storage Protocol Services license, but these interfaces will not become operational until the license is activated.

Physical Fibre Channel Interfaces

Cisco Nexus devices
support up to sixteen physical Fibre Channel (FC) uplinks through the use of two, optional explansion modules. The first module contains eight FC interfaces. The second module includes four Fibre Channel ports and four Ethernet ports.

Each Fibre Channel port can be used as a downlink (connected to a server) or as an uplink (connected to the data center SAN network). The Fibre Channel interfaces support the following modes: E, F, NP, TE, TF, TNP, SD, and Auto.

Like any interface in Cisco NX-OS, vFC interfaces are manipulable objects with properties such as configuration and state. Native Fibre Channel and vFC interfaces are configured using the same CLI commands.

vFC interfaces support only F mode and operate in trunk mode only.

The following capabilities are not supported for virtual Fibre Channel interfaces:

VF Port

vFC interfaces always operate in trunk mode; vFC interfaces do not operate in any other mode. You can configure allowed VSANs on a vFC by using the switchport trunk allowed vsan command under the vfc interface (which is similar to FC TF and TE ports). For vFC interfaces that are connected to hosts, port VSAN is the only VSAN that supports logins (FLOGI). We recommend that you restrict the allowed VSANs for such vFC interfaces to the port VSAN by using the switchport trunk allowed vsan command in the interface mode to configure a VF port.

Includes support for 160 vFC interfaces.

The vFC VSAN assignment and the global VLAN-to-VSAN mapping table enables the Cisco Nexus device to choose the appropriate VLAN for a VF port.

The VF port support over 10G-FEX interfaces feature is supported only in Cisco Nexus Fabric Extender straight-through topologies where each Fabric Extender is directly connected to a Cisco Nexus device.

VE Ports

A virtual E port (VE
port) is a port that emulates an E port over a non-Fibre Channel link. VE port
connectivity between Fibre Channel Forwarders (FCFs) is supported over
point-to-point links. These links can be individual Ethernet interfaces or
members of an Ethernet port-channel interface. For each of the FCF connected
Ethernet interfaces you must create and bind an vFC interface to the Ethernet
interface. Configure vFC interfaces as VE ports by using the
switchport mode e command in interface mode.

VE ports have the
following guidelines:

Auto mode on the
vFC is not supported.

VE Port trunking
is supported over FCoE-enabled VLANs.

VE Port
interface binding to MAC addresses is not supported.

By default the
VE Port is enabled for trunk mode.

You can
configure multiple VSANs on the VE port. You must configure the FCoE VLANs that
correspond to the VE port’s VSANs on the bound Ethernet interface.

The Spanning
Tree Protocol is disabled on the FCoE VLANs on any interface that a vFC
interface is bound to, which includes the interfaces that the VE ports are
bound to.

The number of VE
port pairs that can be supported between a given FCF and a peer FCF depends on
the FCF-MAC advertising capability of the peer FCF:

If a peer FCF
advertises the same FCF-MAC address over all its interfaces, the FCF can
connect to it over one VE port. In such a topology, we recommended that you use
one port-channel interface for redundancy.

If a peer FCF
advertises multiple FCF-MAC addresses, the limits in the table apply.

VE Ports in a vPC Topology

VE ports in a vPC
topology have the following guidelines:

Dedicated links
are required for FCoE VLANs between FCFs connected over a vPC for LAN traffic.

FCoE VLANs must
not be configured on the inter-switch vPC interfaces.

FSPF Parameters

FSPF operates on a
per-VSAN basis over a VE port once it is brought up on the VSAN. The default
FSPF cost (metric) of the vFC interface is as per 10-Gbps bandwidth. For VE
ports that are bound to Ethernet port channels, the cost is adjusted based on
the number of operational member ports.

VE Port Configuration Limits

Interface Type

Platform

Cisco Nexus 6000 Series
Switch

10G Fabric Extenders

VE Port bound to an Ethernet Interface

16 VE Ports

Not supported

VE Port bound to an Ethernet Port-Channel Interface

4 VE Ports

Not supported

VNP Ports

Connectivity from an FCoE NPV bridge to the FCF is only supported over point-to-point links. These links can be individual Ethernet interfaces or members of an Ethernet port channel interface. For each FCF connected Ethernet interfaces, a vFC interface must be created and bound to the Ethernet interface. These vFC interfaces must be configured as VNP ports. On the VNP port, an FCoE NPV bridge emulates an FCoE-capable host with multiple enodes, each with a unique enode MAC address. A VNP port interface binding to MAC address is not supported. By default, the VNP port is enabled in trunk mode. Multiple VSANs can be configured on the VNP port. The FCoE VLANs that correspond to the VNP port VSANs must be configured on the bound Ethernet interface.

The spanning-tree protocol (STP) is automatically disabled in the FCoE VLAN on the interfaces that the VNP port are bound to.

Interface Modes

Each physical Fibre Channel interface in a switch may operate in one of several port modes: E mode, TE mode, F mode, TF mode, TNP mode, and SD mode. A physical Fibre Channel interface can be configured as an E port, an F port, or an SD port. Interfaces may also be configured in Auto mode; the port type is determined during interface initialization.

Each interface has an associated administrative configuration and an operational status:

The administrative configuration does not change unless you modify it. This configuration has various attributes that you can configure in administrative mode.

The operational status represents the current status of a specified attribute such as the interface speed. This status cannot be changed and is read-only. Some values may not be valid when the interface is down (for example, the operational speed).

E Port

In expansion port (E port) mode, an interface functions as a fabric expansion port. This port may be connected to another E port to create an Inter-Switch Link (ISL) between two switches. E ports carry frames between switches for configuration and fabric management. They serve as a conduit between switches for frames destined to remote N ports. E ports support class 3 and class F service.

An E port connected to another switch may also be configured to form a SAN port channel.

F Port

In fabric port (F port) mode, an interface functions as a fabric port. This port may be connected to a peripheral device (host or disk) operating as a node port (N port). An F port can be attached to only one N port. F ports support class 3 service.

NP Port

When the switch is operating in NPV mode, the interfaces that connect the switch to the core network switch are configured as NP ports. NP ports operate like N ports that function as proxies for multiple physical N ports.

TE Port

In trunking E port (TE port) mode, an interface functions as a trunking expansion port. It may be connected to another TE port to create an extended ISL (EISL) between two switches. TE ports connect to another
Cisco Nexus device or a Cisco MDS 9000 Family switch. They expand the functionality of E ports to support the following:

VSAN trunking

Fibre Channel trace (fctrace) feature

In TE port mode, all frames are transmitted in EISL frame format, which contains VSAN information. Interconnected switches use the VSAN ID to multiplex traffic from one or more VSANs across the same physical link. This feature is referred to as VSAN trunking in the
Cisco Nexus device. TE ports support class 3 and class F service.

TF Port

When the switch is operating in NPV mode, the interfaces that connect the switch to the core network switch are configured as NP ports. NP ports operate like N ports that function as proxies for multiple physical N ports.

In trunking F port (TF port) mode, an interface functions as a
trunking expansion port. It may be connected to another trunked N
port (TN port) or trunked NP port (TNP port) to create a link
between a core switch and an NPV switch or an HBA to carry tagged
frames. TF ports expand the functionality of F ports to support VSAN
trunking.

In TF port mode, all frames are transmitted in an EISL frame
format, which contains VSAN information. Interconnected switches
use the VSAN ID to multiplex traffic from one or more VSANs across
the same physical link. This feature is referred to as VSAN trunking in Cisco Nexus devices. TF ports support class 3 and class F service.

TNP Port

In trunking NP port (TNP port) mode, an interface functions as a trunking expansion port. A TNP Port may be connected to a trunked F port (TF port) to create a link to a core NPIV switch from an NPV switch.

SD Port

In SPAN destination port (SD port) mode, an interface functions as a switched port analyzer (SPAN). The SPAN feature monitors network traffic that passes though a Fibre Channel interface. This monitoring is done using a standard Fibre Channel analyzer (or a similar switch probe) that is attached to an SD port. SD ports do not receive frames, instead they transmit a copy of the source traffic. The
SPAN feature is nonintrusive and does not affect switching of network traffic for any SPAN source ports.

Auto Mode

Interfaces configured in auto mode can operate in one of the following modes: E, F, NP, TE, TF, and TNP port. The port mode is determined during interface initialization. For example, if the interface is connected to a node (host or disk), it operates in F port mode. If the interface is attached to a third-party switch, it operates in E port mode. If the interface is attached to another switch in the
Cisco Nexus device or Cisco MDS 9000 Family, it may become operational in TE port mode.

SD ports are not determined during initialization and are administratively configured.

Interface States

Administrative States

The administrative state refers to the administrative configuration of the interface.
The table below describes the administrative states.

Table 1 Administrative States

Administrative State

Description

Up

Interface is enabled.

Down

Interface is disabled. If you administratively disable an interface by shutting down that interface, the physical link layer state change is ignored.

Operational States

The operational state indicates the current operational state of the interface.
The table below describes the operational states.

Table 2 Operational States

Operational State

Description

Up

Interface is transmitting or receiving traffic as desired. To be in this state, an interface must be administratively up, the interface link layer state must be up, and the interface initialization must be completed.

Down

Interface cannot transmit or receive (data) traffic.

Trunking

Interface is operational in TE or TF mode.

Reason Codes

Reason codes are dependent on the operational state of the interface.
The following table describes the reason codes for operational states.

Table 3
Reason Codes for Interface States

Administrative Configuration

Operational Status

Reason Code

Up

Up

None.

Down

Down

Administratively down. If you administratively configure an interface as down, you disable the interface. No traffic is received or transmitted.

Up

Down

See
the table below.

If the administrative state is up and the operational state is down, the reason code differs based on the nonoperational reason code.
The table below
describes the reason codes for nonoperational states.

Note

Only some of the reason codes are listed in
the table.

Table 4 Reason Codes for Nonoperational States

Reason Code (long version)

Description

Applicable Modes

Link failure or not connected

The physical layer link is not operational.

All

SFP not present

The small form-factor pluggable (SFP) hardware is not plugged in.

All

Initializing

The physical layer link is operational and the protocol initialization is in progress.

All

Reconfigure fabric in progress

The fabric is currently being reconfigured.

Offline

The switch software waits for the specified R_A_TOV time before retrying initialization.

Inactive

The interface VSAN is deleted or is in a suspended state.

To make the interface operational, assign that port to a configured and active VSAN.

Hardware failure

A hardware failure is detected.

Error disabled

Error conditions require administrative attention. Interfaces may be error-disabled for various reasons. For example:

Configuration failure.

Incompatible buffer-to-buffer credit configuration.

To make the interface operational, you must first fix the error conditions causing this state and then administratively shut down or enable the interface.

Isolation because limit of active port channels is exceeded.

The interface is isolated because the switch is already configured with the maximum number of active SAN port channels.

Isolation due to ELP failure

The port negotiation failed.

Only E ports and TE ports

Isolation due to ESC failure

The port negotiation failed.

Isolation due to domain overlap

The Fibre Channel domains (fcdomain) overlap.

Isolation due to domain ID assignment failure

The assigned domain ID is not valid.

Isolation due to the other side of the link E port isolated

The E port at the other end of the link is isolated.

Isolation due to invalid fabric reconfiguration

The port is isolated due to fabric reconfiguration.

Isolation due to domain manager disabled

The fcdomain feature is disabled.

Isolation due to zone merge failure

The zone merge operation failed.

Isolation due to VSAN mismatch

The VSANs at both ends of an ISL are different.

port channel administratively down

The interfaces belonging to the SAN port channel are down.

Only SAN port channel interfaces

Suspended due to incompatible speed

The interfaces belonging to the SAN port channel have incompatible speeds.

Suspended due to incompatible mode

The interfaces belonging to the SAN port channel have incompatible modes.

Suspended due to incompatible remote switch WWN

An improper connection is detected. All interfaces in a SAN port channel must be connected to the same pair of switches.

Bound physical interface down

The Ethernet interface bound to a virtual Fibre Channel interface is not operational.

Only virtual Fibre Channel interfaces

STP not forwarding in FCoE mapped VLAN

The Ethernet interface bound to a virtual Fibre Channel interface is not in an STP forwarding state for the VLAN associated with the virtual Fibre Channel interface

Only virtual Fibre Channel interfaces

Buffer-to-Buffer
Credits

Buffer-to-buffer
credits (BB_credits) are a flow-control mechanism to ensure that Fibre Channel
interfaces do not drop frames. BB_credits are negotiated on a per-hop basis.

In Cisco Nexus
devices, the BB_credit mechanism is used on Fibre Channel interfaces but not on
virtual Fibre Channel interfaces. The receive BB_credit determines the receive
buffering capability on the receive side without having to acknowledge the
peer. This is important for links with large bandwidth-delays (long links with
large latency) to be able to sustain line-rate traffic with increased latency.

For virtual Fibre
Channel interfaces, BB_credits are not used. Virtual Fibre Channel interfaces
provide flow control based on capabilities of the underlying physical Ethernet
interface.

Note

The
receive BB_credit value is always 15 for all port modes (F, NP, and E) on this
platform. This value cannot be changed.

You cannot configure the port speed of a virtual Fibre Channel interface.

Note

If this is a
10G breakout port, the
slot/port syntax
is
slot/QSFP-module/port.

Step 3

switch(config-if)# switchport speed 1000

Configures the port speed of the interface to 1000 Mbps.

The number indicates the speed in megabits per second (Mbps). You can set the speed to 1000 (for 1-Gbps interfaces), 2000 (for 2-Gbps interfaces), 4000 (for 4-Gbps interfaces), or auto (default).

Step 4

switch(config-if)# no switchport speed

Reverts to the factory default (auto) administrative speed of the interface.

Autosensing

Autosensing speed is enabled on all 4-Gbps interfaces by default. This configuration enables the interfaces to operate at speeds of 1 Gbps, 2 Gbps, or 4 Gbps on the 4-Gbps ports. When autosensing is enabled for an interface operating in dedicated rate mode, 4-Gbps of bandwidth is reserved, even if the
port negotiates at an operating speed of 1-Gbps or 2-Gbps.

Configuring SD Port Frame Encapsulation

The switchport encap eisl command only applies to SD port interfaces. This command determines the frame format for all frames transmitted by the interface in SD port mode. If the encapsulation is set to EISL, all outgoing frames are transmitted in the EISL frame format, for all SPAN sources.

The switchport encap eisl command is disabled by default. If you enable encapsulation, all outgoing frames are encapsulated, and you will see a new line (Encapsulation is eisl) in the show interface SD_port_interface
command output.

Configuring Receive Data Field Size

You can configure the receive data field size for native Fibre Channel interfaces (but not for virtual Fibre Channel interfaces). If the default data field size is 2112 bytes, the frame length will be 2148 bytes.

If this is a
10G breakout port, the
slot/port syntax
is
slot/QSFP-module/port.

Step 3

switch(config-if)#
switchport ignore
bit-errors

Prevents the detection of bit error threshold
events from disabling the interface.

Step 4

switch(config-if)#
no switchport ignore
bit-errors

Prevents the detection of bit error threshold
events from enabling the interface.

Configuring Global Attributes for Fibre Channel Interfaces

Configuring Switch Port Attribute Default Values

You can configure attribute default values for various switch port attributes. These attributes will be applied globally to all future switch port configurations, even if you do not individually specify them at that time.

To configure switch port attributes, perform this task:

Procedure

Command or Action

Purpose

Step 1

switch# configuration terminal

Enters configuration mode.

Step 2

switch(config)# no system default switchport shutdown san

Configures the default setting for administrative state of an interface as Up. (The factory default setting is Down).

Tip

This command is applicable only to interfaces for which no user configuration exists for the administrative state.

Step 3

switch(config)# system default switchport shutdown san

Configures the default setting for administrative state of an interface as Down. This is the factory default setting.

Tip

This command is applicable only to interfaces for which no user configuration exists for the administrative state.

Step 4

switch(config)# system default switchport trunk mode auto

Configures the default setting for administrative trunk mode state of an interface as Auto.

Note

The default setting is trunk mode on.

Information About N Port Identifier Virtualization

N port identifier virtualization (NPIV) provides a means to assign multiple FC IDs to a single N port. This feature allows multiple applications on the N port to use different identifiers and allows access control, zoning, and port security to be implemented at the application level.
The following figure shows an example application using NPIV.

Figure 1.
NPIV Example

Enabling N Port Identifier Virtualization

You can enable or disable NPIV on the switch.

Before You Begin

You must globally enable NPIV for all VSANs on the switch to allow the NPIV-enabled applications to use multiple N port identifiers.

Note

All of the N port identifiers are allocated in the same VSAN.

Procedure

Command or Action

Purpose

Step 1

configure terminal

Example:

switch# configure terminal
switch(config)#

Enters configuration mode.

Step 2

feature npiv

Example:

switch(config)# feature npiv

Enables NPIV for all VSANs on the switch.

Step 3

no feature npiv

Example:

switch(config)# no feature npiv

Disables (default) NPIV on the switch.

Example Port Channel Configurations

This section shows examples on how to configure an F port channel in shared mode
and how to bring up the link
between F ports on the NPIV core switches and NP ports on the NPV
switches. Before you configure the F port channel, ensure that F port trunking, F port channeling, and NPIV are enabled.

This example shows how to create the port channel:

switch(config)# interface port-channel 2

switch(config-if)# switchport mode F

switch(config-if)# switchport dedicated

switch(config-if)# channel mode active

switch(config-if)# exit

This example shows how to configure the port channel member interfaces on the core switch in dedicated mode:

switch(config)# interface vfc14-16

switch(config-if)# shut

switch(config-if)# switchport mode F

switch(config-if)# switchport speed 4000

switch(config-if)# switchport rate-mode dedicated

switch(config-if)# switchport trunk mode on

switch(config-if)# channel-group 2

switch(config-if)# no shut

switch(config-if)# exit

This example shows how to create the port channel in dedicated mode on the NPV switch:

switch(config)# interface san-port-channel 2

switch(config-if)# switchport mode NP

switch(config-if)# no shut

switch(config-if)# exit

This example shows how to configure the port channel member interfaces on the NPV switch:

switch(config)# interface vfc21-22

switch(config-if)# shut

switch(config-if)# switchport mode NP

switch(config-if)# switchport trunk mode on

switch(config-if)# channel-group 2

switch(config-if)# no shut

switch(config-if)# exit

Verifying Fibre Channel Interfaces

Verifying SFP Transmitter Types

The SPF transmitter type can be displayed for a physical Fibre Channel interface (but not for a virtual Fibre Channel).

The small form-factor pluggable (SFP) hardware transmitters are identified by their acronyms when displayed in the show interface brief command. If the related SFP has a Cisco-assigned extended ID, then the show interface and show interface brief commands display the ID instead of the transmitter type. The show interface transceiver command and the show interface fcslot/porttransceiver command display both values for Cisco supported SFPs.

Verifying Interface Information

The show interface command displays interface configurations. If no arguments are provided, this command displays the information for all the configured interfaces in the switch.

You can also specify arguments (a range of interfaces or multiple, specified interfaces) to display interface information. You can specify a range of interfaces by entering a command with the following example format: interface fc2/1 - 4 , fc3/2 - 3

The following example shows how to display all interfaces:

switch# show interface
fc3/1 is up
...
fc3/3 is up
...
Ethernet1/3 is up
...
mgmt0 is up
...
vethernet1/1 is up
...
vfc 1 is up

The following example shows how to display multiple specified interfaces:

The following example shows how to display transceiver information for a specific interface:

switch# show interface fc3/1 transceiver

Note

The show interface transceiver command is only valid if the SFP is present.

The show running-configuration command displays the entire running configuration with information for all interfaces. The interfaces have multiple entries in the configuration files to ensure that the interface configuration commands execute in the correct order when the switch reloads. If you display the running configuration for a specific interface, all the configuration commands for that interface are grouped together.

The following example shows the interface display when showing the running configuration for all interfaces: