This document describes the caveats and limitations for switches in the Cisco MDS 9000 Family. Use this document in conjunction with documents listed in the "Related Documentation" section.

Note As of Cisco Fabric Manager Release 4.2(1a), Fabric Manager information will no longer appear in the Cisco MDS 9000 Family Release Notes for NX-OS releases. Cisco Fabric Manager Release Notes will include information that is exclusive to Fabric Manager as a management tool for Cisco MDS 9000 Family switches and Cisco Nexus 5000 Series switches. Refer to the following website for Release Notes for Cisco Fabric Manager:http://www.cisco.com/en/US/products/ps10495/prod_release_notes_list.html

Introduction

The Cisco MDS 9000 Family of Multilayer Directors and Fabric Switches provides industry-leading availability, scalability, security, and management, allowing you to deploy high performance storage-area networks with lowest total cost of ownership. Layering a rich set of intelligent features onto a high performance, protocol agnostic switch fabric, the Cisco MDS 9000 Family addresses the stringent requirements of large data center storage environments: uncompromising high availability, security, scalability, ease of management, and seamless integration of new technologies.

2Cisco SAN-OS Release 3.2(1) and later support the 18/4-Port Multiservice Module (MSM-18/4).

Migrating from Supervisor-1 Modules to Supervisor-2 Modules

As of Cisco MDS SAN-OS Release 3.0(1), the Cisco MDS 9509 and 9506 Directors support both Supervisor-1 and Supervisor-2 modules. Supervisor-1 and Supervisor-2 modules cannot be installed in the same switch, except during migration. Both the active and standby supervisor modules must be of the same type, either Supervisor-1 or Supervisor-2 modules. For Cisco MDS 9513 Directors, both supervisor modules must be Supervisor-2 modules.

Caution Migrating your supervisor modules is a disruptive operation.

Note Migrating from Supervisor-2 modules to Supervisor-1 modules is not supported.

To migrate from a Supervisor-1 module to a Supervisor-2 module, refer to the step-by-step instructions in the Cisco MDS 9000 NX-OS Release 4.1(x) and SAN-OS 3(x) Software Upgrade and Downgrade Guide.

Software Download Process

Use the software download procedure to upgrade to a later version, or downgrade to an earlier version, of an operating system. This section describes the software download process for the Cisco MDS NX-OS software and includes the following topics:

Determining the Software Version

To determine the version of Cisco MDS NX-OS or SAN-OS software currently running on a Cisco MDS 9000 Family switch using the CLI, log in to the switch and enter the show version EXEC command.

To determine the version of Cisco MDS NX-OS or SAN-OS software currently running on a Cisco MDS 9000 Family switch using the Fabric Manager, view the Switches tab in the Information pane, locate the switch using the IP address, logical name, or WWN, and check its version in the Release column.

Determining Software Version Compatibility

Table 7 lists the software versions that are compatible in a mixed SAN environment, the minimum software versions that are supported, and the versions that have been tested. We recommend that you use the latest software release supported by your vendor for all Cisco MDS 9000 Family products.

Table 7 Software Release Compatibility

NX-OS Software

Minimum NX-OS or SAN-OS Release

Tested NX-OS and SAN-OS Releases

NX-OS Release 5.0(4)

SAN-OS Release 3.3(1c) or later

SAN-OS Releases 3.3(1c), 3.3(5)

NX-OS Release 4.1(1b) or later

NX-OS Releases 4.1(1b), 4.2(7a)

Fabric Manager Software

Minimum NX-OS or SAN-OS Release

Tested NX-OS and SAN-OS Releases

Fabric Manager Release 5.0(4)

SAN-OS Release 3.3(1c) or later

SAN-OS Release 3.3(1c), 3.3(5)

NX-OS Release 4.1(1b) or later

NX-OS Release 4.1(1b), 4.2(7a)

NX-OS Release 5.0(4) or later

NX-OS Release 5.0(4)

Downloading Software

The Cisco MDS NX-OS software is designed for mission-critical high availability environments. To realize the benefits of nondisruptive upgrades on the Cisco MDS 9500 Directors, we highly recommend that you install dual supervisor modules.

To download the latest Cisco MDS NX-OS software, access the Software Center at this URL:

See the following sections in this release note for details on how you can nondisruptively upgrade your Cisco MDS 9000 switch. Issuing the install all command from the CLI, or using Fabric Manager to perform the downgrade, enables the compatibility check. The check indicates if the upgrade can happen nondisruptively or disruptively depending on the current configuration of your switch and the reason.

Compatibility check is done:

Module bootable Impact Install-type Reason

------ -------- -------------- ------------ ------

1 yes non-disruptive rolling

2 yes disruptive rolling Hitless upgrade is not supported

3 yes disruptive rolling Hitless upgrade is not supported

4 yes non-disruptive rolling

5 yes non-disruptive reset

6 yes non-disruptive reset

At a minimum, you need to disable the default device alias distribution feature using the no device-alias distribute command in global configuration mode. The show incompatibility system bootflash:system image filename command determines which additional features need to be disabled.

Note If you would like to request a copy of the source code under the terms of either GPL or LGPL, please send an e-mail to mds-software-disclosure@cisco.com.

Selecting the Correct Software Image for an MDS 9100 Series Switch

The system and kickstart image that you use for an MDS 9100 series switch depends on which switch you use, as shown in Table 8.

Selecting the Correct Software Image for an MDS 9200 Series Switch

The system and kickstart image that you use for an MDS 9200 series switch depends on which switch you use, as shown in Table 9.

Table 9 Software Images for MDS 9200 Series Switches

Cisco MDS 9200 Series Switch Type

Naming Convention

MDS 9222i

Filename begins with m9200-s2ek9

Selecting the Correct Software Image for an MDS 9500 Series Switch

The system and kickstart image that you use for an MDS 9500 Series are for switches with a Supervisor-2 module, as shown in Table 10. Cisco NX-OS Release 5.x and Release 4.x do not support the Supervisor-1 module.

Table 10 Software Images for Supervisor Type

Cisco MDS 9500 Series Switch Type

Supervisor Module Type

Naming Convention

MDS 9513, 9509, and 9506

Supervisor-2 moduleSupervisor-2A module

Filename begins with m9500-sf2ek9

Use the show module command to display the type of supervisor module in the switch. The following is sample output from the show module command on a Supervisor 2 module:

General Upgrading Guidelines

Note To upgrade to NX-OS Release 5.0(4) from SAN-OS Release 3.2(3a) or earlier, first upgrade to SAN-OS Release 3.3(x), then upgrade to NX-OS Release 4.1(x) or 4.2(x), and then upgrade to NX-OS Release 5.0(4).

Use the following guidelines when upgrading to Cisco MDS NX-OS Release 5.0(4):

•Install and configure dual supervisor modules.

•Issue the show install all impactupgrade-image CLI command to determine if your upgrade will be nondisruptive.

•Be aware that some features impact whether an upgrade is disruptive or nondisruptive:

–Fibre Channel Ports:Fibre Channel portscan be nondisruptively upgraded without affecting traffic on the ports. See Table 11 for the nondisruptive upgrade path for all NX-OS and SAN-OS releases.

–SSM: Intelligent services traffic on the SSM, such as SANTap, NASB, and FC write acceleration, is disrupted during an upgrade. SSM Fibre Channel traffic is not.

–Gigabit Ethernet Ports:Traffic on Gigabit Ethernet ports isdisrupted during an upgrade or downgrade. This includes IPS modules and the Gigabit Ethernet ports on the MPS-14/2 module, the MSM-18/4 module, and the MDS 9222i switch. Those nodes that are members of VSANs traversing an FCIP ISL are impacted, and a fabric reconfiguration occurs. iSCSI initiators connected to the Gigabit Ethernet ports lose connectivity to iSCSI targets while the upgrade is in progress.

–Inter-VSAN Routing (IVR): With IVR enabled, you must follow additional steps if you are upgrading from Cisco SAN-OS Release 2.1.(1a), 2.1(1b), or 2.1.(2a). See the "Upgrading with IVR Enabled" section for these instructions.

Note In addition to these guidelines, you may want to review the information in the "Limitations and Restrictions" section prior to a software upgrade to determine if a feature may possibly behave differently following the upgrade.

Use Table 11 to determine your nondisruptive upgrade path to Cisco MDS NX-OS Release 5.0(4), find the image release number you are currently using in the Current column of the table, and use the path recommended.

1See Table 14 before you begin a software upgrade on a Cisco MDS 9509 or MDS 9506 switch.

FICON Supported Releases and Upgrade Paths

Cisco MDS NX-OS Release 5.0(4) does not support FICON

Table 12 lists the SAN-OS and NX-OS releases that support FICON. Refer to the specific release notes for FICON upgrade path information.

Table 12 FICON Supported Releases

FICON Supported Releases

NX-OS

Release 4.2(1b)

Release 4.1(1c)

SAN-OS

Release 3.3(1c)

Release 3.2(2c)

Release 3.0(3b)

Release 3.0(3)

Release 3.0(2)

Release 2.0(2b)

Use Table 13 to determine your FICON nondisruptive upgrade path to Cisco MDS NX-OS Release 5.0 Find the image release number you are currently using in the Current Release with FICON Enabled column of the table and follow the recommended path.

First upgrade to SAN-OS Release 3.3(1c), and then upgrade to NX-OS Release 4.2(1b).

SAN-OS 3.0(3b)

SAN-OS 3.0(3)

SAN-OS 3.0(2)

SAN-OS 2.0(2b)

Use the interface shutdown command to administratively shut any Fibre Channel ports on Generation 1 modules that are in an operationally down state before nondisruptively upgrading from SAN-OS Release 2.0(2b) to SAN-OS Release 3.0(2) or SAN-OS Release 3.0(3b), and then upgrade to Release 3.3(1c). An operationally down state includes Link failure or not-connected, SFP not present, or Error Disabled status in the output of a show interface command. When an interface is administratively shut it will then show as Administratively down. Interfaces that are currently up or trunking do not need to be shut down.

SAN-OS 1.x

Upgrade to SAN-OS Release 3.0(2). Use the interface shutdown command to shut all the ports operationally down and administratively up on all the Generation 1 modules before nondisruptively upgrading to Release 2.0(2b) and then upgrade to 1.3(4a).

Upgrading Effect on VSAN 4079

If you upgrade to NX-OS Release 5.0(4), and you have not created VSAN 4079, the NX-OS software will automatically create VSAN 4079 and reserve it for EVFP use.

If VSAN 4079 is reserved for EVFP use, the switchport trunk allowed vsan command will filter out VSAN 4079 from the allowed list. In the following example, the allowed list appears on a separate line following the command:

switch(config-if)# switchport trunk allowed vsan 1-4080

1-4078,4080

switch(config-if)#

If you have created VSAN 4079, the upgrade to NX-OS Release 5.0(4) will have no affect on VSAN 4079.

If you downgrade to a release of NX-OS lower than NX-OS Release 4.1(x) after NX-OS Release 5.0(4) creates VSAN 4079 and reserves it for EVFP use, the VSAN will no longer be reserved.

Upgrading with IVR Enabled

An Inter-Switch Link (ISL) flap resulting in fabric segmentation or a merge during or after an upgrade from Cisco MDS SAN-OS Release 2.0(x) to a later image where IVR is enabled might be disruptive. Some possible scenarios include the following:

•FCIP connection flapping during the upgrade process resulting in fabric segmentation or merge.

•ISL flap results in fabric segmentation or merge because of hardware issues or a software bug.

•ISL port becomes part of PCP results in fabric segmentation or merge because of a port flap.

If this problem occurs, syslogs indicate a failure and the flapped ISL could remain in a down state because of a domain overlap.

This issue was resolved in Cisco SAN-OS Release 2.1(2b); you must upgrade to Release 2.1(2b) before upgrading to Release 3.3(1c). An upgrade from Cisco SAN-OS Releases 2.1(1a), 2.1(1b), or 2.1(2a) to Release 2.1(2b) when IVR is enabled requires that you follow the procedure below. If you have VSANs in interop mode 2 or 3, you must issue an IVR refresh for those VSANs.

To upgrade from Cisco SAN-OS Releases 2.1(1a), 2.1(1b), or 2.1(2a) to Release 2.1(2b) for all other VSANs with IVR enabled, follow these steps:

Step 1 Configure static domains for all switches in all VSANs where IVR is enabled. Configure the static domain the same as the running domain so that there is no change in domain IDs. Make sure that all domains are unique across all of the IVR VSANs. We recommend this step as a best practice for IVR-non-NAT mode. Issue the fcdomain domain id static vsan vsan idcommand to configure the static domains.

Note Complete Step 1 for all switches before moving to Step 2.

Step 2 Issue the no ivr virtual-fcdomain-add vsan-rangesvsan-rangecommand to disable RDI mode on all IVR enabled switches. The range of values for a VSAN ID is 1 to 4093. This can cause traffic disruption.

Step 4 Issue the following commands for the isolated switches in Step 3:

switch(config)# vsan database

switch(config-vsan-db)# vsanvsan-idsuspend

switch(config-vsan-db)# no vsanvsan-idsuspend

Step 5 Issue the ivr refresh command to perform an IVR refresh on all the IVR enabled switches.

Step 6 Issue the copy running-config startup-config command to save the RDI mode in the startup configuration on all of the switches.

Step 7 Follow the normal upgrade guidelines for Release 2.1(2b). If you are adding new switches running Cisco MDS SAN-OS Release 2.1(2b) or later, upgrade all of your existing switches to Cisco SAN-OS Release 2.1(2b) as described in this workaround. Then follow the normal upgrade guidelines for Release 3.3(1c).

Note RDI mode should not be disabled for VSANs running in interop mode 2 or interop mode 3.

Upgrading a Cisco MDS 9124 or Cisco MDS 9134 Switch

If you are upgrading from Cisco MDS SAN-OS Release 3.1(1) to Cisco NX-OS Release 4.2(1b) before upgrading to NX-OS Release 5.0(4) on a Cisco MDS 9124 or MDS 9134 Switch, follow these guidelines:

•During the upgrade, configuration is not allowed and the fabric is expected to be stable.

•The Fabric Shortest Path First (FSPF) timers must be configured to the default value of 20 seconds; otherwise, the nondisruptive upgrade is blocked to ensure that the maximum down time for the control plane can be 80 seconds.

•If there are any CFS commits in the fabric, the nondisruptive upgrade will fail.

•If there is a zone server merge in progress in the fabric, the nondisruptive upgrade will fail.

•If a service terminates the nondisruptive upgrade, the show install all failure-reason command can display the reason that the nondisruptive upgrade cannot proceed.

•If there is not enough memory in the system to load the new images, the upgrade will be made disruptive due to insufficient resources and the user will be notified in the compatibility table.

Performing a Disruptive Upgrade on an MDS 9000 Family Switch

If you do not follow the upgrade path when performing a disruptive upgrade on an MDS 9000 Family switch, (for example, you upgrade directly from SAN-OS Release 2.1(2) or earlier version to NX-OS Release 4.2(x)), the binary startup configuration is deleted because it is not compatible with the new image, and the ASCII startup configuration file is applied when the switch comes up with the new upgraded image. When the ASCII startup configuration file is applied, there may be errors. Because of this, we recommend that you follow the nondisruptive upgrade path.

Note You cannot upgrade the software image on an MDS 9120 switch, an MDS 9140 switch, or an MDS 9216i switch to Cisco NX-OS Release 5.x. See Table 3 for the list of switches that support Cisco NX-OS Release 5.0(4).

Resetting SNMP Notifications

Following a software upgrade or downgrade, SNMP notifications will reset as follows:

•When you upgrade from SAN-OS Release 3.x to NX-OS Release 4.1(x), and then to NX-OS Release 5.x, SNMP notifications will reset to their default settings.

•When you upgrade from SAN-OS Release 3.x to NX-OS Release 4.2(x), and then to NX-OS Release 5.x, SNMP notifications will not reset to their default settings.

•When you downgrade from any NX-OS Release 4.2(x), SNMP notifications will reset to their default configuration settings.

Converting Automatically Created PortChannels Before an Upgrade

Before upgrading from NX-OS Release 4.1(x) or 4.2(x) to Release 5.x, ensure that you do not have any automatically created PortChannels present in the switch configuration. Use the port-channel persistent command to convert an automatically created PortChannel to a persistent PortChannel. Failure to convert automatically created PortChannels prior to the upgrade can result in traffic disruption because Autocreation of PortChannels is a deprecated feature as of NX-OS Release 4.1(1b).

Downgrading Your Cisco MDS SAN-OS Software Image

This section lists the guidelines recommended for downgrading your Cisco MDS SAN-OS software image and includes the following topics:

General Downgrading Guidelines

Use the following guidelines to nondisruptivelydowngrade your Cisco MDS NX-OS Release 5.0(4):

•Install and configure dual supervisor modules.

•Issue the system no acl-adjacency-sharing execute command to disable ACL adjacency usage on Generation 2 and Generation 1 modules. If this command fails, reduce the number of zones, IVR zones, TE ports, or a combination of these in the system and issue the command again.

•Disable all features not supported by the downgrade release. Use the show incompatibility systemdowngrade-image command to determine what you need to disable.

•Use the show install all impactdowngrade-image command to determine if your downgrade will be nondisruptive.

•Be aware that some features impact whether a downgrade is disruptive or nondisruptive:

–Fibre Channel Ports:Fibre Channel portscan be nondisruptively downgraded without affecting traffic on the ports. See Table 15 for the nondisruptive downgrade path for all SAN-OS releases.

–SSM: Intelligent services traffic on the SSM, such as SANTap, NASB, and FC write acceleration, is disrupted during a downgrade. SSM Fibre Channel traffic is not.

–Gigabit Ethernet Ports: Traffic on Gigabit Ethernet ports isdisrupted during a downgrade. This includes IPS modules and the Gigabit Ethernet ports on the MPS-14/2 module, the MSM-18/4 module, and the MDS 9222i switch. Those nodes that are members of VSANs traversing an FCIP ISL are impacted, and a fabric reconfiguration occurs. iSCSI initiators connected to the Gigabit Ethernet ports lose connectivity to iSCSI targets while the downgrade is in progress.

–IVR: With IVR enabled, you must follow additional steps if you are downgrading from Cisco SAN-OS Release 2.1.(1a), 2.1(1b), or 2.1.(2a). See the "Upgrading with IVR Enabled" section for these instructions.

Note A downgrade from NX-OS Release 4.2(1b) to SAN-OS Release 3.3(1x) is not supported on MDS switches, when FC-Redirect based applications, such as Data Mobility Manager or Storage Media Encryption, are configured in the fabric if either of the following conditions are satisfied:

1. A target for which FC-Redirect is configured is connected locally and there are Generation 1 modules with ISLs configured in the switch.

2. A host, for which FC-redirect is configured, is connected locally on a Generation 1 module.

If these conditions exist, remove the application configuration for these targets and hosts before proceeding with the downgrade.

See the compatibility information in Table 14 to determine if a downgrade to, or upgrade from Release 5.0(x) software is disruptive or nondisruptive on a Cisco MDS 9509 or 9506 switch.

Table 14 NX-OS Release 5.0(x) Downgrade and Upgrade Matrix

Downgrade Path on a Cisco MDS 9509 or 9506 Switch

Current Release

Desired Release

Expected Behavior

Observed Behavior

5.0(7) or 5.0(8)

5.0(1a), 5.0(1b), or 5.0(4)

Disruptive

NondisruptiveThe user is expected to explicitly reload the switch using the reload command.

5.0(7) or 5.0(8)

4.2(x) or 4.1(x)

Disruptive

Disruptive

5.0(4)

5.0(1a)

Nondisruptive

Nondisruptive

5.0(1a), 5.0(1b), or 5.0(4)

4.2(x) or 4.1(x)

Nondisruptive

Nondisruptive

Upgrade Path on a Cisco MDS 9509 or 9506 Switch

Current Release

Desired Release

Expected Behavior

Observed Behavior

5.0(7) or 5.0(8)

5.2(x)

Nondisruptive Support for Generation 4 modules is available.

NondisruptiveSupport for Generation 4 modules is available.

5.0(1a), 5.0(1b), or 5.0(4)

5.2(x)

NondisruptiveSupport for Generation 4 modules is not available.

NondisruptiveSupport for Generation 4 modules is not available.

Use Table 15 to determine the nondisruptive downgrade path from Cisco NX-OS Release 5.0(4). Find the SAN-OS image you want to downgrade to in the To SAN-OS Release column of the table and use the path recommended.

1See Table 14 before you perform a software downgrade on a Cisco MDS 9509 or MDS 9506 switch.

FICON Downgrade Paths

Table 16 lists the downgrade paths for FICON releases. Find the image release number that you want to downgrade to in the To Release with FICON Enabled column of the table and follow the recommended downgrade path.

Table 16 FICON Downgrade Path from NX-OS Release 4.1(1c)

To Release with FICON Enabled

Downgrade Path

SAN-OS 3.3(1c)

You can nondisruptively downgrade directly from NX-OS Release 4.1(1c).

SAN-OS 3.2(2c)

First downgrade to SAN-OS Release 3.3(1c) and then downgrade to Release 3.2(2c).

SAN-OS 3.0(3b)

First downgrade to SAN-OS Release 3.3(1c) and then downgrade to Release 3.0(3b).

SAN-OS 3.0(2)

First downgrade to SAN-OS Release 3.3(1c) and then downgrade to Release 3.0(2).

SAN-OS 2.0(2b)

Use the interface shutdown command to administratively shut any Fibre Channel ports on Generation 1 modules that are in an operationally down state before nondisruptively downgrading from NX-OS Release 4.1 to SAN-OS Release 3.3(1c) then to SAN-OS Release 3.0(3b) or SAN-OS Release 3.0(2), and then to SAN-OS Release 2.0(2b). An operationally down state includes Link failure or not-connected, SFP not present, or Error Disabled status in the output of a show interface command. When an interface is administratively shut it will then show as Administratively down. Interfaces that are currently up or trunking do not need to be shut down.

SAN-OS 1.3(4a)

Downgrade to SAN-OS Release 3.3(1c) and then to Release 3.0(2). Use the shutdown command to shut all the ports operationally down and administratively up on all the Generation 1 modules before nondisruptively downgrading to Release 2.0(2b) and then downgrade to 1.3(4a).

New Features in Cisco MDS NX-OS Release 5.0(4)

Cisco MDS NX-OS Release 5.0(4) is a maintenance release. It includes bug fixes and the following new feature:

•Slow Drain Device Detection and Congestion Avoidance

The slow drain feature provides various enhancements to detect slow drain devices that cause congestion in the network and lead to an ISL credit shortage in the traffic destined for these devices. The credit shortage affects the unrelated flows in the fabric that use the same ISL link even though destination devices do not experience slow drain. The slow drain feature also includes a congestion avoidance function.

The slow drain feature is not supported on the following Cisco MDS 9000 Family hardware:

NX-OS Feature Descriptions

This section includes descriptions of the major new features of MDS NX-OS Release 5.0(4) and indicates where the feature is documented.

Discontinued Software Features

As of NX-OS Release 5.0(1a), support for Fabric Congestion Control (FCC) is discontinued. If you are currently using FCC in your SAN environment, you should turn it off before upgrading to NX-OS Release 5.0(1a) or later. Use the no fcc enable command to turn off the FCC feature.

Hardware Changes

Starting with Cisco NX-OS Release 5.0(4), the behavior of the management port on the active and standby supervisors has changed as follows:

•On the active supervisor, the management port is up and the status LED is green which indicates that it is up.

•On the standy supervisor, the management port is down and the status LED is off which indicates that it is down.

•Following a supervisor switchover, the management port comes up on the new active supervisor and the LED is turned on and is green.

Prior to NX-OS Release 5.0(4), the status of the standby supervisor was incorrect. The correct behavior is as described in this section.

Licensed Cisco NX-OS Software Packages

Most Cisco MDS 9000 family software features are included in the standard package. However, some features are logically grouped into add-on packages that must be licensed separately, such as the Cisco MDS 9000 Enterprise package, SAN Extension over IP package, Mainframe package, Fabric Manager Server (FMS) package, Storage Services Enabler (SSE) package, Storage Media Encryption package, and Data Mobility Manager package. On-demand ports activation licenses are also available for the Cisco MDS Blade Switch Series and 4-Gbps Cisco MDS 9100 Series Multilayer Fabric switches.

Enterprise Package

The standard software package that is bundled at no charge with the Cisco MDS 9000 Family switches includes the base set of features that Cisco believes are required by most customers for building a SAN. The Cisco MDS 9000 family also has a set of advanced features that are recommended for all enterprise SANs. These features are bundled together in the Cisco MDS 9000 Enterprise package. Refer to the Cisco MDS 9000 Enterprise package fact sheet for more information.

SAN Extension over IP Package

The Cisco MDS 9000 SAN Extension over IP package allows the customer to use FCIP to extend SANs over wide distances on IP networks using the Cisco MDS 9000 family IP storage services. Refer to the Cisco MDS 9000 SAN Extension over IP package fact sheet for more information.

Mainframe Package

The Cisco MDS 9000 Mainframe package uses the FICON protocol and allows control unit port management for in-band management from IBM S/390 and z/900 processors. FICON VSAN support is provided to help ensure true hardware-based separation of FICON and open systems. Switch cascading, fabric binding, and intermixing are also included in this package. Refer to the Cisco MDS 9000 Mainframe package fact sheet for more information.

On-Demand Port Activation License

On-demand ports allow customers to benefit from Cisco NX-OS Software features while initially purchasing only a small number of activated ports on 8-Gbps or 4-Gbps Cisco MDS 9100 Series Multilayer Fabric switches. As needed, customers can expand switch connectivity by licensing additional ports.

Storage Media Encryption Package

The Cisco MDS 9000 Storage Media Encryption package enables encryption of data at rest on heterogeneous tape devices and virtual tape libraries as a transparent fabric service. Cisco SME is completely integrated with Cisco MDS 9000 Family switches and the Cisco Fabric Manager application, enabling highly available encryption services to be deployed without rewiring or reconfiguring SANs, and allowing them to be managed easily without installing additional management software. Refer to the Cisco MDS 9000 Storage Media Encryption package fact sheet for more information. The Storage Media Encryption package is for use only with Cisco MDS 9000 Family switches.

I/O Accelerator Package

The Cisco I/O Accelerator (IOA) package activates IOA on the Cisco MDS 9222i fabric switch, the Cisco MDS 9000 18/4 Multiservice Module (MSM-18/4), and on the SSN-16 module. The IOA package is licensed per service engine and is tied to the chassis. The number of licenses required is equal to the number of service engines on which the intelligent fabric application is used.The SSN-16 requires a separate license for each engine on which you want to run IOA. Each SSN-16 engine that you configure for IOA checks out a license from the pool managed at the chassis level. SSN-16 IOA licenses are available as single licenses.

XRC Acceleration License

The Cisco Extended Remote Copy (XRC) acceleration license activates FICON XRC acceleration on the Cisco MDS 9222i switch and on the MSM-18/4 in the Cisco MDS 9500 Series directors. One license per chassis is required. You must install the Mainframe Package and the SAN Extension over FCIP Package before you install the XRC acceleration license. The Mainframe Package enables the underlying FICON support, and the FCIP license or licenses enable the underlying FCIP support. XRC acceleration is not supported on the SSN-16.

Limitations and Restrictions

This section lists the limitations and restrictions for this release. The following limitations are described:

Support for Generation One Modules

As of Cisco MDS NX-OS Release 5.0(1a), support for Generation One modules has been discontinued.

IPv6

The management port on Cisco MDS switches supports one user-configured IPv6 address, but does not support auto-configuration of an IPv6 address.

User Roles

In SAN-OS Release 3.3(x) and earlier, when a user belongs to a role which has a VSAN policy set to Deny and the role allows access to a specific set of VSANs (for example, 1 through 10), the user is restricted from performing the configuration, clear, execute, and debug commands which had a VSAN parameter outside this specified set. Beginning with NX-OS Release 4.1(1b), these users are still prevented from performing configuration, clear, execute, and debug commands as before, however, they are allowed to perform show commands for all VSANs. The ability to execute the show command addresses the following:

•In a network environment, users often need to view information in other VSANs even though they do not have permission to modify configurations in those VSANs.

•This behavior makes Cisco MDS 9000 Series switches consistent with other Cisco products, such as Cisco Nexus 7000 Series switches, that exhibit the same behavior for those roles (when they apply to the VLAN policy).

Red Hat Enterprise Linux

The Linux kernel core dump is not supported in NX-OS Release 4.1(1b) and later versions and therefore the CLI command has been removed. A syntax error message will be displayed if you import configurations from SAN-OS Release 3.3(x) and earlier to NX-OS Release 4.1(1b) and later. These syntax errors do not affect the application of other commands in the configuration and can be safely ignored. To address this, remove the kernel core configuration from the ASCII configuration file before importing the configuration.

Generation 1 Module Limitation

When a Cisco or other vendor switch port is connected to a Generation 1 module port (ISL connection), the receive buffer-to-buffer credit of the port connected to a Generation 1 module port should not exceed 255.

Schedule Job Configurations

As of MDS NX-OS Release 4.1(1b) and later, the scheduler job configurations need to be entered in a single line with a semicolon(;) as the delimiter.

Job configuration files created with SAN-OS Release 3.3(1c) and earlier, are not supported. However, you can edit the job configuration file and add the delimiter to support Cisco NX-OS Release 4.1(3a).

Maximum Number of Zones Supported in Interop Mode 4

In interop mode 4, the maximum number of zones that is supported in an active zone set is 2047, due to limitations in the connected vendor switch.

When IVR is used in interop mode 4, the maximum number of zones supported, including IVR zones, in the active zone set is 2047.

InterVSAN Routing

When using InterVSAN Routing (IVR), it is recommended to enable Cisco Fabric Services (CFS) on all IVR-enabled switches. Failure to do so may cause mismatched active zone sets if an error occurs during zone set activation.

Java Web Start

When using Java Web Start, it is recommended that you do not use an HTML cache or proxy server. You can use the Java Web Start Preferences panel to view or edit the proxy configuration. To do this, launch the Application Manager, either by clicking the desktop icon (Microsoft Windows), or type ./javaws in the Java Web Start installation directory (Solaris Operating Environment and Linux), and then select Edit> Preferences.

If you fail to change these settings, you may encounter installation issues regarding a version mismatch. If this occurs, you should clear your Java cache and retry.

VRRP Availability

The Virtual Router Redundancy Protocol (VRRP) is not available on the Gigabit Ethernet interfaces on the MSM-18/4 module or module 1 of the MDS 9222i switch, even though it is visible on these modules. The feature is not implemented in the current release.

Using a RSA Version 1 Key for SSH Following an Upgrade

For security reasons, NX-OS Release 4.2(1b) does not support RSA version 1 keys. As a result, if you upgrade to NX-OS Release 4.2(1b) from an earlier version that did support RSA version 1 keys, and you had configured a RSA version 1 key for SSH, then you will not be able to log in through SSH following the upgrade.

If you have a RSA version 1 key configured for SSH, before upgrading to NX-OS Release 4.1(3a), follow these steps:

Step 1 Disable SSH.

Step 2 Create RSA version 2 DSA keys.

Step 3 Enable SSH.

Step 4 Delete any RSA version 1 keys on any remote SSH clients and replace the version 1 keys with the new version 2 keys from the switch.

Proceed with the upgrade to NX-OS Release 4.2(1b).

If you upgrade before disabling SSH and creating RSA version 2 keys, follow these steps:

Reserved VSAN Range and Isolated VSAN Range Guidelines

On an NPV switch with a trunking configuration on any interface, or on a regular switch where the feature fport_channel_trunk command has been issued to enable the Trunking F PortChannels feature, follow these configuration guidelines for reserved VSANs and the isolated VSAN:

•If trunk mode is on for any of the interfaces or NP PortChannel is up, the reserved VSANs are 3040 to 4078, and they are not available for user configuration.

•The Exchange Virtual Fabric Protocol (EVFP) isolated VSAN is 4079, and it is not available for user configuration.

•VSAN 4079 will be impacted by an upgrade to NX-OS Release 4.1(3a), depending on whether or not VSAN 4079 was created prior to the upgrade. See the "Upgrading Effect on VSAN 4079" section for details.

The following VSAN IDs are assigned in the Fibre Channel Framing and Signaling (FC-FS) interface standard:

Applying Zone Configurations to VSAN 1

In the setup script, you can configure system default values for the default-zone to be permit or deny, and you can configure default values for the zone distribution method and for the zone mode.

These default settings are applied when a new VSAN is created. However, the settings will not take effect on VSAN 1, because it exists prior to running the setup script. Therefore, when you need those settings for VSAN 1, you must explicitly issue the following commands:

•zone default-zone permitvsan 1

•zoneset distribute fullvsan 1

•zone mode enhancedvsan 1

Running Storage Applications on the MSM-18/4

The Cisco MDS 9000 18/4-Port Multiservice Module (MSM-18/4) does not support multiple, concurrent storage applications. Only one application, such as SME or DMM, can run on the MSM-18/4 at a time.

RSPAN Traffic Not Supported on CTS Ports on 8-Gbps Switching Modules

An inter-switch link (ISL) that is enabled for Cisco TrustSec (CTS) encryption must be brought up in non-CTS mode to support remote SPAN (RSPAN) traffic on the following modules:

If the ISL link is brought up with CTS enabled, random packets drops of both RSPAN traffic and normal traffic will occur on the receiver port switch.

I/O Accelerator Feature Limitations

IOA does not support the following NX-OS features:

•F port trunking

•F port channeling

•IOA cannot be configured on flows in topologies that have devices with NPV and NPIV enabled. For example, IOA is not supported in a topology where a host logs in from a NPV edge switch and IOA is deployed on a NPV core switch for this host.

Support for FCIP Compression Modes

In Cisco NX-OS Release 4.2(1b) and later, FCIP compression mode 1 and compression mode 3 are not supported on the Cisco MSM-18/4 module and on the SSN-16 module.

Saving Copies of the Running Kickstart and System Images

After you upgrade to MDS NX-OS Release 4.2(1b), you are not allowed to delete, rename, move, or overwrite the kickstart and system images that are in the current system bootvar settings on an active or standby MDS Supervisor-2 module on any Cisco MDS 9500 Series switch. This restriction does not apply to the integrated supervisor module on the MDS 9200 and MDS 9100 series switches.

Configuring Buffer Credits on a Generation 2 or Generation 3 Module

When you configure port mode to auto or E on a Generation 2 module, one of the ports will not come up for the following configuration:

•Port Mode: auto or E for all of the ports

•Rate Mode: dedicated

•Buffer Credits: default value

When you configure port mode to auto or E on a Generation 3 module, one or two of the ports will not come up for the following configuration:

•Port Mode: auto or E for the first half of the ports, the second half of the ports, or for all of the ports

•Rate Mode: dedicated

•Buffer Credits: default value

When you configure port mode to auto or E for all ports in the global buffer pool, you need to reconfigure buffer credits on one or more of the ports. The total number of buffer credits configured for all the ports in the global buffer pool should be reduced by 64.

Features Not Supported on the Cisco MDS 9148 Switch

The Cisco MDS 9148 Multilayer Fabric Switch does not support the following NX-OS features:

•IVR

•Remote Span

•Translative loop support

•FCC - no generation, quench reaction only

•FC-Redirect

In addition, the following features have these limits:

•VSANs - 31 maximum

•SPAN - 1 session maximum

PPRC Not Supported with FCIP Write Acceleration

Configuring a Persistent FCID in an IVR Configuration with Brocade Switches

The following information is relevant if you have a fabric that consists of Cisco MDS 9000 switches and Brocade switches, and the Cisco MDS switches are running either NX-OS Release 4.x or Release 5.x and Brocade is running FOS higher than 6.x. In an IVR configuration, when IVR NAT is enabled on a Cisco MDS 9000 switch, the device in the native VSAN should be configured with a persistent FCID. Assuming the FCID is 0xAABBCC, AA should be configured with the virtual IVR domain ID of the VSAN that contains the ISLs and BB should be configured in the following range:

•1 through 64 if the Brocade switch is operating in native interop mode.

•1 through 30 if the Brocade switch is operating in McData Fabric mode or McData Open Fabric Mode.

This configuration ensures that the devices connected to the Cisco MDS 9000 switch can be seen in the name server database on the Brocade switch.

Caveats

This section lists the open and resolved caveats for this release. Use Table 17 to determine the status of a particular caveat. In the table, "O" indicates an open caveat and "R" indicates a resolved caveat.

Resolved Caveats

Symptom: A RSCN process might fail and cause a system reload. This symptom might occur when many RSCNs are transmitted due to links changing state, but no responses are received from the destinations.

Workaround: This issue is resolved.

•CSCte93754

Symptom: An IOA flow can take a few seconds to become active in certain events such as host or target port flaps. PLOGIs from the hosts are buffered until the IOA flow becomes active. Once the IOA flow becomes active, a RSCN is sent, which forces the host to perform a PLOGI again. Certain target arrays perform a few back-to-back PLOGIs prior to the flow becoming active, which may cause automatic path recovery to fail.

Workaround: This issue is resolved.

•CSCti00670

Symptom: The system reboots when IVR fails.

Workaround: This issue is resolved.

•CSCsx33891

Symptom: In rare circumstances, the SANTap process or SME process might fail on an MDS switch running NX-OS Release 4.1(x) software. If the process fails on an MDS 9222i switch, then a full switch reload might occur.

Workaround: This issue is resolved.

•CSCtb43279

Symptom: The wwn_manager process might fail after ports are flapped because the internal system manager does not received a heartbeat from the process.

Workaround: This issue if resolved.

•CSCte93219

Symptom: If you have an MDS 9000 switch running NX-OS Release 5.0(1a) and also running DMM or Secure Erase, when you downgrade to an NX-OS 4.x release, you see the following errors:

–Syslog messages display the error string PSS_VERSION_MISMATCH for the configuration of an internal service (ilc_helper)

–After the ISSD completes and the switch comes up with the NX-OS 4.x release, an internal process (ilc_helper) fails.

Workaround: This issue is resolved.

•CSCtg11776

Symptom: On a switch where AAA remote authentication was configured, scheduled jobs started failing approximately 24 to 48 hours after they were configured because the scheduler aaa-authentication command did not appear in the running configuration.

Workaround: This issue is resolved.

•CSCtg61169

Symptom: Users are no longer able to log in to an MDS 9000 switch via SSH. The following error is seen in the log file:

unable to lock password file

At the same time the /dev/root file system is 100 percent in use.

This symptom might occur if there are many HTTP requests that go to the built-in MDS web server. The log file of the service continues to grow until it consumes all available space in /dev/root.

Workaround: This issue is resolved.

•CSCtg86355

Symptom: Quoted strings are not permitted in a Call Home configuration for the following values: customer ID and site ID.

Workaround: This issue is resolved.

•CSCth20253

Symptom: The data path processor engine on the MSM-18/4 module occasionally fails after an exchange error occurs between an initiator and target in an IOA cloud. The error occurs because the tape write takes a long time to complete and IOA on the target side MDS 9000 switch tries to abort the exchange.

Workaround: This issue is resolved.

•CSCth21404

Symptom: IP DNS name resolution might be slow. For example, AAA or TACACS+ authentication might time out when the AAA or TACACS+ servers are configured by name. This symptom might be seen when the IP DNS name servers are functioning properly.

Workaround: This issue is resolved.

•CSCth71977

Symptom: LDAP remote authentication will fail if you configure a user with multiple roles and one of the roles is invalid. In such a scenario, AAA skips the invalid role and applies the identifiable roles, but SNMP applies the default network-operator role upon encountering an invalid role.

Symptom: The port group monitor feature does not generate alarms when the configured port ASIC bandwidth threshold is crossed, even when traffic is running that has crossed the threshold. This symptom might be seen on an MDS 9000 Family switch running NX-OS Release 4.2(a) software.

Workaround: This issue is resolved.

•CSCti23777

Symptom: A port on a Generation 3 module is disabled and set to hardware failure state. This symptom can occur when a port MAC has a double bit ECC error (type 87). The output of the show logging onboard exceptionlog command is as follows:

Error Description: IP_FCMAC_ERR Interrupt, port = x, src_bit = 87

Workaround: This issue is resolved.

•CSCti33087

Symptom: Under normal operating conditions, a Cisco MDS 9500 Series switch with Call Home configured might fail, which will cause a supervisor switchover

The following command sequence can cause the failure:

destination-profile new_profile

destination-profile new_profile alert-group License

destination-profile new_profile alert-group Syslog-group-port

destination-profile new_profile message-level 5

Workaround: This issue is resolved.

Open Caveats

•CSCto68011

Symptom: The fcdomain service on both supervisor modules fails, which results in a reload of the device. An error message similar to the following is displayed:

Caution Because the feature in this vulnerability uses UDP as a transport, it is possible to spoof the sender's IP address, which may defeat ACLs that permit communication to these ports from trusted IP addresses.

Although it is often difficult to block traffic that transits a network, it is possible to identify traffic that should never be allowed to target infrastructure devices and block that traffic at the border of networks. Infrastructure Access Control Lists (iACLs) are a network security best practice and should be considered as a long-term addition to good network security as well as a workaround for this specific vulnerability. The iACL example below should be included as part of the deployed infrastructure access-list which will protect all devices with IP addresses in the infrastructure IP address range:

!---

!--- Feature: SNMP

!---

!---

!--- Permit SNMP traffic from trusted sources.

!---

ip access-list 150 permit udp TRUSTED_SOURCE_ADDRESSES WILDCARD

INFRASTRUCTURE_ADDRESSES WILDCARD eq port snmp

ip access-list 150 permit tcp TRUSTED_SOURCE_ADDRESSES WILDCARD

INFRASTRUCTURE_ADDRESSES WILDCARD eq port snmp

!---

!--- Deny SNMP traffic from all other sources.

!---

ip access-list 150 deny udp any any eq port snmp

ip access-list 150 deny tcp any any eq port snmp

!---

!--- Permit/deny all other Layer 3 and Layer 4 traffic in

!--- accordance with existing security policies and

!--- configurations. Permit all other traffic to transit the

!--- device.

!---

access-list 150 permit ip any any

!--- Apply access-list to management interface

interface serial 2/0

ip access-group 150 in

For more information on IP Access Control Lists see the "Configuring IPv4 and IPv6 Access Control List" section in the Cisco MDS 9000 Family NX-OS Security Configuration Guide at the following location:

Symptom: Following an upgrade from Cisco MDS NX-OS Release 4.2(3a) to Release 5.0(1a) on an MDS 9222i switch, the Encapsulating Security Protocol (ESP) configuration is not applied to members of a PortChannel. This issue occurs only on the MDS 9222i switch.

Workaround: Upgrade to software release that has the fix for this issue.

–After performing a software upgrade to a Cisco NX-OS release that contains a fix for this issue, it may be necessary to enter the shut command followed by the noshut command on the affected host ports to regain connectivity.

–If you perform a nondisruptive upgrade or downgrade from a release that contains a fix to a release that does not contain the fix, you need to reload each module affected by this issue.

–If you have a Cisco MDS 9222i swtich that is affected by this issue, and you perform a nondisruptive upgrade or downgrade from a release that contains a fix to a release that does not contain the fix, you need to reload the switch.

•CSCsq20408

Symptom: The show startup command displays aspects of the running configuration when SANTap is configured and/or SANTap objects are created. When a user creates objects such as a CVT or DVT, the configuration is showing in the running-configuration and in the startup-configuration without copying the configuration into the startup-configuration.

Workaround: Issue a copyrunning-config startup-config command whenever you create objects such as a CVT or DVT so that the running-configuration and startup-configuration are synchronized.

•CSCtn68418

Symptom: When you try to save a configuration, you might see the following message:

switch# copy run start

[########################################] 100%

Configuration update aborted: request was aborted

%DAEMON-3-SYSTEM_MSG: ntp:can't open /mnt/pss/ntp.drift.TEMP: No space left on device

- ntpd[xxxx]

%PLATFORM-2-MEMORY_ALERT: Memory Status Alert : MINOR

%PLATFORM-2-MEMORY_ALERT: Memory Status Alert : MINOR ALERT RECOVERED

`show system internal flash` output will display /isan as 100% full.

Mount-on 1K-blocks Used Available Use% Filesystem

/ 204800 54624 150176 27 /dev/root

/proc 0 0 0 0 proc

/isan 409600 409576 24 100 none

This symptom was seen because the Call Home feature had duplicate message throttling disabled and there were flapping interfaces that generated thousands of Call Home messages. These messages filled up the ISAN directory.

Workaround: To work around this issue, enable Call Home duplicate message throttling. If you find that the /isan directory is 100 percent full, open a TAC case to get assistance with deleting the files.

Related Documentation

The documentation set for NX-OS for the Cisco MDS 9000 Family includes the following documents. To find a document online, access the following web site:

Troubleshooting and Reference

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:

Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)