Troubleshooting FC-SP, Port Security, and Fabric Binding

This chapter describes procedures used to troubleshoot Fibre Channel Security Protocol (FC-SP), port security, and fabric binding in Cisco MDS 9000 Family products. It includes the following sections:

FC-SP Overview

Port Security Overview

Fabric Binding Overview

Initial Troubleshooting Checklist

FC-SP Issues

Port Security Issues

Fabric Binding Issues

FC-SP Overview

FC-SP capabilities provide switch-switch and host-switch authentication to overcome security challenges for enterprise-wide fabrics. Diffie-Hellman Challenge Handshake Authentication Protocol (DHCHAP) is an FC-SP protocol that provides authentication between Cisco MDS 9000 Family switches and other devices. You can configure FC-SP to authenticate locally or to use a remote AAA server for authentication.

Port Security Overview

Typically, any Fibre Channel device in a SAN can attach to any SAN switch port and access SAN services based on zone membership. Port security features prevent unauthorized access to a switch port in the Cisco MDS 9000 Family:

All intrusion attempts are reported to the SAN administrator through system messages.

Fabric Binding Overview

The fabric binding feature ensures ISLs are only enabled between specified switches in the fabric binding configuration. Fabric binding is configured on a per-VSAN basis.

This feature helps prevent unauthorized switches from joining the fabric or disrupting current fabric operations. It uses the Exchange Fabric Membership Data (EFMD) protocol to ensure that the list of authorized switches is identical in all switches in the fabric.

Domain IDs are mandatory for FICON-based fabric binding and optional for non-FICON based fabric binding. For non-FICON based fabric binding, not specifying a domain ID means that the switch with the matching WWN can login with any domain ID.

Verify that you have configured MD5 for the hash algorithm if you are authenticating through a RADIUS or TACACS server. RADIUS and TACACS always use MD5 for CHAP authentication. Using SHA-1 as the hash algorithm may prevent RADIUS and TACACS usageâeven if these AAA protocols are enabled for DHCHAP authentication.

Verify that your AAA server is functioning properly.

Begin troubleshooting port security issues by checking the following issues:

Checklist

Check off

Verify that you have the ENTERPRISE_PKG license installed on all switches.

Verify that port security is activated and that the end devices are present in the port security active database.

Verify that no unauthorized devices (host or switch) are connected to a port. (One unauthorized pWWN prevents the port from being active and blocks all other devices on that port.)

Begin troubleshooting fabric binding issues by checking the following issues:

Checklist

Check off

Verify that you have the ENTERPRISE_PKG or the MAINFRAME_PKG license installed on all switches.

Verify that you have activated fabric binding.

Verify that all switches in the fabric have the same fabric binding database settings.

Common Troubleshooting Tools in Fabric Manager

Use the following Fabric Manager procedure to troubleshoot FC-SP issues:

Switches > Security > FC-SP

Use the following Fabric Manager procedure to troubleshoot port security issues:

Fabricxx > VSANxx > Port Security

Use the following Fabric Manager procedure to troubleshoot fabric binding issues:

Fabricxx > VSANxx > Fabric Binding

Common Troubleshooting Commands in the CLI

Use the following CLI commands to troubleshoot FC-SP issues:

Use the following CLI commands to troubleshoot FC-SP issues:

show fcsp interface

show fcsp internal event-history errors

show fcsp dhchap

show fcsp dhchap database

Use the following CLI commands to troubleshoot port security issues:

show port-security status

show port-security database vsan

show port-security database active vsan

show port-security violations

show port-security internal global

show port-security internal info vsan

show port-security internal state-history vsan

show port-security internal commit-history vsan

show port-security internal merge-history vsan

Use the following CLI commands to troubleshoot fabric binding issues:

show fabric-binding status

show fabric-binding database vsan

show fabric-binding database active vsan

show fabric-binding violations

show fabric-binding internal global

show fabric-binding internal info

show fabric-binding internal event-history

show fabric-binding internal efmd event-history

FC-SP Issues

This section describes troubleshooting FC-SP issues and includes the following topic:

Switch or Host Blocked from Fabric

Switch or Host Blocked from Fabric

Symptom Switch or host blocked from joining the fabric.

Table 19-1 Switch or Host Blocked From Fabric

Symptom

Possible Cause

Solution

Switch or host blocked from joining the fabric.

FC-SP not enabled on all switches.

Choose Switches > Security > FC-SP, set the command field to enable, and click Apply Changes on Fabric Manager to enable FC-SP.

Or use the fcsp enable CLI command on all switches in your fabric.

Local switch FC-SP password does not match remote password.

Choose Switches > Security > FC-SP, select the General/Password tab, and set the GenericPassword field in Fabric Manager.

Or use the fcsp dhchap password CLI command to set the local switch password.

FC-SP DHCHAP configuration does not match remote switch or host.

See the "Verifying FC-SP Configuration Using Fabric Manager" section or the "Verifying FC-SP Configuration Using the CLI" section.

Switch or host not in authentication database.

Add switch or host to the local or remote FC-SP database. See the "Verifying Local FC-SP Database Using Fabric Manager" section or the "Verifying Local FC-SP Database Using the CLI" section.

Host or switch does not support FC-SP.

Upgrade host or switch or use the auto-active or auto-passive DHCHAP mode.

Choose Switches > Interfaces > FC logical, select the FC-SP tab, set the Mode field to autoActive or autoPassive, and click Apply Changes in Fabric Manager.

Or use the fcsp auto-active or fcsp auto-passive CLI command in interface mode to set the DHCHAP mode.

Verifying FC-SP Configuration Using Fabric Manager

To verify the FC-SP configuration using Fabric Manager, follow these steps:

3. Choose Switches > Interfaces > FC Logical and select the FLOGI tab to find the pWWN for the host that you want to add to the FC-SP local database.

4. Choose Switches > Security > FC-SP, select the Local Passwords tab, and then click Create Row to add a host or switch to the local database.

5. Fill in the WWN and password fields and click Create.

Verifying Local FC-SP Database Using the CLI

To verify the local FC-SP database using the CLI, follow these steps:

1. Use the show fcsp dhchap database command to view the configured switches and hosts.

switch# show fcsp dhchap database
DHCHAP Local Password:
Non-device specific password:********
Password for device with WWN:29:11:bb:cc:dd:33:11:22 is ********
Password for device with WWN:30:11:bb:cc:dd:33:11:22 is ********
Other Devices' Passwords:
Password for device with WWN:00:11:22:33:44:aa:bb:cc is ********

2. Use the show wwn switch command on the switch that you want to add to the FC-SP local database to find the sWWN.

MDS-9216# show wwn switch
Switch WWN is 20:00:00:05:30:00:54:de

3. Use the show flogi database interface command to find the pWWN for the host that you want to add to the FC-SP local database.

Use the show fcsp asciiwwn sWWN CLI command to get an ASCII equivalent of the sWWN.

On the Cisco ACS server, choose User Setup. Search for the ASCII equivalent of the sWWN in the User column of the User List.

Port Security Issues

This section describes troubleshooting port security issues and includes the following topics:

Device Does Not Log into a Switch When AutoLearn Is Disabled

Cannot Activate Port Security

Unauthorized Device Gains Access to Fabric

Port Security Settings Lost After Reboot

Merge Fails

Note:

After correcting a port security configuration issue, you do not have to disable the interface and reenable it. The port comes up automatically after a port security reactivation if the problem was fixed.

Device Does Not Log into a Switch When AutoLearn Is Disabled

Symptom Device does not log into a switch when autolearn is disabled.

Table 19-3 Device Does Not Log into a Switch When Autolearn Is Disabled

Symptom

Possible Cause

Solution

Device does not log into a switch when autolearn is disabled.

Device pWWN not allowed on port.

Manually add the device to the configured port security database. See the [#wp39672 "Verifying the Active Port Security Database Using Fabric Manager" section] or the [#wp39675 "Verifying the Active Port Security Database Using the CLI" section].

Port not configured for any device.

Add a device to the port in the port security database or turn on autolearn. See the "Configuring Port Security with Autolearn Using Fabric Manager" section or the "Configuring Port Security with Autolearn Using the CLI" section.

Device is configured for some other port.

Manually add the device to the configured port security database. See the "Verifying the Active Port Security Database Using Fabric Manager" section or the "Verifying the Active Port Security Database Using the CLI" section.

Port is shut down because of port security violation.

Remove the device causing the port security violation or add that device to the database. See the "Verifying Port Security Violations Using Fabric Manager" section or the "Verifying Port Security Violations Using the CLI" section.

Device Does Not Log into a Switch When Autolearn Is Enabled

Symptom Device does not log into a switch when autolearn is enabled.

Table 19-4 Device Does Not Log into a Switch When Autolearn Is Enabled

Symptom

Possible Cause

Solution

Device does not log into a switch when autolearn is enabled.

Device is configured for some other port.

Manually remove the device from the configured port security database. See the "Verifying the Active Port Security Database Using Fabric Manager" section or the "Verifying the Active Port Security Database Using the CLI" section.

Port is shut down because of port security violation.

Remove the device causing the port security violation or add that device to the database. See the "Verifying Port Security Violations Using Fabric Manager" section or the "Verifying Port Security Violations Using the CLI" section.

Verifying the Active Port Security Database Using Fabric Manager

To verify the active port security database using Fabric Manager, follow these steps:

c. Use the port-security activate command to copy the configure database to the active database and reactivate port security.

switch(config)# port -security activate vsan 3

d. If CFS distribution is enabled, use the port-security commit command to distribute these changes.

switch(config)# port-security commit vsan 3

e. Use the no shutdown command in interface mode to bring the port back online.

3. Optionally, remove the device from the switch and use the no shutdown command to bring the port back online.

Cannot Activate Port Security

Symptom Cannot activate port security.

Table 19-5 Cannot Activate Port Security

Symptom

Possible Cause

Solution

Cannot activate port security.

Autolearn is enabled.

See the "Disabling Autolearn Using Fabric Manager" section or the "Disabling Autolearn Using the CLI" section.

Conflicting entries in the configure database.

Remove the conflicting entries. Conflicting entries are those that when activated will cause existing logged in devices to logout. See the "Verifying the Active Port Security Database Using Fabric Manager" section or the "Verifying the Active Port Security Database Using the CLI" section.

Disabling Autolearn Using the CLI

StepÂ 2 File:Blank.gifUse the port-security database copy command to copy the active database to the configure database. This ensures that no learned entries are lost.

switch# port-security database copy vsan 2

StepÂ 3 File:Blank.gifIf CFS distribution is enabled, use the port-security commit command to distribute these changes.

switch(config)# port-security commit vsan 2

StepÂ 4 File:Blank.gifCopy the running configuration to the startup configuration, using the fabric option. This saves the port security configure database to the startup configuration on all switches in the fabric.

Port Security Settings Lost After Reboot

SymptomÂ Â Â Port security settings were lost after a reboot.

TableÂ 19-7 Port Security Settings Lost After Reboot

Symptom

Possible Cause

Solution

Port security settings were lost after a reboot.

Autolearn entries not saved to configure database and to startup configuration.

See the [#wp40401 "Disabling Autolearn Using Fabric Manager" section] or the [#wp40404 "Disabling Autolearn Using the CLI" section].

Merge Fails

SymptomÂ Â Â Merge fails.

TableÂ 19-8 Merge Fails

Symptom

Possible Cause

Solution

Merge fails

Activation or autolearn configuration in the separate fabrics do not match.

Disable autolearn. See the [#wp40401 "Disabling Autolearn Using Fabric Manager" section] or the [#wp40404 "Disabling Autolearn Using the CLI" section].

Combined port security database contains more than 2047 entries.

Delete the port security database in one of the fabrics and then relearn the entries after the fabrics merge. See the [#wp42671 "Configuring Port Security with Autolearn Using Fabric Manager" section] or the [#wp42674 "Configuring Port Security with Autolearn Using the CLI" section].

Configuring Port Security with Autolearn Using Fabric Manager

To configure port security with autolearn using Fabric Manager, follow these steps:

StepÂ 7 File:Blank.gifSelect the CFS tab and select commit from the ConfigAction drop-down menu to distribute these changes to all switches in the fabric.

StepÂ 8 File:Blank.gifUncheck the AutoLearn check box and click Apply Changes to disable autolearn after all entries are learned.

StepÂ 9 File:Blank.gifSelect the CFS tab and select commit from the ConfigAction drop-down menu to distribute these changes to all switches in the fabric.

StepÂ 10 File:Blank.gifCheck the CopyActive to Config check box and click Apply Changes to copy the active database to the configure database. This ensures that no learned entries are lost.

StepÂ 11 File:Blank.gifSelect the CFS tab and select commit from the ConfigAction drop-down menu to distribute these changes to all switches in the fabric.

StepÂ 12 File:Blank.gifCopy the running configuration to the startup configuration, using the fabric option. This saves the port security configure database to the startup configuration on all switches in the fabric.

Configuring Port Security with Autolearn Using the CLI

To configure port security with autolearn using the CLI, follow these steps:

StepÂ 4 File:Blank.gifIf CFS distribution is enabled, use the port-security commit command to distribute these changes.

switch(config)# port-security commit vsan 2

StepÂ 5 File:Blank.gifUse the no port-security auto-learn command in EXEC mode to disable autolearn after all entries have been learned.

switch# no port-security auto-learn vsan 2

StepÂ 6 File:Blank.gifIf CFS distribution is enabled, use the port-security commit command to distribute these changes.

switch(config)# port-security commit vsan 2

StepÂ 7 File:Blank.gifUse the port-security database copy command to copy the active database to the configure database. This ensures that no learned entries are lost.

switch# port-security database copy vsan 2

StepÂ 8 File:Blank.gifIf CFS distribution is enabled, use the port-security commit command to distribute these changes.

switch(config)# port-security commit vsan 2

StepÂ 9 File:Blank.gifCopy the running configuration to the startup configuration, using the fabric option. This saves the port security configure database to the startup configuration on all switches in the fabric.

Fabric Binding Issues

This section describes troubleshooting fabric binding issues and includes the following topic:

Note File:Blank.gifAfter correcting a fabric binding configuration issue, you do not have to disable the interface and reenable it. The port comes up automatically after a fabric binding reactivation if the problem was fixed.

Switch Cannot Attach to the Fabric

SymptomÂ Â Â Switch cannot attach to the fabric.

TableÂ 19-9 Switch Cannot Attach to the Fabric

Symptom

Possible Cause

Solution

Switch cannot attach to the fabric.

Fabric binding not activated on local switch. (It is activated on only one side of the ISL).

Fabric binding database has sWWN with a different domain ID configured.

For non-FICON VSANs, you can remove the domain ID from the fabric binding database.

Or update the domain ID in the fabric binding database (for FICON or NON-FICON VSANs).

See the [#wp41172 "Verifying Fabric Binding Violations Using Fabric Manager" section] or the [#wp41175 "Verifying Fabric Binding Violations Using the CLI" section]

The local active fabric binding database is different from the other switches.

Update the fabric binding database and reactivate it. See the [#wp41172 "Verifying Fabric Binding Violations Using Fabric Manager" section] or the [#wp41175 "Verifying Fabric Binding Violations Using the CLI" section]

Switch blocked because of fabric binding violation.

Remove the device causing the fabric binding violation or add that device to the database. See the [#wp41172 "Verifying Fabric Binding Violations Using Fabric Manager" section] or the [#wp41175 "Verifying Fabric Binding Violations Using the CLI" section].

In VSAN 2, the sWWN itself was not found in the list. In VSAN 3, the sWWN was found in the list, but has a domain ID mismatch.

StepÂ 2 File:Blank.gifOptionally, remove the switch and use the no shutdown command to bring the ISL back online.

StepÂ 3 File:Blank.gifOptionally follow these steps to add the switch to the fabric binding database:

a. File:Blank.gifUse the fabric-binding database copy command to copy the active database to the configure database.

switch# fabric-binding database copy vsan 3

b. File:Blank.gifUse the fabric-binding database command to add a new entry into the configure database.

switch(config)# fabric-binding database vsan 3

switch(config-fabric-binding)# swwn 20:11:33:11:00:2a:4a:66

c. File:Blank.gifUse the fabric-binding activate command to copy the configure database to the active database and reactivate fabric binding.

switch(config)# fabric-binding activate vsan 3

d. File:Blank.gifUse the no shutdown command in interface mode to bring the port back online.

Cannot Activate Fabric Binding

SymptomÂ Â Â Cannot activate fabric binding.

TableÂ 19-10 Cannot Activate Fabric Binding

Symptom

Possible Cause

Solution

Cannot activate fabric binding.

Conflicting entries in the configure database.

Remove the conflicting entries. See the [#wp42002 "Verifying the Config Fabric Binding Database Using Fabric Manager" section] or the [#wp42005 "Verifying the Config Fabric Binding Database Using the CLI" section].

Verifying the Config Fabric Binding Database Using Fabric Manager

To verify the config fabric binding database using Fabric Manager, follow these steps:

StepÂ 3 File:Blank.gifSelect the Config Database tab and click Add Row to add a new entry into the configure database.

StepÂ 4 File:Blank.gifFill in the WWNs and Domain ID fields and click Create.

StepÂ 5 File:Blank.gifSelect the Actions tab, select activate from the Action drop-down menu, and click Apply Changes to copy the configure database to the active database and reactivate fabric binding.

StepÂ 6 File:Blank.gifCopy the running configuration to the startup configuration, using the fabric option. This saves the port security configure database to the startup configuration on all switches in the fabric.

StepÂ 5 File:Blank.gifCopy the running configuration to the startup configuration, using the fabric option. This saves the fabric binding configure database to the startup configuration on all switches in the fabric.