General Troubleshooting

Note The terms "Unidirectional Path Switched Ring" and "UPSR" may appear in Cisco literature. These terms do not refer to using Cisco ONS 15xxx products in a unidirectional path switched ring configuration. Rather, these terms, as well as "Path Protected Mesh Network" and "PPMN," refer generally to Cisco's path protection feature, which may be used in any topological network configuration. Cisco does not recommend using its path protection feature in any particular topological network configuration.

This chapter provides procedures for troubleshooting the most common problems encountered when operating a Cisco ONS 15454. To troubleshoot specific ONS 15454 alarms, see "Alarm Troubleshooting." If you cannot find what you are looking for contact the Cisco Technical Assistance Center (TAC, 800-553-2447).

This chapter includes the following sections on network problems:

•Network Troubleshooting Tests—Describes loopbacks and hairpin circuits, which you can use to test circuit paths through the network or logically isolate faults.

•Power and LED Tests—Provides troubleshooting procedures for power supply and LED indicator problems.

1.1 Network Troubleshooting Tests

Use loopbacks and hairpins to test newly created SONET circuits before running live traffic or to logically locate the source of a network failure. All ONS 15454 OC-N cards except some cards allow loopbacks and hairpins. The G-Series Ethernet cards allows terminal and facility loopbacks on the OC-N circuit path, like the OC-N cards. ONS 15454 DWDM cards do not allow loopbacks, but loopback from the transponder cards and line cards can be used to check functionality.

Caution Facility (line) or terminal loopback can be service-affecting. To protect traffic, apply a lockout or force switch to the target loopback port. For more information on these operations, refer to the
Cisco ONS 15454 Procedure Guide.
Caution On OC-N cards, a facility (line) loopback applies to the entire card and not an individual circuit. Exercise caution when using loopbacks on an OC-N card carrying live traffic.

A facility (line) loopback tests the line interface unit (LIU) of a card, the EIA (electrical interface assembly), and related cabling. After applying a facility loopback on a port, use a test set to run traffic over the loopback. A successful facility loopback isolates the LIU, the EIA, or cabling plant as the potential cause of a network problem. Figure 1-1 shows a facility loopback on a DS-N card.

Figure 1-1 Facility (Line) Loopback Process on a DS-N Card

To test the LIU on an OC-N card, connect an optical test set to the OC-N port and perform a facility (line) loopback or use a loopback or hairpin on a card that is farther along the circuit path. Figure 1-2 shows a facility loopback on an OC-N card.

Figure 1-2 Facility (Line) Loopback Process on an OC-N Card

Caution Before performing a facility (line) loopback on an OC-N card, be sure the card contains at least two data communications channel (DCC) paths to the node where the card is installed. A second DCC provides a nonlooped path to log into the node after the loopback is applied, enabling you to remove the facility loopback. Ensuring a second DCC is not necessary if you are directly connected to the ONS 15454 containing the loopback OC-N card.

A terminal loopback tests a circuit path as it passes through the cross-connect card (XC, XCVT, or XC10G) and loops back from the card with the loopback. Figure 1-3 shows a terminal loopback on an OC-N card. The test-set traffic comes in on the DS-N card and goes through the cross-connect card to the OC-N card. The terminal loopback on the OC-N card turns the signal around before it reaches the LIU and sends it back through the cross-connect card to the DS-N card. This test verifies that the cross-connect card and terminal circuit paths are valid, but does not test the LIU on the OC-N card.

Figure 1-3 Terminal Loopback Process on an OC-N Card

Figure 1-4 shows a terminal loopback on a DS-N card. The test-set traffic comes in on the OC-N card and goes through the cross-connect card to the DS-N card. The terminal loopback on the DS-N card turns the signal around before it reaches the LIU and sends it back through the cross-connect card to the OC-N card. This test verifies that the cross-connect card and terminal circuit paths are valid, but does not test the LIU on the DS-N card.

Setting a terminal loopback on the G-Series card may not stop the Tx Packets counter or the Rx Packet counters on the CTC card-level view Performance > Statistics page from increasing. The counters can increment even though the loopbacked port has temporarily disabled the transmit laser and is dropping any received packets.

The Tx Packet statistic continues to increment because the statistic is not based on the packets transmitted by the Tx laser but on the Tx signal inside the G-Series card. In normal in-service port operation, the Tx signal being recorded does result in the Tx laser transmitting packets, but in a terminal loopback this signal is being looped back within the G-Series card and does not result in the Tx laser transmitting packets.

The Rx Packet counter may also continue to increment when the G-Series card is in terminal loopback. Rx packets from any connected device are dropped and not recorded, but the internally looped back packets follow the G-Series card's normal receive path and register on the Rx Packet counter.

Figure 1-4 Terminal Loopback Process on a DS-N Card

ONS 15454 port loopbacks either terminate or bridge the loopback signal. In the ONS 15454 system, all optical, electrical, and Ethernet facility loopbacks are terminated as shown in Table 1-1. During terminal loopbacks, some ONS cards bridge the loopback signal while others terminate it.

If an optical, electrical, or Ethernet port terminates a terminal or facility loopback signal, this means that the signal only loops back to the originating port and is not transmitted downstream. If the port bridges a loopback signal, the signal loops back to the originating port and is also transmitted downstream.

All ONS 15454 card bridging and terminating behaviors are listed in Table 1-1. When a port on a card in the left column of this table originates a terminal or facility loopback, the signal behaves as listed in the middle and right columns.

Note In Table 1-1, no AIS signal is injected if the signal is bridged. If the signal is terminated, an applicable AIS is injected downstream for all cards except Ethernet cards.

1G-Series Ethernet terminal loopback is terminated and Ethernet transmission is disabled. No AIS is inserted for Ethernet, but a TPTFAIL alarm is raised on the far-end Ethernet port.

2G-Series facility loopback is terminated and no AIS is sent downstream. However, the Cisco Link Integrity signal continues to be sent downstream.

The loopback itself is listed in the Alarms window. For example, the window would list the LPBKTERMINAL condition or LPBKFACILITY condition for a tested port.

In addition to the Alarms window listing, the following behaviors occur:

•If a DS-N, OC-N, or EC-1 port is placed in out of service (OOS) state, it injects an AIS signal upstream and downstream.

•If a DS-N, OC-N, or EC-1port is placed in out of service auto in-service (OOS_AINS) state or in the out of service maintenance (OOS_MT) state before loopback testing, the port clears the AIS signal upstream and downstream unless there is a service-affecting defect that would also cause an AIS signal to be injected. For more information about placing ports into alternate states for testing, refer to the Cisco ONS 15454 Procedure Guide.

A hairpin circuit brings traffic in and out on a DS-N port rather than sending the traffic onto the OC-N card. A hairpin loops back only the specific STS or VT circuit and does not cause an entire OC-N port to loop back, thus preventing a drop of all traffic on the OC-N port. The hairpin allows you to test a specific STS or VT circuit on nodes running live traffic. Figure 1-7 shows the hairpin circuit process on a DS-N card.

Figure 1-7 Hairpin Circuit Process on a DS-N Card

A cross-connect loopback tests a circuit path as it passes through the cross-connect card and loops back to the port being tested. Testing and verifying circuit integrity often involves taking down the whole line; however, a cross-connect loopback allows you to create a loopback on any embedded channel at supported payloads at the STS-1 granularity and higher. For example, you can loop back a single STS-1, STS-3c, STS-6c, etc. on an optical facility (line) without interrupting the other STS circuits.

The following restrictions apply to cross-connect loopbacks:

•You can create a cross-connect loopback on all working or protect optical ports unless the protect port is used in a 1+1 protection group and is in working mode.

•If a terminal or facility loopback exists on a port, you cannot use the cross-connect loopback.

1.2 Identify Points of Failure on a DS-N Circuit Path

Facility (line) loopbacks, terminal (inward) loopbacks, and hairpin circuits are often used to test a circuit path through the network or to logically isolate a fault. Performing a loopback test at each point along the circuit path systematically isolates possible points of failure.

The example in this section tests a DS-N circuit on a two-node, bidirectional line switched ring (BLSR). Using a series of facility loopbacks, terminal loopbacks, and hairpins, the path of the circuit is traced and the possible points of failure are tested and eliminated. A logical progression of five network test procedures apply to this sample scenario:

Note The test sequence for your circuits will differ according to the type of circuit and network topology.

1. A facility (line) loopback on the source node DS-N

2. A hairpin on the source node DS-N

3. A terminal (inward) loopback on the destination node DS-N

4. A hairpin on the destination node DS-N

5. A facility (line) loopback on the destination DS-N

Note All loopback tests require on-site personnel.

1.2.1 Perform a Facility (Line) Loopback on a Source DS-N Port

The facility (line) loopback test is performed on the node source port in the network circuit, in this example, the DS-N port in the source node. Completing a successful facility (line) loopback on this port isolates the cabling, the DS-N card, and the EIA as possible failure points. Figure 1-8 shows an example of a facility loopback on a source DS-N port.

Figure 1-8 A Facility (Line) Loopback on a Circuit Source DS-N Port

Caution Performing a loopback on an in-service circuit is service-affecting. To protect traffic, apply a lockout or force switch to the target loopback port. For more information on these operations, refer to the
Cisco ONS 15454 Procedure Guide.

Note DS-3 facility (line) loopbacks do not transmit an AIS condition in the direction away from the loopback. Instead of a DS-3 AIS, a continuance of the signal transmitted to the loopback is provided.

Create the Facility (Line) Loopback on the Source DS-N Port

Note This procedure does not apply to DWDM (Software R4.5).

Step 1 Connect an electrical test set to the port you are testing.

Use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the EIA connectors or DSx panel for the port you are testing. The Tx and Rx terminals connect to the same port. Adjust the test set accordingly.

Step 2 Use CTC to create the facility (line) loopback on the port being tested:

a. In node view, double-click the card where you will perform the loopback.

b. Click the Maintenance > Loopback tabs.

c. Choose OOS_MT from the State column for the port being tested. If this is a multiport card, select the appropriate row for the port being tested.

d. Choose Facility (Line) from the Loopback Type column for the port being tested. If this is a multiport card, select the appropriate row for the port being tested.

e. Click Apply.

f. Click Yes in the Confirmation Dialog box.

Note It is normal for a LPBKFACILITY condition to appear during loopback setup. The condition clears when you remove the loopback.

Test the DS-N Cabling

Note This procedure does not apply to DWDM (Software R4.5).

Step 1 Replace the suspected bad cabling (the cables from the test set to the DSx panel or the EIA ports) with a known-good cable.

If a known-good cable is not available, test the suspected bad cable with a test set. Remove the suspected bad cable from the DSx panel or the EIA and connect the cable to the Tx and Rx terminals of the test set. Run traffic to determine whether the cable is good or defective.

Test the DS-N Card

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

1.2.2 Perform a Hairpin on a Source Node Port

The hairpin test is performed on the cross-connect card in the network circuit. A hairpin circuit uses the same port for both source and destination. Completing a successful hairpin through the card isolates the possibility that the cross-connect card is the cause of the faulty circuit. Figure 1-9 shows an example of a hairpin loopback on a source node port.

Figure 1-9 Hairpin on a Source Node Port

Note The ONS 15454 does not support simplex operation on the cross-connect card. Two cross-connect cards of the same type must be installed for each node.

b. If you are starting the current procedure without the electrical test set hooked up to the DS-N port, use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the DSx panel or the EIA connectors for the port you are testing. The Tx and Rx terminals connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the hairpin on the port being tested:

a. Click the Circuits tab and click Create.

b. In the Circuit Attributes dialog box, give the circuit an easily identifiable name, such as "Hairpin1."

c. Set the Circuit Type and Size to the normal preferences.

d. Uncheck the Bidirectional check box and click Next.

e. In the Circuit Creation Source dialog box, select the same Node, card Slot, Port, and STS or VT where the test set is connected and click Next.

f. In the Circuit Creation Destination dialog box, use the same Node, card Slot, Port, and STS or VT used for the Circuit Source dialog box and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a one-way circuit.

Test the Standby Cross-Connect Card

Note This procedure does not apply to DWDM (Software R4.5).

Step 1 Perform a reset on the standby cross-connect card:

a. Determine the standby cross-connect card. On both the physical node and the CTC window, the ACT/SBY LED of the standby cross-connect card is amber and the ACT/SBY LED of the active cross-connect card is green.

1.2.3 Perform a Terminal (Inward) Loopback on a Destination DS-N Port

The terminal (inward) loopback test is performed on the node destination port in the circuit, in this example, the DS-N port in the destination node. First, create a bidirectional circuit that starts on the source node DS-N port and loops back on the destination node DS-N port. Then proceed with the terminal loopback test. Completing a successful terminal loopback to a destination node DS-N port verifies that the circuit is good up to the destination DS-N. Figure 1-10 shows an example of a terminal loopback on a destination DS-N port.

Figure 1-10 Terminal (Inward) Loopback on a Destination DS-N Port

Caution Performing a loopback on an in-service circuit is service-affecting. To protect traffic, apply a lockout or force switch to the target loopback port. For more information on these operations, refer to the
Cisco ONS 15454 Procedure Guide.

Note DS-3 terminal loopbacks do not transmit an AIS condition in the direction away from the loopback. Instead of a DS-3 AIS, a continuance of the signal transmitted to the loopback is provided.

b. If you are starting the current procedure without the electrical test set hooked up to the DS-N port, use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the DSx panel or the EIA connectors for the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the terminal loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. In the Circuit Attributes dialog box, give the circuit an easily identifiable name, such as "DSNtoDSN."

c. Set Circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Creation Source dialog box, select the same Node, card Slot, Port, and STS or VT where the test set is connected and click Next.

f. In the Circuit Creation Destination dialog box, fill in the destination Node, card Slot, Port, and STS or VT (the DS-N port in the destination node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a 2-way circuit.

Note It is normal for a LPBKTERMINAL condition to appear during a loopback setup. The condition clears when you remove the loopback.

Note DS-3 terminal loopbacks do not transmit a DS-3 AIS (see the "AIS" condition on page 2-24) in the direction away from the loopback. Instead of a DS-3 AIS, a continuance of the signal transmitted to the loopback is provided.

Step 4 Create the terminal (inward) loopback on the destination port being tested:

a. Go to the node view of the destination node:

•Choose View > Go To Other Node from the menu bar.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the DS-N card in the destination node.

c. Click the Maintenance > Loopback tabs.

d. Select OOS_MT from the State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

Test the Destination DS-N Card

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

Step 2 Resend test traffic on the loopback circuit with a known-good card.

Step 3 If the test set indicates a good circuit, the problem was probably the defective card.

1.2.4 Perform a Hairpin on a Destination Node

The hairpin test is performed on the cross-connect card in the network circuit. A hairpin circuit uses the same port for both source and destination. Completing a successful hairpin through the card isolates the possibility that the cross-connect card is the cause of the faulty circuit. Figure 1-11 shows an example of a hairpin loopback on a destination node.

Figure 1-11 Hairpin on a Destination Node

Note The ONS 15454 does not support simplex operation on the cross-connect card. Two cross-connect cards of the same type must be installed for each node.

Create the Hairpin on the Destination Node

Note This procedure does not apply to DWDM (Software R4.5).

Step 1 Connect an electrical test set to the port you are testing.

Use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the EIA connectors or DSx panel for the port you are testing. The Tx and Rx terminals connect to the same port. Adjust the test set accordingly.

Step 2 Use CTC to set up the hairpin on the port being tested:

a. Click the Circuits tab and click Create.

b. In the Circuit Attributes dialog box, give the circuit an easily identifiable name, such as "Hairpin1."

c. Set the Circuit Type and Size to the normal preferences.

d. Uncheck the Bidirectional check box and click Next.

e. In the Circuit Creation Source dialog box, select the same Node, card Slot, Port, and STS or VT where the test set is connected and click Next.

f. In the Circuit Creation Destination dialog box, use the same Node, card Slot, Port, and STS or VT used for the Circuit Source dialog box and click Finish.

Step 3 Confirm that the newly created circuit appears in the Circuits tab list as a one-way circuit.

Test the Standby Cross-Connect Card

Note This procedure does not apply to DWDM (Software R4.5).

Step 1 Perform a reset on the standby cross-connect card:

a. Determine the standby cross-connect card. On both the physical node and the CTC window, the ACT/SBY LED of the standby cross-connect card is amber and the ACT/SBY LED of the active cross-connect card is green.

1.2.5 Perform a Facility (Line) Loopback on a Destination DS-N Port

The facility loopback test is performed on the node source port in the circuit, in this example, the destination DS-N port in the destination node. Completing a successful facility loopback on this port isolates the possibility that the destination node cabling, DS-N card, LIU, or EIA is responsible for a faulty circuit. Figure 1-12 shows an example of a facility loopback on a destination DS-N port.

Figure 1-12 Facility (Line) Loopback on a Destination DS-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Note DS-3 facility (line) loopbacks do not transmit an AIS condition in the direction away from the loopback. Instead of a DS-3 AIS, a continuance of the signal transmitted to the loopback is provided.

b. If you are starting the current procedure without the electrical test set hooked up to the DS-N port, use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the DSx panel or the EIA connectors for the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to create the facility (line) loopback on the port being tested:

a. In node view, double-click the card where the loopback will be performed.

b. Click the Maintenance >Loopback tabs.

c. Select Facility (Line) from the Loopback Type column for the port being tested. If this is a multiport card, select the row appropriate for the desired port.

d. Click Apply.

e. Click Yes in the Confirmation Dialog box.

Note It is normal for a LPBKFACILITY condition to appear during loopback setup. The condition clears when you remove the loopback.

Test the DS-N Cabling

Note This procedure does not apply to DWDM (Software R4.5).

Step 1 Replace the suspect cabling (the cables from the test set to the DSx panel or the EIA ports) with a known-good cable.

If a known-good cable is not available, test the suspected bad cable with a test set. Remove the suspected bad cable from the DSx panel or the EIA and connect the cable to the Tx and Rx terminals of the test set. Run traffic to determine whether the cable is good or defective.

Test the DS-N Card

Note This procedure does not apply to DWDM (Software R4.5).

Step 1 Replace the suspected bad card with a known-good card.

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

Step 5 Resend test traffic on the loopback circuit with known-good cabling, a known-good card, and the replacement EIA.

Step 6 If the test set indicates a faulty circuit, repeat all of the facility loopback procedures.

If the faulty circuit persists, contact Cisco TAC (800-553-2447).

Step 7 If the test set indicates a good circuit, the problem was probably the defective EIA.

Clear the facility (line) loopback:

•Click the Maintenance > Loopback tabs.

•Choose None from the Loopback Type column for the port being tested.

•Choose the appropriate state (IS, OOS, OOS_AINS) from the State column for the port being tested.

•Click Apply.

•Click Yes in the Confirmation Dialog box.

The entire DS-N circuit path has now passed its comprehensive series of loopback tests. This circuit qualifies to carry live traffic.

1.3 Using the DS3XM-6 Card FEAC (Loopback) Functions

The DS3XM-6 card supports FEAC functions that are not available on basic DS-3 cards. Click the Maintenance tab at the DS3XM-6 card view to reveal the two additional function columns. Figure 1-13 shows the DS3 subtab and the additional Send Code and Inhibit FE Lbk function columns.

Figure 1-13 Accessing FEAC Functions on the DS3XM-6 Card

The far end in FEAC refers to the piece of equipment that is connected to the DS3XM-6 card and not the far end of a circuit. In Figure 1-14, if a DS3XM-6 (near-end) port is configured to send a Line Loop Code, the code will be sent to the connected test set, not the DS3XM-6 (far-end) port.

Figure 1-14 Diagram of FEAC

1.3.1 FEAC Send Code

The Send Code column on the maintenance tab of a DS3XM-6 port only applies to out-of-service (OOS_MT, OOS_AINS) ports configured for CBIT framing. The column lets a user select No Code (the default) or Line Loop Code. Selecting Line Loop Code inserts a line loop activate FEAC in the CBIT overhead transmitting to the connected facility (line). This code initiates a loopback from the facility to the ONS 15454. Selecting No Code sends a line-loop-deactivate FEAC code to the connected equipment, which will remove the loopback. You can also insert a FEAC for the 28 individual DS-1 circuits transmuxed into a DS-3 circuit.

1.3.2 FEAC Inhibit Loopback

The DS3XM-6 ports and transmuxed DS-1s initiate loopbacks when they receive FEAC Line Loop codes. If the Inhibit Loopback check box is checked for a DS-3 port, then that port will ignore any received FEAC Line Loop codes and will not loop back. The port can still be put into loopback manually using the Loopback Type column even if the Inhibit Loopback check box is selected. Only DS-3 ports can be configured to inhibit responses to FEAC loopback commands, individual DS-1 ports cannot inhibit their responses.

1.3.3 FEAC Alarms

The node raises a LPBKDS1FEAC-CMD or LPBKDS3FEAC-CMD condition for a DS-1 or DS-3 port if a FEAC loopback code is sent to the far end.

If the ONS 15454 port is in loopback from having received a loopback activate FEAC code, a LPBKDS1FEAC or LPBKDS3FEAC condition occurs. The condition will clear when a loopback deactivate FEAC command is received on that port.

A DS3E card will respond to, and can inhibit, received FEAC DS-3 level loopback codes. A DS3E card cannot be configured to send FEAC codes.

1.4 Identify Points of Failure on an OC-N Circuit Path

Facility (line) loopbacks, terminal (inward) loopbacks, and cross-connect loopback circuits are often used together to test the circuit path through the network or to logically isolate a fault. Performing a loopback test at each point along the circuit path systematically isolates possible points of failure.

The example in this section tests an OC-N circuit on a three-node, bidirectional line switched ring (BLSR). Using a series of facility loopbacks and terminal (inward) loopbacks, the path of the circuit is traced and the possible points of failure are tested and eliminated. A logical progression of seven network test procedures apply to this example scenario:

Note The test sequence for your circuits will differ according to the type of circuit and network topology.

1.4.1 Perform a Facility (Line) Loopback on a Source-Node OC-N or G-Series Port

The facility (line) loopback test is performed on the node source port in the network circuit, in this example, the source OC-N port in the source node. Completing a successful facility (line) loopback on this port isolates the OC-N or G-Series port as a possible failure point. Figure 1-15 shows an example of a facility loopback on a circuit source OC-N port.

Note Facility (line) loopbacks are not available for G-Series cards prior to Release 4.1.

Figure 1-15 Facility (Line) Loopback on a Circuit Source OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Facility (Line) Loopback on the Source OC-N or G-Series Port

Step 1 Connect an optical test set to the port you are testing.

Use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. The Tx and Rx terminals connect to the same port. Adjust the test set accordingly.

Step 2 Use CTC to create the facility (line) loopback circuit on the port being tested:

a. In node view, double-click the card where you will perform the loopback.

b. Click the Maintenance > Loopback tabs.

c. Choose OOS_MT from the State column for the port being tested. If this is a multiport card, select the appropriate row for the desired port.

d. Choose Facility (Line) from the Loopback Type column for the port being tested. If this is a multiport card, select the appropriate row for the desired port.

e. Click Apply.

f. Click Yes in the Confirmation Dialog box.

Note It is normal for a LPBKFACILITY condition to appear during loopback setup. The condition clears when you remove the loopback.

Test the OC-N or G-Series Card

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

1.4.2 Perform a Terminal (Inward) Loopback on a Source-Node OC-N or G-Series Port

The terminal (inward) loopback test is performed on the node destination port. In the circuit in this example, the destination OC-N port in the source node. First, create a bidirectional circuit that starts on the node source OC-N or G-Series port and loops back on the node destination OC-N or G-Series port. Then proceed with the terminal loopback test. Completing a successful terminal loopback to a node destination OC-N or G-Series port verifies that the circuit is good up to the destination OC-N or G-Series. Figure 1-16 shows an example of a terminal loopback on a destination OC-N port.

Note Terminal (inward) loopbacks are not available for DWDM cards in Release 4.5.

Note Terminal (inward) loopbacks are not available for G-Series cards prior to Release 4.0.

b. If you are starting the current procedure without the optical test set hooked up to the OC-N port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the terminal (inward) loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. In the Circuit Attributes dialog box, give the circuit an easily identifiable name, such as "OCN1toOCN2."

c. Set Circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Creation Source dialog box, select the same Node, card Slot, Port, and STS or VT where the test set is connected and click Next.

f. In the Circuit Creation Destination dialog box, fill in the destination Node, card Slot, Port, and STS or VT (the OC-N port in the source node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a 2-way circuit.

Note It is normal for a LPBKTERMINAL condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the terminal (inward) loopback on the destination port being tested:

a. In node view, double-click the card that requires the loopback, such as the destination OC-N card in the source node.

b. Click the Maintenance > Loopback tabs.

c. Select OOS_MT from the State column. If this is a multiport card, select the row appropriate for the desired port.

d. Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

Test the OC-N Card

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

Step 2 Resend test traffic on the loopback circuit with a known-good card.

Step 3 If the test set indicates a good circuit, the problem was probably the defective card.

1.4.3 Create the XC Loopback on the Source OC-N Port

Note This procedure does not apply to DWDM (Software R4.5).

The XC loopback test occurs on the cross-connect card in a network circuit. An XC loopback circuit uses the same port for both source and destination. Completing a successful XC loopback through the cross-connect card isolates the possibility that the cross-connect card is the cause of the faulty circuit. Figure 1-18 shows an example of an XC loopback on a source OC-N port.

Figure 1-18 XC Loopback on a Source OC-N Port

Step 1 Connect an optical test set to the port you are testing.

Note Refer to the manufacturer's instructions for detailed information on connection and setup of the optical test set.

b. If you are starting the current procedure without the optical test set hooked up to the OC-N port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. The Tx and Rx terminals connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to put the circuit being tested out of service:

a. In node view, double-click the card where the test set is connected. The card view appears.

b. In card view, click the Provisioning > Line tabs.

c. Choose OOS or OOS_MT from the Status column for the port being tested.

d. Click Apply.

Step 3 Use CTC to set up the XC loopback on the circuit being tested:

a. In card view, click the Provisioning > SONET STS tabs.

b. Click the check box in the XC Loopback column for the port being tested.

Test the Standby Cross-Connect Card

Note This procedure does not apply to DWDM (Software R4.5).

Step 1 Perform a reset on the standby cross-connect card:

a. Determine the standby cross-connect card. On both the physical node and the CTC window, verify that the active cross-connect card displays a green ACT LED and the standby cross-connect card displays an amber SBY LED.

Step 4 If the test set indicates a good circuit, the cross-connect card might have had a temporary problem that was cleared by the side switch.

Clear the XC loopback circuit:

•Click the Circuits tab.

•Choose the XC loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

1.4.4 Create a Facility (Line) Loopback on an Intermediate-Node OC-N or G-Series Port

The facility (line) loopback test is performed on the node source port in the network circuit, in this example, the source OC-N or G-Series port in the intermediate node. Completing a successful facility loopback on this port isolates the OC-N or G-Series port as a possible failure point. Figure 1-19 shows an example of a facility loopback on a intermediate node circuit source OC-N.

Note Only G-Series loopbacks are possible on DWDM nodes.

Figure 1-19 Facility (Line) Loopback on an Intermediate-Node OC-N

Caution Performing a loopback on an in-service circuit is service-affecting.

Create a Facility (Line) Loopback on an Intermediate-Node OC-N or G-Series Port

b. If you are starting the current procedure without the optical test set hooked up to the OC-N or G-Series port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the facility (line) loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. In the Circuit Attributes dialog box, give the circuit an easily identifiable name, such as "OCN1toOCN3."

c. Set Circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Creation Source dialog box, select the same Node, card Slot, Port, and STS or VT where the test set is connected and click Next.

f. In the Circuit Creation Destination dialog box, fill in the destination Node, card Slot, Port, and STS or VT (the OC-N port in the intermediate node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a 2-way circuit.

Note It is normal for a LPBKFACILITY condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the facility (line) loopback on the destination port being tested:

a. Go to the node view of the intermediate node:

•Choose View > Go To Other Node from the menu bar.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the destination OC-N or G-Series card in the intermediate node.

c. Click the Maintenance > Loopback tabs.

d. Select OOS_MT from the State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

Test the OC-N or G-Series Card

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

The terminal loopback test is performed on the node destination port in the circuit, in this example, the destination OC-N or G-Series port in the intermediate node. First, create a bidirectional circuit that starts on the node source OC-N or G-Series port and loops back on the node destination OC-N or G-Series port. Then proceed with the terminal loopback test. Completing a successful terminal loopback to a node destination OC-N or G-Series port verifies that the circuit is good up to the destination OC-N or G-Series. Figure 1-20 shows an example of a terminal loopback on an intermediate node destination OC-N port.

Note Only G-Series loopbacks are possible on DWDM nodes.

Figure 1-20 Terminal Loopback on an Intermediate-Node OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create a Terminal Loopback on Intermediate-Node OC-N or G-Series Ports

b. If you are starting the current procedure without the optical test set hooked up to the OC-N or G-Series port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the terminal loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. In the Circuit Attributes dialog box, give the circuit an easily identifiable name, such as "OCN1toOCN4."

c. Set Circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Creation Source dialog box, select the same Node, card Slot, Port, and STS or VT where the test set is connected and click Next.

f. In the Circuit Creation Destination dialog box, fill in the destination Node, card Slot, Port, and STS or VT (the OC-N or G-Series port in the intermediate node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a 2-way circuit.

Note It is normal for a LPBKTERMINAL condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the terminal loopback on the destination port being tested:

a. Go to the node view of the intermediate node:

•Choose View > Go To Other Node from the menu bar.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the destination OC-N or G-Series card in the intermediate node.

c. Click the Maintenance > Loopback tabs.

d. Select OOS_MT from the State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

Test the OC-N or G-Series Card

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

Step 2 Resend test traffic on the loopback circuit with a known-good card.

Step 3 If the test set indicates a good circuit, the problem was probably the defective card.

1.4.6 Perform a Facility (Line) Loopback on a Destination-Node OC-N or G-Series Port

The facility (line) loopback test is performed on the node source port in the network circuit, in this example, the source OC-N port in the destination node. Completing a successful facility loopback on this port isolates the port as a possible failure point. Figure 1-21 shows an example of a facility loopback on a destination node circuit source OC-N port. The process works similarly for a G-Series card.

Figure 1-21 Facility (Line) Loopback on a Destination Node OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Facility (Line) Loopback on a Destination Node OC-N or G-Series Port

b. If you are starting the current procedure without the optical test set hooked up to the OC-N or G-Series port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the facility (line) loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. In the Circuit Attributes dialog box, give the circuit an easily identifiable name, such as "OCN1toOCN5."

c. Set Circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Creation Source dialog box, select the same Node, card Slot, Port, and STS or VT where the test set is connected and click Next.

f. In the Circuit Creation Destination dialog box, fill in the destination Node, card Slot, Port, and STS or VT (the OC-N or G-Series port in the destination node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a 2-way circuit.

Note It is normal for a LPBKFACILITY condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the facility (line) loopback on the destination port being tested:

a. Go to the node view of the destination node:

•Choose View > Go To Other Node from the menu bar.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the destination OC-N or G-Series card in the destination node.

c. Click the Maintenance > Loopback tabs.

d. Select OOS_MT from the State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

Test the OC-N or G-Series Card

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

1.4.7 Perform a Terminal Loopback on a Destination Node OC-N or G-Series Port

The terminal loopback test is performed on the node destination port in the circuit, in this example, the destination OC-N port in the destination node. First, create a bidirectional circuit that starts on the node source OC-N port and loops back on the node destination OC-N port. Then proceed with the terminal loopback test. Completing a successful terminal loopback to a node destination OC-N port verifies that the circuit is good up to the destination OC-N. Figure 1-22 shows an example of a terminal loopback on an intermediate node destination OC-N port. The process is similar for a G-Series port.

Note Only G-Series loopbacks are possible on DWDM nodes.

Figure 1-22 Terminal Loopback on a Destination Node OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Terminal Loopback on a Destination Node OC-N or G-Series Port

b. If you are starting the current procedure without the optical test set hooked up to the OC-N or G-Series port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the terminal loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. In the Circuit Attributes dialog box, give the circuit an easily identifiable name, such as "OCN1toOCN6."

c. Set Circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Creation Source dialog box, select the same Node, card Slot, Port, and STS or VT where the test set is connected and click Next.

f. In the Circuit Creation Destination dialog box, fill in the destination Node, card Slot, Port, and STS or VT (the OC-N or G-Series port in the destination node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a 2-way circuit.

Note It is normal for a LPBKTERMINAL condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the terminal loopback on the destination port being tested:

a. Go to the node view of the destination node:

•Choose View > Go To Other Node from the menu bar.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the destination OC-N or G-Series card in the destination node.

c. Click the Maintenance > Loopback tabs.

d. Select OOS_MT from the State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal (Inward) from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

Test the OC-N or G-Series Card

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

Step 2 Resend test traffic on the loopback circuit with a known-good card.

Step 3 If the test set indicates a good circuit, the problem was probably the defective card.

Restore the Database

Note The following parameters are not backed up and restored: node name, IP address, subnet mask and gateway, and IIOP port. If you change the node name and then restore a backed up database with a different node name, the circuits map to the new renamed node. Cisco recommends keeping a record of the old and new node names.

Caution E1000-2 cards lose traffic for approximately 90 seconds when an ONS 15454 database is restored. Traffic is lost during the period of spanning tree reconvergence. The CARLOSS alarm appears and clears during this period.
Caution If you are restoring the database on multiple nodes, wait until the TCC+/TCC2 reboot has completed on each node before proceeding to the next node.

Step 1 In CTC, log into the node where you will restore the database:

a. On the PC connected to the ONS 15454, start Netscape or Internet Explorer.

A Java Console window displays the CTC file download status. The web browser displays information about your Java and system environments. If this is the first login, CTC caching messages appear while CTC files are downloaded to your computer. The first time you connect to an ONS 15454, this process can take several minutes. After the download, the CTC Login dialog box appears.

c. In the Login dialog box, type a user name and password (both are case sensitive) and click Login. The CTC node view window appears.

Step 2 Ensure that no ring or span (four-fiber only) switch events are present; for example, ring-switch east or west, and span-switch east or west. In network view, click the Conditions tab and click Retrieve Conditions to view a list of conditions.

Step 3 If switch events need to be cleared, in node view click the Maintenance > BLSR tabs and view the West Switch and East Switch columns.

a. If a switch event (not caused by a line failure) is present, choose CLEAR from the drop-down menu and click Apply.

b. If a switch event caused by the Wait to Restore (WTR) condition is present, choose LOCKOUT SPAN from the drop-down menu and click Apply. When the LOCKOUT SPAN is applied, choose CLEAR from the drop-down menu and click Apply.

Step 4 In node view, click theMaintenance > Database tabs.

Step 5 Click Restore.

Step 6 Locate the database file stored on the workstation hard drive or on network storage.

Note To clear all existing provisioning, locate and upload the database found on the latest ONS 15454 software CD.

Step 7 Click the database file to highlight it.

Step 8 Click Open. The DB Restore dialog box appears. Opening a restore file from another node or from an earlier backup might affect traffic on the login node.

Step 9 Click Yes.

The Restore Database dialog box monitors the file transfer.

Step 10 Wait for the file to complete the transfer to the TCC+/TCC2 card.

Step 11 Click OK when the "Lost connection to node, changing to Network View" dialog box appears. Wait for the node to reconnect.

Step 12 If you cleared a switch in Step 3, reapply the switch as needed.

1.5.2 Restore the Node to Factory Configuration

Symptom A node has both TCC+/TCC2 cards in standby state, and you are unable reset the TCC+/TCC2 cards to make the node functional.

Caution Cisco strongly recommends that you keep different node databases in separate folders. This is because the reinit tool chooses the first product-specific software package in the specified directory if you use the Search Path field instead of the Package and Database fields. You may accidentally copy an incorrect database if multiple databases are kept in the specified directory.
Caution Restoring a node to the factory configuration deletes all cross-connects on the node.
Caution If you are restoring the database on multiple nodes, wait until the TCC+/TCC2 cards have rebooted on each node before proceeding to the next node.
Caution Restoring a node to factory configuration on a Windows or Unix workstation should only be carried out on a standby TCC+/TCC2 card.
Caution Cisco recommends that you take care to save the node database to safe location if you will not be restoring the node using the database provided on the software CD.

Note The following parameters are not backed up and restored when you delete the database and restore the factory settings: node name, IP address, subnet mask and gateway, and IIOP port. If you change the node name and then restore a backed up database with a different node name, the circuits map to the new renamed node. Cisco recommends keeping a record of the old and new node names.

Note If the software package files and database backup files are located in different directories, complete the Package and Database fields (Figure 1-23).

Note If you need to install or replace one or more TCC+/TCC2 cards, refer to the Cisco ONS 15454 Procedure Guide for installation instructions.

Use the Reinitialization Tool to Clear the Database and Upload Software (Windows)

Caution Restoring a node to the factory configuration deletes all cross-connects on the node.
Caution Restoring a node to factory configuration on a Windows workstation should only be carried out on a standby TCC+/TCC2 card.

Note The TCC+/TCC2 cards reboot several times during this procedure. Wait until they are completely rebooted before continuing.

Step 2 To find the recovery tool file, go to Start > Run > Browse and select the CD drive.

Step 3 On the CD drive, go to the CISCO15454 folderand choose All Files from the Files of Type drop-down menu.

Step 4 Select the RE-INIT.jar file and click Open to open the reinit tool(Figure 1-23).

Figure 1-23 Reinitialization Tool in Windows

Step 5 If the node you are reinitializing is an external network element (ENE) in a proxy server network, enter the IP address of the gateway network element (GNE) in the GNE IP field. If not, leave it blank.

Step 6 Enter the node name or IP address of the node you are reinitializing in the Node IP field (Figure 1-23).

Step 7 If the User ID field does not contain your user ID, enter the ID. Enter your password in the Password field.

Step 8 Verify that the Re-Init Database, Upload Package, and Confirm check boxes are checked. If any one is not checked, check the check box.

Step 9 In the Search Path field, verify that the path to the CISCO15454 folder on the CD drive is listed.

Caution Before you perform the next step, be sure you are uploading the correct database. You cannot reverse the upload process after you click Yes.

Step 10 Click Go. A confirmation dialog box opens.

Step 11 Click Yes.

Step 12 The status bar at the bottom of the screen displays Completewhen the node has activated the software and uploaded the database.

Note The Complete message only indicates that the TCC+/TCC2 successfully uploaded the database, not that the database restore was successful. The TCC+/TCC2 then tries to restore the database after it reboots.

Step 13 If you are logged into CTC, close the browser window and disconnect the straight-through LAN cable from the RJ-45 (LAN) port on the TCC+/TCC2 card or on the hub or switch to which the ONS 15454 is physically connected. Reconnect your straight-through LAN cable to the LAN port and log back into CTC.

Step 14 Manually set the node name and network configuration to site-specific values. Refer to the Cisco ONS 15454 Procedure Guide for information on setting the node name, IP address, mask and gateway, and IIOP port.

Use the Reinitialization Tool to Clear the Database and Upload Software (UNIX)

Caution Restoring a node to the factory configuration deletes all cross-connects on the node.
Caution Restoring a node to factory configuration on a Unix workstation should only be carried out on a standby TCC+/TCC2 card.

Note JRE 1.03_02 must also be installed on the computer you use to perform this procedure.

Note The TCC+/TCC2 cards reboot several times during this procedure. Wait until they are completely rebooted before continuing.

Step 2 To find the recovery tool file, go to the CISCO15454 directory on the CD (usually /cdrom/cdrom0/CISCO15454).

Step 3 If you are using a file explorer, double click the RE-INIT.jar file to open the reinit tool (Figure 1-24). If you are working with a command line interface, runjava -jar RE-INIT.jar.

Figure 1-24 Reinitialization Tool in UNIX

Step 4 If the node you are reinitializing is an external network element (ENE) in a proxy server network, enter the IP address of the gateway network element (GNE) in the GNE IP field. If not, leave it blank.

Step 5 Enter the node name or IP address of the node you are reinitializing in the Node IP field (Figure 1-24).

Step 6 If the User ID field does not contain your user ID, enter the ID. Enter your password in the Password field.

Step 7 Verify that the Re-Init Database, Upload Package, and Confirm check boxes are checked. If one is not checked, check that check box.

Step 8 In the Search Path field, verify that the path to the CISCO15454 folder on the CD-ROM drive is listed.

Caution Before you perform the next step, be sure you are uploading the correct database. You cannot reverse the upload process after you click Yes.

Step 9 Click Go. A confirmation dialog box opens.

Step 10 Click Yes.

Step 11 The status bar at the bottom of the window displays Completewhen the node has activated the software and uploaded the database.

Note The Complete message only indicates that the TCC+/TCC2 successfully uploaded the database, not that the database restore was successful. The TCC+/TCC2 then tries to restore the database after it reboots.

Step 12 If you are logged into CTC, close the browser window and disconnect the straight-through LAN cable from the RJ-45 (LAN) port on the TCC+/TCC2 or on the hub or switch to which the ONS 15454 is physically connected. Reconnect your straight-through LAN cable to the LAN port and log back into CTC.

Step 13 Set the node name and network configuration to site-specific values. Refer to the Cisco ONS 15454 Procedure Guide for information on provisioning the node name, IP address, subnet mask and gateway, and IIOP port.

1.6 PC Connectivity Troubleshooting

This section contains troubleshooting procedures for PC and network connectivity to the ONS 15454.

1.6.1 Unable to Verify the IP Configuration of Your PC

Symptom When connecting your PC to the ONS 15454, you are unable to successfully ping the IP address of your PC to verify the IP configuration.

Use a standard ping command to verify the TCP/IP connection between the PC and the ONS 15454 TCC+/TCC2 card. A ping command will work if the PC connects directly to the TCC+/TCC2 card or uses a LAN to access the TCC+/TCC2 card.

Ping the ONS 15454

a. If you are using a Microsoft Windows operating system, from the Start Menu choose Run, type command in the Open field of the Run dialog box, and click OK.

b. If you are using a Sun Solaris operating system, from the Common Desktop Environment (CDE) click the Personal Application tab and click Terminal.

Step 2 For both the Sun and Microsoft operating systems, at the prompt type:

pingONS-15454-IP-address

For example:

ping 198.168.10.10.

Step 3 If the workstation has connectivity to the ONS 15454, the ping is successful and displays a reply from the IP address. If the workstation does not have connectivity, a "Request timed out" message appears.

Leave one TCC+/TCC2 card in the shelf. Connect a PC directly to the remaining TCC+/TCC2 card and perform a hardware reset of the card. The TCC+/TCC2 card will transmit the IP address after the reset to enable you to capture the IP address for login.

When the CTC software and Netscape are installed, the Help files are associated with Netscape by default. When you remove Netscape, the Help files are not automatically associated with Internet Explorer as the default browser.

Reset Internet Explorer as the default browser so that CTC will associate the Help files to the correct browser.

Reset Internet Explorer as the Default Browser for CTC

Step 4 Click the Internet Explorer should check to see whether it is the default browser check box.

Step 5 Click OK.

Step 6 Exit any and all open and running CTC and Internet Explorer applications.

Step 7 Launch Internet Explorer and open a new CTC session. You should now be able to access the CTC Help.

1.7.2 Unable to Change Node View to Network View

Symptom When activating a large, multinode BLSR from Software Release 3.2 to Software Release 3.3, some of the nodes appear grayed out. Logging into the new CTC, the user is unable to change node view to network view on any and all nodes, from any workstation. This is accompanied by an "Exception occurred during event dispatching: java.lang.OutOfMemoryError" in the java window.

Delete the CTC Cache File Manually

Caution All running sessions of CTC must be halted before deleting the CTC cache. Deleting the CTC cache might cause any CTC running on this system to behave in an unexpected manner.

Step 1 To delete the JAR files manually, from the Windows Start menu choose Search > For Files or Folders.

Step 2 Enter *.jar in the Search for files or folders named field in the Search Results dialog box and click Search Now.

Step 3 Click the Modified column in the Search Results dialog box to find the JAR files that match the date when you downloaded the files from the TCC+/TCC2. These files might include CTC*.jar, CMS*.jar, and jar_cache*.tmp.

Step 4 Highlight the files and press the keyboard Delete key.

Step 5 Click Yes in the Confirm dialog box.

1.7.6 Node Icon is Grey on CTC Network View

Symptom The CTC network view shows one or more node icons as grey in color and without a node name.

You are logging into a node running CTC Software R4.0 or earlier. Releases before R4.1 require a modification to the java.policy file so that CTC JAR files can be downloaded to the computer. The modified java.policy file may not exist on the computer.

1. Install the software CD for the release of the node you are logging into.

Manually Edit the java.policy File

Step 1 Search your computer for java.policy file and open it with a text editor (Notepad or Wordpad).

Step 2 Verify that the end of this file has the following lines:

// Insert this into the system-wide or a per-user java.policy file.

// DO NOT OVERWRITE THE SYSTEM-WIDE POLICY FILE--ADD THESE LINES!

grant codeBase "http://*/fs/LAUNCHER.jar" {

permission java.security.AllPermission;

};

Step 3 If these five lines are not in the file, enter them manually.

Step 4 Save the file and restart Netscape.

CTC should now start correctly.

Step 5 If the error message is still reported, save the java.policy file as .java.policy. On Win95/98/2000 PCs, save the file to the C:\Windows folder. On Windows NT 4.0 or later PCs, save the file to all of the user folders on that PC, for example, C:\Winnt\profiles\joeuser.

The JRE contains the Java virtual machine, runtime class libraries, and Java application launcher that are necessary to run programs written in the Java programming language.

The ONS 15454 CTC is a Java application. A Java application, unlike an applet, cannot rely completely on a web browser for installation and runtime services. When you run an application written in the Java programming language, you need the correct JRE installed. The correct JRE for each CTC software release is included on the Cisco ONS 15454 software CD and on the Cisco ONS 15454 documentation CD. Complete the "Launch CTC to Correct the Core Version Build" procedure.

If you are running multiple CTC software releases on a network, the JRE installed on the computer must be compatible with the different software releases. Table 1-17 shows JRE compatibility with ONS 15454 software releases.

1Software R4.0 will notify you if an older version JRE is running on your PC or UNIX workstation.

Launch CTC to Correct the Core Version Build

Step 1 Exit the current CTC session and completely close the browser.

Step 2 Start the browser.

Step 3 Type the ONS 15454 IP address of the node that reported the alarm. This can be the original IP address you logged on with or an IP address other than the original.

Step 4 Log into CTC. The browser downloads the jar file from CTC.

Note After Release 2.2.2, the single CMS.jar file evolved into core and element files. Core files are common to the ONS 15454, ONS 15454 SDH, and ONS 15327, while the element files are unique to the particular product. For example, the ONS 15327 Release 1.0 uses a 2.3 core build and a 1.0 element build. To display the CTC Core Version number, from the CTC menu bar click Help > About CTC. This lists the core and element builds discovered on the network.

1.7.9 Different CTC Releases Do Not Recognize Each Other

Symptom This situation is often accompanied by the INCOMPATIBLE-SW alarm.

The software loaded on the connecting workstation and the software on the TCC+/TCC2 card are incompatible.

This occurs when the TCC+/TCC2 software is upgraded but the PC has not yet upgraded the compatible CTC jar file. It also occurs on login nodes with compatible software that encounter other nodes in the network that have a newer software version.

Note Remember to always log into the ONS node with the latest CTC core version first. If you initially log into an ONS node running a CTC core version of 2.2 or lower and then attempt to log into another ONS node in the network running a higher CTC core version, the lower version node does not recognize the new node.

Launch CTC to Correct the Core Version Build

Step 3 Type the ONS 15454 IP address of the node that reported the alarm. This can be the original IP address you logged on with or an IP address other than the original.

Step 4 Log into CTC. The browser will download the jar file from CTC.

Note After Release 2.2.2, the single CMS.jar file evolved into core and element files. Core files are common to the ONS 15454, ONS 15454 SDH, and ONS 15327, while the element files are unique to the particular product. For example, the ONS 15327 Release 1.0 uses a 2.3 core build and a 1.0 element build. To display the CTC Core Version number, from the CTC menu bar click Help > About CTC. This lists the core and element builds discovered on the network.

1.7.10 Username or Password Do Not Match

Symptom A mismatch often occurs concurrently with a NOT-AUTHENTICATED alarm.

The username or password entered do not match the information stored in the TCC+/TCC2.

All ONS nodes must have the same username and password created to display every ONS node in the network. You can also be locked out of certain ONS nodes on a network if your username and password were not created on those specific ONS nodes.

For initial logon to the ONS 15454, type the CISCO15 user name in capital letters and click Login (no password is required). If you are using a CTC Software Release 2.2.2 or earlier and CISCO15does not work, type cerent454 for the user name.

Another user has already selected the same source port to create another circuit.

CTC does not remove a card or port from the available list until a circuit is completely provisioned. If two users simultaneously select the same source port to create a circuit, the first user to complete circuit provisioning gets use of the port. The other user will get the "Path in Use" error.

Cancel the circuit creation and start over, or click Back until you return to the initial circuit creation window. The source port that was previously selected no longer appears in the available list because it is now part of a provisioned circuit. Select a different available port and begin the circuit creation process again.

Step 9 Verify that the Ethernet circuit that carries VLAN #1 is provisioned and that ONS 15454 #1 and ONS 15454 #2 ports also use VLAN #1.

1.7.16 VLAN Cannot Connect to Network Device from Untag Port

Symptom Networks that have a VLAN with one ONS 15454 Ethernet card port set to Tagged and one ONS 15454 Ethernet card set to Untag might have difficulty implementing Address Resolution Protocol (ARP) for a network device attached to the Untag port (Figure 1-27). They might also see a higher than normal runt packets count at the network device attached to the Untag port. This symptom/limitation also exists when ports within the same card or ports within the same chassis are put on the same VLAN, with a mix of tagged and untagged.

The Tagged ONS 15454 adds the IEEE 802.1Q tag and the Untag ONS 15454 removes the Q-tag without replacing the bytes. The NIC of the network device categorizes the packet as a runt and drops the packet.

The solution is to set both ports in the VLAN to Tagged to stop the stripping of the 4 bytes from the data packet and prevent the NIC card in the network access device from recognizing the packet as a runt and dropping it. Network devices with IEEE 802.1Q-compliant NIC cards accept the tagged packets. Network devices with non IEEE 802.1Q compliant NIC cards still drop these tagged packets. The solution might require upgrading network devices with non IEEE 802.1Q compliant NIC cards to IEEE 802.1Q compliant NIC cards. You can also set both ports in the VLAN to Untag, but you will lose IEEE 802.1Q compliance.

Dropped packets can also occur when ARP attempts to match the IP address of the network device attached to the Untag port with the physical MAC address required by the network access layer.

Change VLAN Port Tag and Untagged Settings

Step 1 Display the CTC card view for the Ethernet card involved in the problem VLAN.

Step 3 If the port is set toTagged, continue to look at other cards and their ports in the VLAN until you find the port that is set to Untag.

Step 4 At the VLAN port set to Untag, click the port and choose Tagged.

Note The attached external devices must recognize IEEE 802.1Q VLANs.

Step 5 After each port is in the appropriate VLAN, click Apply.

1.7.17 Cross-Connect Card Oscillator Fails

Symptom: The XC, XCVT or XC10G card can be affected by this problem prior to the ONS 15454 maintenance release 4.17. (Subsequent releases automatically detect this problem.) It is indicated by a CTNEQPT-PBPROT or CTNEQPT-PBWORK condition raised against all I/O cards in the node. The following conditions might also be raised on the node:

•SWMTXMOD against one or both cross-connect cards

•SD-L against near-end or far-end line cards

•AIS-L against far-end line cards

•RFI-L against near-end line cards

Table 1-26 describes the potential cause(s) of the symptom and the solution(s).

Step 4 If CTNEQPT-PBPROT does not clear, replace the Slot 10 cross-connect card with a spare card.

Step 5 If CTNEQPT-PBPROT does not clear, replace the spare card placed in Slot 10 with the original cross-connect card.

Step 6 Right-click the Slot 8 card and choose Reset Card.

Step 7 Click OK to activate the Slot 10 card and place the Slot 8 card in standby.

Step 8 If you then see the CTNEQPT-PBWORK condition raised against all I/O cards in the node, verify that CTNEQPT-PBPROT has cleared on all I/O cards. Seeing CTNEQPT-PBWORK on the cards indicates that Slot 8 card has a bad oscillator. If this is indicated, complete the following substeps. Otherwise, go toStep 9.

Step 8 If you have switched the Slot 8 card to active and continue to see CTNEQPT-PBWORK reported against all I/O cards in the node, this indicates the Slot 8 card has a bad oscillator. If this is indicated, complete the following substeps. If not, go to Step 9.

a. Replace the Slot 8 cross-connect card with a spare card. (The Slot 10 card is made active.)

b. Reseat the Slot 10 cross-connect card to make Slot 8 active.

c. Verify that the CTNEQPT-PBWORK condition has cleared on all I/O cards.

Step 9 If you then see the CTNEQPT-PBPROT condition raised against all I/O cards, verify that CTNEQPT-PBWORK has cleared on the I/O cards. This indicates that Slot 10 has a bad oscillator. If so, complete the following substeps:

Log onto the circuit node that did not change to the desired state and determine the version of software. If the software on the node is Software R3.3 or earlier, upgrade the software. Refer to the Cisco ONS 15454 Software Upgrade Guide for software upgrade procedures.

Note If the node software cannot be upgraded to R4.0, the partial state condition can be avoided by only using the circuit state supported in the earlier software version.

During an automatic transition, some path-level defects and/or alarms were detected on the circuit.

Log onto the circuit node that did not change to the desired state and examine the circuit for path-level defects, improper circuit termination, or alarms. Refer to the Cisco ONS 15454 Procedure Guide for procedures to clear alarms and change circuit configuration settings.

Resolve and clear the defects and/or alarms on the circuit node and verify that the circuit transitions to the desired state.

One end of the circuit is not properly terminated.

View the State of OC-N Circuit Nodes

Note This procedure does not apply to DWDM (Software R4.5).

Step 1 Click the Circuits tab.

Step 2 From the Circuits tab list, select the circuit with the *_PARTIAL status condition.

Step 3 Click Edit. The Edit Circuit window appears.

Step 4 In the Edit Circuit window, click the State tab (if you are viewing a SONET circuit).

The State tab window lists the Node, CRS End A, CRS End B, and CRS State for each of the nodes in the circuit.

Step 14 From the Loopback Type list, choose None and then click Apply.

Step 15 Click the Alarms tab and verify that the AIS-V alarms have cleared.

Step 16 Repeat this procedure for all the AIS-V alarms on the DS3XM-6 cards.

1.8.3 Circuit Creation Error with VT1.5 Circuit

Symptom You might receive an "Error while finishing circuit creation. Unable to provision circuit. Unable to create connection object at node_name" message when trying to create a VT1.5 circuit in CTC.

You might have run out of bandwidth on the VT cross-connect matrix at the ONS 15454 indicated in the error message.

The matrix has a maximum capacity of 336 bidirectional VT1.5 cross-connects. Certain configurations exhaust VT capacity with less than 336 bidirectional VT1.5s in a BLSR or less than 224 bidirectional VT1.5s in a path protection or 1+1 protection group. Refer to the Cisco ONS 15454 Reference Manual for more information.

1.8.4 Unable to Create Circuit From DS-3 Card to DS3XM-6 Card

Symptom You cannot create a circuit from a DS-3 card to a DS3XM-6 card.

A DS3XM-6 card converts each of its six DS-3 interfaces into 28 DS-1s for cross-connection through the network. Thus you can create a circuit from a DS3XM-6 card to a DS-1 card, but not from a DS3XM-6 card to a DS-3 card. These differences are evident in the STS path overhead. The DS-3 card uses asynchronous mapping for DS-3, which is indicated by the C2 byte in the STS path overhead that has a hex code of 04. A DS3XM-6 has a VT payload with a C2 hex value of 02.

Note You can find instructions for creating circuits in the Cisco ONS 15454 Procedure Guide.

1.8.5 DS-3 Card Does Not Report AIS-P From External Equipment

Symptom A DS3-12, DS3N-12, DS3-12E or DS3N-12E card does not report STS AIS-P from the external equipment/line side.

This card terminates the port signal at the backplane so STS AIS-P is not reported from the external equipment/line side.

DS3-12, DS3N-12,DS3-12E and DS3N-12E cards have DS3 header monitoring functionality, which allows you to view performance monitoring (PM) on the DS3 path. Nevertheless, you cannot view AIS-P on the STS path. For more information on the PM capabilities of the DS3-12,DS3N-12, DS3-12E or DS3N-12E cards, refer to the Cisco ONS 15454 Procedure Guide.

1.8.7 ONS 15454 Switches Timing Reference

The optical or BITS input is receiving loss of signal (LOS), loss of frame (LOF), or alarm indication signal (AIS) alarms from its timing source.

The ONS 15454 internal clock operates at a Stratum 3E level of accuracy. This gives the ONS 15454 a free-running synchronization accuracy of ±4.6 ppm and a holdover stability of less than 255 slips in the first 24 hours or 3.7 x 10-7/day, including temperature.

The clock is running at the frequency of the last known-good reference input. This alarm is raised when the last reference input fails. See the "HLDOVRSYNC" section for a detailed description of this alarm.

The clock is using the internal oscillator as its only frequency reference. This occurs when no reliable, prior timing reference is available. See the "FRNGSYNC" condition on page 2-92 for a detailed description.

Daisy-chained BITS sources cause additional wander buildup in the network and is therefore not supported. Instead, use a timing signal generator to create multiple copies of the BITS clock and separately link them to each ONS 15454.

1.8.11 Blinking STAT LED after Installing a Card

Symptom After installing a card, the STAT LED blinks continuously for more than 60 seconds.

The card cannot boot because it failed the Power On Shelf Test (POST) diagnostics.

The blinking STAT LED indicates that POST diagnostics are being performed. If the LED continues to blink more than 60 seconds, the card has failed the POST diagnostics test and has failed to boot.

If the card has truly failed, an EQPT alarm is raised against the slot number with an "Equipment Failure" description. Check the alarm tab for this alarm to appear for the slot where the card was installed.

To attempt recovery, remove and reinstall the card and observe the card boot process. If the card fails to boot, replace the card.

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

1.9 Fiber and Cabling

This section explains problems typically caused by cabling connectivity errors. It also includes instructions for crimping CAT-5 cable and lists the optical fiber connectivity levels.

Bit errors on line (traffic) cards usually originate from cabling problems or low optical-line levels. The errors can be caused by synchronization problems, especially if PJ (pointer justification) errors are reported. Moving cards into different error-free slots will isolate the cause. Use a test set whenever possible because the cause of the errors could be external cabling, fiber, or external equipment connecting to the ONS 15454. Troubleshoot cabling problems using the "Network Troubleshooting Tests" section. Troubleshoot low optical levels using the "Faulty Fiber-Optic Connections" section.

Warning Follow all directions and warning labels when working with optical fibers. To prevent eye damage, never look directly into a fiber or connector. Class IIIb laser. Danger, laser radiation when open. The OC-192 laser is off when the safety key is off (labeled 0). The laser is on when the card is booted and the safety key is in the on position (labeled 1). The port does not have to be in service for the laser to be on. Avoid direct exposure to the beam. Invisible radiation is emitted from the aperture at the end of the fiber optic cable when connected, but not terminated.

Step 4 If the power level falls below the specified range for the OC-N card:

a. Clean or replace the fiber patch cords. Clean the fiber according to site practice or, if none exists, follow the procedure in the Cisco ONS 15454 Procedure Guide. If possible, do this for the OC-N card you are working on and the far-end card.

b. Clean the optical connectors on the card. Clean the connectors according to site practice or, if none exists, follow the procedure in the Cisco ONS 15454 Procedure Guide. If possible, do this for the OC-N card you are working on and the far-end card.

c. Ensure that the far-end transmitting card is not an ONS IR card when an ONS LR card is appropriate.

IR cards transmit a lower output power than LR cards.

d. Replace the far-end transmitting OC-N card to eliminate the possibility of a degrading transmitter on this OC-N card.

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

e. If the power level still falls below the specified range with the replacement fibers and replacement card, check for one of these three factors that attenuate the power level and affect link loss (LL):

•Excessive number or fiber connectors; connectors take approximately 0.5 dB each.

•Excessive number of fiber splices; splices take approximately 0.5 dB each.

Note These are typical attenuation values. Refer to the specific product documentation for the actual values or use an optical time domain reflectometer (OTDR) to establish precise link loss and budget requirements.

Step 5 If no power level shows on the fiber, the fiber is bad or the transmitter on the OC-N card failed.

a. Check that the Tx and Rx fibers are not reversed. LOS and EOC alarms normally accompany reversed Tx and Rx fibers. Switching reversed Tx and Rx fibers clears the alarms and restores the signal.

b. Clean or replace the fiber patch cords. Clean the fiber according to site practice or, if none exists, follow the procedure in the Cisco ONS 15454 Procedure Guide. If possible, do this for the OC-N card you are working on and the far-end card.

c. Retest the fiber power level.

d. If the replacement fiber still shows no power, replace the OC-N card.

Caution Removing a card that currently caries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. Consult the
Cisco ONS 15454 Procedure Guide for information.

Step 6 If the power level on the fiber is above the range specified for the card, ensure that an ONS long-range (LR) card is not being used when an ONS intermediate-range (IR) card is appropriate.

LR cards transmit a higher output power than IR cards. When used with short runs of fiber, an LR transmitter will be too powerful for the receiver on the receiving OC-N card.

Receiver overloads occur when maximum receiver power is exceeded.

Tip To prevent overloading the receiver, use an attenuator on the fiber between the ONS OC-N card transmitter and the receiver. Place the attenuator on the receive transmitter of the ONS OC-N cards. Refer to the attenuator documentation for specific instructions.

Tip Most fiber has text printed on only one of the two fiber strands. Use this to identify which fiber is connected to Tx and which fiber is connected to Rx.

1.9.2.1 Crimp Replacement LAN Cables

You can crimp your own LAN cables for use with the ONS 15454. Use a cross-over cable when connecting an ONS 15454 to a hub, LAN modem, or switch, and use a LAN cable when connecting an ONS 15454 to a router or workstation. Use CAT-5 cable RJ-45 T-568B, Color Code (100 Mbps), and a crimping tool. Figure 1-29 shows the layout of an RJ-45 connector.

Note Odd-numbered pins always connect to a white wire with a colored stripe.

1.9.2.2 Replace Faulty GBIC or SFP Connectors

GBICs and small form-factor pluggables (SFP) are hot-swappable and can be installed or removed while the card or shelf assembly is powered and running.

Warning GBICs are Class I laser products. These products have been tested and comply with Class I limits.

Warning Invisible laser radiation may be emitted from the aperture ports of the single-mode fiber optic modules when no cable is connected. Avoid exposure and do not stare into open apertures.

GBICs and SFPs are input/output devices that plug into a Gigabit Ethernet card to link the port with the fiber-optic network. The type of GBIC or SFP determines the maximum distance that the Ethernet traffic can travel from the card to the next network device. For a description of GBICs and SFPs and their capabilities, see Table 1-42 and Table 1-43, and refer to the Cisco ONS 15454 Reference Manual.

Note GBICs and SFPs must be matched on either end by type: SX to SX, LX to LX, or ZX to ZX.

Note DWDM and CWDM GBICs do not function with Software R4.5.

GBICs are available in two different models. One GBIC model has two clips (one on each side of the GBIC) that secure the GBIC in the slot on the E1000-2-G, G-Series, or G1K-4 card. The other model has a locking handle. Both models are shown in Figure 1-32.

The ONS 15454 requires a constant source of DC power to properly function. Input power is -48 VDC. Power requirements range from -42 VDC to -57 VDC.

A newly installed ONS 15454 that is not properly connected to its power supply does not operate. Power problems can be confined to a specific ONS 15454 or affect several pieces of equipment on the site.

A loss of power or low voltage can result in a loss of traffic and causes the LCD clock on the ONS 15454 to default to January 1, 1970, 00:04:15. To reset the clock, in node view click the Provisioning > General tabs and change the Date and Time fields.

Warning When working with live power, always use proper tools and eye protection.

Warning Always use the supplied electrostatic discharge (ESD) wristband when working with a powered ONS 15454. Plug the wristband cable into the ESD jack located on the lower-right outside edge of the shelf assembly.

Caution Operations that interrupt power supply or short the power connections to the ONS 15454 are service-affecting.

Isolate the Cause of Power Supply Problems

Step 1 If a single ONS 15454 show signs of fluctuating power or power loss:

a. Verify that the -48 VDC #8 power terminals are properly connected to a fuse panel. These power terminals are located on the lower section of the backplane EIA under the clear plastic cover.

b. Verify that the power cable is #12 or #14 AWG and in good condition.

c. Verify that the power cable connections are properly crimped. Stranded #12 or #14 AWG does not always crimp properly with Staycon type connectors.

d. Verify that 20-A fuses are used in the fuse panel.

e. Verify that the fuses are not blown.

f. Verify that a rack-ground cable attaches to the frame-ground terminal (FGND) on the right side of the ONS 15454 EIA. Connect this cable to the ground terminal according to local site practice.

g. Verify that the DC power source has enough capacity to carry the power load.

h. If the DC power source is battery-based:

•Check that the output power is high enough. Power requirements range from -42 VDC to -57 VDC.

•Check the age of the batteries. Battery performance decreases with age.

•Check for opens and shorts in batteries, which might affect power output.

•If brownouts occur, the power load and fuses might be too high for the battery plant.

Step 2 If multiple pieces of site equipment show signs of fluctuating power or power loss:

a. Check the uninterruptible power supply (UPS) or rectifiers that supply the equipment. Refer to the UPS manufacturer's documentation for specific instructions.

b. Check for excessive power drains caused by other equipment, such as generators.

c. Check for excessive power demand on backup power systems or batteries when alternate power sources are used.

A lamp test verifies that all the card LEDs work. Run this diagnostic test as part of the initial ONS 15454 turn-up, a periodic maintenance routine, or any time you question whether an LED is in working order.