Field Notice: MGX-FRSM-2CT3 Connectivity Loss on PXM1 Based MGX Systems

Available Languages

September 18, 2001

Products Affected

Problem Description

MGX PXM-1 based systems running any version of MGX Software below version 1.1.32 may lose some port connectivity on MGX-FRSM-2CT3 cards after any softswitch or resetcd.

This anomaly can also occur during the initial upgrade to MGX Software version 1.1.32 or later.

Background

Cisco has determined the root cause to be the FREEDM Buffer Allocation Mechanism used in MGX-FRSM-2CT3 cards (running any firmware version below 10.0.22) prior to MGX Software version 1.1.32.

The number of FREEDM Buffers allocated for a port is dependent upon:

Number of FREEDM Buffers Available

Number of DS0s Available

Number of DS0s in Port to be added

Based upon the above, the number of FREEDM buffers allocated for a Port varies depending on the configuration at the time the port is added. The same port added at different times can have different number of FREEDM buffers allocated.

After a softswitch or resetcd on any MGX-FRSM-2CT3, the ports are added sequentially according to port number; hence it is not the same order in which the ports where originally added. As a result, the FREEDM buffer requirements for the ports are different now, and may use up more FREEDM buffers, thereby leaving no FREEDM buffers for the last few ports.

When there are no FREEDM buffers available the ports are not added.

The solution was first implemented in MGX 1.1.32 containing FRSM-VHS (same as MGX-FRSM-2CT3) firmware version 10.0.22 and contains a new FREEDM buffer allocation scheme as follows:

Each MGX-FRSM-2CT3 has two FREEDM chips.

Each chip has the ability to handle 1 T3 worth of data (672 ds0s).

Freedm chip #1 handles T1s 1 to 14 and 43-56.

Freedm chip #2 handles T1s 15 to 42.

Each chip can handle 128 ports (256 ports total per MGX-FRSM-2CT3 card).

Each port can have a maximum of 24 ds0s or 1 T1.

Freedm buffers are allocated when the port is added, based on the number of ds0s and a port cannot be added if sufficient FREEDM buffers are not available.

The new allocation scheme is as follows:

# of DS0's

# FREEDM Buffers Allocated

1 to 4

3

5 to 6

4

7

5

8 to 11

6

12 to 15

7

16 to 17

8

18 to 19

9

20

10

21

11

22

12

23 to 24

14

Problem Symptoms

Previously allocated ports on MGX-FRSM-2CT3 cards do not appear after any softswitch or resetcd.

Cisco has determined the problem to be on any PXM-1 based system running any version of MGX Software below version 1.1.32.

Workaround/Solution

Workaround

Retrieve and run the script freedmbufchk (see Solution below). If the output shows NOTOK, there are two options:

Option#1 - Move a port (or ports) to the other FREEDM chip. All connections to be moved will need to be deleted, added back and configured.

Option#2 - Move a port (or ports) to a different MGX-FRSM-2CT3 card. All connections on the original MGX-FRSM-2CT3 will need to be deleted and then added back and configured on a different MGX-FRSM-2CT3 card. Additionally, any external devices connected to the Channelized T3 port must have the DS0s regroomed to the T1 port on the different MGX-FRSM-2CT3 card.

Maintain a 20% bandwidth reserve on each MGX-FRSM-2CT3 until scheduling permits upgrading to version 1.1.32 (or later).

Solution

The solution for Customers running MGX PXM-1 based systems below version 1.1.32 is to upgrade to MGX Software version 1.1.32 or later, which has a redesigned FREEDM buffer allocation mechanism.

Contained within 1.1.32 (or later), the file freedmBufChk.tar contains the script freedmbufchk which must be run prior to the upgrade to ensure no issues during the upgrade process.

The README file contained within freedmBufChk.tar reads as follows:

Introduction:

============

freedmbufchk is capable checking the FREEDM buffer in the FRSM-VHS (MGX-FRSM-2CT3) cards.

Files:

=====

There is a single tar file which contains all the files. All the files are organized into 3 dir.

scripts - this contains all the FREEDM buffer checkscript and the inputfile which is the environment file

lib - this contains all the support library scripts

logs - this is where all log files are stored.

Environment:

==========

This script is an expect script. So you need to have expect running in the /usr/local/bin dir on the workstation before this can start working.

Please set the enivornment variable BUFFER_CHK_PATH to the top directory where this script is being installed. In other words, the dir on top of the above mentioned scripts dir.

Usage:

=====

freedmbufchk -ip <ip address> - you can either give ip address or the equivalent DNS name. Output is logged to three files of the format.

log.<shelf_ip>_timestamp - This is the script log. The screen dump while the script is executing on the shelf.

log_<timestamp> - This is the output which contains output created by the script in the format <line #>

log_out_<timestamp> - This is the output which will tell the end user if we haveany problems with the ports on the MGX-FRSM-2CT3 card.

For each MGX-FRSM-2CT3 card on the shelf the output report will summarize the number of ports on each of the two FREEDM chips and it will state either OK or NOTOK.

If the output shows NOTOK there are two options:

Option#1 - Move a port (or ports) to the other FREEDM chip. All connections to be moved will need to be deleted, added back and configured.

Option#2 - Move a port (or ports) to a different MGX-FRSM-2CT3 card. All connections on the original MGX-FRSM-2CT3 will need to be deleted and then added back and configured on a different MGX-FRSM-2CT3 card. Additionally, any external devices connected to the Channelized T3 port must have the DS's regroomed to the T1 port on the different MGX-FRSM-2CT3 card.

DDTS

To follow the bug ID link below and see detailed bug information, you must be a registered user and you must be logged in.