5.1 Case Study 1 Key lessons learned – MTA New York City Transit ATS

System Description

The project provides Automatic Train Supervision (ATS) for the A Division of the New York City subway system. It automates many of the real-time train control functions now performed manually, while generating train location and performance data for travelers and operators. It also allows a single control center to replace several regional manual control towers. The project is nearing completion as this report is written in the summer of 2006.

Future users did not have a clear understanding of screen interactions when only evaluating individual presentation screens. Workstation workspace was purposely planned to be tailor-able to an individual’s preferences therefore a “typical” screen layout could not evaluated for usability.

Prototyping should be demonstrated on an actual workstation mock-up.

Prototype agreements were embedded in “discussions”. The contractor tracked them to the design documents that lacked the appropriate detail to address these issues. Much effort was spent by NYCT to make sure that agreements were not bypassed during acceptance test.

Prototype agreements should clearly be identified following the criteria for requirements: unambiguous, unique, and testable. They should be tracked as supplemental contract requirements.

The independent test team slowly disintegrated. Only developers remained. In addition to the conflict of issues problem is: Developers seldom know more than their own area well; not the whole system. Developers also typically feel testing is not part of their job. Time spent by developers testing impacts project schedule because variances are not being corrected.

As implementation and testing progresses, the design documents are further and further from reality. Other SOWs, letters of direction, memos and e-mails constitute the actual design. Every change such as this should include an equivalent documentation update; the release letter should identify which documents are affected.

Design Document updates – When design is altered or more detail is added due to prototype, variances, AWOs etc. A new release is not expected, however the working copy of the design document should be modified and available on-line .

No time was saved by accepting virtual milestones; rather time was lost in later phases of the project for longer durations.

Design Review milestones should be taken seriously and successful completion should be a prerequisite for proceeding to the next review phase.

Inappropriate skills and competencies of personnel resulted in the wrong individuals inspecting, testing, and accepting equipment/systems; Construction Manager was forced to make decisions without the right technical support.

An NYCT Integrated Project Team should be formed and dedicated to support the Construction Manager.

The requirements were never truly “base-lined”, which posed difficulty to the project team to assess necessary changes. Without a formal change process through a Change Control Board many decisions were made by management without careful evaluation of the impacts to the entire system, and the associated risks to the project.

Change Control Board – Should be instituted early in the process and include NYCT involvement.

Traceability of system performance requirements is time consuming and has yet to be completed.

Requirements Traceability Tool should be utilized from a “systems” perspective.

Although working groups were established, discipline engineers still had a tendency to work with a stove-piped approach to design and implementation of their specific sub-systems.

Need to have a Systems Engineering Management Plan to define roles & responsibilities of the project team.

Mismatch of qualified testers during critical phases of site-acceptance testing; Delays to project schedule also resulted in operations staff having to be re-trained.

Operator training needs to be conducted early enough in the project to provide available and qualified resources to support testing activities.