This is the accessible text file for GAO report number GAO-12-956
entitled 'Next Generation Enterprise Network: Navy Implementing
Revised Approach, but Improvement Needed in Mitigating Risks' which
was released on September 19, 2012.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Requesters:
September 2012:
Next Generation Enterprise Network:
Navy Implementing Revised Approach, but Improvement Needed in
Mitigating Risks:
GAO-12-956:
GAO Highlights:
Highlights of GAO-12-956, a report to congressional requesters.
Why GAO Did This Study:
DON, a component of the Department of Defense (Defense), is replacing
its existing network system with NGEN. Capabilities for the new system
include secure transport of voice and data, data storage, and e-mail,
at a cost of about $38 billion through fiscal year 2024. In March
2011, GAO reported that the approach for acquiring NGEN was not
grounded in a reliable analysis of alternatives, the execution of NGEN
was not based on a reliable schedule, and acquisition decisions were
not always performance- and risk-based. GAO recommended that Defense,
among other things, reconsider its approach. The department has not
yet fully implemented GAO’s recommendations but revised its approach
to include acquiring certain NGEN services simultaneously instead of
staggering their implementation.
GAO was asked to review the revised approach to determine (1) the
extent to which DON’s selected approach to acquire NGEN is the most
cost effective and (2) the current status of and plans for acquiring
NGEN. To do this, GAO reviewed analyses supporting the cost
effectiveness of the acquisition approach, the program’s revised
acquisition strategy, integrated master schedule, key milestone
decisions, and other relevant documents.
What GAO Found:
While the Department of the Navy (DON) has revised its acquisition
approach for its new network system, the Next Generation Enterprise
Network (NGEN), it still has not shown that it has selected the most
cost-effective approach for acquiring NGEN capabilities. Cost
effectiveness is shown by comparing life-cycle costs and quantifiable
and nonquantifiable benefits among alternatives, which can be
accomplished by conducting a thorough analysis of alternatives. GAO
previously identified weaknesses with the NGEN analysis of
alternatives related to cost estimates and analysis of operational
effectiveness and made associated recommendations. However, DON did
not revisit the analysis of alternatives to address the weaknesses
previously identified, nor did it conduct any other analysis that
would show whether its revised approach is the most cost effective.
For example, while DON developed a draft economic analysis in February
2012, the analysis assessed only the status quo and revised approach,
and not other alternatives. As a result, GAO remains concerned with
the analysis measuring NGEN cost effectiveness and DON does not know
whether its revised approach for acquiring NGEN is the most cost
effective.
Even though DON lacks assurance that it is pursuing the most cost-
effective approach to acquiring NGEN capabilities, it has moved
forward with implementing its revised approach. For example, the
agency has completed activities to support the acquisition and
transition to NGEN, prepared plans and analyses required for program
initiation, and conducted oversight reviews to support the release of
the request for proposals for transport and enterprise services
(secure data and e-mail services, among other things). However, the
program’s schedule for acquiring NGEN capabilities has been delayed,
thus making it more likely that DON will not be able to fully
transition by the end of the continuity of services contract in April
2014. For example, the release of the request for proposals was
delayed, and upcoming milestones, such as contract award and program
initiation, have slipped (see table for major delays). Program
officials attributed the delays to the need for additional planning
and to revisions to the request for proposals. Compounding this
situation is that identified risks are not being adequately mitigated.
For example, not all mitigation plans are comprehensive because they
do not always include all the elements of an effective plan (e.g.,
identification of resources needed) nor do they always contain the
current status of the mitigation actions. According to program
officials, weaknesses in these mitigation plans were due, in part, to
the lack of a priority in establishing and maintaining comprehensive
and current mitigation plans. As a result, the program faces an
increased probability that transition from its existing system to NGEN
will face further delays and cost overruns.
Table: Major Delays:
Milestone: Transport services request for proposals release;
Status: Completed;
Delay: 17 months.
Milestone: Enterprise services request for proposals release;
Status: Completed;
Delay: 9 months.
Milestone: Program initiation;
Status: Not yet occurred;
Delay: 15 months.
Milestone: Transport services contract award;
Status: Not yet occurred;
Delay: 14 months.
Milestone: Enterprise services contract award;
Status: Not yet occurred;
Delay: 6 months.
Source: GAO analysis of DON data.
[End of table]
What GAO Recommends:
GAO is recommending that Defense develop comprehensive risk mitigation
plans for program-wide risks. In its comments, Defense concurred with
GAO’s recommendation and noted that it will continue to build on
efforts to improve NGEN risk mitigation.
View [hyperlink, http://www.gao.gov/products/GAO-12-956]. For more
information, contact Valerie C. Melvin at (202) 512-6304 or
melvinv@gao.gov.
[End of section]
Contents:
Letter:
Background:
DON Has Not Reevaluated Alternatives to Ensure It Is Pursuing the Most
Cost-Effective NGEN Acquisition Approach:
DON Is Proceeding with the NGEN Acquisition, but Is Experiencing
Schedule Delays and Not Adequately Mitigating Risks:
Conclusions:
Recommendation for Executive Action:
Agency Comments:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Comments from the Department of Defense:
Appendix III: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Key Purpose of Each DON Gate Review:
Table 2: Summary and Comparison of the Four Alternatives in the NGEN
AOA:
Table 3: Comparison of Previous and Current Plans for Acquiring
Segments:
Table 4: Remaining Key NGEN Program Activities:
Table 5: NGEN Program Critical Risks, as of July 2012:
Figure:
Figure 1: NGEN Major Milestone Delays:
Abbreviations:
AOA: analysis of alternatives:
Defense: Department of Defense:
DON: Department of the Navy:
IT: information technology:
MAIS: Major Automated Information System:
Marine Corps: United States Marine Corps:
NGEN: Next Generation Enterprise Network:
NMCI: Navy Marine Corps Intranet:
USN: United States Navy:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
September 19, 2012:
Honorable Susan M. Collins:
Ranking Member:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
Honorable Claire McCaskill:
Chair:
Subcommittee on Contracting Oversight:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
The Department of the Navy's (DON) Next Generation Enterprise Network
(NGEN) program is expected to replace and improve information
technology (IT) services provided by the Navy Marine Corps Intranet
(NMCI), which currently provides about 382,000 computer workstations
to approximately 700,000 users across 2,500 United States Navy (USN)
and United States Marine Corps (Marine Corps) locations around the
world. As envisioned, NGEN is to provide secure data and IT services,
such as data storage, e-mail, and video teleconferencing, and its
capabilities are to be incrementally acquired through multiple
providers. The first increment is planned to provide comparable NMCI
capabilities, as well as enhanced information assurance and increased
government control over network operations. This increment is expected
to be fully operational by March 2014 and is estimated to cost
approximately $38 billion through fiscal year 2024.
In March 2011, we reported weaknesses in DON's NGEN acquisition,
noting, for example, that its approach for acquiring the network
capabilities was not grounded in a reliable analysis of alternatives
(AOA).[Footnote 1] As a result, we recommended, among other things,
that the department conduct an interim review to reconsider the
acquisition approach. DON subsequently reported that it had made
changes to the NGEN acquisition approach.
At your request, we conducted a study of DON's revised approach for
the NGEN acquisition. Our specific objectives were to determine (1)
the extent to which DON's selected approach to acquire NGEN is the
most cost effective and (2) the current status of and plans for this
acquisition.
To determine the extent to which the selected acquisition approach is
the most cost effective, we reviewed DON documentation describing the
cost effectiveness of the acquisition approach, including the draft
economic analysis and other analyses supporting specific acquisition
approach changes. In addition, we interviewed cognizant DON and Office
of the Secretary of Defense officials about the use of these analyses
in acquisition decision making to ensure that NGEN capabilities are
acquired in the most cost-effective manner.
To determine the current status of and plans for acquiring NGEN, we
analyzed the program's revised acquisition approach, integrated master
schedule, performance assessments, risk reports, and executive
acquisition decision briefings and meeting minutes, among other
things. We also discussed progress made on NGEN acquisition efforts
with DON program officials. Additional details on our objectives,
scope, and methodology are discussed in appendix I.
We conducted this performance audit from November 2011 to September
2012 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Background:
DON is a major component of Department of Defense (Defense), and
consists of USN and Marine Corps service components. It is a large and
complex organization, whose primary mission is to organize, train,
maintain, and equip combat-ready naval forces capable of winning wars,
deterring aggression from foes, preserving freedom of the seas, and
promoting peace and security for the nation. To support this mission,
DON performs a variety of interrelated and interdependent IT-dependent
functions. In fiscal year 2012, DON's IT budget was approximately $7.8
billion for 841 investments. NGEN, with a budget of $1.7 billion in
fiscal year 2012, is one such system investment.
Overview of NGEN:
NGEN is to replace and improve the enterprise network and services
provided by NMCI, which were delivered through a DON-wide network
services contract with a single service provider (Hewlett Packard
Enterprise Services) that ended in September 2010.[Footnote 2] To
bridge the time between the end of the NMCI contract and the full
transition to the first increment of NGEN, DON awarded a $3.4 billion
continuity of services contract[Footnote 3] to Hewlett Packard
Enterprise Services, which is scheduled to run from October 2010
through April 2014. In addition to providing continuity of network
services, the contract includes transition services and the transfer
of NMCI infrastructure and intellectual property to DON.
When implemented, NGEN is to provide secure data and IT services, such
as data storage, e-mail, and video teleconferencing to USN and the
Marine Corps. It is also intended to provide the foundation for DON's
future Naval Networking Environment.[Footnote 4] The network is to be
developed incrementally, with the first increment expected to inherit
the same architecture and design, and provide the same capabilities
and services as does NMCI. In addition, NGEN is to provide increased
DON control over network operations and additional mandatory
information assurance capabilities to meet new Defense security
requirements and the implementation of an independent security
validation function. Future increments of the network have yet to be
defined.
While NGEN's first increment capabilities are not expected to differ
from those of NMCI, the operational environment for the network is
expected to change--from the contractor-owned and contractor-operated
model previously used by both services to a government-owned and
contractor-operated model for USN and to a government-owned and
government-operated model for the Marine Corps. In particular, USN
plans to have ownership and oversight of network operations while it
relies on contractors to execute and provide NGEN services. The Marine
Corps also plans to have ownership and oversight of network
operations, but will serve as its own service provider and obtain
supplemental contractor support as needed. The different operational
models are intended to allow USN and Marine Corps to operate their
respective domains in the manner best suited to support their
different mission needs.
Oversight and Acquisition Processes for NGEN:
To manage the acquisition and deployment of NGEN, DON established a
program management office within the Program Executive Office for
Enterprise Information Systems. In February 2011, DON merged the NGEN
program management office with the NMCI program management office to
form the Naval Enterprise Networks program management office. This
office manages the program's cost, schedule, and performance and is
responsible for ensuring that the program meets its objectives. In
addition, various Defense and DON organizations share program
oversight and review responsibilities. These key entities include the:
* Under Secretary of Defense for Acquisition, Technology, and
Logistics. Serves as the Milestone Decision Authority, which is the
individual designated with overall responsibility for the program, to
include approving the program to proceed through its acquisition cycle
on the basis of, for example, the acquisition strategy, an
independently evaluated economic analysis, and the acquisition program
baseline. The Milestone Decision Authority is accountable for cost,
schedule, and performance reporting, including reporting to Congress.
* Assistant Secretary of the Navy, Research, Development, and
Acquisition. Serves as DON's acquisition oversight organization for
the program, to include implementation of Under Secretary of Defense
for Acquisition, Technology, and Logistics policies and procedures.
Determines when all key milestones are ready to be submitted to the
Milestone Decision Authority.
* Department of the Navy, Program Executive Office for Enterprise
Information Systems. Oversees a portfolio of large-scale projects and
programs designed to enable common business processes and provide
standard capabilities. Reviews the acquisition strategy, economic
analysis, and the acquisition program baseline prior to approval by
the Milestone Decision Authority.
* Department of the Navy Chief Information Officer. Supports DON's
planning, programming, budgeting, and execution processes by ensuring
that the program has achievable and executable goals and conforms to
financial management regulations and to DON, Defense, and federal IT
policies in several areas (e.g., security, architecture, and
investment management). Works closely with the program office during
milestone review assessments.
NGEN is designated as a Major Automated Information System (MAIS)
[Footnote 5] and is subject to both the Office of the Secretary of
Defense's and DON's MAIS acquisition policy and guidance,[Footnote 6]
which require the program to comply with defense acquisition system
requirements. The defense acquisition system consists of five key
program life-cycle phases and three related milestone decision points:
(1) materiel solution analysis, (2) technology development (milestone
A held prior to entering this phase), (3) engineering and
manufacturing development (milestone B held prior to entering this
phase), (4) production and deployment (milestone C held prior to
entering this phase), and (5) operations and support. The Milestone
Decision Authority is to review the initial capabilities document,
which defines operational goals and needed capabilities, and authorize
the phase in which a MAIS program is to enter the defense acquisition
system.
In May 2010, the Under Secretary of Defense for Acquisition,
Technology, and Logistics authorized the NGEN program to enter the
defense acquisition system at production and deployment. NGEN was
approved to enter at this later phase because the technology was
considered mature and already operational under NMCI. Prior to
entering the production and deployment phase, a milestone C review
must be held to review the capability production document and the test
and evaluation master plan, among other things, and to authorize
limited deployment to support operational testing. The purpose of the
phase is to achieve an operational capability that satisfies the
mission needs and is verified through independent operational test and
evaluation, and to implement the system at all applicable locations.
At milestone C, the NGEN program is planned to be initiated and the
acquisition program baseline to be approved, establishing the cost,
schedule, and performance thresholds and objectives for the program.
On approval of milestone C, DON will proceed with award of the NGEN
contracts for the transport and enterprise services segments.
In addition to the defense acquisition system requirements, DON
guidance and policy require all MAIS programs to go through a "two-
pass/six-gate" acquisition review process.[Footnote 7] The first pass,
which consists of gates 1 through 3, is focused on requirements
development and validation and is led by the Chief of Naval Operations
or the Commandant of the Marine Corps. The second pass, which consists
of gates 4 through 6, is focused on developing and delivering a
solution via systems engineering and acquisition and is led by the
Assistant Secretary of the Navy (Research, Development and
Acquisition). In addition to meeting specific criteria for passing a
given gate and proceeding to the next gate, all gate reviews are to
consider program health (i.e., satisfactory cost and schedule
performance, known risks, and budget adequacy) in deciding whether to
proceed. Table 1 lists the key purpose of each gate review.
Table 1: Key Purpose of Each DON Gate Review:
Gate: 1; Purpose: Approve the initial capabilities document and
validate AOA guidance and assumptions.
Gate: 2;
Purpose: Review the AOA and approve the preferred alternatives
resulting from it.
Gate: 3;
Purpose: Authorize the capability development document, which defines
the system's key performance parameters and includes information
necessary to develop an affordable system or system increment.
Gate: 4;
Purpose: Approve the system design specification, which specifies the
system requirements.
Gate: 5;
Purpose: Approve release of a request for proposals.
Gate: 6;
Purpose: Assess overall program health through multiple reviews,
following award of a contract and satisfactory completion of an
integrated baseline review[A] and approve the capability production
document and program health prior to and after milestone C and the
full deployment decision review. Sufficiency (sustainment) reviews
focus on system performance and comparing actual performance and cost
data to estimates.
Source: GAO analysis of DON documentation.
[A] An integrated baseline review is performed to obtain stakeholder
agreement on a contractor's performance measurement baseline, which
represents the cumulative value of planned work and serves as the
baseline against which variances are calculated.
[End of table]
Prior GAO Review Identified Weaknesses in DON's Approach for Acquiring
NGEN:
In March 2011,[Footnote 8] we evaluated DON's AOA for NGEN, which had
examined four acquisition alternatives. All of the alternatives were
assumed to deliver the same NMCI capabilities and the technology
considered for each alternative was assumed to be substantially the
same. As a result, DON officials stated that the AOA was not intended
to be a traditional analysis to determine a system solution, but
rather was an analysis of alternative acquisition approaches. The
primary differences among the alternatives related to how NGEN was to
be acquired, managed, and operated. Specifically, the alternatives
varied in terms of the number of contracts to be awarded and in the
scope of government versus contractor responsibilities. Table 2
provides a description and comparison of each alternative that was
examined in the AOA.
Table 2: Summary and Comparison of the Four Alternatives in the NGEN
AOA:
Alternative: Alternative 1 (status quo);
Description: A recompete of the NMCI contract as a commercial item
under Federal Acquisition Regulation Part 12, in which the contractor
was to be responsible for end-to-end integration of services and
control of the network;
Contractual relationships: 3;
Fiscal years 2011 to 2015 estimated cost (in billions): $10.3;
Relative risk: Least.
Alternative: Alternative 2 (enhanced status quo);
Description: A single contract similar to NMCI but as a noncommercial
item with contract terms and conditions that were to address known
NMCI deficiencies[A] and give the agency more control. The contractor
was to be responsible for end-to-end integration of services and the
government would control the network;
Contractual relationships: 3;
Fiscal years 2011 to 2015 estimated cost (in billions): $10.8;
Relative risk: More.
Alternative: Alternative 3 variant (segmented[B] approach);
Description: Multiple contracts with different vendors, using the same
negotiated approach as alternative 2. The government was to be
responsible for end-to-end integration of services and control of the
network;
Contractual relationships: 10;
Fiscal years 2011 to 2015 estimated cost (in billions): $10.8;
Relative risk: Greater.
Alternative: Alternative 3 (segmented[B] approach);
Description: A greater number of contracts than alternative 3 variant.
The government was to be responsible for end-to-end integration of
services and control of the network;
Contractual relationships: 15;
Fiscal years 2011 to 2015 estimated cost (in billions): $10.7;
Relative risk: Greatest.
Source: NGEN Analysis of Alternatives report, Center for Naval
Analyses, April 2009.
[A] See, for example, GAO, Information Technology: DOD Needs to Ensure
That Navy Marine Corps Intranet Program Is Meeting Goals and
Satisfying Customers, GAO-07-51 (Washington, D.C.: Dec. 8, 2006), for
information on NMCI deficiencies.
[B] A segment represents an allocation of IT services, functions,
tools, and roles and responsibilities associated with end-to-end
service delivery. Segmentation of the network creates interfaces,
which DON refers to as "seams," among the different contractors and
government entities that must be managed effectively to ensure
successful delivery and continuity of services.
[End of table]
However, we reported that the approach pursued by DON did not match
any of the alternatives assessed in the AOA, and it was riskier and
potentially costlier than the alternatives assessed because it
included a higher number of contractual relationships. In particular,
the chosen approach was one that included more contracts, a different
segmentation scheme, and a different transition timeline than any of
the alternatives that had been assessed. We also reported that DON's
November 2009 risk-adjusted preliminary program life-cycle cost
estimate for the approach for fiscal years 2011 through 2015 showed
that this approach would cost at least an estimated $4.7 billion more
than the alternatives assessed in the AOA.[Footnote 9] Moreover, DON
had not analyzed the impact of these differences in terms of how they
compared to the original alternatives. Further, we identified key
weaknesses in the cost estimates and operational effectiveness
analysis included in the NGEN AOA. Specifically, we reported that,
while the AOA cost estimates were substantially well documented, they
were not substantially accurate, and they were neither comprehensive
nor credible. Additionally, we reported that, while the AOA identified
program capabilities and goals, it did not sufficiently assess the
alternatives' ability to satisfy the capabilities and goals.
Given our findings, we recommended that Defense reconsider the
acquisition approach based on a meaningful analysis of all viable
alternative acquisition approaches. The department did not fully
concur with our recommendation and stated that it had concluded that
DON's AOA was sufficient and that the analysis had been approved by
the Office of the Secretary of Defense, Cost Assessment and Program
Evaluation. The department added that it would complete an economic
analysis for milestone C, which would include a follow-on independent
cost estimate and an updated determination of the most cost-effective
solution. However, in response, we pointed out that DON planned to
assess only the status quo and the current approach in the economic
analysis, not other alternatives such as those that had been included
in the AOA, and we maintained that without a meaningful analysis of
alternatives, the department would be unable to determine the most
cost-effective solution.
We also reported that DON's schedule for NGEN did not adequately
satisfy key schedule estimating best practices by, for example,
establishing the critical path (the sequence of activities that, if
delayed, impacts the planned completion date of the project) and
assigning resources to all work activities. Because it did not satisfy
these practices, the schedule did not provide a reliable basis for
program execution. According to program officials, schedule estimating
had been constrained by staffing limitations. However, these
weaknesses contributed to delays in the completion of NGEN events and
milestones, including multiple major acquisition reviews and program
plans. Accordingly, we recommended that Defense ensure that the NGEN
schedule substantially reflect key schedule estimating practices. The
department partially agreed with our recommendation.
Additionally, we reported that NGEN acquisition decisions were not
always performance-and risk-based. In particular, senior executives
had approved the program's continuing progress in the face of known
performance shortfalls and risks. For example, in November 2009, the
program was approved at a key acquisition review despite the lack of
defined requirements, which officials recognized as a risk that would
impact the completion of other key documents, such as the test plan.
According to DON officials, the decision to proceed was based on their
view that they had sufficiently mitigated known risks and issues. We
recommended that the department ensure future NGEN acquisition reviews
and decisions fully reflect the state of the program's performance and
its exposure to risks. The department agreed with our recommendation.
DON Revised the NGEN Acquisition Approach:
Subsequent to the issuance of our March 2011 report,[Footnote 10] DON
reconsidered and made certain changes to the NGEN acquisition
approach. Specifically, in April 2012, the Office of the Secretary of
Defense approved NGEN acquisition approach changes that were intended
to support program executability and reduce program risk for USN. Like
the original approach, the revised approach emphasized segmentation of
the network, with the same five segments that had been defined in the
previous acquisition approach: the two primary segments are enterprise
services and transport services and the remaining three segments are
end user hardware; enterprise software licenses; and verification,
validation, and reporting (see table 3 for details on these segments).
Further, each segment is expected to be delivered by either a
contractor or government provider, with multiple competitive awards.
However, DON made changes to how certain NGEN segments are to be
acquired and transitioned. For example, it plans to solicit transport
and enterprise services using a single request for proposals and has
said it may award a combined contract for both segments; in addition,
it plans to transition both segments to the new provider(s)
simultaneously instead of staggering their implementation. According
to DON officials, these changes were made primarily because the
transport and enterprise services segments were integrally related
under NMCI, so acquiring them simultaneously would potentially reduce
labor costs and administrative burden, and reduce risk. Another change
is that USN is expected to acquire end user hardware as a service from
the enterprise services contractor rather than purchase the equipment
and provide it as government-furnished property to the contractor.
According to program officials, this change was made to mitigate a
critical NGEN risk that the program may not be fully funded if end
user hardware must be purchased in fiscal year 2014 and, in the long
term, procuring the end user hardware as a service is not more
expensive than government-purchased equipment. As an additional change
to the acquisition approach, USN is no longer expected to award a
contract for the verification, validation, and reporting segment
because it now has an internal entity--the Tenth Fleet Cyber Command--
that is to perform this function. Table 3 summarizes the previous and
current plans for acquiring NGEN.
Table 3: Comparison of Previous and Current Plans for Acquiring
Segments:
NGEN segment: Enterprise services;
Purpose of segment: Provide the enterprise service desk, seat services
supporting end user devices, and data center services such as storage
and e-mail, along with hardware and software specific to enterprise
services that are not covered under the end user hardware and
enterprise software licenses segments;
Previous acquisition approach: Award contract for enterprise services
and transition to new provider separately from other segments, for
both USN and Marine Corps;
Revised acquisition approach: Potentially award a combined contract
with transport services and solicit the contract using a single
request for proposals with transport services and transition to new
provider simultaneously with transport services, for both USN and
Marine Corps.
NGEN segment: Transport services;
Purpose of segment: Provide for the operation and sustainment of the
transport infrastructure, associated services, and level-of-effort
support for those services. It includes technology refresh of cable
plant, routers, and switches; some leasehold improvements; and movable
infrastructure associated with local network operations;
Previous acquisition approach: Award contract for transport services
and transition to new provider separately from other segments for both
USN and Marine Corps;
Revised acquisition approach: Potentially award a combined contract
with enterprise services and solicit the contract using a single
request for proposals with enterprise services and transition to a new
provider simultaneously with enterprise services, for both USN and
Marine Corps.
NGEN segment: End user hardware;
Purpose of segment: Provide end user equipment such as computers,
monitors, and keyboards;
Previous acquisition approach: Award contracts for end user hardware
with DON ownership of equipment, for both USN and Marine Corps;
Revised acquisition approach: USN will acquire end user hardware as a
service from the enterprise services contractor. While it may purchase
the initial end user hardware and provide it as government-furnished
property to the enterprise services contractor, it will acquire end
user hardware as a service from the contractor once the initial
equipment requires technology refresh;
No change for Marine Corps from previous approach.
NGEN segment: Enterprise software licenses;
Purpose of segment: Provide software licenses to meet DON-wide
requirements;
Previous acquisition approach: Award contract for end user software
licenses for both USN and Marine Corps;
Revised acquisition approach: No change from previous approach.
NGEN segment: Verification, validation, and reporting;
Purpose of segment: Provide independent third-party security
assessments of NMCI and NGEN to support all DON networks within the
naval network environment;
Previous acquisition approach: Award contract for independent security
assessment services for USN; Marine Corps will perform its own
security function;
Revised acquisition approach: USN will perform its own security
function;
No change for Marine Corps from previous approach.
Source: GAO analysis of DON data.
[End of table]
DON's planned transition from NMCI's contractor-owned and contractor-
operated model to NGEN's government-owned and contractor-operated
model for USN and government-owned and operated model for the Marine
Corps includes establishing government control over network operations
and transferring responsibility for network services from the
incumbent provider to new service providers (contractor or government)
in phases. DON is working to establish government control before award
of the transport and enterprise services contract(s) to help reduce
risk during the transition to new provider(s). Specifically, it has
defined early transition activities as discrete efforts that are
intended to establish government management capabilities, allow for
greater participation in operational decisions, and help expedite the
transition time. In addition, USN plans to demonstrate that it has the
people, processes, and tools in place to execute command and control
and governance in a segmented environment without service degradation
through a series of four government readiness reviews. Government
readiness review 0 is planned to assess general government readiness
prior to milestone C, including determining whether USN has clearly
defined the operating model and whether the program office is ready to
manage and execute the transition with an acceptable level of risk.
Government readiness review 1 is planned to assess readiness after
milestone C and prior to award of the transport and enterprise
services contract(s) to integrate IT service management capabilities.
Government readiness reviews 2 and 3 are planned to assess readiness
after contract award to transition remaining aspects, such as network
and security operation centers.[Footnote 11] Further, USN and Marine
Corps have identified two transition completion milestones--initial
transition complete and final transition complete. DON is working
towards ensuring that all of these transition milestones are met by
the end of the continuity of services contract in April 2014.
DON Has Not Reevaluated Alternatives to Ensure It Is Pursuing the Most
Cost-Effective NGEN Acquisition Approach:
According to cost estimating and acquisition guidance,[Footnote 12]
cost effectiveness is shown by a comparative analysis of all life-
cycle costs and quantifiable and nonquantifiable benefits among the
competing alternatives. Such an analysis should be used to examine
viable alternatives to inform acquisition decision making on the most
promising solution, without assuming a specific means of achieving the
desired result. For example, an AOA is initiated to examine potential
solutions with the goal of identifying the most promising option and
can subsequently be updated, as needed, to refine the proposed
solution and reaffirm the rationale in terms of cost
effectiveness.[Footnote 13] Additionally, an economic analysis
assesses net costs and benefits of the proposed solution relative to
the status quo and can identify and examine additional alternatives
that are considered feasible methods of satisfying the objective.
Even after having revised its acquisition approach, DON has not yet
shown that it is pursuing the most cost-effective approach for
acquiring NGEN capabilities because it did not revisit the AOA to
address the weaknesses we previously identified,[Footnote 14] nor did
it conduct any other analysis that would show that the current
approach is the most cost effective. Officials told us they believe
the approach they are now pursuing remains consistent with the AOA we
previously assessed. However, the revised approach DON is now pursuing
was not one of the alternatives assessed because it differs from the
AOA alternatives in terms of transition timeline, segmentation scheme,
and potentially the number of contracts and the AOA still contains the
issues we identified in our previous report.
Additionally, while DON developed a draft economic analysis for NGEN
in February 2012, the analysis assesses only the status quo (i.e.,
NMCI) and the current acquisition approach for NGEN. As was the
concern noted in our previous report, this analysis does not show
whether DON's approach is the most cost-effective solution. Further,
according to program officials, the draft economic analysis is to be
refined and updated based on a revised service cost position,[Footnote
15] and is not expected to be final until the acquisition program
baseline is to be approved and about 3 months before the planned time
frame for awarding the primary NGEN contracts for transport and
enterprise services, and thus, would be limited in its ability to
inform decision makers on the best NGEN approach to pursue. Program
officials agreed that the final economic analysis would not be able to
show the most cost-effective solution; they stated that the economic
analysis is being prepared because it is a required document for
program initiation (milestone C review).
DON also developed analyses to support changes to its acquisition
approach by examining whether a specific change to a particular
segment would be more cost effective. For example, DON examined
whether it should release one request for proposals instead of two for
the transport and enterprise services segments. Additionally, USN
examined whether it should acquire existing end user hardware owned by
the incumbent and provide it to the enterprise services contractor as
government-furnished property or acquire the end user hardware from
the enterprise services contractor as a service. However, because
these analyses focus on specific changes, they do not provide an
understanding of whether DON's overall acquisition approach is the
most cost effective. Without a meaningful analysis of acquisition
alternatives, DON does not know whether its approach for acquiring
NGEN capabilities and meeting NGEN goals is the most cost effective
among other viable alternatives.
DON Is Proceeding with the NGEN Acquisition, but Is Experiencing
Schedule Delays and Not Adequately Mitigating Risks:
Notwithstanding the lack of assurance that it is pursuing the most
cost-effective acquisition, DON nonetheless has moved forward with its
revised approach for acquiring NGEN. In this regard, the department
has undertaken activities to support its acquisition and transition to
NGEN, prepared plans and analyses required for program initiation at
milestone C, and conducted oversight reviews to support the release of
the request for proposals for transport and enterprise services.
However, the program's schedule for acquiring NGEN capabilities has
been delayed, resulting in a compressed timeline for transitioning to
the new network and increased risks associated with transitioning to
the new network before the end of the continuity of services contract.
Compounding this situation is the fact that identified risks that can
further impact schedule delays are not being adequately mitigated.
Execution of the NGEN Program Is Proceeding, but Major Milestones Have
Slipped:
DON has undertaken activities to support its acquisition and
transition to NGEN, prepared plans and analyses required for program
initiation at milestone C, and conducted oversight reviews to support
the release of the request for proposals for transport and enterprise
services. Specifically,
* As of December 2011, DON had completed early transition activities,
such as developing IT service management strategies, processes,
procedures, and tools to serve as the overarching governance framework
for delivering NGEN capabilities; analyzing and validating the current
NMCI infrastructure inventory; and conducting job task analyses and
assessing learning tools for contractor technical representatives.
Additionally, the Marine Corps assumed control of the NMCI
infrastructure currently supporting its operations and awarded the
Marine Corps Common Hardware Suite contract to procure NGEN end user
hardware in May 2012. Also in May 2012, DON reached agreement on the
first of 12 planned enterprise software license agreements. Further,
the department released NGEN documents and technical data to industry
to ensure all competitors have full access to NMCI technical data and
to reduce the potential for a protest. DON also released multiple
requests for information and solicited input from industry on a draft
request for proposals in order to better understand the capabilities
of the current IT marketplace with respect to NGEN requirements.
Finally, DON released the request for proposals for transport and
enterprise services in May 2012.
* DON has also prepared several plans and analyses required for
program initiation at milestone C, when the acquisition program
baseline is to be approved. In particular, in October 2011, DON
approved the cost analysis requirements description, which defines the
programmatic and technical features of NGEN increment 1 and serves as
the basis for estimating program costs. Additionally, the Naval Center
for Cost Analysis developed the service cost position, which was based
on the reconciliation of a completed program life-cycle cost estimate
and an independent cost estimate. DON also approved the systems
engineering plan and the test and evaluation master plan, which
describes the overall test and evaluation strategy for how the
network's capabilities will be assessed. Subsequently, in November
2011, the program office developed the capability production document,
which clarified and solidified the capabilities for NGEN increment 1
and became the primary source requirements document for the program.
Finally, the revised acquisition strategy, which was required prior to
release of the transport and enterprise services request for
proposals, was approved in April 2012.
* According to Defense and DON policy, acquisition programs must
proceed through a series of gate and milestone reviews (as described
earlier in this report in table 1). Since our prior report,[Footnote
16] DON has conducted two gate reviews and an Office of the Secretary
of Defense-level decision review to support the release of the request
for proposals for transport and enterprise services. In particular, in
October 2011, it completed an acquisition gate review to endorse the
NGEN increment 1 capability production document. Subsequently, DON
conducted a second NGEN acquisition gate review in January 2012 to
approve the transport and enterprise services request for proposals,
during which it reviewed the current status and health of the program
including the key activities remaining to release the request for
proposals. Subsequently, in April 2012, the Office of the Secretary of
Defense, Milestone Decision Authority, reviewed the NGEN program to
approve the updated acquisition strategy and authorize the release of
the transport and enterprise services request for proposals.
* While DON has made progress on these efforts to acquire and
transition to NGEN, key program activities remain to be completed. For
example, DON will need to demonstrate that it is prepared to execute
control and governance of the network through four government
readiness reviews; baseline the program by establishing cost,
schedule, and performance thresholds and objectives; award the primary
NGEN contract(s); and transition to the new NGEN provider(s). Table 4
lists planned completion dates for these remaining key activities.
Table 4: Remaining Key NGEN Program Activities:
Activity: Government readiness review 0;
Planned date: September 2012.
Activity: Gate 6 (sufficiency) review;
Planned date: October 2012.
Activity: Milestone C review (NGEN program initiation);
Planned date: November 2012.
Activity: Government readiness review 1;
Planned date: December 2012.
Activity: Award of transport and enterprise services contract(s);
Planned date: February 2013.
Activity: Government readiness review 2;
Planned date: April 2013.
Activity: USN initial transition complete;
Planned date: May 2013.
Activity: Finalization of software licensing agreements[A];
Planned date: June 2013.
Activity: Marine Corps final transition complete;
Planned date: June 2013.
Activity: Government readiness review 3;
Planned date: July 2013.
Activity: USN final transition complete;
Planned date: March 2014.
Source: DON data.
[A] DON plans to enter into 12 enterprise software license agreements
and this date represents finalization of the last agreement.
[End of table]
However, a number of acquisition activities are facing schedule
delays, even though the incumbent is scheduled to end service delivery
in April 2014. Specifically, while the request for proposals for
transport and enterprise services was issued, as we previously stated,
it was delayed by 17 months and 9 months, respectively, resulting in
current delays in NGEN program milestones, including the dates for
conducting the milestone C review and awarding the contract(s) for
transport and enterprise services. Additionally, the schedule for
assessing USN's readiness to transition (i.e., government readiness
reviews) is tied to milestone C review and contract award, meaning
that they are expected to occur a certain number of days before or
after their associated event; thus, the government readiness reviews
would also be impacted by delays in milestone C and contract award.
Moreover, these delays have compressed the timeline for and increased
the risks associated with transitioning to the new network before the
end of the continuity of services contract. For example, the date for
USN's initial transition from the current service provider to the new
service provider(s) has slipped by 5 months and final transition is
scheduled for March 2014, thus compressing the period for shutting
down network services with the incumbent and transitioning them to the
new NGEN contractor(s) by about 5 months. Further, USN has identified
a number of factors that could impact transition and increase the risk
that NGEN may not be completed on time and may experience cost
overruns, such as proposals not meeting NGEN requirements and lack of
coordination among contractors and the government in operating the
network.
Program officials attributed these schedule delays to the department's
need to conduct more detailed planning before issuing the transport
and enterprise services request for proposals and for addressing
industry comments on the draft request for proposals to reduce the
potential of a bid protest. Figure 1 illustrates the delays in major
NGEN milestones.
Figure 1: NGEN Major Milestone Delays:
[Refer to PDF for image: table]
Activity: Transport services request for proposals release
Status: Completed;
Schedule:
Planned completion date as of August 2010: Late, 2010;
Planned completion date as of April 2012: Mid-2012.
Activity: Enterprise services request for proposals release
Status: Completed;
Schedule:
Planned completion date as of August 2010: Mid-2011;
Planned completion date as of April 2012: Mid-2012.
Activity: Gate 6 (sufficiency review)
Status: Not yet occurred;
Schedule:
Planned completion date as of August 2010: Mid-2011;
Planned completion date as of April 2012: Late 2012.
Activity: Milestone C review
Status: Not yet occurred;
Schedule:
Planned completion date as of August 2010: Mid-2011;
Planned completion date as of April 2012: Late 2012.
Activity: Transport services contract award
Status: Not yet occurred;
Schedule:
Planned completion date as of August 2010: Late 2011;
Planned completion date as of April 2012: Early 2013.
Activity: Finalization of software licensing agreements
Status: Not yet occurred;
Schedule:
Planned completion date as of August 2010: Early 2012;
Planned completion date as of April 2012: Mid-2013.
Activity: Enterprise services contract award
Status: Not yet awarded;
Schedule:
Planned completion date as of August 2010: Mid-2012;
Planned completion date as of April 2012: Early 2013.
Activity: USN transition period
Status: Not yet occurred;
Schedule:
Planned completion date as of August 2010: Late 2012-Early 2014;
Planned completion date as of April 2012: Mid-2013-Late 2014.
Activity: Marine Corps transition periods
Status: Not yet occurred;
Schedule:
Planned completion date as of August 2010: Early 2012-Early 2013;
Planned completion date as of April 2012: Mid-2012-Mid-2013.
Continuity of service contract award: July 2010;
Continuity of service end date: April 2014.
Source: GAO analysis of DON data.
Notes: The schedules, considered to be informal baseline schedules,
have been used to manage the overall scope of work to be conducted
within the program. An official baseline schedule is expected to be
approved as part of the acquisition program baseline at milestone C.
Delays were rounded to the nearest month.
[A] According to Marine Corps officials, new criteria for the initial
and final transition complete milestones resulted in new dates for
planned completion in the April 2012 schedule.
[End of figure]
Our prior report highlighted the significance of DON not having a
reliable schedule for executing NGEN and its contribution to delays in
key program milestones.[Footnote 17] The lack of a reliable schedule,
as we previously noted, and the continuing delays in DON's efforts to
complete the network transition as planned, raise concerns that it
will be unable to complete the transition within the time frames of
the current continuity of services contract. As a result, ensuring
that the NGEN schedule substantially reflects the key estimating
practices, as we discussed and recommended in our previous report,
continues to be a vital step for DON to take.
Program Risk Mitigation Plans Have Not Been Fully Defined:
According to industry best practices,[Footnote 18] an effective risk
management process identifies potential problems before they occur, so
that risk-handling activities may be planned and invoked, as needed,
across the life of the product and project in order to mitigate
adverse impacts on achieving objectives. Key activities of a
comprehensive risk management process include (1) identifying and
analyzing risks, (2) escalating key risks to the attention of senior
management, and (3) developing risk mitigation plans and milestones
for key mitigation deliverables. In particular, effective plans for
risk mitigation should be developed for the most important risks to
the project, which includes a period of performance, identification of
resources needed, and responsible parties. In addition, the status of
each risk should be monitored periodically to determine whether
established thresholds have been exceeded and risk mitigation plans
should be implemented as appropriate to ensure that systems will
operate as intended.
NGEN program-wide and project-specific risks are managed by different
offices, with the program office identifying and tracking program-wide
risks--those that affect the overall NGEN program. In accordance with
best practices, the NGEN program identifies and analyzes program-wide
risks, by assigning a severity rating to risks, tracking these risks
in a database, and planning response strategies for each risk in the
database. In addition, NGEN program officials escalate these risks by
reviewing and evaluating these risks during monthly program risk
management board meetings. As of July 2012, the program office had
identified eight program risks that it considered critical (moderate-
or high-level risks) and that could result in schedule delays and cost
increases. These risks included potential delays in transition from
the incumbent to the new service provider(s) and in contract award for
the transport and enterprise services, as well as the potential lack
of coordination among contractors and the government in operating the
network. Table 5 describes the program-identified critical risks for
NGEN as of July 2012.
Table 5: NGEN Program Critical Risks, as of July 2012:
Program-identified critical risk: NGEN service request tool;
Program risk description from program documentation: The current tools
for ordering requests for services are not prepared within the NGEN
environment. If the development of a functional NGEN ordering tool is
not complete in time to support the NGEN transition, then ordering
will rely on error-prone manual processes. Without automated ordering,
service delivery will degrade and staffing costs to perform ordering
will increase significantly;
Program-identified risk level: High.
Program-identified critical risk: Transition delays for USN;
Program risk description from program documentation: Significant
delays could impact the ability to transition from the incumbent to
the new service provider(s) before the continuity of services contract
performance period ends in April 2014. If USN does not fully
transition in time, it will have to extend the continuity of services
contract, resulting in cost increases and additional schedule delays;
Program-identified risk level: High.
Program-identified critical risk: Seam management;
Program risk description from program documentation: Roles and
relationships among segments and government functions have not been
defined or agreed on. If these seams are not accurately identified and
characterized, DON will not be able to manage them effectively and the
transition will take longer;
Program-identified risk level: High.
Program-identified critical risk: Purchase of end user hardware for
USN;
Program risk description from program documentation: The NGEN program
will not be fully funded if end user hardware must be purchased in
fiscal year 2014 for approximately $150 million;
Program-identified risk level: Moderate.
Program-identified critical risk: Delayed transport and enterprise
contract(s) award;
Program risk description from program documentation: A day-to-day slip
in completing the final transition will be realized if award of
transport and enterprise service contract(s) is delayed;
Program-identified risk level: Moderate.
Program-identified critical risk: Systems engineering;
Program risk description from program documentation: Service
performance may degrade, systems integration costs may increase, and
time to resolve engineering issues may increase if the program is
unable to execute design authority and problem management;
Program-identified risk level: Moderate.
Program-identified critical risk: Service offering descriptions;
Program risk description from program documentation: Service
availability and performance will degrade if DON cannot effectively
manage the level of service quality and direct the resources in
support of service delivery;
Program-identified risk level: Moderate.
Program-identified critical risk: Potential protest is sustained;
Program risk description from program documentation: A protest against
any of the NGEN contract awards may impact the schedule for transition
completion by as much as 6 months, resulting in an extension of the
continuity of services contract;
Program-identified risk level: Moderate.
Source: DON data.
[End of table]
While DON is working to mitigate seven of the eight program risks, its
mitigation plans did not always include all the elements of an
effective plan (e.g., identification of resources needed, responsible
parties, and period of performance). Specifically, the reported
mitigation strategies did not fully identify the resources needed,
such as the staff and funds, nor fully identify organizations that are
responsible and accountable for accomplishing risk mitigation
activities. Additionally, while five of the seven mitigation plans had
activities with planned completion dates, most did not include an
estimated start date; thus, the plans did not fully define the period
of performance to ensure that the mitigation activities are being
implemented appropriately. Moreover, three of the seven plans did not
identify the status of activities for which completion dates had
already occurred. In particular, to mitigate the risk of potential
lack of coordination among contractors and the government in operating
the network, DON was to develop, implement, and automate key processes
by February 15, 2012. However, the plan does not reflect whether this
activity has been completed or, otherwise discuss its status.
Additionally, two of the seven plans did not fully reflect the current
status of the program. For example, to mitigate transition risks, DON
officials identified that the enterprise and transport services
contract(s) must be awarded no later than December 2012 in order to
ensure continuous network availability during the transition from the
continuity of services contract to the NGEN contract(s). However, the
current mitigation plan does not document this milestone or reflect
the current status of the program, which now plans to award the
contract(s) in February 2013.
Further, according to program documentation, a mitigation plan was
required and was being updated for the service offering descriptions
risk; however, according to other program documentation, the plan to
mitigate service offering descriptions is still under development,
even though it has been identified as a program risk since August
2011. According to program officials, weaknesses in these mitigation
plans were due, in part, to the lack of a priority in establishing and
maintaining comprehensive and current plans.
Several of the risks identified are significant to ensure that NGEN
transition occurs as planned and within the estimated costs. Therefore
it is essential to ensure that for a given risk, techniques and
methods will be invoked to avoid, reduce, and control the probability
of occurrence.
Conclusions:
Even though DON does not know whether it is pursuing the most cost-
effective approach to acquiring NGEN capabilities, it has proceeded to
implement its revised acquisition approach, completed various plans
and analyses including the first official program life-cycle cost
estimate, and held oversight reviews to support the issuance of the
request for proposals for the two primary NGEN segments. While these
steps have been taken, DON faces delays in upcoming milestones, which
have resulted in a compressed transition timeline and increased risks
associated with transitioning to the new network before the end of the
continuity of services contract. Compounding this are weaknesses in
DON's risk mitigation efforts that could further impact schedule
delays and result in cost increases. Without a well-defined schedule,
as we previously reported, and adequate risk mitigation, DON cannot
ensure that needed NGEN capabilities will be in place in time to
ensure that services will continue to operate when the incumbent is
scheduled to shut down its services.
Recommendation for Executive Action:
To strengthen risk mitigation activities for the NGEN program, we
recommend that the Secretary of Defense direct the Secretary of the
Navy to develop comprehensive mitigation plans and strategies for
program-wide critical risks that identify the mitigation period of
performance, resources needed, and responsible parties, and that fully
reflect the current status of the program.
Agency Comments:
The Department of Defense provided written comments on a draft of this
report, signed by the Deputy Assistant Secretary of Defense (C3 and
Cyber), and reprinted in appendix II. In its comments, the department
agreed with our recommendation and noted that the program office will
continue to build on efforts to improve NGEN's risk management and
mitigation process. For example, the department stated that it plans
to increase the speed at which NGEN risk management board action items
are closed.
We are sending copies of this report to the appropriate congressional
committees; the Director, Office of Management and Budget; the
Congressional Budget Office; the Secretary of Defense; and the
Secretary of the Navy. In addition, the report is available at no
charge on the GAO website at [hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff who made major
contributions to this report are listed in appendix III.
Signed by:
Valerie C. Melvin:
Director, Information Management and Technology Resources Issues:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
Our objectives were to determine (1) the extent to which the
Department of the Navy's (DON) selected approach to acquiring the Next
Generation Enterprise Network (NGEN) is the most cost effective and
(2) the current status of and plans for acquiring NGEN.
To determine the extent to which DON's approach to acquiring NGEN is
the most cost effective, we reviewed our prior work evaluating the
NGEN analysis of alternatives[Footnote 19] and analyzed current
documentation that DON had completed to describe and justify the cost
effectiveness of its acquisition approach. These included the draft
economic analysis and analyses to support specific acquisition
approach changes. We assessed DON's supporting analyses against
relevant Department of Defense guidance and our Cost Estimating and
Assessment Guide.[Footnote 20] In this regard, we evaluated the
purpose and use of these analyses in examining viable alternatives to
inform acquisition decision making on the most cost-effective
solution. We also interviewed cognizant DON program officials and
Office of the Secretary of Defense Cost Assessment and Program
Evaluation officials about the use of these analyses in acquisition
decision making.
To determine the current status of and plans for acquiring NGEN, we
analyzed the revised NGEN acquisition strategy, integrated master
schedule, program performance assessments, risk reports, transport and
enterprise services request for proposals, planned system
requirements, cost estimates, draft operational readiness plan, and
executive acquisition decision briefings and meeting minutes, among
other things. We also reviewed these documents to determine how the
program had changed since our prior review of NGEN.[Footnote 21] To
assess the status of risk management for the NGEN program initiative,
we compared the risk management plans and supporting documentation
against leading practices, such as Carnegie Mellon Software
Engineering Institute's Capability Maturity Model® Integration for
Services,[Footnote 22] to determine whether such practices had been
specified in the plans. Further, we interviewed relevant DON program
officials and Office of the Secretary of Defense Cost Assessment and
Program Evaluation officials to clarify information in documents we
reviewed and to more fully understand the program's progress to
acquire NGEN. To assess the reliability of the data that we used to
support the findings in this report, we reviewed relevant program
documentation to substantiate evidence obtained through interviews
with agency officials. We determined that the data used in this report
are sufficiently reliable. We have also made appropriate attribution
indicating the sources of the data used.
We conducted this performance audit from November 2011 to September
2012 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
[End of section]
Appendix II: Comments from the Department of Defense:
Office of The Under Secretary of Defense:
Acquisition, Technology and Logistics:
3000 Defense Pentagon:
Washington, DC 20301-3000:
September 11, 2012:
Ms. Valerie Melvin:
Director, Information Management and Technology Resources Issues:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Ms. Melvin:
This is the Department of Defense (DoD) response to the GAO Draft
Report, GAO-12-956, "Next Generation Enterprise Network: Navy
Implementing Revised Approach, but Improvement Needed in Mitigating
Risks," dated August 24, 2012 (GAO Code 310977). Detailed comments on
the report recommendations are enclosed.
Sincerely,
Signed by:
Dr. Ronald C. Jost:
Deputy Assistant Secretary of Defense (C3&Cyber):
Enclosure: As stated.
GAO Draft Report Dated August 24, 2012:
GAO-12-956 (GAO Code 310977):
"Next Generation Enterprise Network: Navy Implementing Revised
Approach, But Improvement Needed In Mitigating Risks"
Department Of Defense Comments To The GAO Recommendations:
Recommendation: To strengthen risk mitigation activities for the NGEN
project, GAO recommends that the Secretary of Defense direct the
Secretary of the Navy to develop comprehensive mitigation plans and
strategies for programwide critical risks that identify the mitigation
period of performance, resources needed, responsible parties, and that
fully reflect the current status of the program.
DoD Response: Concur. The Naval Enterprise Networks (NEN) Program
Office has already taken several steps to bolster NGEN's risk
management and mitigation program. The Navy has implemented Risk
Exchange as the enterprise risk tool, replacing previously used Risk
Registry (Risk Radar). This web-based tool has provided greater access
and significantly improved visibility into risk mitigation strategies,
timelines, status and required resources for responsible parties and
other stakeholders. Now program office personnel along with NGEN
stakeholders can effectively collaborate on broader mitigation
strategies tied to internal and external factors that potentially have
direct and indirect cost impacts to program execution. The NGEN Program
Office has combined the risk management efforts of acquisition and
operations functions, further enhancing collaboration among Risk
Managers and Risk Coordinators across the entire program office.
The Program Office continues to place greater emphasis on the
timeliness of risk information. The number of Risk Management Board
(RMB) meetings has increased from 8 in FY 2011 to 10 in FY 2012 (to
date). The speed at which RMB actions are closed has increased by 28%.
In FY 2011 the average number of calendar days to close an action was
85; in FY 2012 (to date) it was reduced to 61, to include a large
number of backlogged actions. The goal for FY 2013 is to close actions
within 30 days. The number of working days to release RMB minutes has
been reduced by 43%, from 7 days to 4 days.
Going forward, the Navy will continue to build upon these efforts. The
Navy will be rolling out an updated NEN Risk Management Plan that
incorporates the major changes that have occurred since its last
iteration of September, 2011. The NGEN Program Office will continue
the integration of acquisition and operations risk management efforts
to promote visibility and communication and build a more risk-aware
culture across the program.
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Valerie C. Melvin, (202) 512-6304, or melvinv@gao.gov:
Staff Acknowledgments:
In addition to the individual named above, key contributors to this
report were Eric Winter (assistant director), Nabajyoti Barkakati,
Harold Brumm, Neil Doherty, Nancy Glover, Madhav Panwar, Jeanne Sung,
and Niti Tandon.
[End of section]
Footnotes:
[1] GAO, Information Technology: Better Informed Decision Making
Needed on Navy's Next Generation Enterprise Network Acquisition,
[hyperlink, http://www.gao.gov/products/GAO-11-150] (Washington, D.C.:
Mar. 11, 2011).
[2] NMCI was composed of transport infrastructure such as cables,
routers, and switches; end-user equipment such as computers, monitors,
and keyboards; and software. It provided, among other things, data
storage, e-mail, transport of voice and data, and video
teleconferencing.
[3] In August 2012, DON issued a sole source modification notice to
increase the contract ceiling to $5.5 billion. According to DON
officials, the increase is needed because additional funds will be
necessary to maintain NMCI services through the transition to NGEN.
Officials stated that the modification had not yet been approved.
[4] The Naval Networking Environment is to be an iterative set of
integrated, phased programs that share a common enterprise
architecture and standards. It includes NGEN and the Consolidated
Afloat Networks and Enterprise Services program, among others, and is
to be in place by 2016.
[5] Defense uses acquisition categories, where programs of increasing
dollar value and management interest are subject to increasing levels
of oversight. MAIS programs are the highest level category for
automated information systems.
[6] Department of Defense Instruction 5000.02, Operation of the
Defense Acquisition System (Dec. 8, 2008), and Secretary of the Navy
Instruction 5000.2E, Implementation and Operation of the Defense
Acquisition System and the Joint Capabilities Integration and
Development System (Sept. 1, 2011).
[7] Secretary of the Navy Instruction 5000.2E.
[8] [hyperlink, http://www.gao.gov/products/GAO-11-150].
[9] According to program documentation, the November 2009 risk-
adjusted preliminary program life-cycle cost estimate of approximately
$50 billion was developed when program definition was still evolving
and program documentation was very recent or did not exist. The NGEN
life-cycle cost estimate has since been refined and is now
approximately $38 billion through fiscal year 2024.
[10] [hyperlink, http://www.gao.gov/products/GAO-11-150].
[11] In addition to the government readiness reviews, USN plans to
conduct a series of accompanying reviews with the contractor(s) to
validate contractor readiness to transition.
[12] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March
2009); Defense Acquisition University, Defense Acquisition Guidebook
(accessed January 10, 2012).
[13] Defense Acquisition Guidebook and DON Acquisition and
Capabilities Guidebook (May 2012).
[14] [hyperlink, http://www.gao.gov/products/GAO-11-150].
[15] The service cost position is DON's first official cost estimate
for NGEN, reflecting total life-cycle NGEN costs of about $38 billion
through fiscal year 2024, and was developed by reconciling the NGEN
program life-cycle cost estimate with an independent cost estimate
developed by the Naval Center for Cost Analysis.
[16] We previously reported in March 2011 that DON had completed gates
1 through 4 and conducted a gate 5 review in October 2010 of its
transport services request for proposals but had not yet exited the
gate because approval to proceed was conditional based on satisfactory
completion of open action items [hyperlink,
http://www.gao.gov/products/GAO-11-150].
[17] [hyperlink, http://www.gao.gov/products/GAO-11-150].
[18] Carnegie Mellon Software Engineering Institute, Capability
Maturity Model® Integration for Services (CMMI-SVC), version 1.3
(November 2010).
[19] GAO, Information Technology: Better Informed Decision Making
Needed on Navy's Next Generation Enterprise Network Acquisition,
[hyperlink, http://www.gao.gov/products/GAO-11-150] (Washington, D.C.:
Mar. 11, 2011).
[20] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March
2009); Defense Acquisition University, Defense Acquisition Guidebook
(accessed January 10, 2012); and DON Acquisition and Capabilities
Guidebook (May 2012).
[21] [hyperlink, http://www.gao.gov/products/GAO-11-150].
[22] Carnegie Mellon Software Engineering Institute, Capability
Maturity Model® Integration for Services (CMMI-SVC), version 1.3
(November 2010).
[End of section]
GAO’s Mission:
The Government Accountability Office, the audit, evaluation, and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the
performance and accountability of the federal government for the
American people. GAO examines the use of public funds; evaluates
federal programs and policies; and provides analyses, recommendations,
and other assistance to help Congress make informed oversight, policy,
and funding decisions. GAO’s commitment to good government is
reflected in its core values of accountability, integrity, and
reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each
weekday afternoon, GAO posts on its website newly released reports,
testimony, and correspondence. To have GAO e-mail you a list of newly
posted products, go to [hyperlink, http://www.gao.gov] and select
“E-mail Updates.”
Order by Phone:
The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black
and white. Pricing and ordering information is posted on GAO’s
website, [hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
Connect with GAO:
Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov].
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm];
E-mail: fraudnet@gao.gov;
Automated answering system: (800) 424-5454 or (202) 512-7470.
[End of document]
Congressional Relations:
Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, DC 20548.
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, DC 20548.