This is the accessible text file for GAO report number GAO-14-522R
entitled 'Department of Defense's Waiver of Competitive Prototyping
Requirement for the Air Force's B-2 Defensive Management System
Modernization Program' which was released on April 22, 2014.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
GAO-14-522R:
United States Government Accountability Office:
GAO:
441 G St. N.W.
Washington, DC 20548:
April 22, 2014:
Congressional Committees:
Department of Defense's Waiver of Competitive Prototyping Requirement
for the Air Force's B-2 Defensive Management System Modernization
Program:
The Weapon Systems Acquisition Reform Act of 2009, as amended (WSARA),
requires the Secretary of Defense to modify guidance to ensure that
the acquisition strategy for each major defense acquisition program
provides for competitive prototypes before Milestone B approval--which
authorizes entry into system development--unless the Milestone
Decision Authority waives the requirement.[Footnote 1] Competitive
prototyping, which involves commercial, government, or academic
sources producing early prototypes of weapon systems or critical
subsystems, can help Department of Defense (DOD) programs reduce
technical risk, refine requirements, validate designs and cost
estimates, and evaluate manufacturing processes prior to making major
commitments of resources. It can also help reduce the time it takes to
field a system, and as a result, reduce its acquisition cost. WSARA
states that the Milestone Decision Authority may waive the competitive
prototyping requirement only on the basis that (1) the cost of
producing competitive prototypes exceeds the expected life-cycle
benefits (in constant dollars) of producing such prototypes, including
the benefits of improved performance and increased technological and
design maturity that may be achieved through competitive prototyping;
or (2) but for such a waiver, DOD would be unable to meet critical
national security objectives.
WSARA also provides that whenever a Milestone Decision Authority
authorizes a waiver of the competitive prototyping requirement on the
basis of what WSARA describes as "excessive cost," the Milestone
Decision Authority is required to submit notification of the waiver,
together with the rationale, to the Comptroller General of the United
States at the same time it is submitted to the congressional defense
committees. WSARA further provides that no later than 60 days after
receipt of a notification of a waiver, we are mandated to review the
rationale for the waiver and submit a written assessment of that
rationale to the congressional defense committees.[Footnote 2]
On December 3, 2013, we received notice from DOD that it had waived
the competitive prototyping requirement for the Air Force's B-2
Defensive Management System Modernization (DMS-M) program. The Air
Force's B-2 bomber is a low-observable, long-range strike aircraft
capable of entering heavily defended areas to deliver both
conventional and nuclear weapons. The defensive management system
detects, identifies, and locates enemy radar systems and provides real-
time threat avoidance, threat warning, and threat situational
awareness information to the aircrew. The DMS-M program is expected to
upgrade the aircraft's analog defensive management system to a digital
capability, update pilot displays, enhance the in-flight capability to
avoid unanticipated air defense threats, and improve frequency
coverage and the reliability and maintainability of the system. The
DMS-M consists of three critical subsystems--antenna, electronic
support measures, and avionics and graphics processors--that will be
integrated onto existing B-2 aircraft. The program is in the second of
two technology development phases and plans to gain approval to enter
system development in late fiscal year 2015.
In this report, we assess DOD's rationale for waiving the competitive
prototyping requirement for the B-2 DMS-M program and the analysis
used to support it. To conduct our assessment, we compared the
rationale in the waiver to the WSARA requirement to determine the
extent to which the waiver is consistent with the statute. In
addition, we reviewed the Air Force's cost-benefit analysis, which
provides the data and assumptions on which the waiver is based, the
technology development strategy, and other relevant documentation. We
did not independently verify the Air Force's data on cost and
benefits. We also submitted written questions to B-2 program officials
to clarify information in program documentation, as necessary.
We conducted this performance audit from March 2014 to April 2014 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Results in Brief:
DOD's rationale for waiving WSARA's competitive prototyping
requirement for the B-2 DMS-M program addresses one of the two bases
provided in the statute; namely that the cost of producing competitive
prototypes exceeds the expected life-cycle benefits (in constant
dollars) of producing the prototypes. The Air Force's B-2 DMS-M
program is well into technology development and according to the cost-
benefit analysis supporting the waiver, there is little opportunity to
gain significant cost savings by prototyping. According to the Air
Force, the prime contractor has already held competitions and selected
suppliers that will leverage existing, technically mature subsystems
for two of three DMS-M critical subsystems. The prime contractor has
also conducted competitive prototyping for antennas, which are not yet
technically mature, to reduce technical, cost, and schedule risk. In
its cost-benefit analysis, the Air Force concluded that it will
achieve significant operations and support cost reductions through its
strategy of leveraging existing subsystems for the DMS-M and any
additional improvements from prototyping would be on the margins. In
the waiver, DOD concluded that the Air Force's prototyping cost-
benefit analysis was reasonable. We also found that the Air Force's
cost-benefit analysis was consistent with key principles in DOD's
policy on economic analysis, which states that each feasible
alternative for meeting an objective must be considered and its life-
cycle costs and benefits evaluated.[Footnote 3] The Air Force analysis
examined four different scenarios including the system-and subsystem-
level prototyping options outlined in WSARA and DOD's implementing
guidance. The Air Force concluded these strategies would increase the
program's development costs by between $28.2 million and $524.8
million (in base year 2011 dollars) depending on the type and number
of prototypes. The Air Force also estimated that the various
prototyping strategies could achieve an estimated $1.3 million to $6.3
million (in base year 2011 dollars) in life-cycle benefits by
improving the reliability of the system or key subsystems.
Waiver Rationale Is Consistent with WSARA and the Supporting Analysis
Considered a Reasonable Set of Prototyping Alternatives:
DOD's rationale for waiving WSARA's competitive prototyping
requirement for the B-2 DMS-M program addressed one of the two bases
provided for a waiver in the statute, namely that the cost of
producing competitive prototypes exceeds the expected life-cycle
benefits, including the benefits of improved performance and increased
technological and design maturity that may be achieved through
competitive prototyping. The Air Force's B-2 DMS-M program is well
into technology development and according to the cost benefit analysis
supporting the waiver, there is little opportunity to gain significant
cost savings by prototyping. During technology development, the prime
contractor Northrop Grumman held competitions and selected suppliers
for critical DMS-M subsystems. According to the Air Force, the chosen
suppliers for the electronic support measures and avionics and
graphics processors subsystems will leverage existing, technically
mature subsystems and will also produce single prototypes of these
subsystems during technology development. During technology
development, the prime contractor also conducted competitive
prototyping for B-2 DMS-M antennas, which are not yet technically
mature, to reduce technical, cost, and schedule risk. According to the
Air Force, there were four competitive prototypes produced for one
part of the antenna subsystem and three competitive prototypes
produced for the other part. In its cost-benefit analysis, the Air
Force concluded that its strategy of leveraging existing subsystems
for the DMS-M will achieve a 47 percent or $155.8 million (in base
year 2011 dollars) reduction in operations and support costs over the
current system and any additional improvements from prototyping would
be on the margins. We also noted that the Air Force plans to use a
cost-type contract for system development, which, according to DOD
acquisition regulations, requires a written determination by the
Milestone Decision Authority that a program is so complex and
technically challenging that it would not be practicable to reduce
program risk to a level that would permit the use of a fixed-price
type contract.[Footnote 4] The Air Force recommended a cost-type
contract because of uncertainty related to software development,
system integration, testing, and potential funding volatility.
In the waiver, DOD found reasonable the Air Force's cost-benefit
analysis, which examined multiple prototyping strategies including
system-and subsystem-level prototyping. The Air Force's cost-benefit
approach was consistent with key principles in DOD's policy on
economic analysis, which states that each feasible alternative for
meeting an objective must be considered and its life-cycle costs and
benefits evaluated.[Footnote 5] In all, the Air Force examined four
different scenarios, including the system-and subsystem-level
prototyping options outlined in WSARA and DOD's implementing guidance,
and compared the life-cycle cost of each prototyping scenario to the
potential life-cycle benefits. The Air Force concluded these
strategies would increase the program's development costs by between
$28.2 million to $524.8 million (in base year 2011 dollars) depending
on the type and number of prototypes. The Air Force's prototyping cost
estimates were based primarily on the actual and planned cost of the
program's two technology development phases. For example, the Air
Force assumed a second contractor would need to develop a DMS-M system
prototype at the same cost as Northrop Grumman's system delivered
during technology development. In addition, the system-level
competitive prototyping scenario included $54 million for data rights
because access to proprietary Northrop Grumman data is needed to act
as the system integrator for B-2 related efforts. According to the Air
Force, this estimate was developed using historic cost estimating
factors. In practice, Northrop Grumman is unwilling to provide a price
to the government for the data, so this prototyping scenario might not
be feasible. The Air Force also estimated that the various prototyping
strategies could achieve an estimated $1.3 million to $6.3 million (in
base year 2011 dollars) in life-cycle benefits by improving the
reliability of the system or key subsystems. According to the Air
Force, it also considered the positive impacts of prototyping with
respect to integration and production costs, but it did not include a
dollar value for these benefits in any of its prototyping scenarios.
Agency Comments and Our Evaluation:
We provided a draft of this report to DOD for comment. In response,
DOD stated that it had no comments.
We are sending copies of this report to interested congressional
committees, the Secretary of Defense and the Secretary of the Air
Force. In addition, the report will be available at no charge on the
GAO website at [hyperlink, http://www.gao.gov].
If you or your staff have any questions, please contact me at (202)
512-4841 or sullivanm@gao.gov. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last
page of this report. GAO staff who made key contributions to this
report were Ron Schwenn, Assistant Director; Kristine Hassinger;
Matthew B. Lea; Kenneth E. Patton; Carol D. Petersen; and Don M.
Springman.
Signed by:
Michael J. Sullivan:
Director, Acquisition and Sourcing Management:
List of Committees:
The Honorable Carl Levin:
Chairman:
The Honorable James M. Inhofe:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Richard J. Durbin:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable Howard P. "Buck" McKeon:
Chairman:
The Honorable Adam Smith:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable Rodney Frelinghuysen:
Chairman:
The Honorable Pete Visclosky:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
[End of section]
Footnotes:
[1] Pub. L. No. 111-23, § 203(a), as amended by the Ike Skelton
National Defense Authorization Act for Fiscal Year 2011, Pub. L. No.
111-383, § 813. DOD modified its guidance related to the operation of
its acquisition system through Directive-Type Memorandum (DTM) 09-027,
"Implementation of Weapon Systems Acquisition Reform Act of 2009,"
(Dec. 4, 2009, incorporating Change 4, Jan. 11, 2013). This DTM also
defined major defense acquisition programs as those estimated by DOD
to require an eventual total expenditure for research, development,
test, and evaluation or for procurement--including all planned
increments--of more than $365 million or more than $2.19 billion,
respectively, in fiscal year 2000 constant dollars. DOD issued an
Interim Instruction 5000.02, "Operation of the Defense Acquisition
System," on November 25, 2013 which canceled DTM 09-027 and modified
the thresholds for major defense acquisition programs. However for
purposes of this report we used DTM 09-027 because the competitive
prototyping waiver for the B-2 Defensive Management System
Modernization program was completed prior to issuance of the Interim
DOD Instruction. The Milestone Decision Authority for major defense
acquisition programs is the Under Secretary of Defense for
Acquisition, Technology, and Logistics; the head of a DOD component;
or, if delegated, the component acquisition executive.
[2] Pub. L. No. 111-23, § 203(b)(1) & (2).
[3] DOD Instruction 7041.3, Economic Analysis for Decisionmaking,
Encl. 3, para. E3.1.1. (Nov. 7, 1995).
[4] Defense Federal Acquisition Regulation Supplement §
234.004(2)(i)(C)(1). Cost-reimbursement contracts, also known as cost-
type contracts, provide for payment of allowable incurred costs, to
the extent prescribed in the contract. This contract type places most
of the risk on the government, which may pay more than budgeted should
incurred costs be more than expected when the contract is signed.
[5] DOD Instruction 7041.3, Economic Analysis for Decisionmaking,
Encl. 3, para. E3.1.1. (Nov. 7, 1995).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation, and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the
performance and accountability of the federal government for the
American people. GAO examines the use of public funds; evaluates
federal programs and policies; and provides analyses, recommendations,
and other assistance to help Congress make informed oversight, policy,
and funding decisions. GAO's commitment to good government is
reflected in its core values of accountability, integrity, and
reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's website [hyperlink, http://www.gao.gov]. Each
weekday afternoon, GAO posts on its website newly released reports,
testimony, and correspondence. To have GAO e-mail you a list of newly
posted products, go to [hyperlink, http://www.gao.gov] and select
"E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO's actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black
and white. Pricing and ordering information is posted on GAO's
website, [hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
Connect with GAO:
Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov].
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm];
E-mail: fraudnet@gao.gov;
Automated answering system: (800) 424-5454 or (202) 512-7470.
Congressional Relations:
Katherine Siggerud, Managing Director, siggerudk@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, DC 20548.
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, DC 20548.
[End of document]