This is the accessible text file for GAO report number GAO-12-311
entitled 'Homeland Defense: Continued Actions Needed to Improve
Management of Air Sovereignty Alert Operations' which was released on
January 31, 2012.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Requesters:
January 2012:
Homeland Defense:
Continued Actions Needed to Improve Management of Air Sovereignty
Alert Operations:
GAO-12-311:
GAO Highlights:
Highlights of GAO-12-311, a report to congressional requesters.
Why GAO Did This Study:
In the 11 years since September 11, 2001, the U.S. government has put
forth extensive efforts to protect the nation’s aviation sector and
airspace. These efforts include air sovereignty alert (ASA)
operations, for which the Air Force provides personnel and fully
fueled, fully armed aircraft sitting on constant alert at 18 sites
across the United States. In 2009, GAO found shortcomings in the
Department of Defense’s (DOD) management of ASA operations, leading to
a number of GAO recommendations. For this report, GAO examined the
extent to which (1) the Air Force has implemented GAO’s 2009
recommendations, (2) DOD has implemented a risk-based management
approach for ASA operations, and (3) the Air Force has accurately
identified expenditures for ASA operations. To do so, GAO analyzed
relevant strategies, planning documents, guidance, and expenditure
data; and interviewed North American Aerospace Defense Command
(NORAD), Air Force, National Guard Bureau, and other DOD officials.
What GAO Found:
The Air Force has not fully implemented the recommendations from GAO’s
2009 report. With regard to GAO’s recommendation that the military
services should formally assign ASA duties to the units that
consistently conduct them and ensure that the readiness of those units
is fully assessed, the Air Force did so. However, the National Guard
Bureau is considering reversing that action because it believes that
the recommendation can be better addressed through the Air Force’s
standard deployment process. The Air Force has also not established a
timetable to implement ASA as a steady-state mission; has not
developed and implemented a plan to recapitalize the aging fighter
aircraft that conduct ASA operations before the end of their service
lives; and, when ASA units are deployed to support other ongoing
operations, the Air Force continues to identify replacement units to
perform the ASA mission on an ad hoc basis. All of the above were
related to recommendations GAO made to the Air Force in its 2009
report. Separately, GAO found considerable confusion about the
capabilities associated with ASA operations in part because, in
September 2011, NORAD stopped using the term “air sovereignty alert”
and created a new term, “aerospace control alert” (ACA), without
clearly defining ACA or the missions that are now included within it.
DOD has taken a series of actions for ASA operations that are
consistent with a risk-based management approach. However, several key
actions have yet to be taken that would enable the department to
better balance risk and costs. Risk-based management includes
conducting routine risk assessments that evaluate threats,
vulnerabilities, and criticality of assets, as recommended in GAO’s
2009 report, and selecting between alternative courses of action to
mitigate risk and make decisions about allocating resources. Although
threats to the nation’s air sovereignty continue to emerge and evolve,
GAO found that DOD is unable to measure the extent to which ASA helps
to achieve the department’s homeland-defense goal of securing the
United States from direct attack because DOD has not established
performance measures. NORAD has not conducted routine risk assessments
of ASA operations. DOD has also yet to conduct a cost-benefit analysis
for two of the three alternatives to current ASA operations that GAO
evaluated. Adopting a more-rigorous risk-based management approach—-
including balancing risk and costs-—would help policymakers within DOD
and elsewhere more effectively allocate finite DOD resources.
Weak internal controls limit the ability of the Air Force and National
Guard Bureau to accurately identify ASA expenditures. GAO analyzed the
fiscal year 2010 expenditure information that the Air Force and
National Guard Bureau submitted to Congress along with DOD’s fiscal
year 2012 budget justification and found the reported expenditures of
more than $246 million to be inaccurate. For example, GAO found that
the Air Force overstated ASA flying-hour expenditures by at least $22
million and included expenditures related to national special-security
events, which are not part of ASA operations. GAO found that the Air
Force’s ability to identify ASA expenditures is limited by unclear
roles and responsibilities for programming and budgeting and a lack of
guidance on defining and tracking ASA expenditures. These types of
internal controls are important to ensuring basic accountability,
maintaining funds control, and preventing fraud and abuse.
What GAO Recommends:
Congress may wish to consider requiring the Air Force to fully
implement GAO’s 2009 recommendations. In addition, GAO recommends that
DOD improve its risk management of ASA operations and improve the Air
Force’s ability to accurately identify ASA expenditures. DOD fully or
partially agreed with all of GAO’s recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-12-311]. For more
information, contact Brian J. Lepore at (202) 512-4523 or
leporeb@gao.gov.
[End of section]
Contents:
Letter:
Background:
The Air Force Has Not Fully Implemented the Recommendations from Our
2009 Report, and Mission Confusion Persists:
DOD Has Not Fully Implemented a Risk-Based Management Approach That
Balances Risk and Costs for ASA Operations:
Weak Internal Controls Limit the Ability of the Air Force and National
Guard Bureau to Accurately Identify ASA Expenditures:
Conclusions:
Matter for Congressional Consideration:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Air Force's Implementation of Our 2009 Recommendations as
of November 2011:
Appendix III: Comments from the Department of Defense:
Appendix IV: GAO Contacts and Staff Acknowledgments:
Related GAO Products:
Table:
Table 1: Nonexhaustive GAO-Developed List of Potential Measures of
Effectiveness and Measures of Performance for ASA Operations:
Figures:
Figure 1: GAO's Risk-Based Management Framework:
Figure 2: National Infrastructure Protection Plan's Risk-Management
Framework:
Figure 3: Air Force's Implementation of Our 2009 Recommendations as of
November 2011:
Figure 4: Air Sovereignty Alert (ASA) versus Aerospace Control Alert
(ACA):
Abbreviations:
ACA: Aerospace Control Alert:
ASA: Air Sovereignty Alert:
DOD: Department of Defense:
NORAD: North American Aerospace Defense Command:
OMB: Office of Management and Budget:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
January 31, 2012:
Congressional Requesters:
Since the September 11, 2001, terrorist attacks, the U.S. government
has taken extensive efforts to protect the aviation sector and
airspace over the United States--including the establishment of
restricted flight zones; installation of hardened cockpit doors;
enhanced screening of airline passengers and baggage before flights;
increased use of air marshals on selected flights; the vigilance of
civilian passengers and flight attendants aboard commercial aircraft;
and having fully fueled, fully armed jets ready to scramble at any
time in response to airborne threats. As a result of these efforts,
the Transportation Security Administration believes that a hijacking
scenario over the United States remains a possibility although it is
less likely to occur today. Other threats to the nation's air
sovereignty continue to emerge and evolve.[Footnote 1] As a part of
the federal government's overall efforts to protect U.S. airspace, the
Department of Defense (DOD) performs Operation Noble Eagle, which
consists of several operations, including the maintenance of fighter
aircraft and trained personnel on alert 24 hours a day, 365 days a
year, at 18 alert sites across the United States.[Footnote 2] Since
its inception in 2003, these efforts have been commonly referred to as
air sovereignty alert (ASA) operations. The ASA sites serve to deter,
respond to, and if necessary, defeat airborne threats over the United
States and Canada. In effect, these fighter jets can be the last line
of multiple layers of defense.
As a component of Operation Noble Eagle, ASA operations are conducted
by several DOD components under the command of North American
Aerospace Defense Command (NORAD) for the airspace over North America,
which includes Alaska and the continental United States, and under the
command of U.S. Pacific Command for the airspace over the state of
Hawaii and U.S. territories in the Pacific. The Joint Chiefs of Staff
have directed the Air Force (among other components) to provide
support for ASA operations. Generally, Air National Guard aircraft and
personnel are used for alert, since the National Guard Bureau believes
that ASA operations are consistent with the Air National Guard's
homeland mission. Both the Air Force's Air Combat Command and the
National Guard Bureau[Footnote 3] believe that Air National Guard
units can conduct alert duty with less effect on overseas missions and
at a lower cost than active-duty Air Force units. Consequently, Air
National Guard units are on alert at 17 of the 18 ASA sites. In
addition, units at all 18 ASA sites (i.e., both Air National Guard and
active-duty personnel) are dual-tasked to conduct both expeditionary
missions, such as Operation Enduring Freedom in Afghanistan, and ASA
operations.
In January 2009, we issued a report that identified a number of
shortcomings in DOD's management of ASA operations.[Footnote 4] First,
we found that NORAD faced difficulty determining the appropriate
levels and types of units, personnel, and aircraft for ASA operations.
We also found that, after 8 years of operations, the Air Force had not
yet established ASA as a steady-state mission, which would include
integrating ASA operations into the Air Force's planning, programming,
and budgeting cycle. Partly resulting from the absence of a steady-
state mission, NORAD had only partially assessed the readiness of
units that consistently conduct ASA operations to ensure that those
units were organized, trained, and equipped to perform ASA operations.
In addition, we found that the Air Force faced significant challenges
replacing or extending the service life of aging fighter aircraft, to
the extent that, if aircraft were not replaced, 11 of the 18 ASA sites
could be without aircraft by 2020. On the basis of those findings, we
identified a number of actions that DOD needed to take to improve
management of ASA operations. Specifically, we recommended that DOD
should conduct routine risk assessments as part of a risk-based
management approach, implement ASA as a steady-state mission, and
develop plans to address future challenges.
GAO and other federal agencies have long advocated the use of a risk-
based management approach to enhance preparedness against potential
terrorist threats.[Footnote 5] Risk management is a systematic,
analytical process to determine the likelihood that a threat will harm
physical assets or individuals and then to identify actions to reduce
risk and mitigate the consequences of an attack. The principles of
risk management acknowledge that while risk generally cannot be
eliminated, enhancing protection from known or potential threats can
serve to reduce risk. Key elements of risk management include
assessing threats, vulnerabilities, and criticality of assets and
selecting between alternative courses of action to mitigate risk. As
we have previously reported, risk management can help policymakers
make informed decisions, take actions, and prioritize resource
investments under conditions of uncertainty. For additional
information, see the Related GAO Products list at the end of this
report.
On the basis of your request that we review DOD's actions on our
January 2009 recommendations, other issues associated with ASA
operations, and the Air Force's ability to sustain those capabilities,
this report examines the extent to which (1) the Air Force has
implemented the recommendations made to the service in our 2009
report, (2) DOD has implemented a risk-based management approach for
ASA operations--including our 2009 recommendation for routine risk
assessments--that balances risk with cost, and (3) the Air Force
accurately identified expenditures for its ASA operations. We are also
issuing a classified version of this report. That version includes a
classified appendix of our analysis of NORAD's ASA basing model.
To determine the extent to which the Air Force has implemented the
recommendations from our 2009 report, we reviewed ASA-related
guidance, plans, and other documents from the Office of the Assistant
Secretary of Defense for Homeland Defense and Americas' Security
Affairs, Air Force's homeland defense office, Air Force's Air Combat
Command, National Guard Bureau, and 3 of the 18 ASA units. We also
interviewed officials from these organizations. To determine the
extent to which DOD has adopted a risk-based management approach to
balance risk and costs for ASA operations, we evaluated documents and
interviewed officials from NORAD, the Office of the Assistant
Secretary of Homeland Defense and Americas' Security Affairs, the Air
Force, and the National Guard Bureau. On the basis of our analyses of
these documents and discussions, we compared actions that DOD has
taken to three of the five phases within GAO's risk-based management
framework.[Footnote 6] To determine the extent to which the Air Force
was able to accurately identify ASA expenditures, we analyzed the
fiscal year 2010 expenditure information the Air Force included in ASA
budget justification materials provided to Congress in support of the
fiscal year 2012 budget and documented the steps taken to identify
those expenditures through interviews, e-mails, and related
documentation. We also compared the practices used by the Air Force to
prepare the ASA budget justification against best practices for
producing cost estimates.[Footnote 7] To determine the expenditures of
each ASA site, we developed a survey that we sent out to the Air
National Guard and active-duty Air Force ASA units to gather fiscal
year 2009 and 2010 expenditures. We were provided information from 18
of the 19 ASA sites that incurred expenditures for ASA
operations.[Footnote 8] Of the 18 sites that provided us information,
3 of the sites provided us incomplete information.[Footnote 9]
We conducted this performance audit from March 2011 to January 2012 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives. Additional
information on our scope and methodology appears in appendix I.
Background:
ASA Operations:
ASA operations consist of ground operations that take place before
fighter aircraft take off, including such activities as maintaining
the fighter aircraft. They also include those activities that may take
place after a unit receives an alert from NORAD, but before the
aircraft are airborne. For example, pilots and maintenance personnel
may rush from their nearby lodging facility to the alert aircraft
facility, where maintenance personnel conduct final preparations while
the pilots sit in their aircraft awaiting further instruction (battle
station). Alternatively, pilots may taxi the aircraft to the end of
the runway and await further instruction (runway alert) or take off in
response to the alert (scramble).
Once aircraft take off, an Air National Guard pilot converts from
Title 32 status under the command and control of the state governor to
federal Title 10 status under the command and control of NORAD.
[Footnote 10] Within limits, NORAD can increase personnel, aircraft,
and the number of ASA sites based on changes in the threat conditions.
Risk-Based Management Approach:
GAO has previously reported that a risk-based management approach
helps policymakers make informed decisions and prioritize resource
investments. Risk management is a widely endorsed strategy for helping
decision makers allocate finite resources and take action under
conditions of uncertainty. We have previously recommended a five-phase
risk-based management approach as shown in figure 1.
Figure 1: GAO's Risk-Based Management Framework:
[Refer to PDF for image: illustration]
GAO's Risk Management Framework:
[Refer to PDF for image: illustration]
Interlocking circular segments:
Strategic Goals, Objectives, and Constraints:
Risk Assessment:
Alternatives Evaluation:
Management Selection:
Implementation and Monitoring:
Source: GAO.
[End of figure]
The framework is dynamic and new information can be entered at any
phase. The framework can be used to inform agency officials and
decision makers of the basic components of a risk-based management
approach or can be used as a stand-alone guide. The risk-based
management approach, as outlined above is designed to be flexible in
that the approach may be applied at various organizational levels from
a department or a multiagency organization down to specific projects
or operations, such as ASA operations.
As we previously reported, because there is no one uniformly accepted
approach to risk management, terms and activities may differ across
applications. For example, the Department of Homeland Security's 2009
National Infrastructure Protection Plan identifies a six-phase risk-
management framework as shown in figure 2.[Footnote 11] The overall
difference between this framework and GAO's is that the National
Infrastructure Protection Plan's framework splits GAO's risk-
assessment phase into two separate phases.
Figure 2: National Infrastructure Protection Plan's Risk-Management
Framework:
[Refer to PDF for image: illustration]
Assets, systems, and networks: (Physical, Cyber, Human:
Set security goals;
Identify asses, systems, networks, and functions;
Assess risks (consequences, vulnerabilities, and threats);
Prioritize;
Implement programs;
Measure effectiveness.
Continuous feedback: occurs at each step.
Source: Department of Homeland Security.
[End of figure]
According to the 2009 National Infrastructure Protection Plan, the
core criteria for risk assessments include information needed to
understand and address each of the following components of risk--
threat, vulnerability, and consequence. Any approach that omits the
substance of the steps may result in resources that are not targeted
to the highest security needs. In addition, failing to monitor the
implementation of countermeasures, including those implemented by
other agencies, may result in a misallocation of resources. Similarly,
failing to conduct routine or periodic assessments of programs or
operations could result in missed opportunities to increase their
efficiency and effectiveness.
DOD, itself, has identified risk management, in this context, as a
process of systematically identifying, assessing, and managing risks
arising from operational factors and making decisions that balance
cost with mission benefits. For example, DOD's defense critical
infrastructure program is a risk-management program that seeks to
ensure the availability of certain identified assets critical to DOD
missions. Activities include the identification, assessment, and
security enhancement of assets essential for executing the National
Military Strategy. The Defense Logistics Agency similarly established
its enterprise risk-management approach to identify, assess,
communicate, and manage risk in a cost-effective manner.
The Air Force Has Not Fully Implemented the Recommendations from Our
2009 Report, and Mission Confusion Persists:
The Air Force has fully implemented one recommendation from our 2009
report, but has not fully implemented the remaining four
recommendations. In addition, confusion over the definition and scope
of the mission perpetuates confusion about the capabilities and
expenditures associated with the mission.
Air Force Has Taken Partial Steps to Implement Recommendations:
As of November 2011, the Air Force had fully implemented one of the
five recommendations from our 2009 report[Footnote 12] and taken
partial steps to implement the remaining four recommendations, as
shown in figure 3 below. We made the following recommendations: (1) to
formally assign ASA duties to the appropriate units; (2) to implement
ASA as a steady-state mission; (3) to establish a timetable for
implementation of the preceding recommendation; (4) to develop and
implement a plan to address any projected capability gaps; and (5) to
develop and implement a formal method to replace deploying units.
Figure 3: Air Force's Implementation of Our 2009 Recommendations as of
November 2011:
[Refer to PDF for image: interactive table]
Interactivity instructions:
Click on a recommendation to see the actions taken. Click on the “x”
to clear. See appendix II for the non-interactive, printer-friendly
version.
Recommendation: We recommend that the Secretary of Defense direct the
military services with units that consistently conduct ASA operations
to formally assign ASA duties to these units and then ensure that the
readiness of these units is fully assessed, to include personnel,
training, equipment, and ability to respond to an alert;
Grade: Recommended action complete;
Actions taken: Refer to Appendix II.
Recommendation: We recommend that the Secretary of Defense direct the
Secretary of the Air Force to implement ASA as a steady state
mission according to NORAD, DOD, and Air Force guidance by:
(1) incorporating ASA operations within the Air Force submissions for
the 6-year Future Years Defense Program; (2) updating the Air Force
homeland defense policy, homeland operations doctrine, and concept of
operations to incorporate and define the roles and responsibilities
for ASA operations; and; (3) updating and implementing the ASA program
action directive; Grade: Recommended action complete;
Actions taken: Refer to Appendix II.
Recommendation: We recommend that the Secretary of Defense direct the
Secretary of the Air Force to establish a timetable to implement ASA
as a steady-state mission [by taking the actions identified in the
preceding recommendation];
Grade: [Empty];
Actions taken: Refer to Appendix II.
We recommend that the Secretary of Defense direct the Secretary of the
Air Force to develop and implement a plan to address any projected
capability gaps in ASA units due to the expected end of the useful
service lives of their F-15 and F-16 fighter aircraft;
Grade: Recommended action underway;
Actions taken: Refer to Appendix II.
Recommendation: We recommend that the Secretary of Defense direct the
Secretary of the Air Force to develop and implement a formal method to
replace deploying units that still provides unit commanders
flexibility to coordinate replacements;
Grade: Recommended action underway;
Actions taken: Refer to Appendix II.
Source: GAO analysis of DOD information.
[End of figure]
We continue to believe that the prior recommendations have merit and
that by not fully addressing the recommendations, DOD could leave the
long-term sustainability of ASA operations at risk. Since the 2009
report was issued, some issues have arisen that might have been
averted had the Air Force implemented the recommendations. For
example, since the Air Force canceled the ASA program action directive
that clearly identified the service's Air Combat Command as
responsible for programming for ASA operations, the National Guard
Bureau assumed programming and budgeting for its ASA sites to ensure
that its units that were conducting ASA operations were funded. This
in turn has led to the National Guard Bureau having to fund emergent
expenses with funds that were programmed for other purposes. For
example, according to National Guard Bureau officials, when an ASA
site is undergoing certain unplanned construction, the National Guard
Bureau must fund the transfer of ASA personnel and equipment to a
separate base.
Undefined New Terminology Regarding ASA Operations Perpetuates
Confusion about the Mission:
Further complicating DOD's management approach to ASA operations, we
found considerable confusion about the capabilities and expenditures
associated with ASA operations. In September 2011, NORAD stopped using
the term "air sovereignty alert" (ASA) and created a new term,
"aerospace control alert" (ACA) without clearly defining ACA or the
capabilities that are now included within it. When we asked NORAD
officials which capabilities are included under ACA, they told us that
ACA includes--but is not limited to--DOD's traditional scope of ASA
operations (i.e., the dedicated fighter aircraft and personnel at 18
steady-state alert sites across the United States), as well as a more-
comprehensive list of capabilities used in their aerospace control
mission.[Footnote 13] As figure 4 shows, according to NORAD, ACA
capabilities include air defense and air sovereignty operations
personnel and equipment at command and control centers at NORAD and
elsewhere, ground-based defense systems, air patrols, airborne radar,
and air-to-air refuelers--in addition to ASA fighters sitting on
ground alert. However, Air Force and National Guard Bureau officials
told us that while they were uncertain as to the definition of ACA,
they believe the new term is interchangeable with ASA (i.e., the
dedicated fighter aircraft and personnel at 18 steady-state alert
sites across the United States).
Figure 4: Air Sovereignty Alert (ASA) versus Aerospace Control Alert
(ACA):
[Refer to PDF for image: interactive illustration with 10 photographs]
Interactivity instructions: Click on a photo thumbnail to view larger.
Air Sovereignty Alert:
Fighters on ground alert.
Aerospace Control Alert:
Aerospace Control is the combination of air sovereignty mission (i.e.,
know and control what is flying in U.S. and Canadian airspace) and air
defense mission (i.e. measures designed to defend against attacking
enemy aircraft or missiles). According to NORAD officials, ACA is the
capabilities (i.e., personnel and equipment) that are on some type of
alert to support the broader mission.
Fighters on ground alert;
Refuelers;
Air patrols;
Airborne radar;
Ground-based defense system;
Joint Air Defense Operations Center (JADOC);
Western Air Defense Sector (WADS) and Eastern Air Defense Sector
(EADS);
Continental NORAD Region Command’s (CONR) and Alaskan NORAD Region
Command’s (ANR) Air Operations Centers;
NORAD/NORTHCOM Command and Control Center.
Source: NORAD and DOD.
[End of figure]
While NORAD's new terminology may better reflect its overall aerospace-
control mission, without a clear definition of what capabilities and
expenditures are related to ACA--compared to ASA--Congress will lack a
complete picture of the funding and other resources needed for DOD to
conduct ASA operations (i.e., the dedicated fighter aircraft and
personnel at 18 steady-state alert sites across the United States).
DOD is required by law to submit a consolidated budget justification
document annually that includes all funding requirements for ASA
operations. As will be discussed in more detail later in this report,
even before the change in terminology, the Air Force and National
Guard Bureau faced difficulties providing Congress with complete and
accurate information regarding expenditures associated with ASA
operations. During our review, officials from NORAD, the Office of the
Assistant Secretary of Defense for Homeland Defense and Americas'
Security Affairs, the Joint Chiefs of Staff, U.S. Pacific Command, the
Air Force, and the National Guard Bureau could not consistently define
what capabilities and expenditures should be considered part of ASA.
Until DOD defines ACA; identifies the specific capabilities that
support it--including personnel and equipment; and distinguishes the
place of ASA within that mission for the purposes of providing
consistent information in response to their budget-justification
requirement, Congress, NORAD, and other entities within DOD could be
making important homeland security decisions based on inconsistent
information.
DOD Has Not Fully Implemented a Risk-Based Management Approach That
Balances Risk and Costs for ASA Operations:
DOD has taken a series of actions for ASA operations that are
consistent with a risk-based management approach; however, several key
actions have yet to be taken to fully implement a risk-based
management approach that would enable the department to better balance
risk and costs.
Risk Management Helps Balance Risk and Costs:
According to our prior work, an ongoing risk-based management approach
is a best practice that enhances an organization's decision making,
including determining operational requirements, and helps to guide the
use of limited resources. We have previously recommended a five-phase
approach to a risk-management framework, including: strategic goals
and objectives, risk assessment, evaluation of alternatives,
management selection, and implementation and monitoring.
Risk management is a widely endorsed strategy for decision makers to
allocate finite resources and take action under conditions of
uncertainty. For example, the 2009 National Infrastructure Protection
Plan states that its risk management framework integrates and
coordinates strategies, capabilities, and governance to enable risk-
informed decision making related to the nation's critical
infrastructure and key resources. This framework is applicable to
threats such as natural disasters, manmade safety hazards, and
terrorism, although different information and methodologies may be
used to understand each.
DOD Has Established and Linked Its Homeland Defense Strategic Goal to
Its ASA Objectives, but Has Not Established Performance Measures:
DOD's overarching homeland defense goal is to secure the United States
from direct attack, but DOD is unable to measure the extent to which
ASA operations help to achieve that goal. In its 2005 Strategy for
Homeland Defense and Civil Support, DOD linked homeland defense with
several objectives associated with ASA operations--including
deterring, detecting, and if necessary, defeating airborne threats.
[Footnote 14] While these actions are initial steps in the first phase
of a risk-based management approach, NORAD has not identified
performance measures that could identify the extent to which its ASA
operations are progressing toward DOD's homeland defense goal and
objectives.[Footnote 15] While U.S. Pacific Command has established
one type of performance measure identified in Joint Chiefs of Staff
doctrine on joint planning (measures of effectiveness), the command
has not established the other type of performance measure (measures of
performance). U.S. Pacific Command officials also recognized that the
command's performance measures may not be comprehensive since its
single ASA site does not deal with the same operational tempo as the
other 17 ASA sites. According to Joint Chiefs of Staff doctrine on
planning, continuous evaluation of operations against measures of
performance and measures of effectiveness to assess the component's
progress toward accomplishing a task or achieving an objective is a
key component of DOD's planning process. Measures of effectiveness are
criteria used to assess changes in system behavior, capability, or
operational environment that are tied to measuring the attainment of
an end state or an objective, or creation of an effect. Measures of
performance are criteria to assess friendly actions that are tied to
measuring task accomplishment. To provide examples of performance
measures that could be applied to ASA operations, we created potential
measures of effectiveness and measures of performance after analyzing
NORAD and DOD planning documents and talking with NORAD and DOD
officials, including ASA units. Table 1 lists some of the potential
measures we developed. NORAD and DOD officials agreed that these
examples could be a viable start for a more comprehensive list of
performance measures to evaluate the effectiveness and performance of
ASA operations.
Table 1: Nonexhaustive GAO-Developed List of Potential Measures of
Effectiveness and Measures of Performance for ASA Operations:
Measures of effectiveness: Number of track-of-interest infractions
within the Special Flight Restricted Area within a certain period of
time;
Measures of performance: Number (or percentage) of times ASA aircraft
launch within Operation Noble Eagle requirements.
Measures of effectiveness: Number and type of aircraft declared to be
of interest within an established Temporary Flight Restricted Area
within a certain period of time;
Measures of performance: Number (or percentage) of times ASA aircraft
launch within NORAD established requirements.
Measures of effectiveness: Number of track-of-interest aircraft that
divert after certain tactics, techniques, and procedures are performed;
Measures of performance: Number (or percentage) of times ASA pilots
are able to accurately identify and communicate track-of-interest
aircraft tail number to command and control entities within 5 minutes.
Measures of effectiveness: Total number of commercial and general-
aviation aircraft declared to be a track of interest within a certain
metropolitan area within a certain period of time;
Measures of performance: Number of Alert Force Evaluations failed each
year.
Source: GAO analysis of DOD data.
[End of table]
Measuring the status of tasks, effects, and objectives becomes the
basis for reports to senior commanders and civilian leaders on the
progress of the operation. NORAD could then advise the President and
the Secretary of Defense accordingly and adjust operations as
required. When we discussed the lack of performance measures with DOD
officials, they acknowledged that such information would be helpful
yet did not exist for ASA operations.
DOD officials pointed out that NORAD evaluates each ASA unit generally
every 20 months. As we previously reported, NORAD's unit assessments
evaluate the quality of alert aircraft, to include the overall
condition of the aircraft, and a unit's ability to respond to
different air sovereignty scenarios.[Footnote 16] While these
assessments evaluate the equipment and operations of a single ASA unit
against a planned scenario, the individual assessments do not provide
NORAD or DOD situational awareness of actual ASA operations that occur
every day (as opposed to planned evaluations); do not allow either
NORAD or DOD to identify trends across ASA units; and do not provide
NORAD or DOD criteria with which to evaluate the effectiveness and
performance of ASA operations against DOD's overall homeland-defense
goal. Without ASA performance measures, NORAD and DOD are limited in
their ability to assess ASA operations.[Footnote 17]
NORAD Has Not Conducted Routine Risk Assessments, and Its ASA Model
Has Limitations:
Although we have previously recommended that NORAD should conduct
routine risk assessments of ASA operations[Footnote 18]--the second
phase of a risk-based management approach--we found that it has not
implemented this recommendation. Specifically, our 2009 ASA report
stated that by undertaking routine risk assessments, NORAD could
better evaluate the extent to which previous threats have been
mitigated by DOD or other government agencies; better evaluate current
and emerging threats to determine which ones require the most urgent
attention; and determine operational requirements to address changing
conditions. Moreover, routine risk assessments could help NORAD
evaluate alternatives to current operations, especially in a resource-
restricted environment. Further, such assessments could enhance
NORAD's ability to determine and apply the appropriate level and type
of resources--including units, personnel, and aircraft--to ASA
operations.
Although NORAD has not conducted routine risk assessments, it took
steps toward implementing our 2009 recommendation by developing a
computer model to evaluate current ASA basing locations. However, we
found that the model had certain limitations that limit its ability to
provide NORAD commanders, DOD decision makers, and Congress risk-based
information. Specifically, it did not include a prioritized list of
metropolitan areas and critical infrastructure locations that NORAD
should protect. The 2009 National Infrastructure Protection Plan
states that identifying and prioritizing key assets to protect, such
as metropolitan areas and critical infrastructure, is a key component
for risk management. According to NORAD officials, the model omitted
this information because DOD had not issued additional guidance since
the initial lists were distributed after the September 11, 2001,
terrorist attacks. The model also did not incorporate data that are
associated with risk: threat, vulnerability, and consequence because
NORAD designed the model to assess the ability of its aircraft to
respond to a single type of threat regardless of its location in the
United States.
We also found some limitations with the assumptions and reliability of
the input data used by NORAD's computer model.[Footnote 19] NORAD
officials agreed with our observations and agreed to take action to
address them. Further, we found that while NORAD officials regularly
run queries using the model, they do not generate reports documenting
the results of their analyses. GAO's Standards for Internal Control in
the Federal Government recommends ongoing monitoring, in addition to
separate evaluations, to assess the quality of performance over time
and ensure that the findings of audits and other reviews are promptly
resolved.[Footnote 20] A report that documents the model's analyses
may allow analysts to routinely identify trends over a period of time
so that commanders could determine the extent to which their
operations are effective or to which a threat might be evolving. We
believe that NORAD's computer model, if properly designed, could
sufficiently address our 2009 recommendation to conduct routine risk
assessments, as well as allow NORAD to assess the specifics of
detailed ASA scenarios. These areas of improvement are consistent with
DOD methodology used in other DOD risk-management efforts, such as
force protection and defense critical infrastructure, as well as the
U.S. Coast Guard's and the Transportation Security Administration's
risk-based efforts. Similarly, The National Strategy for the Physical
Protection of Critical Infrastructures and Key Assets states that
responsible authorities using models and simulations can evaluate the
risk associated with particular vulnerabilities more accurately and
subsequently make more-informed protection decisions.[Footnote 21]
Without conducting routine risk assessments or maintaining a risk-
based computer model that includes key components identified above,
DOD will be unable to identify and assess the level of risk that it is
taking; evaluate its ASA posture as the threat evolves or is mitigated
by other U.S. government efforts; and make risk-based decisions,
especially in a constrained fiscal environment. In addition, without
documenting the results of risk assessments or the analyses that are
completed using data from the computer model, DOD will be limited in
its ability to identify trends in threats, vulnerabilities, and
potential consequence over time.
DOD Has Considered Costs and Benefits of One of Three Alternatives to
ASA Operations:
We identified three alternatives to DOD's current ASA operations that
the department has considered since 2009, and DOD has analyzed costs
and benefits of these alternatives to varying extents. During this
review, we found that DOD has considered the following three
alternatives to its current ASA posture: establish a new ASA site to
support the President while in Chicago; acquire additional
capabilities to address a perceived threat; and modify the current
number and location of ASA sites. Of these three alternatives, we
determined that DOD conducted a cost-benefit analysis for one of the
alternatives, but had not conducted a cost-benefit analysis on the
other two alternatives at the time of our review. According to GAO's
risk-based management framework, weighing costs and benefits is a
critical component of risk management when choosing among alternative
security measures because it links the benefits of alternatives to the
costs associated with implementing and maintaining them. The actions
that DOD should take in the "risk-assessment" phase should be the
basis for identifying and considering alternatives. Capital
investments of federal expenditures generally require a cost-benefit
or cost-effectiveness approach.[Footnote 22] We analyzed the three
alternatives to ASA operations that DOD officials told us they had
considered since 2009:
* Establish a new ASA site to support the President while in Chicago.
DOD evaluated the possibility of establishing a new ASA site in the
Chicago metropolitan area to support the President's visits to his
hometown. In evaluating this option, DOD identified several sites that
were considered and analyzed potential costs for each of these sites.
In addition, DOD analyzed the benefits, such as cost savings, and
drawbacks associated with the various sites and concluded that the
cost outweighed the benefit of an additional ASA site at this location.
* Acquire capabilities to address a perceived threat. NORAD wants DOD
to acquire a number of capabilities for ASA operations to address a
capability gap that NORAD believes exists based on its ASA computer
model. While NORAD has identified some of the potential costs
associated with one of the new capabilities, it did not include all of
the costs associated with that capability; nor did it include costs
associated with other capabilities that could be acquired. However,
since NORAD requested these additional capabilities through DOD's
Joint Capabilities Integration and Development System process, a cost-
benefit analysis is not scheduled to occur until NORAD's proposal has
been assessed and approved as part of that process.[Footnote 23]
* Modify the number and location of ASA sites. In 2010, NORAD analyzed
its ASA basing strategy to identify whether it could change the number
and location of ASA sites without affecting its ability to defend the
homeland from aviation-related attack. NORAD's analysis determined
that the probability of success of an attack increased as the number
of ASA sites decreased. The analysis also identified a "point of
diminishing returns" below which the number of ASA sites could not be
further reduced without assuming an unacceptable level of risk.
However, the analysis did not identify potential cost savings that
could result from eliminating a given number of sites. Should NORAD,
DOD, or Congress consider modifying the number and location of ASA
sites in the future, without an analysis that balances both risk and
costs, decision makers will be unable to make fully informed decisions
about whether the potential cost savings (or increase) warrants the
corresponding increase (or decrease) in risk.
We could not evaluate the extent to which DOD implemented the last two
phases--management selection, and implementation and monitoring--
because (1) DOD had not fully implemented the first three phases of
GAO's risk-based management framework, which are essential for the
last two phases, and (2) DOD was still in the managerial-selection
phase for the alternatives that we identified.
While we are encouraged that DOD has examined some alternatives to its
current ASA operations, adopting a more-rigorous risk-based management
approach--including balancing risk and costs--would help decision
makers within DOD and elsewhere more-effectively allocate finite DOD
resources.
Weak Internal Controls Limit the Ability of the Air Force and National
Guard Bureau to Accurately Identify ASA Expenditures:
Internal control weaknesses limit the Air Force's ability to
accurately identify expenditures with regard to Air Force management
of ASA operations. Without accurate expenditure information, Congress,
NORAD, and DOD will not have an accurate basis for decision making and
realistic budget formulation.
Air Force and National Guard Bureau Officials Have Not Accurately
Identified ASA Expenditures:
We analyzed the fiscal year 2010 expenditure information the Air Force
and National Guard Bureau submitted to Congress[Footnote 24] and found
that it did not accurately identify all ASA expenditures.[Footnote 25]
According to the Air Force's submission to Congress, more than $246
million was expended on ASA operations in fiscal year 2010. While the
Air Force included all of the mandated funding requirements categories
in its budget-justification document, we found three areas in which
the expenditures reported by the Air Force were inaccurate.
First, we found that the Air Force overstated its ASA flying-hour
expenditures by at least $22 million. According to Air Force
officials, they mistakenly used a placeholder for the ASA flying-hour
expenditure of approximately $57.6 million, instead of using the
actual flying-hour expenditure data. During the course of our review,
the Air Force provided us with actual flying-hour data, which showed
that the service expended $35.6 million--38 percent less than reported.
Second, the ASA budget-justification document included expenditures
that were not associated with ASA operations. For example, it included
flying-hour and personnel expenditures related to non-ASA homeland-
defense missions, such as air patrols protecting the President and
national special-security events. Hence, it is likely that the Air
Force further overstated its ASA flying-hour expenditures beyond the
$22 million discussed above.
Third, the budget justification document the Air Force provided to
Congress did not include some expenditures that are related to ASA.
While the budget justification included one military construction
project for a munitions storage unit at the Toledo, Ohio, ASA site, it
did not include any other expenditures associated with constructing,
maintaining, and upgrading ASA facilities in fiscal year 2010. For
example, when asked if the officials included the construction
expenditures related to establishing an ASA site at McEntire Joint
National Guard Base, South Carolina, in fiscal year 2010, such as
constructing hangers for the alert fighter jets and upgrading crew
quarters, the officials stated that these expenditures were not
included in the ASA budget justification. However, we asked budget
officials at each site about ASA expenditures for fiscal year 2010 in
a survey. According to their responses and additional information from
command officials, ASA sites as a whole spent at least $3.8 million in
facilities construction, maintenance, and equipment that was not
included in the budget justification.
When asked to identify the expenditures included in the budget
justification and the process by which the ASA expenditures were
compiled, the Air Force and National Guard Bureau could not answer
many of our questions regarding which detailed expenditures were
included under the expenditure totals listed in the budget
justification. Since the Air Force and National Guard Bureau were
unable to accurately identify the expenditures to conduct ASA
operations, we conducted a survey of all ASA sites in an effort to
identify approximate expenditures for fiscal year 2010. According to
the information obtained from this survey, the expenditures for each
unit to conduct ASA operations during fiscal year 2010 ranged from $2
million to $8 million, with the average site expenditure being
approximately $5 million.
In addition, Air Force and National Guard Bureau officials told us
they did not have documentation or guidance outlining how the ASA
expenditures were defined, identified, and compiled for the budget
justification. DOD is required by various statutes to improve its
financial-management processes, controls, and systems to ensure that
complete, reliable, consistent, and timely information is prepared and
responsive to the financial information needs of agency management and
oversight bodies.[Footnote 26] Additionally, GAO's Standards for
Internal Control in the Federal Government cite the importance of
developing detailed internal-controls policies, procedures, and
practices to ensure basic accountability, maintain funds control,
[Footnote 27] and prevent fraud and abuse.[Footnote 28] Such efforts
should include clearly defined roles and responsibilities and accurate
and complete guidance on defining and tracking ASA expenditures.
Without adequate internal controls in place, the Air Force's ability
to accurately identify ASA expenditures will be limited.
Unclear ASA Roles and Responsibilities Contributed to Weaknesses in
the Identification of ASA Expenditures:
The Air Force and National Guard Bureau lack clearly defined
programming and budgeting roles and responsibilities for ASA
operations, which in turn limited the Air Force's ability to identify
ASA expenditures. We found that the Air Force and National Guard
Bureau have not clearly defined programming and budgeting roles and
responsibilities since the Air Force canceled its 2003 ASA program
action directive in 2009. The directive required the service's Air
Combat Command to program for ASA operations. As a result, Air Combat
Command programmed and budgeted for ASA operations between 2003 and
2009. However, since the directive was canceled and no additional
guidance was issued, Air Combat Command stopped programming, planning,
and budgeting for ASA operations at Air National Guard sites. National
Guard Bureau officials told us that they consequently assumed these
responsibilities. However, Bureau officials have questions about the
bureau's authority to fund and conduct ASA operations in the manner in
which they are currently conducted. We have requested the Air Force's
official position regarding whether the Air National Guard should be
conducting this mission under a Title 10 or Title 32 status. As of
January 2012, the Air Force had not responded to our inquiry. We also
found other internal-control weaknesses, such as a lack of oversight
of funds expended for ASA operations. According to a National Guard
Bureau internal-review report, a lack of oversight and other internal-
control weaknesses, such as poor guidance, led to improper
compensation at several ASA sites that are operated by Air National
Guard units[Footnote 29]. Without clearly defined roles and
responsibilities, questions about who is responsible for programming,
planning, and budgeting will remain; and the ability to accurately
identify ASA expenditures will be limited. In addition, the Air Force
and National Guard Bureau may remain vulnerable to improper use of
limited resources.
The Air Force and National Guard Bureau's Lack of Guidance on Defining
and Tracking ASA Expenditures Limited Its Ability to Accurately
Identify ASA Expenditures:
Air Force and National Guard Bureau financial-management officials
told us that they did not accurately capture the ASA expenditures
because they lacked guidance that defines ASA operations and the
process to track the associated expenditures. First, as mentioned
previously in the report, there is no agreed-upon definition of ASA
operations among NORAD and DOD components that contribute to ASA
operations. Consequently, the financial-management guidance does not
clearly define the expenditures that should be associated with ASA
operations. Instead, the guidance that is provided to the Air Force by
the Office of the Under Secretary of Defense (Comptroller) contains
broad categories that do not clearly state what specific expenditures
should or should not be included in the budget exhibit.[Footnote 30]
For example, the guidance associated with identifying ASA military-
personnel expenditures states that Air Force officials should provide
military-personnel costs associated with the military full-time
equivalents supporting the ASA mission. Since the guidance does not
clearly identify the type or group of military personnel who are
associated with ASA, financial managers could respond to this
reporting requirement with a range of different personnel, such as
personnel that work at NORAD's air-defense sectors and DOD's refueling
units since they assist in ASA operations, but are not necessarily
included within the 18 alert sites.
Second, Air Force and National Guard Bureau financial-management
officials told us that they had not issued any guidance explaining the
process for tracking ASA expenditures at the local level. According to
Air Force guidance, financial managers and budget officers at all
levels must make sure that operating personnel are made aware of the
importance of properly identifying and capturing all costs associated
with emergency and special programs, like ASA, because only accurate
cost data can provide the documentation to meet reporting
requirements. Guidance to the local level on the process of tracking
ASA expenditures would be important because Air Force and National
Guard Bureau officials told us that they develop the ASA budget
justification using the information that the financial managers at the
ASA sites enter into the service's accounting systems. While the
National Guard Bureau's annual financial-management guidance does not
include any information on tracking ASA expenditures, we found that it
does contain specific financial-management guidance for a similar
mission. Specifically, the Air National Guard's counterdrug mission is
similar to the ASA mission in that they both involve Guard personnel
supporting a federal mission and have specific accounting codes
dedicated to tracking their expenditures. When we asked about this
disparity, National Guard Bureau financial-management officials
acknowledged that there was not a good reason for having not included
more guidance about tracking ASA expenditures and told us that the
bureau would address this in the next version of the guidance.
However, a new version of the National Guard Bureau's financial-
management guidance was subsequently issued and it did not include
guidance on tracking ASA expenditures. National Guard Bureau financial-
management officials stated that changes to the guidance will be
driven by a number of factors, including appropriations law, questions
from their comptrollers in the field, and our findings. Without clear
guidance on defining and tracking ASA expenditures, unit officials
will be unable to provide Air Force and National Guard Bureau
officials accurate and consistent ASA expenditure information.
Conclusions:
Since the September 11, 2001, terrorist attacks, the U.S. government
has taken extensive efforts to protect the airspace over the United
States, aviation threat profiles have changed, and fiscal resources
have become more constrained. Nevertheless, NORAD and DOD have not
fully evaluated how to most-effectively manage ASA operations in this
current environment. Until the Air Force fully implements the
recommendations identified in our January 2009 report, including
establishing ASA as a steady-state mission by updating Air Force
homeland-defense policy and doctrine, concerns about the long-term
sustainability of ASA operations will remain. Additionally, until DOD
defines ACA; identifies the specific capabilities that support it,
including personnel and equipment; and distinguishes the place of ASA
within that mission for the purposes of meeting Congress's budget-
justification requirement, Congress, NORAD, and other entities within
DOD could be making important homeland-security decisions based on
different definitions and a resulting variance in understanding of
what is required to accomplish that mission.
Further, in this environment, it is critical that DOD makes basing,
acquisition, and other decisions about ASA operations on the basis of
risk-management principles that balance risk and costs. These
principles include establishing performance measures; identifying and
prioritizing locations that should be protected; utilizing accurate,
comprehensive, and documented risk assessments; and conducting cost-
benefit analyses for alternatives. With a more-rigorous, risk-based
management approach, DOD and congressional decision makers would be in
a better position to effectively allocate resources. Finally, unless
the Air Force addresses internal-control weaknesses that limit its
ability to identify ASA expenditures, Congress, NORAD, and DOD will
not have an accurate basis for investment decision making and
realistic budget formulation.
Matter for Congressional Consideration:
In order to ensure that the Air Force is taking action that addresses
the long-term sustainability of ASA operations, Congress may wish to
consider requiring the Secretary of the Air Force to fully implement
the remaining actions identified in our 2009 report within a time
period that Congress believes most prudent.
Recommendations for Executive Action:
To address these issues, we recommend that the Secretary of Defense
take the following seven actions.
In order that all DOD components may better understand the mission
they have been asked to perform, we recommend the Secretary of Defense
direct the Assistant Secretary of Defense for Homeland Defense and
Americas' Security Affairs, in coordination with the Joint Chiefs of
Staff, to clearly define ASA operations and its relationship to
NORAD's ACA operations.
In order to implement a more-complete risk-based management approach
that balances risk and costs for ASA operations, we recommend the
Secretary of Defense take the following five actions:
* Direct the Joint Chiefs of Staff, in coordination with the U.S.
element of NORAD and U.S. Pacific Command, to develop performance
measures for ASA operations and then use these measures to evaluate
the mission and make adjustments, as warranted, on the basis of the
performance-measure results.
* Direct the Office of the Assistant Secretary of Defense for Homeland
Defense and Americas' Security Affairs, in coordination with the Joint
Chiefs of Staff, to issue updated guidance, which includes a
prioritized list of metropolitan areas and critical infrastructure
that NORAD is supposed to protect.
* Direct the U.S. element of NORAD to update its ASA computer model to
address identified areas for improvement to include incorporating the
elements of risk--threat, vulnerability, and consequence.
* Direct the U.S. element of NORAD to document the results of its risk
assessments so that NORAD and DOD can identify trends over time.
* Identify the appropriate DOD entities that should conduct a
comprehensive cost-benefit analysis of NORAD's ASA basing strategy and
then have those entities conduct such an analysis.
In order to accurately identify ASA expenditures and address other
internal control weaknesses, we recommend that the Secretary of
Defense direct the Secretary of the Air Force and the Director of the
National Guard Bureau to issue guidance that:
* defines ASA programmatic and budgeting roles and responsibilities;
* defines all expenditures that should be identified as ASA
expenditures in financial-management systems; and:
* identifies the proper procedures to track ASA expenditures in their
financial-management systems.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, DOD stated that it
concurred with one of our recommendations and partially concurred with
the other six recommendations. However, DOD did not consistently
identify specific actions that it would take to implement the
recommendations and it did not identify time frames for implementing
the recommendations. DOD's comments are reprinted in appendix III. The
Department of Homeland Security and the Federal Bureau of
Investigation also reviewed a draft of this report but did not provide
any comments.
DOD concurred with our recommendation to clearly define ASA operations
and its relationship to NORAD's ACA operations and stated that all
organizations involved in the air sovereignty mission would use
standard terminology and that the new terminology would be reflected
in subsequent plans, execute orders, and DOD publications.
DOD partially concurred with our recommendation to develop performance
measures for ASA operations, but did not identify any specific actions
that it would take to implement the recommendation. In its comments,
DOD stated that it recognizes the merit of, and routinely employs
performance measures that are incorporated into military service
training requirements. However, NORAD officials told us during our
review that the command had not developed any performance measures for
ASA operations. Consequently, we believe that the development and use
of ASA-related performance measures would improve DOD's ability to
assess the overall effectiveness of its ASA operations and enhance the
performance measures to which DOD referred to in its comments.
DOD partially agreed with our recommendation that it issue updated
guidance, which includes a prioritized list of metropolitan areas and
critical infrastructure that NORAD is supposed to protect. DOD stated
that the current guidance on prioritized metropolitan areas and
critical infrastructure was produced by the National Security Staff
and the criteria used to produce it are still valid. However, this
guidance has not been updated since shortly after September 11, 2001,
and DOD has produced other similar lists within the department that it
uses in some contexts. For these reasons, we continue to believe that
DOD should consider updating existing guidance or developing new
guidance concerning a prioritized list of metropolitan areas and
critical infrastructure locations. We anticipate that NORAD and DOD
could seek additional guidance from the National Security Staff if
they believe the Staff's guidance needs to be reviewed or updated.
Such actions would better ensure that key national security
organizations have consistent and up to date expectations.
DOD partially concurred with our recommendation to direct the U.S.
element of NORAD to update its ASA computer model to address
identified areas for improvement to include incorporating the elements
of risk--threat, vulnerability, and consequence. In its comments, DOD
stated that NORAD determines mission risk by evaluating its computer
model results (based on vulnerability, threat capability, and targets
of consequence) and comparing the results to intelligence assessments
of threat intent. However, our analysis of the model found that it did
not incorporate threat, vulnerability, and consequence (or
criticality) data because NORAD designed the model to assess the
ability of its aircraft to respond to a single type of threat
regardless of its location in the United States. DOD's comments also
stated that NORAD would explore improvements to the ASA model in
support of NORAD/NORTHCOM decision making as targets and threats
evolve and capabilities improve. We continue to believe it appropriate
that NORAD's future improvements to the model should include the three
components that DOD has long-considered essential in risk assessments:
threat, vulnerability, and consequence (or criticality). Such actions
would be consistent with DOD's other efforts to protect its people,
property, and systems from attacks. We also expect that future
improvements to the model will include the specific areas we
identified in the classified version of this report.
DOD partially concurred with the recommendation to direct the U.S.
element of NORAD to document the results of its risk assessments so
that NORAD and DOD can identify trends over time. In its response, DOD
identified several NORAD studies or assessments--including a 2009
Collateral Damage Study and the 2010 ASA Basing Study--that are
subject to trend analysis. We believe that NORAD's efforts to study or
analyze different aspects of its missions are positive actions.
However, while NORAD issued a one-time report to Congress in March
2011 that identified the results of its 2010 ASA Basing Study, a one-
time study is not a trend analysis and the command has not documented
the results of its analyses of the ASA model since that time. We
continue to believe that DOD should document over time the results of
analyses stemming from the ASA basing model itself. This would provide
a valuable reference of the model's inputs and outputs over time and
improve the ability of current and future leaders to identify and
address evolving threats within U.S. airspace.
DOD partially concurred with our recommendation that the Secretary of
Defense should identify the appropriate DOD entities that should
conduct a comprehensive cost-benefit analysis of NORAD's ASA basing
strategy and then have those entities conduct such an analysis. The
response stated that multiple cost-benefit analyses have been
conducted on NORAD's ASA basing strategy by NORAD, OSD Cost Assessment
and Program Evaluation, the Joint Staff, the Department of Air Force,
and the Office of the Assistant Secretary of Defense for Homeland
Defense and Americas' Security Affairs. However, when we asked NORAD
and DOD officials, including some of those listed above, for the cost-
benefit analysis of its basing strategy (i.e. potential cost savings
that could result from eliminating a given number of sites), the
officials told us that such an assessment had not been completed.
Further, when we asked which organization would be responsible for
completing such studies, the officials provided us conflicting views
of whose responsibility it would be. Nonetheless, DOD's comments also
recognize that such cost-benefit analyses are critical for NORAD's
force posturing. As such, we believe that NORAD and DOD's ability to
demonstrate its use of these cost-benefit analyses in future decision
making efforts will help to ensure effective mission capability
consistent with the current fiscal environment.
DOD partially concurred with our recommendation to direct the
Secretary of the Air Force and the Director of the National Guard
Bureau to issue guidance that addresses internal control weaknesses
that kept the Air Force from providing accurate ASA expenditure
information to Congress. Such weaknesses included unclear roles and
responsibilities; lack of guidance that identified expenditures that
should be identified as ASA expenditures in financial-management
systems; and proper procedures to track ASA expenditures in their
financial-management systems. In its response, DOD stated that it
believes that programmatic and budgetary roles are clearly understood
and well defined in DOD's planning, programming, budgeting, and
execution process. However, the Air Force's inability to provide
Congress an accurate reporting of resources used for ASA operations
indicates that internal control weaknesses exist that have not been
mitigated by DOD's standard process. For example, officials from the
Air Force's homeland defense office, the National Guard Bureau, and
the Air Force's Combat Command could not provide us any documentation
of which organization is responsible for programming and budgeting for
ASA operations since the Air Force canceled its ASA program action
directive in 2009.
As a result of these weakness, the Air Force provided Congress
inaccurate information about its fiscal year 2010 ASA expenditures--
ranging from overstating its flying hour costs by at least 38 percent
to understating its facilities construction, maintenance, and
equipment expenditures. DOD's response did not indicate that it would
issue guidance to address these internal control weaknesses. Given the
current fiscal environment, DOD should identify and take specific
actions immediately to address our findings so that Congress, NORAD,
and DOD will have an accurate basis for decision making and realistic
budget formulation.
We are sending copies of this report to appropriate congressional
committees, the Secretary of Defense, the Commander of NORAD, the
Secretary of the Air Force, the Commissioner of Customs and Border
Protection, the Director of the Federal Bureau of Investigation, the
Administrator of the Transportation Security Administration, the
Commandant of the U.S. Coast Guard, and the Director of OMB. In
addition, the report is available at no charge on the GAO website at
[hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-4523 or leporeb@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff who made key contributions
to this report are listed in appendix IV.
Signed by:
Brian J. Lepore:
Director, Defense Capabilities and Management:
List of Requesters:
The Honorable Adam Smith:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable J. Randy Forbes:
Chairman:
The Honorable Madeleine Z. Bordallo:
Ranking Member:
Subcommittee on Readiness:
Committee on Armed Forces:
House of Representatives:
The Honorable Roscoe G. Bartlett:
Chairman:
The Honorable Silvestre:
Reyes Ranking Member:
Subcommittee on Tactical Air and Land Forces:
Committee on Armed Forces:
House of Representatives:
[End of section]
Appendix I: Scope and Methodology:
To determine the extent to which the Air Force has implemented the
recommendations from our 2009 report, we interviewed officials from
the Assistant Secretary of Defense for Homeland Defense and Americas'
Security Affairs, the Air Force's homeland-defense office, the Air
Force's Air Combat Command, the National Guard Bureau, and three air
sovereignty alert (ASA) units. We gathered information about steps
taken related to the recommendations and obtained supporting
documentation, when possible. In those cases where the Air Force had
taken action, we asked the officials about the effect that the action
had on their operations. In those cases where the Air Force had not
taken action, we asked the officials to identify the rationale for
lack of action and the plan that they had in place--including time
frames--to implement the recommendations.
To determine the extent to which the Department of Defense (DOD) has
adopted a risk-based management approach to balance risk and costs for
ASA operations, we reviewed and analyzed applicable documentation and
interviewed relevant officials. Specifically, we reviewed prior GAO
reports that recommended organizations use a risk-based management
approach to manage risk and costs.[Footnote 31] We reviewed documents
and interviewed officials from the Office of the Assistant Secretary
of Defense for Homeland Defense and Americas' Security Affairs, the
Air Force, and the North American Aerospace Defense Command (NORAD) to
determine the extent to which NORAD has followed a risk-based
management approach for ASA operations. We reviewed NORAD's ASA
assessments and compared them to the elements required for a risk
assessment. We reviewed acquisition documentation to determine whether
the command considered the cost and benefits when considering
alternatives for ASA operations. During our discussions with
officials, we asked how and whether they measure the performance of
ASA operations and whether they consider cost for ASA operations. To
help us better understand whether DOD incorporates risk into ASA
operations, we reviewed documentation on how other agencies
incorporate risk into their missions. We then reviewed documentation
on NORAD's Operation Noble Eagle Fighter Basing Location Model and
analyzed the model's inputs and assumptions.[Footnote 32] We
interviewed officials from NORAD on how it operationalizes the model
in support of ASA operations.
In order to determine the extent to which the Air Force was able to
accurately identify ASA expenditures, we analyzed the fiscal year 2010
expenditure information the Air Force included in its fiscal year 2012
ASA budget-justification document. We documented the steps taken to
identify and compile the ASA expenditures through interviews, e-mails,
and related documentation. We then compared the information and
practices used by the Air Force to prepare the ASA budget
justification against cost-evaluation best practices consistently
applied by organizations throughout the federal government.
Specifically, we evaluated the budget justification to determine
whether it included all expenditures associated with ASA operations;
[Footnote 33] whether it contained few, if any, mistakes; and whether
the underlying documentation and expenditure calculations were well
documented so that they could be easily repeated or updated. To
determine the expenditures of each ASA site we developed a survey,
which we sent out to the Air National Guard and active-duty ASA units
to gather fiscal year 2009 and 2010 expenditures. We developed the
expenditure survey by gathering ASA operations-requirement
documentation that outlined the personnel, operational, and facilities
requirements for conducting ASA operations. In order to ensure greater
reliability in the expenditure information we gathered from the ASA
sites, we instructed the sites to separate non-ASA Operation Noble
Eagle expenditures from ASA expenditures; include all of the ASA
construction and maintenance expenditures; and to remove duplications
in their responses. We also consulted with two Air National Guard and
two active duty ASA sites to verify the accuracy and completeness of
the survey's instructions, parameters, and expenditure categories. We
found the data gathered in this survey to be sufficiently reliable for
our purposes.
[End of section]
Appendix II: Air Force's Implementation of Our 2009 Recommendations as
of November 2011:
Recommendation: We recommend that the Secretary of Defense direct the
military services with units that consistently conduct Air Sovereignty
Alert (ASA) operations to formally assign ASA duties to these units
and then ensure that the readiness of these units is fully assessed,
to include personnel, training, equipment, and ability to respond to
an alert;
Actions taken: Fully Implemented: The National Guard Bureau, in
coordination with Air Combat Command, initially implemented this
recommendation by placing ASA operations on the units' secondary
mission statements.[A] However, the National Guard Bureau told us that
it does not believe that the Secretary of Defense has formally
assigned ASA operations to the Air National Guard units. As of
November 2011, bureau officials told us that they were taking action
to remove the reference to ASA operations from these statements
because they now believe that they could better address the findings
identified in the 2009 report through the Air Force's standard
deployment process. However, North American Aerospace Defense Command
(NORAD) does not believe that the National Guard Bureau should remove
the reference to ASA operations from the units' secondary mission
statements. NORAD officials told us that they do not support the
bureau's position because NORAD experienced some challenges when it
placed units that do not normally sit alert on alert for the 10th
anniversary of the September 11, 2001, terrorist attacks. While our
assessment gives the Air Force credit for having taken action, we are
also concerned that removing the reference to ASA operations from the
units' secondary mission statements could have adverse effects, such
as the units not reporting the extent to which they are able to
conduct both their ASA operations and overseas deployments in their
appropriate readiness systems. It is too early to determine the
consequences of the bureau's decision, although a bureau official
acknowledged that the problems we identified in our 2009 report could
reoccur if this new approach with NORAD was not fully implemented.
Recommendation: We recommend that the Secretary of Defense direct the
Secretary of the Air Force to implement ASA as a steady-state mission
according to NORAD, Department of Defense (DOD), and Air Force
guidance by: (1) incorporating ASA operations within the Air Force
submissions for the 6-year Future Years Defense Program; (2) updating
the Air Force homeland defense policy, homeland operations doctrine,
and concept of operations to incorporate and define the roles and
responsibilities for ASA operations; and; (3) updating and
implementing the ASA program action directive;
Actions taken: Partially Implemented: The Air Force has taken one of
three steps to implement ASA as a steady-state mission. Specifically:
(1) The National Guard Bureau has incorporated ASA operations in its
fiscal year 2013 Future Years Defense Program; (2) The Air Force had
not updated any of the identified documents. Consequently, overall
roles and responsibilities remain undefined; (3) Rather than updating
and implementing the ASA program action directive, the Air Force
canceled the program action directive. This leaves the Air Force and
the National Guard Bureau without clear roles and responsibilities for
operational planning, programming, and budgeting for ASA operations,
which GAO has identified as an internal control weakness.
Recommendation: We recommend that the Secretary of Defense direct the
Secretary of the Air Force to establish a timetable to implement ASA
as a steady-state mission [by taking the actions identified in the
preceding recommendation];
Actions taken: Not Implemented: The Air Force has not established a
timetable to implement ASA as a steady-state mission.
Recommendation: We recommend that the Secretary of Defense direct the
Secretary of the Air Force to develop and implement a plan to address
any projected capability gaps in ASA units due to the expected end of
the useful service lives of their F-15 and F-16 fighter aircraft;
Actions taken: Minimally Implemented: The Air Force has taken some
action that could mitigate potential challenges, if proactively
managed, but has not developed a single plan to manage the
recapitalization of the aircraft that conduct ASA operations.
Specifically, the service has initiated efforts to test the effective
service life of its remaining F-15 and F-16 fleets. In addition, the
service has requested funding to update both platforms with radar and
communication equipment that will support ASA operations. The Air
Force has also identified the extent to which it could cover its alert
sites if either the F-15 or F-16 fleet were unavailable. However, the
potential grounding of ASA aircraft--such as the grounding of F-15s
during 2007 and F-22s during 2011, the ongoing delays in the F-35
program, and the unspecified fielding locations for the F-35 aircraft
could pose challenges in the future. Further, the Air Force could have
challenges if the retirement of the current fleet of F-15s and F-16s
are not managed in a manner that considers the effect on ASA
operations. For example, if the Air Force were to decrease the number
of older models of F-16s in the inventory, this could affect ASA
operations since these aircraft are flown by units that conduct these
operations. Also, if the Air Force retires the fleet of F-15s and F-
16s with the longest remaining effective service lives before those
with shorter service lives, the service could experience a shortage in
fighter aircraft available for ASA operations before the 5th-
generation aircraft are available[B].
Recommendation: We recommend that the Secretary of Defense direct the
Secretary of the Air Force to develop and implement a formal method to
replace deploying units that still provides unit commanders
flexibility to coordinate replacements;
Actions taken: Minimally Implemented: In response to this
recommendation, DOD noted that it has a global force-management system
that it uses to formally manage its forces around the world. However,
the approach that DOD uses to backfill personnel and units for ASA
operations is still managed on an ad hoc basis. NORAD and the Air
Force have developed some planning documents, such as a plan for
backfilling ASA units on a large scale. For example, one of the
planning documents identifies potential units that could conduct ASA
operations at bases that are equipped with F-15s if those aircraft
were grounded for a period of time as they were in 2008. In addition,
Air Combat Command and the National Guard Bureau developed a
memorandum of understanding that identifies a process by which Air
National Guard units will be scheduled to support non-ASA Operation
Noble Eagle missions, such as national special-security events,
throughout the year, if called upon. The agreement identifies a
process by which Air National Guard units will be identified and
scheduled to support these missions, if called upon, and the time
frame during which each unit will be ready to support these missions.
It also identifies Air Combat Command as the command that will be
responsible for providing funds to those units when called upon.
However, the Air Force has not developed similar Operation Noble Eagle
planning documents to replace ASA units that deploy to support
overseas operations or units that are taken off alert for a short
period of time when they fail an alert force evaluation. Consequently,
the National Guard Bureau, in coordination with Air Combat Command,
must try to find personnel and equipment that are available to support
these units. In addition, since it is unclear who is responsible for
funding these backfill efforts, the National Guard Bureau must use
funds that were programmed for other purposes to cover these costs.
Source: GAO analysis of DOD information.
[A] The Air Force issues mission designed operational capabilities
statements that identify the unit's mission(s) and related
requirements (e.g., type and number of personnel). The unit's
readiness is based on these requirements.
[B] Fifth-generation aircraft, such as the F-22 and F-35, have stealth
characteristics, fused sensor data, and advanced radars.
[End of table]
[End of section]
Appendix III: Comments from the Department of Defense:
Note: GAO received Department of Defense comments on January 27, 2012.
Assistant Secretary of Defense:
Homeland Defense & Americas' Security Affairs:
2006 Defense Pentagon
Washington, D.C. 20301-2600:
Mr. Brian Lepore:
Director, Defense Capabilities and Management:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, DC 20548:
Dear Mr. Lepore:
This is the Department of Defense (DoD) response to the GAO final
report, "(U) Homeland Defense: Continued Actions Needed to Improve
Management of Air Sovereignty Alert Operations," dated January 11, 2012
(GAO Code 351594/GA0-12-311).
Thank you for the opportunity to comment on this important GAO report.
Defense of the United States and its citizens is and will remain DoD's
number one priority. Although DoD works with many interagency partners
on overall air security, it has the sole responsibility to deter and
defeat airborne threats to the homeland. Air Sovereignty is a critical
component of our Homeland Defense posture, and we look forward to
studying this report in depth to identify changes that could
strengthen the air defense of the United States.
We greatly appreciate GAO's efforts to ensure DoD's official written
comments are included despite the expedited release timeline.
Our point of contact for this action is LtCol Jeff Blackwell, (571)
256-8322, or thomas.blackwell@osd.mil.
Sincerely,
Signed by:
Paul N. Stockton:
Enclosure: 1. DoD Response to FAO-12-311.
[End of letter]
GAO Report — Dated January 11, 2012:
GAO Code 351594/GAO-12-311:
"(U) Homeland Defense: Continued Actions Needed to Improve Management of
Air Sovereignty Alert Operations"
Department Of Defense Comments To The Recommendations:
Recommendation 1: "In order that all DoD components may better
understand the mission they have been asked to perform, GAO recommends
that the Secretary of Defense direct the Assistant Secretary for
Homeland Defense and Americas' Security Affairs, in coordination with
the Joint Chiefs of Staff, to clearly define ASA operations and its
relationship to NORAD'S ACA operations."
DoD Response: Concur. OSD will coordinate with the Joint Staff,
NORAD/USNORTHCOM, and the National Guard Bureau (NGB) to ensure
standard terminology is used by all organizations involved in the Air
Sovereignty mission and is reflected in subsequent plans, execute
orders, and DoD publications. This standardization will also assist in
implementing GAO Recommendation #3 as it relates to reporting and
accounting for this mission.
Recommendation 2: "In order to implement a more complete risk-based
management approach that balances risk and costs for ASA operations,
GAO recommends that the Secretary of Defense take the following
actions:
* "Direct the Joint Chiefs of Staff, in coordination with the U.S.
element of NORAD and U.S. Pacific Command, to develop performance
measures for ASA operations and then use these measures to evaluate
and make adjustments, as warranted, based on the performance measure
results."
DoD Response: Partially Concur. DoD recognizes the merit of, and
routinely employs measures of performance. The Military Departments
are tasked with preparing and certifying combat units for use by
Combatant Commanders. Each Military Department has long-established
procedures and measures of performance to certify units as combat
ready. Combatant Command-specific measures of effectiveness and
performance deemed critical to mission execution are incorporated into
Service training requirements.
* "Direct the Office of the Assistant Secretary of Defense for
Homeland Defense and Americas' Security Affairs, in coordination with
the Joint Chiefs of Staff, to issue updated guidance, which includes a
prioritized list of metropolitan areas and critical infrastructure
that NORAD is supposed to protect."
DoD Response: Partially Concur. The National Security Staff (NSS)
produced the current guidance on prioritized metropolitan areas and
critical infrastructure. The criteria used to produce the guidance are
still valid. OSD welcomes the opportunity to review and update the
guidance as required.
* "Direct the U.S. element of NORAD to update its ASA computer model
to address identified areas for improvement to include incorporating
the elements of risk-threat, vulnerability, and consequence."
DoD Response: Partially Concur. NORAD determines mission risk by
evaluating its computer model results (based on vulnerability, threat
capability, and targets of consequence) and comparing the results to
intelligence assessments of threat intent. NORAD does, and will
continue to explore improvements to the ASA model in support
of NORAD/USNORTHCOM decision making, as targets and threats evolve,
and capabilities improve.
* "Direct the U.S. element of NORAD to document the results of its
risk assessments so that NORAD and DOD can identify trends over time."
DoD Response: Partially Concur. NORAD conducts numerous formal
studies/risk assessments (e.g., 2009 Collateral Damage Study, 2010 ASA
Basing Study) that are subject to both formal and informal trend
analysis. DoD recognizes the value of this trend analysis and will
continue to work with NORAD to understand current risk and to track
changes over time.
* "Identify the appropriate DOD entities that should conduct a
comprehensive cost-benefit analysis of NORAD's ASA basing strategy and
then have those entities conduct such an analysis."
DoD Response: Partially Concur. Multiple cost-benefit analyses have
been and will continue to be conducted on NORAD's ASA basing strategy,
as they are critical to NORAD force posturing. Previous efforts
include those led by NORAD, OSD Cost Assessment and Program Evaluation
(CAPE), the Joint Staff, the Department of the Air Force, and the
OASD(HD&ASA).
Recommendation 3: "In order to accurately identify ASA expenditures
and address other internal control weaknesses, GAO recommends that the
Secretary of Defense direct the Secretary of the Air Force and the
Director of the National Guard Bureau to issue guidance that:
* Defines ASA programmatic and budgeting roles and responsibilities;
* Defines all expenditures that should be identified as ASA
expenditures in financial management systems; and;
* Identifies the proper procedures to track ASA expenditures in their
financial management systems."
DoD Response: Partially Concur. Programmatic and budgetary roles are
clearly understood. Responsibilities are well defined and established
in the DoD Planning, Programming, Budgeting, and Execution (PPBE)
process. The Secretary of the Air Force is responsible for ASA
programming and budgeting and submits the ASA request as part of the
annual Air Force budget request. OSD will continue to work with the
Secretary of the Air Force to clarify and standardize tracking and ASA
expenditures.
[End of section]
Appendix IV: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Brian J. Lepore, (202) 512-4523 or leporeb@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, the following individuals made
significant contributions to this report: Davi M. D'Agostino, Director
(retired); Joseph Kirschbaum, Assistant Director; James D. Ashley;
Tommy Baril; Mark Braza; Susan C. Ditto; Brandon Jones; David F.
Keefer; Charles W. Perdue; Karen A. Richey; Jeff M. Tessin; Amie M.
Steele; Amanda G. Weldon; and Michael Willems.
[End of section]
Related GAO Products:
Homeland Defense:
Homeland Defense: Actions Needed to Improve Management of Air
Sovereignty Alert Operations to Protect U.S. Airspace. [hyperlink,
http://www.gao.gov/products/GAO-09-184]. Washington, D.C.: January 27,
2009.
Homeland Defense: Actions Needed to Address Management of Air
Sovereignty Alert Operations to Protect U.S. Airspace. [hyperlink,
http://www.gao.gov/products/GAO-09-612T]. Washington, D.C.: April 22,
2009.
DOD Tactical Aircraft:
Tactical Aircraft: Air Force Fighter Reports Generally Addressed
Congressional Mandates, but Reflected Dated Plans and Guidance, and
Limited Analyses. [hyperlink,
http://www.gao.gov/products/GAO-11-323R]. Washington, D.C.: February
24, 2011:
Tactical Aircraft: DOD's Ability to Meet Future Requirements Is
Uncertain, with Key Analyses Needed to Inform Upcoming Investment
Decisions. [hyperlink, http://www.gao.gov/products/GAO-10-789].
Washington, D.C.: July 29, 2010.
Aviation Security:
Aviation Security: TSA Has Made Progress, but Additional Efforts Are
Needed to Improve Security. [hyperlink,
http://www.gao.gov/products/GAO-11-938T]. Washington, D.C.: September
16, 2011.
Aviation Security: TSA's Revised Cost Comparison Provides a More
Reasonable Basis for Comparing the Costs of Private-Sector and TSA
Screeners. [hyperlink, http://www.gao.gov/products/GAO-11-375R].
Washington, D.C.: March 4, 2011.
Transportation Security: Comprehensive Risk Assessments and Stronger
Internal Controls Needed to Help Inform TSA Resource Allocation.
[hyperlink, http://www.gao.gov/products/GAO-09-492]. Washington, D.C.:
March 27, 2009.
Risk Management:
Intelligence, Surveillance, and Reconnaissance: DOD Needs a Strategic,
Risk-Based Approach to Enhance Its Maritime Domain Awareness.
[hyperlink, http://www.gao.gov/products/GAO-11-621]. Washington, D.C.:
June 20, 2011.
Defense Management: Comprehensive Cost Information and Analysis of
Alternatives Needed to Assess Military Posture in Asia. [hyperlink,
http://www.gao.gov/products/GAO-11-316]. Washington, D.C.: May 25,
2011.
Defense Critical Infrastructure: Actions Needed to Improve the
Identification and Management of Electrical Power Risks and
Vulnerabilities to DOD Critical Assets. [hyperlink,
http://www.gao.gov/products/GAO-10-147]. Washington, D.C.: October 23,
2009.
Risk Management: Strengthening the Use of Risk Management Principles
in Homeland Security. [hyperlink,
http://www.gao.gov/products/GAO-08-904T]. Washington, D.C.: June 25,
2008.
Highlights of a Forum Convened by the Comptroller General of the
United States: Strengthening the Use of Risk Management Principles in
Homeland Security. [hyperlink,
http://www.gao.gov/products/GAO-08-627SP]. Washington, D.C.: April 15,
2008.
Homeland Security: Applying Risk Management Principles to Guide
Federal Investments. [hyperlink,
http://www.gao.gov/products/GAO-07-386T]. Washington, D.C.: February
7, 2007.
Risk Management: Further Refinements Needed to Assess Risks and
Prioritize Protective Measures at Ports and Other Critical
Infrastructure. [hyperlink, http://www.gao.gov/products/GAO-06-91].
Washington, D.C.: December 15, 2005.
Financial Management:
Department of Defense: Financial Management Improvement and Audit
Readiness Efforts Continue to Evolve. [hyperlink,
http://www.gao.gov/products/GAO-10-1059T]. Washington, D.C.: September
29, 2010.
[End of section]
Footnotes:
[1] Airborne threats to the nation can come from a range of vehicles
(e.g., commercial, general aviation, cargo, ultra light, and military
aircraft and cruise missiles) and modes of attack (e.g., detonating
explosives on board, hijacking, crashing the aircraft, and launching
of weapons).
[2] Operation Noble Eagle, which was initiated after the terrorist
attacks of September 11, 2001, is DOD's umbrella homeland defense
operation for North America and Hawaii. There are a variety of
missions that could be conducted underneath this umbrella operation.
For example, North American Aerospace Defense Command (NORAD) could
conduct irregular air patrols above major metropolitan areas and
critical infrastructure facilities as well as maintain an alert force
of fighter, tanker, and control aircraft. Aircraft sorties and alert
commitments are based on a tiered response system. As threat levels
intensify, the number of aircraft on alert and on patrol increases.
[3] The National Guard Bureau is a joint function within the DOD that,
among other things, participates with staffs from the Department of
Army and the Department of Air Force in the formulation, development,
and coordination of all programs, policies, concepts, and plans
pertaining to or affecting the National Guard, the Army National Guard
of the United States, and the Air National Guard of the United States.
The Air National Guard of the United States is a reserve component of
the U.S. Air Force.
[4] GAO, Homeland Defense: Actions Needed to Improve Management of Air
Sovereignty Alert Operations to Protect U.S. Airspace, [hyperlink,
http://www.gao.gov/products/GAO-09-184] (Washington, D.C.: Jan. 27,
2009). In addition, GAO testified on this topic in April 2009, GAO,
Homeland Defense: Actions Needed to Address Management of Air
Sovereignty Alert Operations to Protect U.S. Airspace, [hyperlink,
http://www.gao.gov/products/GAO-09-612T] (Washington, D.C.: Apr. 22,
2009).
[5] GAO, Homeland Security: Key Elements of a Risk Management
Approach, [hyperlink, http://www.gao.gov/products/GAO-02-150T]
(Washington, D.C.: Oct. 12, 2001); Intelligence, Surveillance, and
Reconnaissance: DOD Needs a Strategic, Risk-Based Approach to Enhance
Its Maritime Domain Awareness, [hyperlink,
http://www.gao.gov/products/GAO-11-621] (Washington, D.C.: June 20,
2011); Joint Strike Fighter: Strong Risk Management Principles
Essential as Program Enters Most Challenging Phase, [hyperlink,
http://www.gao.gov/products/GAO-09-711T] (Washington, D.C.: May 20,
2009); DOD Instruction, 3020.45, Defense Critical Infrastructure
Program (DCIP) Management (Apr. 21, 2008).
[6] We could not evaluate the extent to which DOD implemented the last
two phases--management selection and implementation and monitoring--
because (1) DOD had not fully implemented the first three phases of
GAO's risk-based management framework, which are essential for the
latter two phases, and (2) DOD was still in the decision-making stage
for the alternatives that we identified.
[7] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009).
[8] There are 18 steady-state ASA sites. However, in fiscal year 2010,
McEntire Joint National Guard Base, South Carolina, constructed and
upgraded facilities since it was going to take on alert duty from Shaw
Air Force Base, South Carolina, at the beginning of fiscal year 2011.
Therefore, we included McEntire Joint National Guard Base's fiscal
year 2010 expenditures in our survey efforts. Budget officials from
Elmendorf Air Force Base, Alaska, told us that they were unable to
provide us accurate ASA expenditure information because the base does
not track ASA expenditures separately from its other mission
expenditures.
[9] Budget officials from Shaw Air Force Base, South Carolina;
Homestead Air Force Base, Florida; and Hickam Air Force Base, Hawaii,
were unable to provide us information for some aspects of the
operation, such as flying hour or personnel expenditures.
[10] Title 32 and Title 10 refer to sections of the United States Code.
[11] Department of Homeland Security, National Infrastructure
Protection Plan (2009).
[12] [hyperlink, http://www.gao.gov/products/GAO-09-184].
[13] NORAD has three overall missions: aerospace warning, aerospace
control, and maritime warning. According to NORAD's most-recent
chartering agreement, aerospace control includes providing
surveillance and exercising operational control of the airspace of
Canada and the United States.
[14] DOD, Strategy for Homeland Defense and Civil Support (Washington,
D.C.: March 2005).
[15] For purposes of this report, performance measures refer to both
measures of effectiveness and measures of performance.
[16] [hyperlink, http://www.gao.gov/products/GAO-09-184].
[17] According to DOD's joint doctrine on planning, an assessment with
performance measures should answer two questions: (1) Is the Joint
Force Commander doing things right? and (2) Is the Joint Force
Commander doing the right things?
[18] [hyperlink, http://www.gao.gov/products/GAO-09-184].
[19] Specific information about the model's inputs and areas for
improvement are provided in the classified version of this report.
[20] GAO, Standards for Internal Control in the Federal Government,
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]
(Washington, D.C.: November 1999).
[21] White House, The National Strategy for the Physical Protection of
Critical Infrastructures and Key Assets (February 2003).
[22] Office of Management and Budget (OMB) circulars A-11 (sect. 7), A-
94, and A-130 generally apply to capital investments. For example, OMB
Circular A-94 outlines procedures for cost-benefit analysis that are
suggested for use in the internal planning of executive-branch
agencies, but are mandatory in certain contexts, such as for analyses
submitted to OMB in support of budget programs pursuant to OMB
Circular A-11.
[23] The Joint Capabilities Integration and Development System
analysis process is a requirements-generation system intended to
prioritize and ensure that the most-essential needs of the warfighter
are met. According to this process, when a material solution is
recommended to mitigate a capability gap, an analysis of alternatives
is generally conducted that evaluates the performance, operational
effectiveness, operational suitability, and the estimated costs of an
alternative to meet a mission capability.
[24] In 2008, Congress mandated that for fiscal year 2010 and each
year thereafter, the Secretary of Defense submit to the President for
consideration by the President for inclusion with the budget materials
submitted to Congress a consolidated budget justification that covers
all programs and activities of the ASA mission of the Air Force,
including military-personnel expenditures; flying hours; and any other
associated ASA mission expenditures of the Air Force and the Air
National Guard.
[25] For the purpose of this review, we evaluated the ASA fiscal year
2010 expenditures included in the fiscal year 2012 budget-
justification documents because at the time of our review it was the
most-recent budget justification reflecting resources spent on ASA
operations.
[26] These statutes include the Chief Financial Officers Act of 1990,
the Government Management Reform Act of 1994, the Federal Financial
Management Improvement Act of 1996, and various annual authorization
and appropriation act provisions.
[27] Funds controls are controls over the use of public funds,
including (1) using public funds efficiently and effectively and for
the purposes and within the time frames and amounts prescribed by law;
(2) making payments to the right parties in the correct amount within
allowable time frames and recouping any improper payments; and (3)
accurately recording and reporting on transactions and use of public
funds.
[28] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1].
[29] According to the National Guard Bureau, federal authorities are
working to recover the improper compensation.
[30] The Office of the Under Secretary of Defense (Comptroller) could
not find the original ASA budget-justification guidance and template
so it recreated the template for the purposes of our audit.
[31] GAO, Joint Strike Fighter: Strong Risk Management Principles
Essential as Program Enters Most Challenging Phase, [hyperlink,
http://www.gao.gov/products/GAO-09-711T] (Washington, D.C.: May 20,
2009); Intelligence, Surveillance, and Reconnaissance: DOD Needs a
Strategic, Risk-Based Approach to Enhance Its Maritime Domain
Awareness, [hyperlink, http://www.gao.gov/products/GAO-11-621]
(Washington, D.C.: June 20, 2011); Defense Acquisitions: Managing Risk
to Achieve Better Outcomes, [hyperlink,
http://www.gao.gov/products/GAO-10-374T] (Washington, D.C.: May 20,
2009); Aviation Security: DHS and TSA Have Researched, Developed and
Begun Deploying Passenger Checkpoint Screening Technologies, but
Continue to Face Challenges, [hyperlink,
http://www.gao.gov/products/GAO-10-128] (Washington, D.C.: Oct. 7,
2009); and Risk Management: Further Refinements Needed to Assess Risks
and Prioritize Protective Measures at Ports and Other Critical
Infrastructure, [hyperlink, http://www.gao.gov/products/GAO-06-91]
(Washington, D.C. Dec. 15, 2005).
[32] The Operation Noble Eagle Fighter Basing Location Model is a
Monte Carlo simulation that creates events and evaluates how effective
the response will be, given a predefined fighter posture and location.
[33] We defined ASA operations as the fully fueled, fully armed
aircraft and trained personnel on alert 24 hours a day, 365 days a
year, and the infrastructure, equipment, and supplies required to
maintain them at the 18 ASA sites across the United States.
[End of section]
GAO’s Mission:
The Government Accountability Office, the audit, evaluation, and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the
performance and accountability of the federal government for the
American people. GAO examines the use of public funds; evaluates
federal programs and policies; and provides analyses, recommendations,
and other assistance to help Congress make informed oversight, policy,
and funding decisions. GAO’s commitment to good government is
reflected in its core values of accountability, integrity, and
reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO’s website [hyperlink, http://www.gao.gov]. Each
weekday afternoon, GAO posts on its website newly released reports,
testimony, and correspondence. To have GAO e mail you a list of newly
posted products, go to [hyperlink, http://www.gao.gov] and select “E-
mail Updates.”
Order by Phone:
The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black
and white. Pricing and ordering information is posted on GAO’s
website, [hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
Connect with GAO:
Connect with GAO on facebook, flickr, twitter, and YouTube.
Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts.
Visit GAO on the web at [hyperlink, http://www.gao.gov].
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm];
E-mail: fraudnet@gao.gov;
Automated answering system: (800) 424-5454 or (202) 512-7470.
Congressional Relations:
Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, DC 20548.
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, DC 20548.