Report to the Chairman,
Committee on National Security,
House of Representatives

August 1996

United States General Accounting Office
National Security and International Affairs Division

B-274120

August 30, 1996

The Honorable Floyd D. Spence
Chairman, Committee on National Security
House of Representatives

Dear Mr. Chairman:

This report responds to your letter of February 28, 1996, asking us to evaluate certain National Intelligence Estimates (NIE) prepared by the U.S. Intelligence Community (IC) that analyze the threat to the United States from foreign missile systems. As arranged with your office, our reporting objectives were to compare the content and conclusions of NIE 95-19, Emerging Missile Threats to North America During the Next 15 Years, November 1995, with the content and conclusions of two previous NIEs prepared in 1993; to evaluate whether these three NIEs appear to be objective and supported by facts; and to describe the conclusions of recent, unclassified studies on the threat to the United States from foreign
missile systems.

This report supplements our June 12, 1996, briefing to you and is an unclassified version of our classified report. All of our findings are contained in this report; the omitted classified information concerned detailed examples drawn from the NIEs to support our findings and observations.

Background

NIEs analyze issues of major importance and long-term interest to the United States and are the IC's most authoritative projection of future developments in a particular subject area. NIEs are intended to help policymakers and military leaders think through critical issues by presenting the relevant key facts, judgments about the likely course of events in foreign countries, and the implications for the United States. In this regard, former Director of Central Intelligence (DCI) William Casey stated: "the highest duty of a Director of Central Intelligence is to produce solid and perceptive national intelligence estimates relevant to the issues with which the President and the National Security Council need to concern themselves."

NIEs are produced by the National Intelligence Council (NIC), an organization composed of 12 National Intelligence Officers who report directly to the DCI. To prepare an NIE, the NIC brings together analysts from

Page 1 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

all the intelligence agencies that have expertise on the issue under review.1 However, in the final analysis, an NIE is the DCI's assessment with which the heads of the U.S. intelligence agencies concur, except as noted in the NIE'S text.

Based on a synthesis of the published views of current and former senior intelligence officials, the reports of three independent commissions, and a Central Intelligence Agency (CIA) publication that addressed the issue of national intelligence estimating, an objective NIE should meet the following standards:

quantify the certainty level of its key judgments by using percentages or "bettors' odds,"2 where feasible, and avoid overstating the certainty of judgments;

identify explicitly its assumptions and judgments;

develop and explore "alternative futures:" less likely (but not impossible) scenarios that would dramatically change the estimate if they occurred;

allow dissenting views on predictions or interpretations; and

note explicitly what the IC does not know when the information gaps could have significant consequences for the issues under consideration.

All or part of the three NIEs we reviewed addressed the nature of the current and future threat to the United States from foreign missiles. NIE 95-19 was specifically prepared by the IC to support decisions on missile defense systems for North America. In the United States, this issue is a critical one for the Congress and the administration as they debate the desirability and planned characteristics of a proposed multibillion dollar national missile defense system. Such a system would aim to protect the United States from limited ballistic missile attacks, whether accidental, unauthorized, or deliberate.3

Ballistic missiles are self-propelled missiles guided in the ascent of a high-arch trajectory and freely falling in the descent. If launched from any of the 18 countries analyzed in NIE 95-19 (except Cuba), such missiles would have to travel between 5,000 and 13,000 kilometers (3,100 to

Page 2 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

8,100 miles) to reach North America, classes them as intercontinental ballistic missiles (ICBM).4

Figure 1: Ranges to the United States and Canada

Source: National Intelligence Council.

Results in Brief

The main judgment of NIE 95-19 -- "No country, other than the major declared nuclear powers, will develop or otherwise acquire a ballistic missile in the next 15 years that could threaten the contiguous 48 states or Canada."5. -- was worded with clear (100 percent) certainty. We believe this level of certainty was overstated, based on the caveats and the intelligence gaps noted in NIE 95-19.

NIE 95-19 had additional analytic shortcomings. It did not (1) quantify the
certainty level of nearly all of its key judgments, (2) identify explicitly its

Page 3 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

critical assumptions, and (3) develop alternative futures. However, in
accordance with standards for producing objective NIES, NIE 95-19 acknowledged dissenting views from several agencies and also explicitly noted what information the IC does not know that bears upon the foreign missile threat. The 1993 NIEs met more of the standards than NIE 95-19 did.

NIE 95-19 worded its judgments on foreign missile threats very differently than did the 1993 NIEs, even though the judgments in all three NIEs were not inconsistent with each other. That is, while the judgments were not synonymous, upon careful reading they did not contradict each other.

NIE 95-19 Overstated Certainty of Its Main Judgment

The main judgment of NIE 95-19 was worded with clear (100 percent) certainty. We believe this level of certainty was overstated, based on the caveats and intelligence gaps noted in NIE 95-19.

On the issue of certainty in judgments, in 1992 then-DCI Robert Gates opined: "While we strive for sharp and focused judgments for a clear assessment of likelihood, we must not dismiss alternatives or exaggerate our certainty under the guise of making the 'tough calls.' We are analysts, not umpires, and the game does not depend on our providing a single judgment."

The wording of NIE 95-19's main judgment implies a 100-percent level of certainty that the predicted outcome will hold true during the next 15 years. However, the caveats and intelligence gaps noted in the NIE do not support this level of certainty. For example, at the begiming of NIE 95-19, the estimate notes "as with all projections of long-term developments, there are substantial uncertainties." A 1993 NIE stated its view that substantial uncertainties cloud the IC's ability to project developments, especially beyond 10 years. Finally, in NIE 95-19's Intelligence Gaps section, it noted several shortcomings in the IC's collection of information on foreign plans and capabilities.

NIE 95-19 Had Additional Analytic Shortcomings

NIE 95-19 did not (1) quantify the certainty level of nearly all of its key judgments, (2) identify explicitly its critical assumptions, and (3) develop alternative futures. However, in accordance with standards for producing objective NIEs, NIE 95-19 acknowledged dissenting views from several agencies and also explicitly noted what information the IC does not know that bears upon the foreign missile threat.

Page 4 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

Given the important role NIEs play in the national security decision-making process, U.S. policymakers require, and expect, objective estimates. "The paramount value [in NIEs] is objectivity," according to a former NIC Vice Chairman. Adds the CIA, "dedication to objectivity -- tough-minded evaluation of information, description of sources, and explicit defense of judgments -- provides [an estimate with] credibility on uncertain and often controversial policy issues."

We believe that five standards, previously discussed, apply to an objective
NIE. These standards were synthesized from our review of the published views of nine current or former senior intelligence officials, three independent commissions, and a CIA publication that addressed the issue of national intelligence estimating.6 We were unable to obtain the DCI's current, official standards (if any exist) for the essential elements of an
objective NIE, because the DCI refused to grant us access to the NIC. (See our Scope and Methodology section for more details on this scope impairment.)

NIE 95-19 Did Not Quantify Certainty Levels of Key Judgments

NIE 95-19 did not quantify the certainty level associated with its key judgments, by either using bettors' odds or percentages.7 It used unquantified words or phrases such as 'unlikely," "likely," "probably," "normally," "sometimes," "some leakage,' and "feasible, but unlikely."

The CIA has told its analysts to be predse in conveying the levels of confidence they have in their conclusions because policymakers and others rely on these assessments as they define and defend U.S. interests. Different people can hear very different messages from the same words, especially about probabilities, and therefore good estimates should use quantitative measures of confidence, according to a former NIC Vice Chairman. For example, a 'small but significant' chance could mean one chance in a hundred to one person; for another it may mean one chance in
five. Similarly, a former NIC Chairman wrote that NIEs with only words such as "possibly" are not of much help to someone trying to make an important decision. Instead, where feasible, NIES should use a percentage, a

Page 5 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

percentage range, or bettors' odds to better serve policymakers -- a controversial, but necessary, approach, according to this former official. Some intelligence judgments, such as estimating foreign economic developments well into the future, may not easily lend themselves to specifying a meaningful level of confidence, using numbers.

NIE 93-17 quantified the certainty of one of its key judgments by estimating
a "small but significant chance (10 to 30 percent)" that an event would occur. The certainty levels of its other key judgments were not quantified. NIE 93-19 did not quantify the certainty levels of any of its key judgments.

NIE 95-19's Critical Assumptions Not Explicitly Identified

NIE 95-19 did not explicitly identify its critical assumptions either by separately listing them in one place or by introducing them throughout the text with wording such as "we have assumed . . ."

Critical assumptions, also known as "linchpin assumptions," are defined by CIA as analysts' debatable premises that hold the argument together and warrant the validity of judgments. Therefore, as previously mentioned, assumptions should be explicitly distinguished from other information, including judgments. Estimative judgments are to be defended by fully laying out the evidence and carefully explaining the analytic logic used, according to a former Deputy Director for Intelligence, CIA.8 Writing about NIEs, a former Vice Chairman of the NIC agreed. As a general rule, the more complex and controversial an issue, the more analytic effort is required to ensure that critical assumptions are precisely stated and well defended, according to the CIA. Good analysis will clearly identify its key assumptions so that policymakers are aware of the "foundations" of the estimate and can therefore judge for themselves the appropriateness of the assumptions and the desirability of initiating actions to hedge against a failure of one or more assumptions.

From our reading of NIE 95-19, we identified what appear to be its implicit
cxitical assumptions.9 Most of these assumptions first appear in the NIE'S Key Judgrnents section, leading the reader to believe that the IC considers these assumptions to be fact-based judgments. However, we did not find a body of evidence in NIE 95-19 that would allow us to consider these

Page 6 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

statements as judgments, rather than assumptions. NIE 95-19 had only one
explicit assumption, which was not a critical one, concerning Iraq.

Some of NIE 95-19's implicit critical assumptions are listed below. Three
other assumptions that we identified included classified information.

The Missile Technology Control Regime (MTCR)10 will continue to significantly limit international transfers of missiles, components, and related technology, but some leakage of components and critical technologies will likely continue.

No country with ICBMs will sell them.

Three countries -- all of which were assessed as being "high" in both technical ability and economic resources -- will not be interested in developing an ICBM that could reach the United States (and elsewhere).

A flight test program lasting about 5 years is essential to the development of an ICBM.

An attack against the United States from off-shore ships using cruise missiles, while feasible, is unlikely to occur...

In addition, NIE 95-19 did not specify its assumption about the payload weight or weights the IC used in forecasting the range for North Korea's Taepo Dong 2 ballistic missile. Publicly, the NIC's Chairman has stated that the Taepo Dong 2 missile could have a range sufficient to reach Alaska, some U.S. territories in the Pacific, and the far westem portion of the 2,000 km-long Hawaiian Island chain. NIE 95-19 did, however, specify payload weights for the Taepo Dong 1 missile. NIE 93-19 explicitly analyzed the effects of changes in payload weight on the estimated range of ballistic missiles. The payload weight directly affects the range of a missile -- that is, a lighter payload allows any given missile to travel farther. For example, the IC judges that a certain country could increase the range of its existing intermediate range ballistic missile by 90 percent, if it decreased its
payload weight by 70 percent.

Like NIE 95-19, the 1993 NIEs did not explicitly identify their critical
assumptions, as a rule. However, in one case, the text of NIE 93-17 prefaced its judgment with a clear assumption about the current nuclear practices in one country.

Page 7 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

NIE 95-19 Did Not Develop Alternative Futures

NIE 95-19 did not develop alternative futures: less likely (but not
impossible) scenarios that would dramatically change the estimate if they
occurred.

NIEs should "describe the range of possible outcomes, including relatively
unlikely ones that could have major impact on American interests, and indicate which outcomes they think are most likely and why... The job,vafter all, is not so much to predict the future as to help pohcymakers think about the future," according to a forrner NIC Chairman. The CIA, then-DCI Robert Gates, and other senior NIC officials agree that NIEs should analyze alternative futures. A senior intelligence official told us that an alternative future takes a fundamental analytic assumption and varies it to explore different potential outcomes; for example, "What if countries do not honor the MTCR?"

Both 1993 NIEs explored alternative futures. NIE 93-19 mentioned them in the NIE's text and explored them in detail in a separate annex. NIE 93-17's Key Judgments included alternative futures, which were further developed through detailed scenarios. These alternative futures are classified.

NIE 95-19 disclosed that it did not account for alternative economic and political futures. NIE 95-19 did address some less likely technical options, including the characteristics and implications of a potential ICBM program of one country.

NIE 95-19 Offered Dissenting Views

NIE 95-19 had 12 dissents in the estimate.11 NIE 93-19 and NIE 93-17 had 23 and 2 dissents, respectively. There were qualitative differences in the nature of the dissents in the NIEs.

According to a February 1996 statement by the current Chairman of the NIC, "The process for producing NIEs is directed particularly at ensuring presentation of all viewpoints. We do not impose consensus; in fact we encourage the many agencies that participate in NIEs to state their views and we display major differences of view in the main text. Lesser reservations are expressed in footnotes."

While all three NIEs included dissenting views, the dissents were qualitatively different among the NIEs. For example, NIE 93-19's Key Judgments contained two fundamental disagreements by one department

Page 8 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

on the overall potential for proliferation of nuclear weapons and on the nuclear weapons program of a specific country. Other dissents in the body of this estimate were also of a fundarnental nature. In one instance, one department took an "alternative view" to NIE 93-19's forecasts about ICBM and space launch vehicle development and transfers. This alternative view from 1993 is very similar to the consensus view of NIE 95-19's main judgment.

Both NIE 95-19 and NIE 93-17 had no dissents in their Key Judgments. The dissents in the body of these NIEs were mostly on technical issues and contained classified information.

NIE 95-19 Explicitly Noted Information Gaps

NIE 95-19 and the 1993 NIEs explicitly noted information gaps at places in the estimates' text and in a separate Intelligence Gaps section.

Estimates should reveal what intelligence analysts do not know that could have significant consequences for the issue under consideration, according to several sources. This disclosure not only helps alert policymakers to the limits of the estimate, but also informs intelligence collectors of needs for further information, according to a former NIC Chairman.

In their Intelligence Gaps sections, the three NIEs each noted shortfalls in
the IC's collection of information on the issues they examined.

Differences and Similarities Between NIE 95-19 and 1993 NIEs

NIE 95-19 worded its judgments on foreign missile threats very differently
than did the 1993 NIEs, even though the judgments in all three NIEs were not inconsistent with each other. In addition, the evidence in NIE 95-19 was
qualitatively and quantitatively different compared to the 1993 NIEs. Details
of other differences and the wording of judgments do not appear in this
report because they contain classified information. Finally, the NIEs agreed
on several points.

Judgments on Missile Threats Worded Very Differently, but Were Not Inconsistent

NIE 95-19 worded its judgments on foreign missile threats very differently than did the 1993 NIEs, even though the judgments in all three NIEs were not inconsistent with each other. That is, while the judgments were not synonymous, upon careful reading they did not contradict each other. Because the DCI denied us access to officials responsible for the NIES, we

Page 9 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

were unable to obtain their reasons for the different wording chosen in the
three NIEs.

In general, the 1993 NIEs pointed out unfavorable and unlikely outcomes associated with foreign ICBMs more often than did NIE 95-19. A table that compares the exact wording of judgments on foreign missile threats in the three NIEs does not appear in this report because it contains classified information.

NIE 95-19 Presented Less Evidence Compared to 1993 NIEs

The evidence in NIE 95-19 is considerably less than that presented in the earlier NIEs, in both quantitative and qualitative terms.

Laying out the evidence is important because it allows readers to judge for
themselves how much credence to give the judgments, according to a former Vice Chairman of the NIC.

In quantitative terms, the earlier NIEs had at least one supporting volume with additional evidence and judgments. Each of the 1993 NIEs was over three times as long as NIE 95-19. The 1993 NIEs backed each of their key judgments with more support than did NIE 95-19. For example, NIE 93-19, which unlike NIE 95-19, was not focused on foreign missile threats, had almost twice the supporting evidence on missile threats than NIE 95-19 did when comparing the same countries.12 In addition, and in contrast to NIE 95-19, both of the 1993 NIEs referred readers to other IC studies for additional evidence or information.

In qualitative terms, we believe the earlier NIEs provided more convincing support for their key judgments. For example, NIE 95-19 stated that "no countries with ICBMS will sell them." For support, the NIE included one paragraph that cited a multi-national counter-proliferation policy (MTCR) and the theory that countries with ICBMS would probably be concerned that any missiles they sell might be turned against them. The NIE provided very little evidence to support its position that membership in the MTCR (or pledges to abide by the MTCR in China's case) would necessarily prevent a country from selling missiles. The NIE asserted that the MTCR had helped terminate missile programs in specific countries, but it provided no
evidence to support its view. The NIE did not cite additional evidence such
as intelligence on whether MTCR members have or have not sold missiles or missile technology in the past, or whether countries have refrained from

Page 10 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

selling such technology because of the MTCR. In addition, the NIE provided no evidence or detailed analysis to support its position that countries will not sell ICBMs because they would probably fear that the missiles could be turned against them.

In contrast to NIE 95-19, the earlier NIEs supported their judgments more thoroughly. Detailed examples contain classified information and do not appear in this report.

We were unable to identify the reasons why NIE 95-19 presented less evidence to support its judgments than the 1993 NIE, because NIC Officials refused to meet with us to discuss the preparation Of NIE 95-19. The reasons could include limitations on NIE 95-19's length, its SECRET/Releasable to "Country X" security classification (compared to the TOP SECRET/Codeword classification of the 1993 NIEs), and/or a
smaller evidentiary base.

NIEs Agreed on Several Points

In addition to the similarities between the NIEs on some judgments, the NIEs agreed on several other points, including the impact of foreign technology assistance on ICBM development, and the capabilities and intentions of two countries with respect to ICBM development.

Unclassified Studies on Foreign Missile Threats to the United States

The conclusions of unclassified goverment, or government-sponsored, studies on foreign missile threats to the United States were generally consistent with the conclusions Of NIE 95-19. However, whereas NIE 95-19's main judgment was that there will be no new missile threats to the contiguous 48 states during the next 15 years, two studies estimated some possibility -- "low" and "quite low" -- of such missile threats. The private studies we reviewed differed significantly from NIE 95-19's assessment of threats; these studies raised more immediate concerns about foreign missile threats to the United States. For example, the Heritage Foundation's Missile Defense Study Team concluded that ballistic missiles pose a clear, present, and growing threat to the United States.

We reviewed several recent unclassified studies on foreign missile threats to the United States and its interests. We identified these studies through a literature search of several databases that include defense and intelligence information. We limited our review to complete studies on this topic, and we did not include newspaper or journal articles. While we compared the conclusions of these studies to NIE 95-19, we did not review the quality of

Page 11 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

their evidence or attempt to reconcile any differences they had with NIE 95-19.

Government Studies

In a November 1993 letter to the Chairman of the House Committee on Armed Services, the CIA provided the declassified findings of its report entitled Prospects for the Worldwide Development of Ballistic Missile Threats to the Continental United States. The study's scope excluded countries with a current capability to strike the continental United States(CONUS) -- China and strategic forces in several states of the former Soviet Union. The study concluded that the "probability is low that any other country will acquire this capability in the next 15 years." Also, the study found that "no evidence exists that any of the countries examined in this study are developing missiles -- especially ICBMs -- for the purpose of attacking CONUS." There were no recommendations identified in the letter.

In June 1995, the Congressional Research Service issued a report for the Congress entitled Ballistic and Cruise Missile Forces of Foreign Countries. The report was written by Robert Shuey, a specialist in U.S. foreign policy and national defense. The report stated that "other than the declared nuclear powers (the United States, China, France, Russia, and the United Kingdom) few countries have long-range missiles." It also said that North Korea is in the process of developing longer range ballistic missiles, including the Taepo Dong 2. The report concluded that "the production or international transfer of more and better ballistic and cruise missiles will potentially have serious negative implications for the security of U.S. citizens and facilities..." The report contained no recommendations.

In April 1996, the Office of the Secretary of Defense released a study entitled Proliferation: Threat and Response. The key finding in the report was that the threat was changing from global to regional. The report did not address the current ballistic missile threat to the United States. The report did note, however, that "... unlike during the Cold War, those who possess nuclear, biological, and chemical weapons may actually come to use them." The report concluded that "The end of the Cold War has reduced the threat of a global nuclear war, but today a new threat is rising from the global spread of nuclear, biological, and chemical weapons." The report had no recommendations. The report had no indications that there was an increasing missile threat to the United States itself.

Government-Sponsored Study

In February 1993, a report commissioned by the Strategic Defense Initiative Organization of the Department of Defense was released entitled

Page 12 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

The Emerging Ballistic Missile Threat to the United States. The report was prepared by the Proliferation Study Team, chaired by Lieutenant General William E. Odom, USA (ret), Director of National Security Studies at the Hudson Institute. The report found that at this point there is no indication that Brazil, India, Italy, Israel, Germany, Japan, and Sweden -- countries that possess the potential to develop ICBMs during the 1990s -- have any intention of initiating an ICBM program. The report estimated that, if current trends continue, the probability of new ICBM threats during the 1990s or in the very early years of the next decade is quite low. In reaching its conclusion that "the prospects for an increase in bauistic missile threats to the United States during this decade are limited," the study team identified three uncertainties that affected their ability to forecast confidently 10 to 20 years into the future. First, intelligence indicators are often ambiguous. Second, a number of events could alter the capabilities or intentions of some states to field long-range ballistic missiles. Third, dramatic and rapid changes in U.S. political relations with states possessing or capable of fielding long-range missiles could occur. The report made no recommendations.

Private Studies

In July 1991, the Cato Institute published Foreign Policy Briefing No. 10 entitled Countdown to Disaster: The Threat of Ballistic Missile Proliferation. This study was prepared by Channing R. Lukefahr, an associate defense policy analyst at the Cato Institute, as part of the Institute's regular series evaluating government policies and offering proposals for reform. The key findings of the study were that "As the horizontal proliferation of ballistic missile technology continues, the threat of an accidental launch rises," and that "while the threat that unstable or antagonistic regimes will achieve the ability to launch intercontinental ballistic missiles... moves rapidly toward reality, attempts to reverse that destabimng trend have been merely exercises in delay." The study concluded that "the days when weapons of mass destruction and the systems to deliver them are possessed by only the two super-powers... are rapidly drawing to a close" and that "although there is no imminent threat to the United States from any of those [friendly] nations, continuation of that state of affairs cannot be guaranteed... an ally can become an enemy in a matter of months." The report cited stronger secessionist forces in the Soviet Union as undermining the central control of nuclear weapons and making the accidental launch of a few dozen or even a few hundred missiles possible as is the possibility of a limited launch by rogue elements. The report's sources were congressional testimony and articles in journals, magazines, and newspapers. The report

Page 13 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

recommended the development and deployment of antiballistic missile systems.

In March 1996, the Heritage Foundation released a document entitled
Defending America: Ending America's Vulnerability to Ballistic Missiles.
This was an update to a June 1995 report entitled Defending America: A Near- and Long-Term Plan to Deploy Missile Defenses. The Missile Defense Study Team was chaired by Ambassador Henry Cooper, former Director of the Strategic Defense Initiative Organization. The main finding of the reports was that the United States had no defense against ICBMs. The initial report said that ICBMs marketed as space launchers could provide rogue states with the ability to attack the United States. The update cited, but did not identify, authoritative administration officials as having testified to the Congress in May 1995, that rogue states could threaten U.S.
cities with long-range missile attacks in 3 to 5 years. The reports concluded that ballistic missiles pose a clear, present, and growing threat to America and her allies overseas. The report recommended a decision to deploy, when technically feasible, the Navy's Upper Tier interceptor system and the Brilliant Eyes space-based sensor system.

Agency Comments

The NIC did not comment on our draft report. On July 10, 1996, we wrote to the NIC's Chairman and requested his views on our draft report. On July 22, 1996, the DCI's Director of Congressional Affairs replied to us and stated that they would not comment on the substance or accuracy of our draft report because these issues "fall under the purview of intelligence oversight arrangements established by the Congress." As requested, the DCI's staff provided us with a security classification review, which we have incorporated into our final report.

Scope and Methodology

Our scope included a detailed review of NIE 95-19, and a comparison of this NIE to NIE 93-17, NIE 93-19, and recent unclassified studies. We did not attempt to independently evaluate foreign missile threats to the United States. To assess the objectivity of the NIE, we used various IC and other sources to develop standards for producing objective NIEs. Then we carefully reviewed NIE 95-19 and the two earlier NIEs to determine whether they met those standards. To compare NIE 95-19 to the 1993 NIEs, we conducted detailed comparisons of the judgments, evidence, and structure of the NIEs. The 1993 NIEs had a different focus than NIE 95-19, so we could not make direct comparisons in some areas. For example, unlike NIE 95-19,
the earlier NIEs did not address the Third World cruise missile threat.

Page 14 GAO/NSIAD-96-225 National Intelligenct- Estimates
B-274120

To compare NIE 95-19 to other unclassified studies, we conducted a variety of literature searches to identify such studies. Where possible, we identified the sources of data used by these studies; however, we did not evaluate the quality of their evidence or attempt to reconcile any differences they had with NIE 95-19.

Our scope was significantly impaired by a lack of cooperation by officials from the CIA, NIC, and the Departments of Defense and State. The Departments of Defense and State would not allow us access to their records. Defense and State spokespersons referred us to the DCI on all matters concerning NIEs. On March 6, 1996, we wrote to the DCI's Director of Congressional Affairs and requested access to CIA and NIC officials and documents. On June 17, 1996, he replied to us and declined to cooperate with our review. Ihs letter argued that our review of certain NIEs would be contrary to oversight arrangements for intelligence that the Congress has established. Specifically, he stated that "such subjects are under the direct purview of Congressional entities that have been charged with overseeing the Intelligence Community." Therefore, we were unable to discuss preparation of the NIEs with cognizant officials or review supporting documentation at the departments and agencies previously mentioned. Due to this lack of access, we also could not review other NIEs that may have covered similar topics as NIE 95-19. Except as previously mentioned, our review was conducted from April to June 1996 in accordance with generally accepted goverrunent auditing standards.

At your request, we plan no further distribution of this report until 30 days
after its issue date. At that time, we will provide copies to other congressional committees; the Chairman, President's Foreign Intelligence Advisory Board; the Secretaries of State, Defense, and Energy; Chairman, NIC; and the Director of Central Intelligence. Copies will also be made available to others on request.

Page 15 GAO/NSIAD-96-225 National Intelligence Estimates
B-274120

Please contact me at (202) 512-3504 if you or your staff have any questions
concerning this report. Major contributors to this report were Gary K Weeter, Assistant Director; Douglas M. Horner, Evaluator-in-Charge; Stephen L. Caldwell, Senior Evaluator, and James F. Reid, Senior Evaluator.

Sincerely yours,

Richard Davis
Director, National Security Analysis

(701098)

Page 16 GAO/NSIAD-96-225 National Intelligence Estimates

Notes

1. The following organizations may participate in preparing an NIE: the NIC, CIA, Defense Intelligence Agency, National Security Agency, State Department's Bureau of Intelligence and Research, Federal Bureau of Investigation, the intelligence organizations of the Departments of Treasury and Energy, and the military services.

2. Bettors' odds state the chance as, for example, "one out of three."

3. For more information on national missile defense, see Ballistic Missile Defense: Evolution and Current Issues (GAO/NSIAD-93-229, July 16,1993).

4. The distance depends on the launch site and the chosen U.S. target. For example, portions of Alaska are about 6,000 kilometers from North Korea; Honolulu is about 7,000 kilometers from North Korea. However, with forward-deployed missile launchers, the distance to the United States would be less.

5. The declared nuclear powers are Russia, China, France, the Urdted Kingdom, and the United States. However, U.S. capabilities and intentions are out of the scope of foreign intelligence estimates.

6. 0ur sources included the published views of Robert M. Gates, former DCI and Deputy Director for Intelligence, CIA; Joseph S. Nye, Jr., former Chairman, NIC; Harold P. Ford, former Acting Chairman, NIC; Gregory F. Treverton, former Vice Chairman, NIC; reports by the Vice President's National Performance Review, the Commission on the Roles and Capabilities of the United States Intelligence Community, and a study group on intelligence sponsored by the Council on Foreign Relations; and A Compendium of Analytic Tradecraft Notes, Vol. I, March 1996, published by the CIAs Product Evaluation Staff, Directorate of Intelligence.

7. Except for the 100-percent certainty implied by its main judgment previously discussed.

8.The Tradecraft of Analysis: Challenge and Change in the CIA, Douglas J. MacEachin, 1994, Consortium for the Study of Intelligence.

9. In our analysis of NIE 95-19's assumptions, we were assisted by an expert in the missile proliferation field, Dr. Richard H. Speier, an independent consultant. Previously, Dr. Speier worked in the Office of the Under Secretary of Defense for Policy, Department of Defense, and in the Non-Proliferation Bureau, U.S. Arms Control and Disarmament Agency.

10. The MTCR, begun in 1987, is the primary international regime aimed at stemming the proliferation of unmanned delivery systems (including missiles and space launch vehicles) and related technologies. The regime is not an international treaty, but rather a set of identical policies announced by member governments, to be implemented in parallel.

11. In counting dissents, we counted discrete topics of dissent. Sometimes more than one agency would dissent on a certain topic, and sometimes the dissent would appear multiple times (i.e., in the executive summary and supporting volumes).

12. We compared the treatment of 11 countries that both NIE 95-19 and NIE 93-19 analyzed. In describing the results of our comparison, we only used NIE 93-19's volume II (supporting analysis) to avoid double-counting information contained in NIE 93-19's volume I.