2
Committee Methodology

With the context and scope of its assignment established, the committee turned its attention to defining a robust methodology for technology warning that would be suitable for the diverse inquiries likely to stem from ongoing engagement between a standing committee of the National Research Council (NRC) and the intelligence community’s (IC’s) technology warning components. The proposed methodology is described in this chapter and tested through application in subsequent chapters.

KEY FEATURES OF THE METHODOLOGY

A robust methodology for technical inquiry should have four key features. First, to be accepted, it must be presented in a lexicon and structure appropriate for the user’s culture—in this case, for the culture in which the Defense Intelligence Agency (DIA) Technology Warning Division (the sponsor of this study) operates. Any communication of findings, conclusions, or recommendations offered by the committee must be expressed accordingly. The division makes use of weather-forecasting terminology (Futures, Watch, Warning, Alert)1 in the issuance of technology assessments, making the overall warning message regarding all products readily interpretable by any reader. The committee adopted and adapted the DIA’s vocabulary to characterize the relative status—and recommended action—for each technology.

The second key feature is that, to be relevant, the study methodology must be tied in a fundamental way to top-level Department of Defense (DOD) strategies. For example, the committee reviewed Joint Vision 2020 (JCS, 2000) to validate its selection of the technology topics addressed in this report. In future studies, to facilitate integration into the larger body of intelligence materials, the committee proposes that technology selections be derived through a more disciplined, RED team2 review of top-

1

The definitions used by the DIA for these terms are as follows: Futures—Create a technology roadmap and forecast; identify potential observables to aid in the tracking of technological advances. Technology Watch—Monitor global communications and publications for breakthroughs and integrations. Technology Warning—Positive observables indicate that a prototype has been achieved. Technology Alert—An adversary has been identified and operational capability is known to exist.

2

“RED” is used in this report to denote the adversary or an adversarial perspective (e.g., “RED team”).

Citation Manager

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 20
Avoiding Surprise in an Era of Global Technology Advances
2
Committee Methodology
With the context and scope of its assignment established, the committee turned its attention to defining a robust methodology for technology warning that would be suitable for the diverse inquiries likely to stem from ongoing engagement between a standing committee of the National Research Council (NRC) and the intelligence community’s (IC’s) technology warning components. The proposed methodology is described in this chapter and tested through application in subsequent chapters.
KEY FEATURES OF THE METHODOLOGY
A robust methodology for technical inquiry should have four key features. First, to be accepted, it must be presented in a lexicon and structure appropriate for the user’s culture—in this case, for the culture in which the Defense Intelligence Agency (DIA) Technology Warning Division (the sponsor of this study) operates. Any communication of findings, conclusions, or recommendations offered by the committee must be expressed accordingly. The division makes use of weather-forecasting terminology (Futures, Watch, Warning, Alert)1 in the issuance of technology assessments, making the overall warning message regarding all products readily interpretable by any reader. The committee adopted and adapted the DIA’s vocabulary to characterize the relative status—and recommended action—for each technology.
The second key feature is that, to be relevant, the study methodology must be tied in a fundamental way to top-level Department of Defense (DOD) strategies. For example, the committee reviewed Joint Vision 2020 (JCS, 2000) to validate its selection of the technology topics addressed in this report. In future studies, to facilitate integration into the larger body of intelligence materials, the committee proposes that technology selections be derived through a more disciplined, RED team2 review of top-
1
The definitions used by the DIA for these terms are as follows: Futures—Create a technology roadmap and forecast; identify potential observables to aid in the tracking of technological advances. Technology Watch—Monitor global communications and publications for breakthroughs and integrations. Technology Warning—Positive observables indicate that a prototype has been achieved. Technology Alert—An adversary has been identified and operational capability is known to exist.
2
“RED” is used in this report to denote the adversary or an adversarial perspective (e.g., “RED team”).

OCR for page 20
Avoiding Surprise in an Era of Global Technology Advances
level strategy documents (e.g., Joint Vision 2020) with an eye to identifying technologies that could be used to deny a BLUE3 capability deemed critical to U.S. military success.
The third key feature is that, to maintain focus and ensure timeliness, the study methodology must yield assessments built on a solid understanding of the technical feasibility of potential technology-based threats. This requirement leads to a capability-based approach for investigating and categorizing candidate technologies. Furthermore, the technical peer review process to which all NRC reports are subjected provides additional assurance of the technical quality of committee assessments.
Lastly, to be enduring, the methodology should accommodate evolving realities of science and technology (S&T) leadership, driven by the synergistic trends of globalization and commercialization described in Chapter 1. Traditionally, the United States has assumed that it leads the world in S&T. This perspective leads the technology warning community to look for indications that external actors are trying to “catch up,” or to exploit known technologies in new ways. Projected trends suggest that it should no longer automatically be assumed that the United States will lead technological advances in all relevant technologies. This reality imposes a new burden on the technology warning community, generating the need for it to search in different places and in different ways for the information needed to warn against technology surprise.
FOUNDATION OF THE METHODOLOGY
The committee believes that the Technology Warning Division can most effectively prioritize its limited resources by utilizing a capabilities-based approach with respect to assessing technologies. The landscape of potentially important emerging technologies is both vast and diverse. Ideally, the division should assess whether a given technology has the potential to pose a viable threat prior to commissioning in-depth analyses. Since the division is keenly interested in when specific technologies may mature to the point that they pose a threat to U.S. forces, a functional decomposition from an adversarial, or RED, perspective is most useful. The methodology defined by the committee begins with the following focus question: What capabilities does the United States have that, if threatened, impact U.S. military preeminence?
In general, U.S. capabilities could be threatened either through direct denial of or disruption of BLUE capabilities or via RED capabilities that negate or significantly diminish the value of BLUE capabilities (e.g., improvised explosive devices (IEDs) being employed by insurgent forces in Iraq).
Joint Vision 2020 was used to define the basic framework for U.S. military capabilities deemed vital to sustained success (JCS, 2000). The overarching focus of this vision is Full Spectrum Dominance—achieved through the interdependent application of four operational concepts (Dominant Maneuver, Precision Engagement, Focused Logistics, and Full Dimensional Protection) and enabled through Information Superiority, as illustrated in Figure 2-1 (JCS, 2000). The committee selected the four operational concepts, together with Information Superiority, as the foundation for its assessment methodology. Joint Vision 2020 provides the definitions presented in Box 2-1.
The committee also noted the importance of technology warning with respect to the “Innovation” component of Joint Vision 2020 shown in Figure 2-1, since “leaders must assess the efficacy of new ideas, the potential drawbacks to new concepts, the capabilities of potential adversaries, the costs versus benefits of new technologies, and the organizational implications of new capabilities” (JCS, 2000).
From this foundation the committee then identifies specific capabilities in accordance with the
3
“BLUE” is used in this report to denote U.S. military forces.

OCR for page 20
Avoiding Surprise in an Era of Global Technology Advances
FIGURE 2-1 Concepts constituting the basic framework for U.S. military capability as defined by Joint Vision 2020. (See Box 2-1.) SOURCE: JCS (2000).
previously defined focus question—What capabilities does the United States have that, if threatened, impact U.S. military preeminence?
While the U.S. military has devoted significant time to the definition of vital capabilities in alignment with Joint Vision 2020, the committee made no effort in this first report to synchronize its derivations or definitions, or to provide a complete decomposition of the operational concepts and enablers into their underlying capabilities. Rather, committee members selected a few evolving technologies and assessed the potential for those technologies to threaten important U.S. capabilities. Given that the committee’s proposed basic methodology is adopted, future studies will analyze more comprehensively the threats to a taxonomy of U.S. military capabilities that derives from the operational concepts envisioned by Joint Vision 2020. The basic methodology developed by the committee is summarized in Box 2-2 and is described in greater detail in subsequent sections.
IDENTIFY
The next step of the proposed assessment methodology is performed from the RED perspective. The central question here is as follows: What are the evolving technologies that, in the hands of U.S. adversaries, might be used to threaten an important U.S. military capability? A corollary question is, What technologies, if rapidly exploited by the U.S. military, are likely to yield sustained technological superiority? However, this issue was addressed only peripherally, given the division’s focus on technology warning.
Having identified a technology of potential interest, the next challenge becomes the derivation of “indicators” or “observables” that may suggest adversarial adoption or exploitation of that technology.

OCR for page 20
Avoiding Surprise in an Era of Global Technology Advances
BOX 2-1
Relevant Definitions from Joint Vision 2020 Serving as Foundation for Assessment Methodology
Information Superiority is the capability to collect, process, and disseminate an uninterrupted flow of information while exploiting or denying an adversary’s ability to do the same. Information superiority is achieved in a noncombat situation or one in which there are no clearly defined adversaries when friendly forces have the information necessary to achieve operational objectives.
Dominant Maneuver is the ability of joint forces to gain positional advantage with decisive speed and overwhelming operational tempo in the achievement of assigned military tasks. Widely dispersed joint air, land, sea, amphibious, special operations and space forces, capable of scaling and massing force or forces and the effects of fires as required for either combat or noncombat operations, will secure advantage across the range of military operations through the application of information, deception, engagement, mobility and counter-mobility capabilities.
Focused Logistics is the ability to provide the joint force the right personnel, equipment, and supplies in the right place, at the right time, and in the right quantity, across the full range of military operations. This will be made possible through a real-time, web-based information system providing total asset visibility as part of a common relevant operational picture, effectively linking the operator and logistician across Services and support agencies.
Precision Engagement is the ability of joint forces to locate, surveil, discern, and track objectives or targets; select, organize, and use the correct systems; generate desired effects; assess results; and reengage with decisive speed and overwhelming operational tempo as required, throughout the full range of military operations.
Full Dimensional Protection is the ability of the joint force to protect its personnel and other assets required to decisively execute assigned tasks. Full dimensional protection is achieved through the tailored selection and application of multilayered active and passive measures, within the domains of air, land, sea, space, and information across the range of military operations with an acceptable level of risk.
SOURCE: JCS (2000).
Although targeted intelligence-collection methods remain important, in this report the committee focuses on observables that may be derived from open source analysis—leveraging the effects of the information revolution and acknowledging that the twin forces of globalization and commercialization provide new sources of relevant information. At the same time, however, the committee recognizes the difficulty of discerning when technological advances portend emerging threats rather than societal benefits.
A sample chart—Chart 2-1—exemplifies how each technology is assessed.

OCR for page 20
Avoiding Surprise in an Era of Global Technology Advances
BOX 2-2
Proposed Methodology for Technology Warning
Foundation
Joint Vision 2020a Operational Concepts and Information Superiority
•
Focus
What capabilities does the United States have that, if threatened, impact U.S. military preeminence?
•
Identify
What are the evolving technologies that, in the hands of U.S. adversaries, might be used to threaten an important U.S. military capability?
What are the observables that may indicate adversarial adoption or exploitation of such technologies?
•
Assess
Accessibility: How difficult would it be for an adversary to exploit the technology?
Maturity: How much is known about an adversary’s intentions to exploit the technology?
Consequence: What is the impact on U.S. military capability should the technology be employed by an adversary?
•
Prioritize
Identify: What are the relative resources to be applied to each emerging technology to support the technology warning process?
•
Task
Establish and assign intelligence-information-collection requirements.
aSOURCE: JCS (2000).
CHART 2-1 Example of Technology Assessment Chart
Technology
Observables
Brief description of technology
Brief description of observables.
Accessibility
Maturity
Consequence
Level 1, 2, or 3
Technology Futures
Technology Watch
Technology Warning
Technology Alert
Short characterization.

OCR for page 20
Avoiding Surprise in an Era of Global Technology Advances
ASSESS
The committee’s assessment methodology involves characterization of a technology in terms of three variables: Accessibility, Maturity, and Consequence. Priorities for more detailed analyses may derive from any individual variable or any combination of the three.
Accessibility
The Accessibility variable focuses on the question, How difficult would it be for an adversary to exploit the technology? It addresses the ability of an adversary to gain access to and exploit a given technology. This assessment is divided into three levels:
Level 1. The technology is available through the Internet, being a commercial off-the-shelf item; low sophistication is required to exploit it.
Level 2. The technology would require a small investment (hundreds of dollars to a few hundred thousand dollars) in facilities and/or expertise.
Level 3. The technology would require a major investment (millions to billions of dollars) in facilities and/or expertise.
In general, Level 1 technologies are those driven by the global commercial technology environment; they are available for exploitation by a diverse range of potential adversaries. Level 3 technologies, by contrast, are typically accessible only to state-based actors. The indicators likely to be of value in determining an adversary’s actual access to a given technology vary by level as well as by the type of technology.
Maturity
The Maturity variable focuses on the question, How much is known about an adversary’s intentions to exploit the technology? It integrates what is known about an adversary’s actions, together with an evaluation of the state of play with respect to the technology of interest. At the highest level, called Technology Alert, an adversary has been identified and an operational capability has been observed. At the lowest level, Technology Futures, the potential for a technology-based threat has been identified, but no positive indicators have been observed. The Maturity assessment is divided into four categories: the first two (the lower levels) suggest further actions for the technology warning community; the other two indicate the need for immediate attention by military leadership:
Futures. Create a technology roadmap and forecast; identify potential observables to aid in the tracking of technological advances.
Technology Watch. Monitor (global) communications and publications for breakthroughs and integrations.
Technology Warning. Positive observables indicate that a prototype has been achieved.
Technology Alert. An adversary has been identified and operational capability is known to exist.
Given the potential for disruptive advances through technological breakthroughs or innovative integration, as well as the difficulty of identifying and tracking meaningful indicators, any particular technology is unlikely to progress sequentially through the various categories of Maturity listed above.

OCR for page 20
Avoiding Surprise in an Era of Global Technology Advances
As indicated at the beginning of this chapter, the committee adopted and adapted the DIA’s terminology in defining these categories. The definitions are likely to evolve as the process matures. The committee sees significant value in this basic approach, however, since it divorces the challenge of technology warning from the discrete time lines associated with “prediction,” which are almost invariably inaccurate.
Consequence
Characterization of a technology in terms of the Consequence variable involves addressing the question, What is the impact on military capability should the technology be employed by an adversary? It involves assessing the impact of the postulated RED technology on the capability of BLUE forces. This impact can range from denial or negation of a critical capability to the less-consequential level of annoyance or nuisance. A corollary assessment may be made as to the locus of impact—that is, whether the technology affects a single person, as in the case of an assassination, or creates a circumstance of mass casualty and attendant mass chaos.
PRIORITIZE
The objective of the prioritization step of the methodology is to respond to the question, What are the relative resources to be applied to each emerging technology to support the technology warning process? This step is intended to harmonize the distinct nodes of observed capability, demonstrated intent, resources available, and the inherent cost of inaction. Prioritization is key to the technology warning methodology, since the Technology Warning Division lacks the resources to fully analyze every conceivable evolving technology. It is equally important to recognize that prioritization is an integral part of each methodology parameter. The prioritization of individual parameters is based on the levels of change detection and potential impact. By prioritizing the parameter, the division can focus subsequent analyses over a smaller subset of an assigned change detection domain. Priority assignment is essential to enable the focusing of more sophisticated information-gathering tools and analytic techniques on the areas of highest potential concern.
The prioritization methodology lends itself to any number of commercially available tools and techniques designed for assistance in establishing and maintaining a logical and consistent focus as well as the flexibility to react to the dynamics of technology change and country-of-interest variability. During the prioritization process, it will be important to establish measures of performance to allow critical analysis as well as change management in order to improve the overall process. The end result of the prioritization process is to provide for actionable awareness with which to influence analysis and tasking, the last of the methodology parameters.
The committee envisions that prioritization would be accomplished in close consultation with the technology warning community. It made no attempt to further develop the prioritization process in this report.
TASK
The Technology Warning Division will inevitably have unmet needs for additional information and/ or intelligence relating to the prioritized list of evolving technologies. Although some needs may be met through division-chartered research, others will require the assistance of the broader intelligence community.

OCR for page 20
Avoiding Surprise in an Era of Global Technology Advances
The task step—Establish and assign intelligence-information-collection requirements—involves the dissemination of collection requirements to other IC components and subordinate agencies. Such requirements must provide sufficient specificity to enable interpretation by collectors who are not necessarily literate in the specific technology. The requirements may include general instructions for accomplishing the mission. It is envisioned that some of the observables postulated in the Identify step of the methodology will provide a useful basis for such tasking.
The results from collection efforts will be integrated back into the assessment step in order to refine, reprocess, and update the division’s understanding of a given technology. This analysis may stimulate the issuance of a new report to the division’s customers to inform them of changes in the assessed maturity of that technology.
USING THE METHODOLOGY IN THIS REPORT
To test the robustness of the proposed technology warning methodology, the committee applied it in order to assess four key areas in this initial report. It should be noted that this initial exercise was necessarily circumscribed by the domain expertise represented in the committee members and by the shortness of time for broader outreach to the technical community at large. Furthermore, since the methodology emerged in parallel with the committee’s technology assessments, the approaches taken were not entirely consistent.
The foundation provided by Joint Vision 2020 and augmented by the military and professional backgrounds of committee members was used to select the following four key capabilities to assess:
Information superiority (Chapter 3),
Air superiority (Chapter 4),
Discrimination between friends/foes/neutrals (Chapter 5), and
Battle readiness and communications superiority (Chapter 6).
Chapters 3 through 6 each address the “Identify” activity with examples of evolving technologies that may threaten the capability and potential indicators that such technology development is underway. The “Assess” activity then examines opportunity and motivation for adversarial technology development and/or employment, posits change detection relative to the indicators, and assesses likely impact. Preliminary characterizations of accessibility, maturity, and consequence are provided for most evolving technologies, although the level of specificity is variable.
Subsequent steps (i.e., “Prioritize” and “Task”) of the proposed methodology require customer inputs and actions and are left to future study efforts.
REFERENCE
JCS (Joint Chiefs of Staff). 2000. Joint Vision 2020. Director for Strategic Plans and Policy, J5, Strategy Division. U.S. Government Printing Office, Washington, D.C. June.