FORCE FIELD ANALYSIS

Foreword to the Second Edition

Jack Davis, CIA Trailblazer

ome fifty years ago, Sherman Kent, legendary Chairman

of the Board of National Estimates, sent an early advocate of structured analysis to make his case to a new butwell-regarded member of his Estimates staffJack Davis.I listened, with feigned interest, as the advocate spelledout the virtues of externalizing and evaluating the assumptions supporting key judgments of assessments. To put itdirectly, I saw no need to change the way I did analysis.I rather abruptly terminated the meeting by averring,There is no piece of paper big enough to hold all thethoughts influencing my predictions of future developmentsin [the countries I work on]. A response that while nothelpful was not unreasonable at a time when computers hadnot yet replaced typewriters and my ego had not yet beentempered by several avoidable misjudgments.It took some twenty years for me fully to appreciate andvigorously promote the analytic benefits of structured analysis, especially the insurance provided against the hazardsof judgments based solely on internalized critical thinking,unstructured peer debate, and subjective boss review.Several factors abetted the growing influence within theIntelligence Community (IC) of what was first called Alternative Analysis and is now called Structured Analytic Techniques (SATs). A string of highly publicized intelligence failures setoff calls for changes in the conduct of analysis thatgave advocates of structured analysis a foot in thedoor. A small but influential cadre of intelligenceprofessionals began teaching and preaching aboutthe mental, bureaucratic, and political obstacles tosound analysis spelled out with authority by RobertJervis in the foreword to the first and presenteditions of Cases in Intelligence Analysis. Leading students of analytic methodology, includingprominently the two authors of this book, developed,

tested, and refined through case studies an

impressive array of SATs to address said obstacles.These personal observations serve as a preface to whatI see as the valuable contributions to the practice of analysis of the second edition of Cases in Intelligence Analysis:Structured Analytic Techniques in Action. SATs are notsilver bullets that automatically improve the assessmentat hand and simultaneously enhance the critical thinking ofthe responsible analyst(s). The well-tested procedures followed in the book hold promise of achieving both goals. The cases range in challenge from reducinguncertainty on data-rich issues by structuredorganization of what is known (e.g., chronologies),to reducing uncertainty on data-poor issues bystructured assessments of multiple plausibleoutcomes (e.g., Scenarios Analysis). The case texts start with stating the nature of analyticchallenges, the essence of likely correctives, costbenefit expectations from structuring, per se, andonly then the effectiveness of selected SATs. Each case has a list of recommended substantivereadings, a reminder to participants that expertknowledge serves to facilitate effective execution ofstructured analysis. The focus of learning is on sound analytic processfor example, changing the lens for viewing the caseissuerather than on coming up with the correctanswer. In the same vein, the book shows the perils ofoverconfidence and heavy reliance on existingparadigms as well as the rewards of doubting andchallenging the conventional wisdom.For these and other reasons the book serves well potential and practicing analysts not only in intelligence but in all

xv

xvi Foreword to the Second Edition

fields of endeavor where the charge is, in effect, managingsubstantive uncertainty to serve clients charged with decision making and action taking.A brief assessment of the books potential value for onesuch group:

As in the 1960s, veteran analysts assigned to craft the

most important (cant fail) assessments out of respectfor their substantive expertise and critical thinking skillstend to resist intrusion of formal structuring. Some analysts see SATs as unnecessary if not also disruptive.Managers may temper this resistance by raising from

I believe that combining the best of substantive expertise and critical thinking with the best of structuredanalysis provides the best protection against avoidableanalytic shortfalls. Cases in Intelligence Analysis providesthe wherewithal for helping IC analysts move toward thatgoal.

Preface

heres an old anecdote about a tourist who stops a New

Yorker on the street and asks, How do you get toCarnegie Hall? The New Yorker replies, Practice, practice,practice. The humor in the anecdote highlights an important truth: the great musicians who play at Carnegie Hallhave a lot of innate talent, but none of them got there without a lot of practice.Really great analysts have a lot of innate talent too.Whether in government, academia, or business, analysts areusually curious, question-asking puzzle solvers who havedeep expertise in their subject matter. Not surprisingly, theylike to be right, and they frequently are. And yet, the IraqWMD Commission Report shows that analysts can bewrong. Analytic failures often are attributed to a range ofcognitive factors that are an unavoidable part of beinghuman, such as faulty memory, misperception, and a rangeof biases. Sometimes the consequences are unremarkable.Other times, the consequences are devastating. Structuredanalysis gives analysts a variety of techniques they can useto mitigate these cognitive challenges and potentially avoidfailures, if analysts know when and how best to apply them.This book is designed to give analysts practice usingstructured analytic techniques.Improving ones cognitive processes by using thetechniques discussed in this book can be challenging butalso rewarding. The techniques themselves are not thatcomplicated, but they can push us out of our intuitive andcomfortablebut not always reliablethought processes.They make us think differently in order to generate newideas, consider alternative outcomes, troubleshoot our ownwork, and collaborate more effectively.This process is like starting a fitness regimen for thebrain. At the beginning, your muscles burn a little. Butover time and with repetition, you become stronger, andthe improvements you see in yourself can be remarkable.Becoming a better thinker, just like becoming a betterathlete, requires practice. We challenge you to feelthe burn.

AUDIENCEThis book is for anyone who wants to explore new ways ofthinking more deeply and thoroughly. It is primarilyintended to help up-and-coming analysts in colleges anduniversities, as well as intelligence professionals, learntechniques that can make them better analysts throughouttheir careers. But this book is just as salient for seasonedintelligence veterans who are looking for ways to brush upon skillsor even learn new ones. The cases also areintended for teams of analysts who want to rehearse andrefine their collaboration skills so that when reallife situations arise, they are prepared to rise to thechallenge together.CONTENT AND DESIGNWe chose the case study format because it provides anopportunity to practice the techniques with real-lifecontemporary issues. It is also a proven teaching method inmany disciplines. We chose subject matter that is relativelyrecentusually from within the past decadeand thatcomprises a mix of better- and lesser-known issues. In allcases, we strove to produce compelling and historicallyaccurate portrayals of events; however, for learning purposes,we have tailored the content of the cases to focus on keylearning objectives. For example, we end many of the caseswithout revealing the full outcome. Several cases, such asWho Murdered Jonathan Luna?, have no known outcome.But whether or not the outcome is known, we urge students tojudge their performance on the merits of their analytic process.Like mathematics, just arriving at a numerical value orcorrect outcome is not enough; we need to show our work.The value of the cases lies in the process itself and in learninghow to replicate it when real-life analytic challenges arise.The seventeen cases and analytic exercises in this bookhelp prepare analysts to deal with the authentic problems andreal-life situations they encounter every day. Taken as awhole, the seventeen cases walk through a broad array of

xvii

xviii Prefaceissues such as how to identify mindsets, mitigate biases,challenge assumptions, think expansively and creatively,develop and test multiple hypotheses, create plausiblescenarios, identify indicators of change, validate those indicators, frame a decision-making process, and troubleshootanalytic judgmentsall of which reinforce the main elementsof critical thinking that are so important for successfulanalysis. Individually, each chapter employs a consistentorganization that models a robust analytic process bypresenting the key questions in the case, a compelling andwell-illustrated narrative, and carefully chosen recommendedreadings. Each also includes question-based analyticexercises that challenge students to employ structuredanalytic techniques and to explicate the value added byemploying structured techniques.INSTRUCTOR RESOURCESAs instructors ourselves, we understand how important it isto provide truly turnkey instructor resources. The InstructorMaterials that accompany this book are free to all readers ofthis book as a downloadable .pdf, and graphics from boththe case book and the Instructor Materials are available asfree, downloadable .jpeg and PowerPoint slides. We haveclassroom-tested each case study and applied what we havelearned to enhance the Instructor Materials and betteranticipate the instructors needs. We believe they are just asuseful to working analysts and students seeking to learnhow best to apply the techn iques. Just like the casesthemselves, the Instructor Materials employ a consistentorganization across all cases that puts the case and theanalytic challenges in context, offers step-by-step solutionsfor each exercise, and provides detailed conclusions and keytakeaways to enhance classroom discussion.

DISCLAIMERAll statements of fact, opinion, or analysis expressed in thisbook are those of the authors and do not reflect the officialpositions of the Office of the Director of NationalIntelligence (ODNI), the Central Intelligence Agency (CIA),and the Federal Bureau of Investigation (FBI), or any otherUS government agency. Nothing in the contents should beconstrued as asserting or implying US gove rnmentauthentication of information or agency endorsement of theauthors views. The materials in the book have beenreviewed by the ODNI, FBI, and CIA only to prevent thedisclosure of classified material.

About the Authors

Sarah Miller Beebe began thinking about a book of cases

during her career as an analyst and manager at the CentralIntelligence Agency. A variety of broadening experiences,including an assignment as director for Russia on theNational Security Council staff and a position as a nationalcounterintelligence officer at the Office of the NationalCounterintelligence Executive, drove home the need forrigorous and effective approaches to intelligence analysis.It became apparent to her that cases could not only teachimportant analytic lessons surrounding historical events butalso give analysts experience using a question-based thinkingapproach underpinned by practical techniques to improvetheir analyses. Now, as owner of Ascendant Analytics, shehelps organizations apply such techniques against their specific analytic problems.Randolph H. Pherson has spearheaded teaching and developing analytic techniques and critical thinking skills in theIntelligence Community. He is the author of the Handbook ofAnalytic Tools and Techniques and has coauthored Structured

Analytic Techniques for Intelligence Analysis with Richards

J. Heuer Jr., Critical Thinking for Strategic Intelligence withKatherine Hibbs Pherson, and the Analytic Writing Guidewith Louis M. Kaiser. Throughout his twenty-eight-yearcareer at the Central Intelligence Agency, where he lastserved as national intelligence officer for Latin America, hewas an avid supporter of ways to instill more rigor in theanalytic process. As president of Pherson Associates, LLCsince 2003 and chief executive officer of Globalytica, LLCsince 2009, he has been a vigorous proponent of a casebased approach to analytic instruction.Together, Beebe and Pherson have developed and tested newanalytic tools and techniques, created interactive analytictradecraft courses, and facilitated analytic projects. In theirwork as analytic coaches, facilitators, and instructors, theyhave found the case approach to be an invaluable teachingtool. This second edition of case studies is their most recentcollaboration and one that they hope will help analysts ofall types improve both the quality and impact of their work.

xix

Introduction

or the past two decades, a quiet movement has been

gathering momentum to transform the ways inwhich intelligence analysis is practiced. Prior to thismovement, analysts generally approached their tradecraftas a somewhat mysterious exercise that used their expertjudgment and inherent critical thinking skills. Althoughsome analysts produced solid reports, this traditionalapproach was vulnerable to a large number of commoncognitive pitfalls, including unexamined assumptions,confirmation bias, and deeply ingrained mindsets thatincreased the chances of missed calls and mistaken forecasts. 1 Without a means of describing these invisiblemental processes to others, instruction in analysis wasdifficult, and objective assessments of what worked andwhat did not work were nearly impossible. Moreover, thistraditional approach tended to make analysis an individual process rather than a group activity; when conclusions were reached through internal processes that wereessentially intuitive, groups of analysts could notapproach problems on a common basis, and consumers ofanalysis could not discern how judgments had beenreached. Absent systematic methods for making the analytic process transparent, problems that required collaboration across substantive disciplines and geographicregions were particularly prone to failure.The desire for change has been propelled by a growingawareness that analytic performance has too often fallenshort. Former Central Intelligence Agency (CIA) DeputyDirectors of Intelligence Robert Gates and Doug MacEachindid much to spark this awareness within the IntelligenceCommunity during the 1980s and 1990s, criticizing whatthey regarded as flabby thinking and insisting that CIAanalysts employ evidence and argumentation in much morerigorous and systematic ways. To address these problems,

Gates focused on raising the quality of analytic reviews, and

MacEachin established a set of standard corporate practicesfor analytic tradecraft, which were disseminated and taughtto CIA analysts.2 Subsequent investigations into the failureto anticipate Indias 1998 nuclear test, the surprise terroristattacks of 11 September 2001 in the United States, and theerroneous judgments about Iraqs possession of weapons ofmass destruction brought the need for analyticimprovements into broader public view.But simply realizing that improvements in analysis wereneeded was not sufficient to produce effective change. Anunderstanding of the exact nature of the analytic problems,as well as a clear sense of how to address them, wasrequired. Richards J. Heuer Jr., a longtime veteran of theCIA, provided the theoretical underpinnings for a newapproach to analysis in his pioneering work Psychology ofIntelligence Analysis.3 In this, Heuer drew upon the work ofleading cognitive psychologists to explain why the humanbrain constructs mental models to deal with inherentuncertainty, tends to perceive information that is consistentwith its beliefs more vividly than it sees contradictory data,and is often unconscious of key assumptions that underpinits judgments. Heuer argued that these problems couldbest be overcome by increasing the use of tools andtechniques that structure information, challengeassumptions, and explore alternative interpretations. Thesetechniques have since come to be known collectively asstructured analytic techniques, or SATs. He developed oneof the earliest techniques, called Analysis of CompetingHypotheses, to address problems of deception inintelligence analysis. It now is being used throughout thecommunity to address a variety of other analytic problemsas well, helping to counter the natural tendency towardconfirmation bias.4

2 IntroductionSince the pioneering efforts of Heuer to understand andaddress common cognitive pitfalls and analyticpathologies, considerable progress has been made indeveloping a variety of new SATs and defining the waysthey may be used. In 2011, Heuer joined one of the authorsof this volume, Randolph H. Pherson, in publishing themost comprehensive work on this subject to date,Structured Analytic Techniques for Intelligence Analysis.5The book describes how structured analysis compares toother analytic methods, including expert judgment andquantitative methods, and provides a taxonomy of eightfamilies of SATs and detailed descriptions of some fiftyfive techniques. By including an in-depth discussion ofhow each technique can be used in collaborative teamprojects and a vision for how the techniques can besuccessfully integrated into analysis done in theintelligence, law enforcement, and business communities,Heuer and Pherson challenged analysts from all disciplinesto harness the techniques to produce more rigorous andinformative analysis.WHY A BOOK OF CASES?The books published by Heuer and Pherson have helpedanalysts become familiar with the range of availablestructured analytic techniques and their purposes, but littlework has been done to provide analysts with practicalexercises for mastering the use of SATs. This book isdesigned to fill that gap. As such, it is best regarded as acompanion to both Psychology of Intelligence Analysis andStructured Analytic Techniques for Intelligence Analysis. Thecases in this bookvivid, contemporary issues coupledwith value-added analytic exercisesare meant to bridgethe worlds of theory and practice and bring analysis to life.They compel readers to put themselves in the shoes ofanalysts grappling with very real and difficult challenges.Readers will encounter all the complexities, uncertainties,and ambiguities that attend real-life analytic problems and,in some cases, the pressures of policy decisions that hang inthe balance.We have chosen a case study approach for severalreasons. First, the techn ique has proved an effectiveteaching tool in a wide variety of disciplines, fosteringinteractive learning and shifting the emphasis frominstructor-centric to student-centric activity while usuallysparking interest in issues previously unfamiliar tostudents.6 The use of the case study approach also allows

students to tackle problems on either an individual or a

group basis, facilitating insights into the strengths andweaknesses of various approaches to independent andcollaborative analysis. Although the seventeen cases in thisbook are used to illustrate how structured analysis can aidthe analytic process, they also can be used to catalyzebroader discussions about current issues, such as foreignpolicy decision making, international relations, lawenforcement, homeland security, and many other topicscovered in the book. It is through these types of practicalexercises and discussions that analysts learn to put problemsin context and develop and execute clear and effectiveanalytic frameworks.The cases cover recent events and include a mix offunctional and regional issues from across the world. Westrive to present compelling and historically accurateportrayals of eventsalbeit tailored for learning purposesto demonstrate how SATs can be applied in the fastbreaking and gritty world of real-life events and policydecisions. To discourage students from gaming theiranalysis, however, we end many of the cases withoutrevealing the full outcome in the main text, and severalsuch as Who Murdered Jonathan Luna?have no knownoutcome. But whether or not the outcome is known, thepurpose of the exercises is not simply to arrive at thecorrect judgment or forecast contained in the InstructorMaterials or to make the analysis mirror the actual outcome.As with exercises in mathematics, arriving at the propernumerical value or outcome does not demonstrate mastery;that can only be demonstrated by showing the math that ledone to the proper outcome. The value of the cases lies inlearning the analytic processes themselves and how to applythem to real-life problems.ORDER AND ORGANIZATIONThe order of the cases roughly mirrors the hierarchy ofproblems that analysts face when assuming responsibilityfor a new portfolio or account. Typically, when starting anew assignment, analysts are asked to become familiarwith past analytic reports and judgments on the topic.When done well, such a process will uncover preexistingmindsets and expose unsupported assumptions. Thefirst cases in the bookWho Poisoned KarinnaMoskalenko?, The Anthrax Killer, Cyber H2O, Jousting with Cuba over Radio Marti, Is Wen Ho Lee a Spy?,The Road to Tarin Kowt, and Who Murdered Jonathan

Introduction 3

Luna?are designed to teach SATs that challenge prevailing mindsets and develop alternative explanations forevents.As analysts gain more familiarity with the issues forwhich they are respons ible, they often encounter newdevelopments for which no line of analysis has beendeveloped. In such circumstances, analysts requiretechniques for developing and testing new hypothesesand for visualizing the data in creative and thoughtprovoking ways. The Assassination of Benazir Bhutto,Death in the Southwest, The Atlanta OlympicsBombing, and The DC Sniper are designed with thesegoals in mind.Finally, as analysts master their subjects, they are askedto tackle problem sets that are arguably the most difficultanalytic challenges: understanding the perceptions andplans of foreign adversaries and forecasting uncertain futuredevelopments shaped by dynamic sets of drivers. InColombias FARC Attacks the US Homeland, Understanding Revolutionary Organization 17 November, andDefending Mumbai from Terrorist Attack, students putthemselves in the shoes of the adversary and develop arange of plausible future outcomes, while in Iranian Meddling in Bahrain and Shades of Orange in Ukraine students not only develop scenarios but also actively consider arange of future outcomes and specific indicators that a particular outcome is emerging. Violence Erupts in Belgraderounds out the cases by placing students in a direct decisionsupport role in which they must not only provide assessments about the forces and factors that will drive events butalso develop a decision framework and troubleshoot theiranalysis.Each of our case studies employs a consistent internalorganization that guides the student through an analyticprocess. We begin each case study by listing severaloverarching Key Questions. These questions are designedas general reading guides as well as small-group discussionquestions. The questions are followed by the CaseNarrative, which tells the story of the case. This isfollowed by a Recommended Readings section. The finalsection, Structured Analytic Techniques in Action, presentsfocused intelligence questions and exercises to guide thestudent through the use of several structured analytictechniques and toward self-identification of the valueadded by SAT-aided analysis. The turnkey InstructorMaterials, which are available to analysts, students, andinstructors via download, put the learning points for the

cases in context, present detailed explanations of how to

successfully apply the techn iques, and provide caseconclusions and additional key takeaways that may beused in instruction.TECHNIQUE CHOICEThe techniques are matched to the analytic tasks in eachcas e. For example, in Who Pois oned Kar innaMoskalenko?, there are many unanswered questions thatrequire the kind of divergent and imaginative thinking thatStarbursting can prompt. In Violence Erupts in Belgrade,Force Field Analysis helps the analyst make a judgmentabout the prospect of additional violencean analyticjudgment that will shape decisions about what to do toprotect the US Embassy. Each case includes at least threetechnique-driven exercises, and each exercise begins with adiscussion of how the technique can be used by analysts totackle the kind of problem presented in the exercise. Spaceconstraints preclude the inclusion of all techniques thatmight be applicable for each case; we chose those that wefelt were most salient and illustrative. For example, nearlytwo-thirds of the cases implicitly or explicitly include a KeyAssumptions Check or Structured Brainstorming, butthese core techniques could easily be applied to all thecases. Overall, we strove to include a variety of SATsthroughout the book that are representative of each of theeight families of techniques. To help orient readers, wehave included a secondary, matrixed table of contents thatdetails the cases and the full complement of techniquesthat each utilizes.HOW CAN THESE CASES BESTFACILITATE LEARNING?Whether students are working alone or in small groups, thecases are most effective when students and instructors viewthem as opportunities to test and practice new ways ofthinking that can help them break through the cognitivebiases and mindsets that are at the core of so many analyticfailures. Viewed this way, the techniques are a means bywhich analysts can practice robust analytic approaches, notan end in and of themselves. Our goal was to give analysts afun and effective way to hone their cognitive skills. We hopewe have hit the mark, and we welcome feedback on thecases and the techniques as well as suggestions for theirrefinement and further development.

4 IntroductionNOTES 1. See Rob Johnston, Analytic Culture in the U.S. IntelligenceCommunity: An Ethnographic Study (Washington, DC: Center forthe Study of Intelligence, Central Intelligence Agency, 2005), http://www.fas.org/irp/cia/product/analytic.pdf, 2223. What tends tooccur is that the analyst looks for current data that confirms theexisting organizational opinion or the opinion that seems mostprobable and, consequently, is easiest to support....This tendencyto search for confirmatory data is not necessarily a consciouschoice; rather, it is the result of accepting an existing set of hypotheses, developing a mental model based on previous corporateproducts, and then trying to augment that model with current datain order to support the existing hypotheses. 2. See Jack Davis, Introduction: Improving IntelligenceAnalysis at CIA; Dick Heuers Contribution to IntelligenceAnalysis, in Psychology of Intelligence Analysis, ed. Richards J.Heuer Jr. (Washington, DC: Center for the Study of Intelligence,Central Intelligence Agency, 1999, and reprinted in 2007 byPherson Associates, LLC, Reston, VA, http://www.pherson.org),

1 Who Poisoned Karinna Moskalenko?

TECHNIQUE 1: PREMORTEM ANALYSIS AND

STRUCTURED SELF-CRITIQUE

his case has been written to approximate the

information environment that analysts confronted inthinking about this case as it unfolded in 2008. To producesound analysis, students must consciously go beyond themental framework established by the media coverage andknown history that surrounded the case. The exercise isaimed at pushing the student to challenge the existingmindset that prevailed at the time and to question theinformation presented in the media coverage.The Karinna Moskalenko case study details the challengesposed by quickly moving events punctuated by anomalousevidence, ingrained mindsets, misleading reports, and subconsciously held biases. As students begin their analysis ofthis case, the court of public opinion has already spoken;Western press coverage has pointed its finger at Moscow evenas it has raised and then dismissed out of hand the possibilitythat it could perhaps...[be] an unfortunate accident.1Task 1.Conduct a Premortem Analysis and Structured Self-Critique2of the reigning view in the case study that KarinnaMoskalenko is the latest victim in a series of alleged Russianattacks on Kremlin critics.Step 1: Imagine that a period of time has passed since youpublished your analysis that contains the reigning view juststated. You suddenly learn from an unimpeachable sourcethat the judgment was wrong. Then imagine what couldhave caused the analysis to be wrong.

The first two steps in the Premortem Analysis are rightbrain-led, creative brainstorming. This process asks analysts to imagine a future in which they have been provedwrong and work backward to try to identify the possiblecauses. In essence, they are identifying the weak links intheir analysis in order to avoid these potential pitfalls priorto publishing the analysis. Most analysts are more leftbrained than right brained, which often makes imaginationtechniques like brainstorming challenging. However, whencoupled with the systematic, left-brained checklist thatcomprises the second half of the Premortem Analysis,brainstorming can be the first step toward identifyingsometimes fatal analytic flaws. It is important to encouragestudents to be as creative as possible when brainstorming,keeping all ideas in play.In this case, a brainstorming session might prompt students to consider the following: New evidence comes to light that suggestssomeone other than the Russians is behind thepoisoning (e.g., her husband, her children, anacquaintance, a colleague at work, or a case ofmistaken identity). The toxicology reports were faked. She isnt ill. The mercury was accidentally placed in the vehicle(e.g., by her kids, the former owner of the vehicle, orsomeone else).Step 2: Use a brainstorming technique to identify alternative hypotheses for how the poisoning could have occurred.Keep track of these hypotheses.

6 Chapter 1In this case, students might identify a number of alternativeperpetrators of the crime. They could include the following: Karinna Moskalenkos husband. Moskalenko herself, who staged the poisoning withor without the assistance of her husband to put theRussian government on the defensive. A jealous work colleague. An acquaintance not connected to her legal work. Someone connected to a previous or pending case.

Table 1.4 Key Assumptions in the Karinna

Moskalenko CaseKey AssumptionMoskalenko was a target of theRussians because of her work asa human rights lawyer.

Unsupported. There is noevidence that the Russianstargeted her.

The Russians are the

perpetrators because they haveintentionally poisoned theirenemies in the past.

Unsupported. This is a non

sequitur. There is no evidenceof Russian involvement.

This was intentional poisoning.

Unsupported. There is noevidence of intent; there areother possible explanations.

An accident or fluke.The alternatives should not include scenarios that contradict known facts in the case. Instructors may advise students that facts such as the presence of mercury in the carand that Moskalenko and her family are truly suffering fromsymptoms of mercury poisoning may be accepted as accurate for the purposes of the case study. As a result, any alternative hypothesis that the Moskalenko family poisoning is ahoax or that the mercury is not present would be discarded.Step 3: Identify key assumptions underlying the consensus view. Could any of these be unsubstantiated? Do someassumptions need caveats? If some are not valid, how muchcould this affect the analysis?The most important aspect of this step is the conversation it produces about the effect of assumption on the analysts confidence level in the mainline judgment itself.In this case, when assumptions are explicated in thismanner, it becomes apparent that the key assumptions areunsupported by evidence. This lack of evidence suggeststhat analysts should be prepared to track down additionalinformation, consider alternative explanations, and potentially add a caveat to or revise the mainline judgment.Some key assumptions and notional assessments arelisted in Table 1.4.Step 4: Review the critical evidence that provides thefoundation for the argument. Is the analysis based on anycritical item of information? On a particular stream ofreporting? If any of this evidence or the source of thereporting turned out to be incorrect, how much would thisaffect the analysis?The Moskalenko case is short on hard evidence. Studentsshould note this dearth, as well as the fact that the directevidence in this case is based on two main sources: Frenchpolice and Karinna Moskalenkos comments to the press.

Assessment

Other evidence is really historical information, speculation on the part of Moskalenkos friends and colleagues, andconclusions based on inference.Step 5: Is there any contradictory or anomalous information? Was any information overlooked that is inconsistentwith the lead hypothesis?The key pieces of hard evidence in the case are the mercury found in Moskalenkos car and the press reports confirming that she suffered from mercury poisoning. Eventhese hard facts, however, are anomalous when examinedmore closely. Other information, such as the discrepancybetween press headlines and actual substance of their reports,is contradictory. A notional analysis is presented in Table 1.5.

Table 1.5 Evidence Assessment in the Karinna

Moskalenko CaseEvidence

Assessment

Mercury foundin car

Anomalous. Why use mercury when in the past

the Russians have allegedly used highly effective techniques? Mercury used in this manner isnot effective. It requires specific conditions overtime to poison someone.

Moskalenkosillness

Anomalous. Causing illness is an ineffective

scare tactic if being used by the Russians tothwart her participation in the trial. To wit,she must get sick and know how and why atprecisely the right time in order to prevent hertravel. She fell ill Tuesday and went to the policetwo days after her husband found the mercury.

Headlineversus facts

Contradictory. The press headlines read poison

fell Moskalenko, but the French Police arecited as cautious about the poison claim.

Who Poisoned Karinna Moskalenko? 7

Step 6: Is there a potential for deception? Does anyone

have motive, opportunity, and means to deceive you?In this case there is no evidence that the Russians wereintentionally trying to deceive. Moskalenkos statements tothe pressand various press analysesthat the Russians arethe perpetrators of the poisoning, however, could easilymislead an analyst. Although technically no deception waspresent because no one deliberately tried to promote a falsehood, it is useful to explore the deception question becauseit can prompt a discussion of whether one should take atface value what is being reported in the press and whatMoskalenko is saying publicly. In this case, the judgmentthat the perpetrators were most likely Russianfueled byMoskalenko herselfis a key and unsupported assumption.Assumptions masquerading as facts can reinforce preexisting mindsets and bias the analysis of other information relevant to a case. Both Moskalenko and journalists may havehad motives for their allegations of Russian involvement;their motives, however, are not relevant to the question ofwhether there is independent evidence to substantiate theclaims.Step 7: Is there an absence of evidence, and does it influence the key judgment? (See Table 1.6)

bias, satisficing, premature closure, anchoring, and historical analogy? (See Table 1.7)Step 9: Based on the answers to the themes of inquiryoutlined, list the potential deficiencies in the argument inorder of potential impact on the analysis.Analysts should recognize that there are potential deficiencies in most elements of the Premortem Analysis of thiscase, including the following: Unsupported assumptions. Absence of evidence. Contradictory information. Presence of analytic pitfalls.Analytic Value Added: As a result of analysis, wouldyou retain, add a caveat to, or dismiss the mainline judgment, and why? Students should seek to dismiss the mainline judgment that the Russians poisoned Moskalenkobecause of the unsupported statements by the press andMoskalenko herself, and the likelihood that analytic pitfalls biased the judgment. They should cite the gaps intheir information base as well as the potential for other,

Table 1.6 Absence of Evidence Assessment in the

There could be another

perpetrator or possible hypothesis(e.g., someone other than theRussians, accidental poisoning,self-inflicted poisoning, someoneshe knows who is unconnected tothis case or her work).

No other sources ofinformation other thanMoskalenkos statements,the mercury found in thecar, and the laboratoryreports confirming that shehas mercury poisoning

The dearth of information should

alert us to the need for moreinformation and at the very leastaffect our confidence level inour assessment pending additional, corroborative information.We should prepare collectionrequirements and indicate thepresence of these gaps in ouranalysis.

Using past events as a model to explain

Assuming that the subject of the analysis

Coming to a conclusion too quickly based

on initial and incomplete information.

Satisficing

Generating a quick response that satisfies all

stakeholders associated with the issue.

8 Chapter 1

WHO?

Figure 1.3 Starbursting the Karinna Moskalenko Case

Task 2.Rewrite the lead judgment of the case so that it reflects anychanges you would incorporate as a result of the PremortemAnalysis.Important elements that students should use to revisethe judgment include these: While Moscow has a long history of targetingits opponents, the involvement of the Russiangovernment in this case is unclear at this time.

WH

?AT

Y?

WH

HOW?

RE

HE

WHEN?

plausible alternative hypotheses. More information is

needed about family dynamics, any history of maritalstrains, how the mercury was distributed in the car, andany potential adversaries of Moskalenko other than theRussian government.

We lack direct evidence that would link the Russian

government to the poisoning or that proves this wasan intentional poisoning. If this is an intentional poisoning, there are a range ofpossible suspects, including the Russian government,professional associates, or even family members. Finally, hypotheses attributing the poisoning to anaccident cannot be ruled out.TECHNIQUE 2: STARBURSTINGUsing Starbursting to brainstorm a robust list of questionsabout a topic can help analysts explore the same questionfrom many different angles. It is particularly useful in thiscase because there preexists a firm mindset and a fairlyuncontested assessment of the cause and perpetrator of thealleged poisoning.In addition, the process of drawing a Starburst diagramforces analysts to array the questions graphically around thestar rather than simply list the questions. Doing so presentsthe analysts with a blank canvas to fill with as many questions as possible. As a result, it stimulates discussion abouteach point of the star and makes it more difficult for analysts to dismiss or overlook one or more angles.Task 3.Starburst the case Who Poisoned Karinna Moskalenko?Step 1: Use the template in Figure 1.3 or draw a sixpointed star and write one of the following words at eachpoint of the star: Who? What? How? When? Where? Why?

Step 2: Start the brainstorming session, using one of the

words at a time to generate questions about the topic. Donot try to answer the questions as they are identified; justfocus on generating as many questions as possible.Students should be able to develop at least two to fourquestions per point in the star, as reflected in the notionalFigure 1.4.Step 3: After generating questions that start with eachof the six words, the group should either prioritize thequestions to be answered or sort the questions into logicalcategories.Depending on the specific questions they develop, students may choose to categorize the questions on the basisof a known factor, such as supporting evidence. Forinstance, they could form three groups of questions: onegroup for questions that have evidence to support theanswer, another for which there is only indirect evidence orassumptions, and another for which there is no supportingevidence at all. Alternatively, students could prioritize thequestions on the basis of known unknowns, or gaps theyseek to fill.Analytic Value Added: As a result of your analysis,which questions or categories deserve further investigation?

Who Poisoned Karinna Moskalenko? 9

Figure 1.4 Starbursting the Karinna Moskalenko Case

Who poisoned Moskalenko?

WHO?

Who else besides the Russians?

Why was Moskalenko a target?

e Why was there a lapse between thems?discovery and the onset of symptoms?n Why would the Russians employ anindirect method to poison her?

WhWhat was the location?

WH

T?HA

Y?

HOW?

RE

HE

WHEN?

Where was the mercury found?

WhWhat was the substance?

Where could it have come from?

In what form was it?

WhWhat is the toxicity of this amount?

Ho did the family find the substance?

HowHo did they know it was mercury? How

When was it found?

When could it have been put there?

Analysts could focus their assessment on those questions

for which there is the least information or for which thereare alternative explanations. In this case, these mightinclude the following: Who else besides the Russians could be interested inpoisoning Moskalenko? Where else could the mercury have come from? When could the mercury have been placed in the car? Why was there a lapse between the discovery of themercury and the onset of symptoms?This process raises the overall issue that there is no directevidence to answer the Starburst questions for many of thekey points on the star, including Who? Where? When? andWhy? This should cause analysts to reassess their confidence in the overall assessment that the Russians poisonedMoskalenko with mercury because of her work as a humanrights lawyer.CONCLUSIONOn 22 October 2008, only eight days after the case broke inthe news media and ten days after Moskalenko and her

husband discovered mercury in their car, media outlets

reported that Karinna Moskalenkos poisoning wasaccidental.3 The New York Times reported that Frenchinvestigators have concluded that the mercury found in thecar of a prominent Russian human rights lawyer had beenaccidentally spilled from a thermometer that had beenbroken in the car before the lawyer bought the vehicle.4The assistant prosecutor in the case said that the amount ofmercury in the car was not toxic and that the amount ofmercury in Moskalenkos blood was insignificant.5 Headded that mercury must be ingested or injected to be toxic.KEY TAKEAWAYS Avoid a rush to judgment, even if what is happeningseems obvious. Slow down the momentum in a crisissituation by always asking why a judgment could beincorrect. Ensure that the line of analysis is underpinned bya strong evidentiary base. Track down key gaps toavoid potentially catastrophic analytic vulnerabilities. Always be alert to the analytic trap of satisficing,especially when under pressure to confirm a popularviewpoint or generate an analysis rapidly.

10 Chapter 1NOTES 1. More Poison: Another Prominent Adversary of VladimirPutin Is Mysteriously Exposed to Toxins [editorial], WashingtonPost, October 22, 2008, http://www.washingtonpost.com/wp-dyn/content/article/2008/10/21/AR2008102102342.html. 2. The steps as outlined in this case combine the processes fora Premortem Analysis and Structured Self-Critique. This combination is particularly helpful in cases that require analysts to thinkbroadly, imaginatively, and exhaustively about how they might havebeen wrong. The Premortem Analysis taps the creative brainstorming process, and the Structured Self-Critique provides a step-by-stepassessment of each analytic element. To aid students learning process, the questions in this case have already been narrowed from thefuller set of Structured Self-Critique questions found in Richards J.

2 The Anthrax Killer

n the following exercises, students put themselves in the

shoes of an FBI analyst who must unravel how events inthe anthrax case unfolded, present the information to asenior policy maker in a succinct and effective format, andtroubleshoot the judgment that Steven Hatfill is most likelythe anthrax killer prior to the announcement that he is theFBIs person of interest.Analysts are often called upon to support government taskforce investigations in which the fast pace of events, scrutinyby high-level officials, and sheer quantity of information canbe overwhelming. In the face of this kind of challenge,Chronologies frame the problem and bring order to thejumble of data points, helping analysts identify assumptionsand gaps that form the case. Combined with Timelines, thisordering puts key facts and events in context so that individualanalysts can easily track large amounts of data andmultiperson task forces can maintain a commonunderstanding of developments, day or night. Timelines andChronologies can also be the basis for tailored products orgraphics such as Maps that can be used to bring seniorofficials up to speed efficiently and effectively. The PremortemAnalysis and Structured Self-Critique help analysts avoid arush to judgment and illuminate important areas for furtherconsideration by challenging assumptions, identifying biases,and closely examining the evidentiary base.TECHNIQUES 1, 2, & 3: CHRONOLOGY,TIMELINE, AND MAPChronologies are a simple but useful tool that helps orderevents sequentially; display the information graphically;and identify possible gaps, anomalies, and correlations. The

technique pulls the analyst out of the evidentiary weeds to

view a data set from a more strategic vantage point. AChronology places events or actions in the order in whichthey occurred. A Timeline is a visual depiction of thoseevents, showing both the time of events and the timebetween events. Chronologies can be paired with Timelineand mapping software to create geospatial products thatdisplay multiple layers of information such as time, location,and multiple parallel events. The geographic scope andmany details of this case make a Chronology, Timeline, andMap particularly useful in understanding how the caseunfolded both temporally and spatially.In the case narrative, students pick up the case on15 October, well after the anthrax letters are sent. Bycreating the Chronology, the analyst develops a deeperunderstanding of each relevant event or piece of data. TheTimeline, in turn, illustrates different temporal aspects ofthe case. In the following exercise, the key is to correlatethe timing of the onset of illness with the letters themselves.By using the Timeline, it becomes apparent that the timingof the onset of illness overlapped significantly in NewYork, New Jersey, and Florida, which corresponded withthe first mailing, while a separate grouping of New Jerseyand Washington, D.C., cases emerges around the time ofthe second mailing. Also, the cutaneous cases emergedmore rapidly after known exposure than the inhalationcases, which is consistent with the clinical descriptionsprovided by the Centers for Disease Control. The use ofthese techniques also highlights the importanceof arranging the data by date of information, not the date ofacquisition or the date of reporting. For example, theanthrax cases are tracked by date of illness onset or by date

11

12 Chapter 2that treatment was sought, not by the date the case wasreported in the press. In fact, the FBI used a similarchronology to illustrate this point in the officialAmerithrax Investigative Summary, noting, the evidencesupports the conclusions that the mail attacks occurred ontwo separate occasions.1Task 1.Create a Chronolog y of the anthrax attacks andinvestigation.Step 1: Identify the relevant information from the casenarrative with the date and order in which it occurred.Step 2: Review the Chronology by asking the followingquestions:

What does the timing of the appearance of symptoms

tell me about when the letters were mailed? Could there be any other letters than the four in thegovernments possession? What additional information should we seek? Are there any anomalies in the timing of events?Task 2.Create a Timeline of the victims of the attacks based ongeographic location.Step 1: Identify the relevant information about the victimsfrom the Chronology with the date and order in which theevents occurred. Consider how best to array the data alongthe Timeline. Can any of the information be categorized?

Ernesto Blanco is released from the hospital.

The Centers for Disease Control confirms that the strains of anthrax in the Daschle and Brokaw letters match, as do thehandwriting in the letters. Also in October, Northern Arizona University microbiologist Dr. Paul Keim pinpoints the strain asAmes, a strain developed in US government labs. The CDC confirms the find.

FBI Press Briefing provides linguistic and behavior assessment of a potential anthrax killer and asks for the publics help.

14 November 2001

Ottilie Lundren, a 94-year-old CT woman, develops inhalation anthrax.

15 November 2001

Investigators find an anthrax-laced letter to Senator Leahy in a bag of quarantined mail that was postmarked 9 October.

21 November 2001

Ottilie Lundren dies of inhalation anthrax.

June 2002

FBI releases information that radiocarbon dating indicates the spores used in the attacks were made within the last two years.

June 2002

FBI drains pond near Ft. Detrick in search of anthrax evidence.

25 June 2002

Investigators search Hatfills apartment.

July 2002

FBI profile of the anthrax killer leaks to the press.

August 2002

Investigators pinpoint a mailbox in Princeton, NJ from which the anthrax letters were sent.

1 August 2002

Investigators search Hatfills apartment and trash bins.

6 August 2002

Attorney General John Ashcroft names Hatfill a person of interest.

11 August 2002

Investigators search Hatfills apartment again.

Step 2: Review the timeline by asking the following

questions: Do any of the events appear to occur too rapidly ortoo slowly to have reasonably occurred in the orderor timing suggested by the data (e.g., the letters andtheir postmarks)? Are there any underlying assumptions about theevidence that merit attention?

Does the case study contain any anomalous data or

information that could be viewed as an outlier? Whatshould be done about it?Task 3.Create an annotated Map of the letters and twenty-twoanthrax cases based on your Chronology. Visually displaythe information on a Map so that it could be used as agraphic for a briefing with a high-level official.

Students may elect to use another scheme to represent

the locations and timing of the attacks. Their performanceshould be judged on the accuracy and effectiveness of theirchosen approach, not the degree to which they reproducethe map used in this example.Step 1: Use publicly available software of your choosing tocreate a Map of the area.Step 2: Overlay the route (location, case type, prognosis).Step 3: Annotate the Map with appropriate times andlocations presented in the case.Analytic Value Added: What do the locations andsequence of events tell you? What additional information

should you seek? Do you agree with investigators findings

that the four letters to date and a fifth unknown letter aremost likely responsible for the anthrax cases to date? Thecases in New York, New Jersey, and Florida overlapped significantly both in exposure and onset of illness, while theWashington, D.C., cases emerged some weeks later. Thissupports the understanding that the attacks took place intwo tranches, with letters postmarked 18 September and 9October.Seek additional information on the Florida case. Werethere any eyewitnesses? Does Blanco remember theenvelope? How did the letters travel from New Jersey totheir final destinations? Do those modes of transport revealany clues about additional letters?Is there any significance to the timing of the letters,either the postmark or the day of the week? Both 18

September and 9 October are Tuesdays. The letter could

have been dropped into the mailbox anytime between thelast pickup on Monday and Tuesday. Where is thepostbox located? What are the surrounding businesses orhomes? Are there any cameras in the area?What about the two outlier cases: Kathy Nguyen in NewYork and Ottilie Lungren in Connecticut? What explanationsare there for these cases? Did any mail destined for these twovictims travel via the Hamilton Township mail center inTrenton, New Jersey? There are potentially knowable answersto these questions. Given the uncertainties surrounding thecase, it is essential to track down information that would helpanswer these questions. Investigators never found the sourceof exposure in the Nguyen case, and they later announcedthat the Lundgren case was most likely a result of secondarycontamination of her mail.

TECHNIQUE 4: PREMORTEM ANALYSIS

AND STRUCTURED SELF-CRITIQUEThe goal of these techniques is to challengeactively andexplicitlyan established mental model or analytic consensusin order to broaden the range of possible explanations orestimates that are seriously considered. This process helpsreduce the risk of analytic failure by identifying and analyzingthe features of a potential failure before it occurs.2Task 1.Conduct a Premortem Analysis Assessment and StructuredSelf-Critique of the reigning view that Steven Hatfill is theanthrax killer.Step 1: Imagine that a period of time has passed since youpublished your analysis that contains the reigning view. You

16 Chapter 2

Map 2.1 Example of a Map Graphic Depicting the Spatial and Temporal Aspects of the Attacks

Bold Italics = fatal inhalation case

One date = symptom onset/

suddenly learn from an unimpeachable source that the

judgment above was wrong. Then imagine what could havecaused the analysis to be wrong. One possibility is a problem with the physicalevidence in the case. The main physical evidenceis the anthrax itself, so any problem with the chainof custody or analysis of the spores could cause aspectacular failure.

Also, a lack of evidence directly linking Hatfill to the

crime could undermine the case.Step 2: Use a brainstorming technique to identifyalternative hypotheses for how the poisoning could haveoccurred. Keep track of these hypotheses. The FBI has taken a painstaking approach todevelop a full profile of the killer that stipulates the

The Anthrax Killer 17

Table 2.2 Common Analytic Pitfalls

Pitfall

Definition

Analytic mindset

A fixed view or attitude that ignores new

data inconsistent with that view or attitude

Anchoring

The tendency to rely too heavily on one

trait or piece of information when makingdecisions

Confirmationbias

The tendency to favor information that

confirms ones preconceptions or hypotheses,independently of whether they are true

Historicalanalogy

Using past events as a model to explain

current events or to predict future trends

Mirror imaging

Assuming that the subject of the analysis

would act in the same way as the analyst

Prematureclosure

Coming to a conclusion too quickly based

on initial and incomplete information

Satisficing

Generating a quick response that satisfies

all stakeholders associated with the issue

key criteria required for the killer to produce the

anthrax, such as access and scientific expertise. Asa result, they have been able to narrow the list ofpotential persons of interest to less than fifty, and byworking to rule out potential suspects. As a result,other possible hypotheses could be that anotherscientist at the US Army Medical Research Instituteof Infectious Diseases (USAMRIID) could be thekiller. Also, someone outside the lab could havegained access to the Ames strain through the normalcourse of scientific inquiry and collaboration. Doany other facilities in the United States have Amesstrain anthrax? Does USAMRIID conduct scientificexchanges with foreign countries? These hypothesespoint to gaps such as chain of control and securityprocedures that investigators should fill in order torule out these other possible explanations.Step 3: Identify key assumptions underlying the consensusview. Could any of these be unsubstantiated? Do someassumptions need caveats? If some are not valid, how muchcould this affect the analysis?Step 4: Review the critical evidence that provides thefoundation for the argument. Is the analysis based on anycritical item of information? On a particular stream ofreporting? If any of this evidence or the source of thereporting turned out to be incorrect, how would this affectthe analysis?

The critical pieces of evidence against Hatfill include:

Biology student/currently a virologistSpent time in Africa during anthrax outbreaks Worked at USAMRIID from 1997 to 1999 Had virtually unrestricted access to USAMRIIDfacilities Possessed specialized knowledge about how toweaponize bubonic plague Knew how to disseminate anthrax via mail Oversaw construction of a model Iraq mobilebioweapons lab Helped prepare a brochure in 1999 on how tohandle anthrax attacks Went to medical school in Zimbabwe near a suburbcalled Glendale, the same name that was on two ofthe envelopes Was taking Cipro in September

Taken together, these form a circumstantial case that

raises suspicion about Hatfill.Step 5: Is there any contradictory or anomalousinformation? Was any information overlooked that isinconsistent with the lead hypothesis? Hatfill is a virologistan expert in viruses suchas Ebola, HIV, hemorrhagic fever, etc.not amicrobiologist who has expertise in bacteria. Thereis no evidence that he has the requisite skills toproduce highly purified anthrax spores of thisstrain. The FBI profile describes the suspect as anintroverted person who prefers being by himselfmore often than not, but Hatfill is an extroverted exmilitary member who has lived and worked overseasin Africa for most of his life.Step 6: Is there a potential for deception? Does anyonehave motive, opportunity, and means to deceive you? Any of the scientists under scrutiny have motive,opportunity, and means to deceive investigators whoare not scientific experts themselves. If a scientistother than Hatfill at USAMRIID or elsewhere werethe true killer, that person would certainly seek tominimize his or her own profile, perhaps even byassisting investigators or falsely identifying Hatfill asthe main suspect.Step 7: Is there an absence of evidence, and does itinfluence the key judgment? There is no physical evidence that we know of linkingHatfill to the anthrax. There is physical evidence

18 Chapter 2linking the anthrax to USAMRIID. This lack ofevidence should challenge the level of certaintythat Hatfill should be named as a person of interestuntil the circumstantial evidence can be thoroughlyreviewed. Neither is there evidence, either direct or indirect,linking Hatfill to NBC or Tom Brokaw, the New YorkPost, or Senators Daschle and Leahy.Step 8: Have you considered the presence of commonanalytic pitfalls such as analytic mindsets, confirmationbias, satisficing, premature closure, anchoring, andhistorical analogy? Confirmation bias. The case against Hatfill couldrepresent confirmation bias. No physical evidencelinks Hatfill to the crime, yet he is publicly nameda person of interest. The evidence against him isentirely circumstantial and deserves greater scrutiny.The presence of several pieces of circumstantialevidence that the government found once it focusedon him as a suspect may have had the unintendedconsequence of raising the governments confidencein Hatfills guilt. As a result, each piece of evidencedeserves greater scrutiny to ensure that the decisionto name Hatfill as a person of interest is not aresult of confirmation bias. For example, are therealternative explanations for why Hatfill was takingCipro in 2001? Satisficing/Premature Closure. The governmentinterviewed Hatfill and searched his home on25 June. No charges were brought against himat that time. As pressure mounted to identifythe perpetrator, however, the government againsearched his home on 1 August. Pressurewhetherexplicit or implicitmay have caused investigatorsto come to the first, most plausible explanation(satisficing) without fully investigating the otherpossible suspects or tracking down questions aboutcircumstantial or anomalous evidence (prematureclosure). In law enforcement spheres, this is calleddetective myopia.Step 9: Based on the answers to the themes of inquiry justoutlined, list the potential deficiencies in the argument inorder of potential impact on the analysis. The lack of physical evidence linking Hatfill to thecrime raises uncertainty about his guilt, even in theface of other circumstantial evidence.

Each of the points above can be used to develop a

prioritized collection strategy to obtain informationthat would help corroborate or refute the questionsraised by the Premortem Analysis and StructuredSelf-Critique.Analytic Value Added: As a result of your analysis,what are the strengths and weakness of the case againstHatfill? What additional information should you seekout? Do any assumptions underpin the case? Do theychange or reinforce your level of certainty? The caseagainst Steven Hatfill is based on several pieces ofcircumstantial evidence that, taken together, could indicatehe is the anthrax killer. They could also simply form ahouse of cards that will collapse upon further scrutiny. Forexample, the evidence that he was taking Cipro inSeptember could indicate that he was using the drug as aprophylactic measure for anthrax exposure, but he couldalso have been taking it for a common infection. Apotentially key deficiency in the case against Hatfillsurrounds his access to the Ames strain anthrax stored atUSAMRIID. Until this assumption is substantiated, itraises uncertainty about Hatfills access to the material andany role he could have played in the attacks. Also, it isunclear what Hatfills motive could have been; and, if hewas trained as a virologist, he may have lacked theexpertise to produce highly purified and dried anthraxspores.CONCLUSIONOn 8 August 2008, the government officially excludedSteven J. Hatfill as a suspect. The announcement cametwo weeks after the Department of Justice settled aninvasion of privacy lawsuit by Hatfill for over $5 million.This was one of several lawsuits brought by Hatfillagainst the government and media in connection withthe media frenzy surrounding his identification as aperson of interest. 3 The courts dismissed several libelsuits brought by Hatfill, including one against the NewYork Times. According to a letter the Department ofJustice sent to Hatf ills law yer, the governmentconcluded, based on lab access records, witnessaccounts, and other information, that Dr. Hatfill did nothave access to the particular anthrax used in the attacks,and that he was not involved in the anthrax mailings.4Some of the most anomalous evidence was easilyexplained:

The Anthrax Killer 19

Hatfill had chronic sinus infections for years as a result

of an injury sustained while serving as a volunteer medic inAfrica, and he took Cipro to manage the infection. He neverhad access to the BLS-3 lab at USAMRIID, a fact supportedby the lab access records. Also, he completed his doctoralresearch but left Africa before receiving his diploma.5 In theend, new scientific methods developed after the attacks andin conjunction with the case helped to prove Hatfillsinnocence. In 2007, investigators had used new geneticmethods to determine that a flask of RMR-1029 Amesstrain anthrax found at USAMRIID was the parent materialfor the anthrax spores. According to the Department ofJustice Amerithrax Investigative Summary, investigatorssubsequently were able to rule out Hatfill as a suspectbecause:Early in the investigation, it was assumed thatisolates of the Ames strain were accessible toany individual at USAMRIID with access to thebio-containment lab. Later in the investigation,when scientific breakthroughs led investigatorsto conclude that RMR-1029 was the parentmaterial to the anthrax powder used in themailings, it was determined that Dr. Hatfillcould not have been the mailer because he neverhad access to the particular bio-containmentsuites at USAMRIID that held the RMR-1029.In other words, although Dr. Hatfill had accessto Ames strain anthrax while at USAMRIID, henever had access to the particular spore-batchused in the mailings.6Other scientists at USAMRIID did have access to theRMR-1029 Ames strain anthrax, but only a very limitednumber. Investigators used traditional law enforcementmethods such as interviews, alibi checks, and polygraphsto rule out all but one suspect: the very scientist who haddeveloped RMR-1029 and who had been aiding theinvestigation from the start, Dr. Bruce Ivins. Asinvestigators prepared to seek authorization to ask afederal grand jury to return an indictment charging Dr.Ivins with Use of a Weapon of Mass Destruction inviolation of Title 18, United States Code 2332a andrelated charges, Ivins took a lethal dose of Tylenol anddied on 29 July 2008.7Investigators indicated that Ivins had motive,opportunity, and means to commit the crime, in addition tosuffering from severe mental health issues. They found thatIvins was under intense personal and professionalpressure because the anthrax vaccine program to which he

Flask of RMR-1029 found in Ivinss Lab

SOURCE: Courtesy of the Department of Justice.

had devoted his career was failing. Short of some major

breakthrough or intervention, he feared that the vaccineresearch program was going to be discontinued. Followingthe anthrax attacks, his program was suddenlyrejuvenated.8Not only had Ivins developed the spore batch for RMR1029, laboratory logs indicated that he had spent anabnormal number of late-night and off-hours in his lab,where the RMR-1029 was stored along with highlysophisticated lab equipment capable of creating the anthraxpowder. He was one of the few researchers nationwide withthe knowledge and ability to create the highly purifiedspores used in the mailings.9In addition, the envelopes used in the mailings wereprestamped envelopes from a batch distributed only to postoffices in Maryland and Virginia. Investigators found thatthe envelopes most similar to those used in the attackswere distributed to the Frederick, Maryland, post office thatwas only blocks from Ivinss home. He also took steps tocover his tracks: he decontaminated his office and failed toreport it; sent nonsensical explanations for the firstinhalation anthrax case to the Centers for Disease Control,

20 Chapter 2presumably to throw investigators off his trail; threw out abook on codes that he may have used to embed codes intothe anthrax letters; and gave the FBI questionable samplesof RMR-1029 in order to conceal his activities frominvestigators.10Investigators also pointed to Ivinss mental health status,noting his use of alternate identities, his 40-year-longobsession with the Kappa Kappa Gamma (KKG) sororityduring which he burglarized chapter houses, and hisinability to explain his own suspicious behavior. The taskforce found that not only were the anthrax letters sent froma New Jersey mailbox outside a KKG chapter at PrincetonUniversity, but also Ivins was unable to provide reasonableor consistent explanations for his behavior, such as his latenight hours and submission of questionable samples ofRMR-1029.11Still, given Ivinss untimely death, and the fact that thegovernment could not take the case to trial, not everyoneaccepted the governments explanations. Ivinss lawyersposthumously defended their client, calling the chargesheaps of innuendo and a total absence of proof that hecommitted this crime.12 Some of his colleagues accusedthe government of hounding an innocent man tosuicide.13 Later, when the government closed the case inFebruary 2010 and released to the public thousands ofdocuments related to the case, his colleagues still raiseddoubts that he could have perpetrated the crime. In anemail quoted in the documents released by thegovernment, Ivins posthumously offers his ownexplanation for some of his erratic behavior, blaming analter ego, Crazy Bruce, who surfaces periodically asparanoid, severely depressed and ridden with incredibleanxiety.14Over a decade after the attacks, questions still remain. A2010 report by the National Research Council found that itis not possible to reach a definitive conclusion about theorigins of the anthrax in letters mailed to New York Cityand Washington, D.C., based solely on the availablescientific evidence. 15 The report specifically calls intoquestion the RMR-1029 flask, indicating that while the

anthrax in the letters and the flask share a number of

genetic similarities...the committee found that otherpossible explanations for the similaritiessuch asindependent, parallel evolutionwere not definitivelyexplored during the investigation.16 Also, while the RMR1029 flask was identified as the parent material for theanthrax in the letters, the National Academy of Sciencesreport indicated that it was not the immediate source ofspores used in the letters, noting, the contents of the NewYork and Washington letters had different physicalproperties.17The FBI, however, is confident that it found its anthraxkiller. In response to questions about the science behindthe case that were raised by the National ResearchCouncil report, the FBI reiterated the point from thereport that it was not possible to reach a definitiveconclusion about the origins of the samples based onscience alone, and added that, even so, investigators andprosecutors have long maintained that while scienceplayed a significant role, it was the totality of theinvestigative process that ultimately determined theoutcome of the anthrax case.18 Despite ongoing questionssurrounding Ivinss guilt and the science behind theinvestigation, the case remains closed.KEY TAKEAWAYS Chronologies and Timelines are useful toolsfor tracking key events and evidence. They helpindividual analysts organize their thinking andprovide a transparent framework for groups ofanalysts to track the progress of a case. They areparticularly useful for identifying gaps and puttingfast-breaking events in context. Use the Premortem Analysis and Structured SelfCritique to troubleshoot your analysis and avoida rush to judgment. The technique will help youidentify assumptions, biases, and evidentiaryinconsistencies that otherwise could undermine theanalysis.

NOTES 1. Amerithrax Investigative Summary, Department ofJustice, February 19, 2010, www.justice.gov/amerithrax, 3. 2. The steps as outlined in this case combine the processesfor a Premortem Analysis and Structured Self-Critique. This

combination is particularly helpful in cases that require analysts to

think broadly, imaginatively, and exhaustively about how theymight have been wrong. The Premortem Analysis taps the creativebrainstorming process, and the Structured Self-Critique

Decomposition and Visualization

Key Assumptions Check

Assessment of Cause and Effect

nalysts are often asked to conduct their analyses under

tight time frames on breaking issues. In situationswhere time is of the essence and the pressure to deliver theanalysis to stakeholders is high, the onus is on analysts toensure that relevance and accuracy are not sacrificed fortimeliness. The Getting Started Checklist, Key AssumptionsCheck, and Devils Advocacy are quick and effectivetechniques that help analysts to focus on the relevantquestions, consider alternative outcomes, reveal unsupportedassumptions, and troubleshoot their final analysis.In this case, analysts must contend not only with thepressure to produce an analytic product quickly, but alsowith the insufficiency of the evidence at hand, the presenceof unchallenged assumptions in the initial analyticjudgment, and the need for information sharing andcollection with other stakeholders. Each of the techniquesutilizes a different approach to troubleshoot these aspects ofthe analysis. Once analysts have uncovered one or twodeficiencies with the initial judgment, they may be temptedto address only these and move on. The presence of threetechniques that emphasize different aspects of the analysisencourages analysts to overcome this temptation bythoroughly examining the problem through various prismsafforded by the techniques. The result is a much morenuanced and thorough understanding of the problem,impact, stakeholders, underlying assumptions, informationgaps, and evidentiary base.

TECHNIQUE 1: GETTING STARTED CHECKLIST

Getting off to the right start is key to any successful analysis. The Getting Started Checklist can help to explicate

important aspects regarding the audience, central analytic

question, evidentiary base, alternative explanations, andother resources that could be brought to bear on the problem. By getting these fundamentals correct at the start of aproject, analysts can avoid having to change course later on.This groundwork can save time and greatly improve thequality of the final product.Task 1.Put yourself in the shoes of the Illinois Statewide Terrorismand Intelligence Center analysts who have just learnedabout the pump incident at the Curran-Gardner waterplant. Use the following Getting Started Checklist questionsto launch your analysis:Step 1: What has prompted the need for the analysis? Forexample, was it a news report, a new intelligence report, anew development, a perception of change, or a customerrequest?This analysis was prompted by a new development onthe basis of a report by Curran-Gardner to the EPA. Thefusion center is responsible for analysis and informationsharing with federal, state, local, tribal, and industrystakeholders.Step 2: What is the key question that needs to be answered?What caused the pump to fail?Step 3: Why is this issue important, and how can analysismake a meaningful contribution?This issue is important because one possible explanationis that the supervisory control and data acquisition(SCADA) system has been remotely accessed and controlled

23

24 Chapter 3via a foreign-based IP address. The implications of this arefar-reaching because it would be the first such reportedincident and could signal a new trend in activity that couldhave reverberations across not only the water sector, butalso other sectors that utilize industrial control systems.Step 4: Has your organization or any other organizationever answered this question or a similar question before,and, if so, what was said? To whom was this analysisdelivered, and what has changed since that time?This is a first for the water sector and for USinfrastructure, but there have been other instances, such asin Australia, in which an insider has compromised a wastewater system.Step 5: Who are the principal customers? Are thesecustomers needs well understood? If not, try to gain abetter understanding of their needs and the style of thereporting they like.The customer set includes federal, state, and localofficials, as well as industry. At the federal level, interest willbe high because of the possible implications of such an attackfor other types of infrastructure, the broader economicimpact, and the potential national security implications. Atthe state and local level, interests will center on theimplications for the water customers and the economiceffects. Industry will be interested in all of these issues.Step 6: Are there other stakeholders who would have aninterest in the answer to this question? Who might see theissue from a different perspective and prefer that a differentquestion be answered? Consider meeting with others whosee the question from a different perspective.At the federal level, DHS Cyber Emergency ResponseTeam (CERT) is an important resource for cyberforensics.At the industry level, the WaterISAC may have expertisethat could be brought to bear. The Curran-Gardneremployees and contract staff may also be able to providemore context for analysts regarding the timing, location,pump type, and SCADA system logs.Step 7: From your first impressions, what are all thepossible answers to this question? For example, whatalternative explanations or outcomes should be consideredbefore making an analytic judgment on the issue?While the initial reports suggest that a hacker caused thepump failure, other possible explanations could include acyber-savvy insider or a mechanical failure.

Step 8: Depending on responses to the previous questions,

consider rewording the key question. Consider addingsubordinate or supplemental questions.What is the most likely cause of the pump failure?What does the range of possible causes mean for CurranGardners customers?What does it mean for industrial control system securitymore broadly?Step 9: Generate a list of potential sources or streams ofreporting to be explored. Curran-Gardner staff and contractors WaterISAC DHS CERT Previous reporting on tests, experiments, knownintrusions for other sectorsStep 10: Reach out and tap into the experience andexpertise of analysts in other organizationsboth withinand outside governmentwho are knowledgeable on thistopic. For example, call a meeting or conduct a virtualmeeting to brainstorm relevant evidence and to develop alist of alternative hypotheses, driving forces, key indicators,or important players.Consider convening a teleconference with DHS CERT,the WaterISAC, and knowledgeable Intelligence Communityprofessionals who may be able to help provide context aboutthe threat environment, suggest new sources of information,or brainstorm possible hypotheses or driving forces.Analytic Value Added: How do the answers to thequestions listed affect the prevailing judgment that thepump failure was caused by a Russian-based intrusionusing stolen SCADA system log-on credentials? TheGetting Started Checklist suggests that more work is neededbefore publication, such as reaching out to knowledgeablestakeholders in industry and government who may haverelevant knowledge or expertise, seeking additional information about the incident from Curran-Gardner employeesand contract staff, and more closely examining other possible explanations for the pump failure.TECHNIQUE 2: KEY ASSUMPTIONS CHECKThe Key Assumptions Check is a systematic effort to makeexplicit and question the assumptions that guide an analysts

Cyber H20 25

interpretation of evidence and reasoning about any particular problem. Assumptions are usually a necessary andunavoidable means of filling gaps in the incomplete,ambiguous, and sometimes deceptive information withwhich the analyst must work. They are driven by the analysts education, training, and experience, including the cultural and organizational contexts in which the analyst livesand works. It can be difficult to identify assumptions,because many are sociocultural beliefs that are unconsciously or so firmly held that they are assumed to be truthand not subject to challenge. Nonetheless, identifying keyassumptions and assessing the overall impact should theybe invalid are critical parts of a robust analytic process.

Step 4: Elicit additional assumptions. Work from the

prevailing analytic line back to the key arguments thatsupport it. Use various devices to help prod participantsthinking. Ask the standard journalistic questions: Who?What? How? When? Where? and Why?Phrases such as will always, will never, or would haveto be suggest that an idea is not being challenged and perhapsshould be. Phrases such as based on or generally the caseusually suggest that a challengeable assumption is being made.Step 5: After identifying a full set of assumptions,critically examine each assumption. Ask: Why am I confident that this assumption is correct? In what circumstances might this assumption beuntrue?

Task 2.Conduct a Key Assumptions Check of the prevailing judgment that the pump failure was caused by a Russian-basedintrusion using stolen SCADA system log-on credentials.Step 1: Gather a small group of individuals who areworking on the issue along with a few outsiders. Theprimary analytic unit already is working from an establishedmental model, so the outsiders are needed to bring otherperspectives.

Could it have been true in the past but no longer be

true today? How much confidence do I have that this assumptionis valid? If the assumption turns out to be invalid, how muchimpact would this have on the analysis?Step 6: Using Table 3.2, place each assumption in one ofthree categories:

Step 2: Ideally, participants should be asked to bring a list

of assumptions when they come to the meeting. If not, startthe meeting with a silent brainstorming session. Ask eachparticipant to write down several assumptions on 3 x 5cards.Step 3: Collect the cards and list the assumptions on awhiteboard for all to see. A simple template can be used, asshown in Table 3.2.

1. Basically supported2. Correct with some caveats3. Unsupported or questionablethe keyuncertaintiesStep 7: Refine the list, deleting those assumptions that donot hold up to scrutiny and adding new assumptions thatemerge from the discussion.

Table 3.2 Key Assumptions Check Template

Key Assumption

Commentary

Solid

With Caveat

Unsupported

26 Chapter 3

Table 3.3 Cyber H20 Key Assumptions Check Example

Key Assumption

Commentary

Supported

With Caveat

Unsupported

The pump failure was a result

of a computer network attackoriginating in Russia.

There are other possible explanations for the

failure that do not include a computer networkattack originating in Russia, such as an insider ora mechanical failure. There is no direct reportingthat indicates the failure was a result of an attack.

The Russian IP address and

user log-on in the SCADA logindicate that the hacker usedstolen log-on credentials.

The Russian IP address simply indicates that it

was the last IP address used to access the system.Hackers based somewhere else could havebounced off the IP address in order to obfuscatetheir true location. This person could be not onlya Russian-based hacker, but also a computersavvy insider who used his or her own log-oncredentials, or someone based in a third countrywho stole the credentials.

The information reported to the

EPA is a sufficient basis to ruleout other possible causes.

The information reported to the EPA is a starting

point, but we cannot assume that this informationis accurate or exhaustive at this point.

Steps 8: Consider whether key uncertainties should be

converted into collection requirements or researchtopics.Analytic Value Added: What impact could unsupported assumptions have on your analysis of the pumpfailure? How confident are you in your analysis of thecause of the failure? All of the unsupported assumptionscould have an impact on the original analysis of the pumpfailure (see Table 3.3). Most important, the assumption thatthe SCADA system log-on information indicates a Russianbased intrusion using stolen credentials is particularly perilous because there are a number of other possibleexplanations for the activity. All of the unsupportedassumptions should, therefore, be treated as collectionrequirements prior to publication; or, at the very least, theanalysis should be amended to reflect these uncertainties.TECHNIQUE 3: DEVILS ADVOCACYDevils Advocacy can be used to critique a proposed analytic judgment, plan, or decision. Devils Advocacy is oftenused before a final decision is made, when a policy makeror military commander asks for an analysis of what couldgo wrong. The Devils Advocate builds the strongest possible case against the proposed decision or analytic judgment, often by examining critical assumptions andsources of uncertainty, among other issues.

Task 3.Build the strongest possible case against the prevailing judgment that the pump failure was caused by a Russian-basedintrusion using stolen SCADA system log-on credentials.Steps: Although there is no prescribed procedure for aDevils Advocacy, begin with the analytic judgment,assumptions, and gaps. These can serve as a useful startingpoint from which to build the case against the originaljudgment that the pump failure was caused by a Russian-basedintrusion using stolen SCADA system log-on credentials.Next, build a logical argument that undermines each goal.It is too early to conclude that the pump failure wascaused by a Russian-based intrusion using stolen SCADAsystem log-on credentials. The basis for the judgment is anunsupported assumption that the so-called attack originatedin Russia and was conducted using stolen log-oncredentials. While previous government- and industrysponsored experiments have demonstrated this capabilityon the part of hackers, we cannot rule out other possibleexplanations at this time. Barring further investigation andcollection of information from the site of the pump failureand US government cyberforensic specialists, it is just aslikely that the cause of the failure is attributable to aninsider or a simple equipment malfunction.Analytic Value Added: Which issues could underminethe analysis, and why? Unsupported assumptions and

Cyber H20 27

critical information gaps raise the level of uncertainty about

the initial analysis. Given that a case can be made thatundermines this initial analysis even in the absence ofadditional information, analysts should reserve judgment orcaveat their analysis to reflect the deep level of uncertaintyabout the cause of the pump failure. Using the results of theDevils Advocacy, analysts can create a collectionrequirements list that would help them to rule out othercauses. Doing so could help raise or lower the level ofuncertainty about the actual cause of the pump failure.CONCLUSIONOn 10 November 2012, just two days after the pump failureat the Curran-Gardner plant, the Illinois StatewideTerrorism and Intelligence Center issued a DailyIntelligence Notes report entitled Public Water DistrictCyber Intrusion. The report detailed initial findings ofanomalous behavior in a supervisory control and dataacquisition (SCADA) system at a Central Illinois publicwater district. This report also alleged a malicious cyberintrusion from an IP address located in Russia that causedthe SCADA system to power on and off, resulting in a waterpump to burn out.1 Joe Weiss, a well-known computer engineer, broke the story when he posted information about thereport on his blog and spoke to press outlets, warning,there very easily could be other utilities as we speak whohave their networks compromised.2 The media reportedthe failure as the first-ever US SCADA system attack, akinto the Stuxnet attack that targeted the industrial control system at Irans Bushehr nuclear power plant. Within twoweeks, and after intense scrutiny by the media, theDepartment of Homeland Security (DHS), Federal Bureauof Investigation (FBI), and water sector stakeholders, however, DHS reported that the pump had failed because ofphysical and mechanical issues over a period of time ratherthan from a cyber attack.3During the two-day period between the initial pumpfailure and the publication of the fusion center report, thefailure to challenge faulty assumptions and missedopportunities to share and corroborate information seem tohave produced a perfect storm. When the pump failed, aCurran-Gardner employee requested help from a computerrepairman, who subsequently reviewed the SCADA systemlogs and noted that the system had been remotely accessedby a system username via a Russian IP address during thepreceding months. Curran-Gardner reported theinformation to the Environmental Protection Agency,

which is the lead sector-specific agency, and the information

made its way to the Illinois Statewide Terrorism andIntelligence Center. The fusion center, just two days later,released the report, indicating that the event was caused bya Russian-based intrusion using stolen SCADA systemlog-on credentials. 4 It is unclear whether the CurranGardner employee, the computer repairman, or the fusioncenter made the judgment that the failure was linked to theremote access from Russia, and that this represented anintrusion using stolen credentials.The DHS computer forensic specialists at the CERTlearned about the incident a week later, on 16 November.5Upon subsequent on-site analysis of the logs, CERT couldnot validate the claims made in the report, according to ajoint DHSFBI statement that was issued on 22 November.6The user whose username appeared in the log alongside theRussian IP address and who was an employee of the SCADAsystem maintenance company used by Curran-Gardner wasnot consulted. The user, Jim Mimlitz, later told a populartechnology magazine, I could have straightened it up withjust one phone call.7 Mimlitz was on vacation in Russia inJune 2011 when he received a cell phone call asking him toexamine the SCADA computer at Curran-Gardner. He didso using remote access from Russia, and again on a flightlayover in Germany. The so-called account breach wasactually the user himself. After reading about the intrusionin the press, Mimlitz realized what had happened. Heworked with the CERT team to scour the logs and found thatall indications pointed to an electromechanical problem asthe source of the pump failure, not a SCADA systemproblem. In addition, Mimlitz told the press that the systeminstability, or glitches noted by the plant in the monthspreceding the problem, were actually due to the age of thesystem and modifications that had been made a year earlierby another contractor.8On 22 November, the industry-run WaterISAC releaseda bulletin stating, after detailed analysis, DHS and FBI havefound no evidence of a cyber intrusion into the SCADAsystem of the Curran-Gardner Public Water District inSpringfield. 9 In an ICSCERT Information Bulletinreleased on 23 November, the DHS and FBI confirmed:In addition, there is no evidence to support claims madein the initial Illinois STIC reportwhich was based onraw, unconfirmed data and subsequently leaked to themediathat any credentials were stolen, or that the vendor was involved in any malicious activity that led to apump failure at the water plant. In addition, DHS and

28 Chapter 3the FBI have concluded that there was no malicious orunauthorized traffic from Russia or any foreign entities,as previously reported.10

Luckily for Curran-Gardners 2,000 customers, the

ICSCERT bulletin also noted, At no time were there anyimpacts to customers served by the water district due to thepump failure.11KEY TAKEAWAYS Before you write, use the Getting Started Checklistto ensure that you have fully considered thequestion, alternative explanations, assumptions,

gaps, evidentiary base, and stakeholders to be

consulted. Doing so can save time and lead to a moreproductive and thorough analysis. A Key Assumptions Check is a vital part of anyanalysis. Use it not only to identify unsupportedassumptions, but also to explore how changes inyour assumptions could affect your bottom-linejudgments. A Key Assumptions Check will also helpyou identify what information is needed to raise orlower your confidence in in your analysis. When the stakes are high, but time is short, useDevils Advocacy as a quick and effective way to findholes in your logic or judgments that are not wellsupported by the facts.

4 Is Wen Ho Lee a Spy?

sing this case, analysts can build a good argument

that Wen Ho Lee is a spy. They can also build a goodargument that he is not a spy. This case illustrates howimportant it is for analysts to consider all the data, notsimply build a case to suit their perspective. The techniquesin this case help analysts evaluate both sides of the argumentabout Wen Ho Lees activities, dig deeper into the possibilityof deception surrounding a key piece of evidencethewalk-in documentthat catalyzed the case again him, andtroubleshoot their final analysis by conducting a PremortemAnalysis. This combination of techniques helps analystsidentify important assumptions, gaps, and avenues forfurther research that can improve the overall rigor of theiranalysis and avoid the temptation to go with their gut,especially when doing so can have such significantconsequences.TECHNIQUE 1: FORCE FIELD ANALYSISA Force Field Analysis helps analysts identify and assess allof the forces and factors for and against an outcome andavoid premature or unwarranted focus only on one side ofthe analysis. It is particularly helpful at the beginning of aproject or investigation as a tool to sort and consider all evidence as an evidentiary base is amassed. Furthermore, theweighting mechanism allows analysts to more easily identify the strongest and weakest forces or factors and recommend strategies to reduce or strengthen the effect of forcesthat support or work toward a given outcome.In this case, investigators amassed a long list of countsagainst Wen Ho Lee, but Lee pled guilty toand wasconvicted ofonly one relatively minor count ofmishandling a controlled document. Many observers

questioned the governments case; the government

remained solid in its conviction that Wen Ho Lee was a spy.A Force Field Analysis helps to illuminate both sides of thecase.Task 1.Conduct a Force Field Analysis of the arguments for andagainst Wen Ho Lee being guilty of passing nuclear secretsto China.Step 1: Define the problem, goal, or change clearly andconcisely.Step 2: Use form of brainstorming to identify the mainfactors that will influence the issue.Two key considerations would be Wen Ho Lees ethnicloyalty to China and a history of interactionssome ofthem unreportedwith Chinese scientists. Note, however,that Lee was of Taiwanese descent, and this could influencehow he views his relationship with the mainland. Somewould argue that Hu Sides hug of Lee and praise for Leeshelp indicated that Lee was providing valuable informationto the Chinese. However, if Lee had been a clandestinesource, it is unlikely that the Chinese government wouldhave wanted to draw undue attention to its relationshipwith Lee.Another key factor is the lack of any hard evidence ofespionage; Lee was never observed providing any materialsto the Chinese, nor was he overheard revealing any secrets.Lee and his wife served as informants for the FBI. Somewould argue this proved his loyalty, while others would sayhe was operating as a double agent and that serving as aninformant provided him with a good feedback channel.

29

30 Chapter 4There is no doubt that Lee moved large quantities of datafrom a classified computer to an unclassified computer. Thequestion is why. Was he told to archive the data? Was heafraid of losing his job and did he want to keep a copy of hisnotes? Did he put the data on tape drives to pass to theChinese? Although Lee requested remote access to aclassified system while in Taiwan, he did not do sosurreptitiously. Some would point to his questionablesecurity practices as evidence that he was trying to concealclandestine activities; others would point out that he wassimply absentminded.The case study does not include information about Leesfinancial situation or whether his colleagues at the labexhibited similar behavior and security lapses. Neither doesthe case contain any information about Wen Ho Leesattitude toward the management at Los Alamos NationalLaboratory (LANL) nor whether he felt denied opportunityor otherwise disadvantaged. These potential driving forceswould be topics of investigation and analysis and at the veryleast represent gaps that should be discussed.Step 3: Make one list showing the strongest argumentssupporting Wen Ho Lees innocence and another listshowing the strongest arguments showing his guilt.Step 4: Array the lists in a table like Table 4.2 in the book.Table 4.5 shows an example response.

Step 5: Assign a value to each factor or argument for and

against to indicate its strength. Assign the weakest-intensityscores a value of 1 and the strongest a value of 5. The sameintensity score can be assigned to more than one factor ifyou consider the factors equal in strength.Step 6: Calculate a total score for each list to determinewhether the arguments for or against are dominant.In this case, the total points arguing for his guilt are 17and for innocence are 20. It should be noted that this doesnot necessarily mean that he is innocent. If other factorsare added to the Arguments For column, the overallscore would increase. For this reason, it is important tomaintain some balance in terms of how many factors areincluded on each list. In some cases, even one factor couldmake the case compelling, for example, if Wen Ho Lee hadconfessed that he had committed espionage when beinginterrogated.Step 7: Examine the two lists to determine whether any ofthe factors balance each other out.In addition to the Hu Side hug, the question of Leesloyalties to China or Taiwan balance out. Our assessmentmight change if we had additional information that Lee wasobserved making public anti-China statements or,contrarily, that most of his family still resided on themainland and he maintained close ties to them.

Table 4.5 Wen Ho Lee Force Field Analysis Example

Issue: Wen Ho Lee Is a Chinese SpyWeight

Arguments For

Arguments Against

Weight

China targets ethnic Chinese Americans.

Lee is Taiwanese American.

Frequent contacts with high-level Chinese nuclear scientists.

Lee and his wife were FBI informants.

Did not report contacts with Chinese; failed to get clearance

to pass an unclassified document to the Taiwanese.

No evidence that Lee passed any documents or tapes to China.

Tried to get remote access via the help desk to a classified

computer network while in Taiwan.

Chinese able to obtain most information from unclassified

sources.

When visiting LANL, Hu Side hugged Lee and thanked him

for his help.

When visiting LANL, Hu Side hugged Lee and thanked him forhis help.

Lee took the PARD data on the tapes home.

Lee was asked to archive the data.

Financial trouble?

Total

Total

17

20

Is Wen Ho Lee a Spy? 31

Step 8: Analyze the lists to determine how changes in

factors might affect the overall outcome. If the technique isbeing used as a decision tool, devise a manageable course ofaction to strengthen those forces that lead to the preferredoutcome and weaken the forces that would hinder thedesired outcome.Analytic Value Added: What are the strongestarguments for and against Lees guilt in your analysis ofthe issue? Do any factors deserve further investigation?Have you identified any information gaps that shouldbe further investigated? Strong arguments can be madeboth for and against Wen Ho Lees guilt. The USgovernment was unable to substantiate a case that hecommitted espionage, but some of his behavior (like goinghome to erase computer documents) suggested that hewas feeling guilty about or afraid of something. Viablealternative explanations for Wen Ho Lees behavior includethat he was: Simply a sloppy scientist, just like his peers at the labwho often overlook security regulations because theyare too focused on their research. Part of a soft spy network that providedunclassified information to the Chinese but neverengaged in espionage.

deception is well done, one should not expect to see evidence of it. There are, however, some indicators that shouldalert analysts that they may be the targets of deception, suchas the timing of reporting or the bona fides of a source, orwhen there are known and potentially serious consequencesif the source is believed.For illustrative purposes, we have focused this DeceptionDetection example on the provenance of the walk-indocument that catalyzed the case. The same process,however, could be used to examine the possibility ofdeception surrounding any of the actors or evidence in thecase.Task 2.Use Deception Detection to determine whether deceptionmay be occurring in the case of Wen Ho Lee.Step 1: Using Table 4.3 in the book as your guide,determine whether Deception Detection should beconducted. Assuming that the United States and the FBIwould be the target, who would be the most likelyperpetrators of deception? If a case can be made thatsomeone may have a motive to deceive, state this as ahypothesis to be proved or disproved. Note which indicatorsbest apply to this case. Table 4.6 shows a sample response.

Afraid of losing his job and wanted to retain access to

files that documented his research activities shouldthey prove useful in a new job. Dutifully archiving records as instructed, needingto move the files from a classified to an unclassifiedsystem because the classified system did not have anytape drives.In this case, several key information gaps can be identifiedthat would help investigators resolve the case, includingLees financial situation and any evidence of unexplainedwealth, whether his security lapses were serious breachesor similar to the behavior of most of his colleagues, exactlywhat materials were downloaded from the classified system,and the extent of his ties to mainland China.TECHNIQUE 2: DECEPTION DETECTIONAnalysts should routinely consider the possibility thatadversaries are attempting to mislead them or to hideimportant information. The possibility of deception cannotbe rejected simply because there is no evidence of it; if

Table 4.6 When to Use Deception Detection:

The Wen Ho Lee CaseAnalysts should be concernedabout the possibility ofdeception when:

Information suggestingindicators may be true:

The potential deceiver has a

history of conducting deception.

China has a long-standing

tradition of deploying deception.

Key information is received at a

critical time, that is, when eitherthe recipient or the potentialdeceiver has a great deal togain or to lose.

China could have planted the

walk-in to throw the UnitedStates off the scent of a morevalued intelligence source. Itprobably knew an investigationwas underway.

Information is received from a

source whose bona fides arequestionable.

The FBI and the CIA questioned

the bona fides of the walk-in.

Analysis hinges on a single

critical piece of information orreporting.

The W-88 sketch was viewed as

a critical piece of evidence byNotra Trulock.

(Continued)

32 Chapter 4

Table 4.6 When to Use Deception Detection:

The Wen Ho Lee Case (Continued)Analysts should be concernedabout the possibility ofdeception when:Accepting new informationwould require the analyst toalter a key assumption or keyjudgment.

Accepting the new information

would cause the IntelligenceCommunity, the US government,or the client to expend or divertsignificant resources.The potential deceiver mayhave a feedback channel thatilluminates whether and howthe deception information isbeing processed and to whateffect.

Information suggestingindicators may be true:Analysts may have assumedprior to the walk-in that theChinese could have receivedhelp from the Russians or couldhave developed the warheadon their own. The walk-ininformation would lead themto consider an espionagehypothesis more seriously.The walk-in informationprompted both the Departmentof Energy and the FBI toexpend substantial resourcesinvestigating LANL and WenHo Lee.The Chinese almost certainlyhave other sources at DOE andthe National Labsor peoplein contact with employeestherewho could report thatan investigation was underway.

Step 2: Consider Motive, Opportunity, and Means; Past

Opposition Practices; Manipulability of Sources; andEvaluation of Evidence for the potential deceiver. Use thetemplates and questions in Table 4.4 in the book as yourguide. Table 4.7 shows an example response.

When discussing Past Opposition Practices (POP), the

question sometimes arises as to whether others besides theChinese should be considered adversaries. For example,could the adversary be the Taiwanese or Wen Ho Leehimself? It is a good question and should prompt a usefuldiscussion. The fact that such questions arise demonstratesthe value of using structured techniques, which help theanalyst think critically about the issue, sometimes outsidethe context of the specific question at hand.Analytic Value Added: Summarize the results of allfour matrices in terms of whether they tend to prove ordisprove the deception hypothesis. Did the techniqueexpose any embedded assumptions or critical gaps thatneed to be examined more critically?Task 3.Assess whether the overall potential for deception is aninsignificant threat, a possibility but one with no significantpolicy or resource implications, or a serious concern thatmerits attention and warrants further investigation.A relatively strong case can be made here to consider thepossibility of a deception operation. Further investigation iswarranted, and any final analysis should await the outcomeof that investigation.TECHNIQUE 3: PREMORTEM ANALYSISAND STRUCTURED SELF-CRITIQUEThe goals of these techniques1 is to challengeactively andexplicitlyan established mental model or analytic consensus

Table 4.7 Wen Ho Lee Deception Detection Example

Motive, Opportunity, and Means (MOM)

Motive: What are the goals and motives of the

potential deceiver?

To protect a real or more productive spy by casting suspicion on someone else, namelyWen Ho Lee.To get rid of Wen Ho Lee if he was becoming a troublesome source.To confuse any investigation while continuing to procure valuable intelligence.

Channels: What means are available to the

potential deceiver to feed information to us?

Double agents feeding information to a known intelligence organization such as the FBI orthe CIA.Providing the US government with authentic documentation through a walk-in, forexample, a report with drawings that contained more than public information.Participating in routine scientific exchanges with national lab personnel.

Risks: What consequences would the adversary

suffer if such a deception were revealed?

Possible loss of scientific exchanges.

The discovery of informant networks in labs.The real source becoming frightened and no longer cooperating.

Is Wen Ho Lee a Spy? 33

Table 4.7 (Continued)

Costs: Would the potential deceiver need tosacrifice sensitive information to establish thecredibility of the deception channel?

Not reallymuch information publicly available.

Feedback: Does the potential deceiver have a

feedback mechanism to monitor the impact ofthe deception operation?

Scientific delegations making inquiries.

Engineering flaws in document could be deliberate.

Social conversation with lab personnel.

Wen Ho Lee himself.Other sources throughout the scientific community and working in the national labs andthe US government.Past Opposition Practices (POP)

Does the adversary have a history of engaging

in deception?

Classic Chinese military doctrine espouses deception.

Does the current circumstance fit the pattern of

past deceptions?

China has history of recruiting ethnic Chinese to give it information inadvertently or by

If not, are there other historical precedents?

The entire system of Chinese intelligence gathering offers deniability or the option of casting suspicion on multiple actors.

If not, are there changed circumstances that

would explain the use of this form of deceptionat this time?Manipulability of Sources (MOSES)Is the source vulnerable to control ormanipulation by the potential deceiver?

No information about the sources background; not a recruited asset.

What is the basis for judging the source to be

reliable?

Only basis is the actual documentation provided, but that could be part of the deceptionoperation.

Does the source have direct access or only

indirect access to the information?

Little information about the access or background of the source; not a recruited source.

How good is the sources track record of

reporting?

Source is a walk-in and has no previous track record.

Does the source have personal reasons for

providing faulty information, for example, toplease the collector, promote a personal agenda,or gain more revenue? Or could a well-meaningsource just be nave?

Unlikely the source would be trying to please the collector or obtain more revenue becausethere is no established relationship between the source and the collector; it is feasible,however, that the source may have been promoting a personal agenda.

The walk-in probably has relatives on the mainland.

Evaluation of Evidence (EVE)

How accurate is the sources reporting? Has thewhole chain of evidence, including translations,been checked?

Shows a high level of detail but not entirely consistent with what we know Wen Ho Lee tohave worked on.

Does the critical evidence check out? Remember,

the subsource can be more critical than the source.

The sketches could be authentic; they reveal a convincing level of detail.

Does evidence from one source of reporting

(e.g., human intelligence) conflict with thatcoming from another source (e.g., signalsintelligence or open source reporting)?

No other sources of information to collaborate what was provided by the walk-in. Noconflicts but also no independent collaboration.

Do other sources of information provide

corroborating evidence?

No other sources of information to collaborate what was provided by the walk-in. Noconflicts but also no independent collaboration.

Care was taken to translate the documents well; the sketches speak for themselves.

34 Chapter 4in order to broaden the range of possible explanations orestimates that are seriously considered. This process helpsreduce the risk of analytic failure by identifying and analyzingthe features of a potential failure before it occurs.Task 4.Conduct a Premortem Analysis and Structured SelfCritique of the reigning view in the case study that Wen HoLee passed nuclear secrets to the Peoples Republic of China.Step 1: Imagine that a period of time has passed since youconcluded that Wen Ho Lee was guilty of espionage. Yousuddenly learn from an unimpeachable source that thejudgment was wrong. Then imagine what could havehappened to cause the analysis to be wrong.The first two steps comprise the Premortem Analysis.This right-brain-led, creative brainstorming process asksanalysts to imagine a future in which they have been provedwrong and work backward to try to identify the possiblecauses. In essence, they are identifying the weak links intheir analysis in order to avoid these potential pitfalls priorto publishing the analysis or, in this case, bringing a case toprosecution. Most analysts are more left brained than rightbrained, which often makes imagination techniques likebrainstorming challenging. However, when coupled withthe Structured Self-Critique, the systematic, left-brainedchecklist that comprises steps three through eight,brainstorming can be the first step toward identifyingsometimes fatal analytic flaws. It is important to encouragestudents to be as creative as possible when brainstorming,keeping all ideas in play.In this case, a brainstorming session might promptstudents to consider the following: Was Wen Ho Lees behavior any different than thatof his colleagues? For example, were his securityindiscretions atypical, or did his colleagues often actin the same way, forgetting to report meetings orrevealing controlled but not classified information toforeign nationals without permission? Was it suspicious or insignificant that Wen Ho Leeentered the lab at 3:30 a.m. Christmas Eve? Was he aChristian who celebrated Christmas? Did he and hiscolleagues often work late hours? Was Wen Ho Lee a member of a broader networkthat was exploited by Chinese intelligence but did notprovide any actual secret information to the Chinese?If so, who else might be in this network? Who else

attended the conferences in China along with Wen

Ho Lee?Step 2: Use a brainstorming technique to identifyalternative hypotheses that might explain Wen Ho Leespattern of behavior. Keep track of these hypotheses.In this case, students might identify a number ofalternative explanations that could be consistent with WenHo Lees known activities. They could include alternativehypotheses that Wen Ho Lee was: Simply a sloppy scientist, just like his peers at the labwho often overlook security regulations because theyare too focused on their research. Part of a soft spy network that providedunclassified information to the Chinese but neverengaged in espionage. Afraid of losing his job and wanting to retain accessto files that documented his research activities shouldthey prove useful in a new job. Dutifully archiving records as instructed and hadto move the files from a classified to an unclassifiedsystem because the classified system did not have anytape. Actually a double agent that US intelligence wasrunning against the Chinese and could not, forcounterintelligence purposes, tell others within theanalytic or law enforcement community.The alternatives should not include scenarios thatobviously contradict known facts in the case. Instructorsmay advise students that some facts, such as the movementof large quantities of information from a classified to anunclassified computer and the presence of job applicationletters that were drafted but not sent, should be acceptedas accurate for the purposes of the case study. As a result,any alternative hypothesis that Wen Ho Lee was conductingindustrial espionage for a company that recently hired himwould be discarded.Step 3: Identify key assumptions underlying the consensusview that Wen Ho Lee was guilty of passing nuclear secretsto the Chinese. Could any of these be unsubstantiated? Dosome assumptions need caveats? If some are not valid, howmuch could this affect the analysis?The most important aspect of this step is theconversation it produces about the effect of assumptions onanalysts confidence level in the mainline judgment itself.

Is Wen Ho Lee a Spy? 35

In this case, when assumptions are explicated in this

manner, it becomes apparent that some of the keyassumptions are unsupported by evidence or have caveats.This lack of evidence suggests that analysts should beprepared to track down additional information, consideralternative explanations, and potentially add caveats to orrevise the mainline judgment.Some key assumptions and notional assessments arelisted in Table 4.8.Table 4.8 Wen Ho Lee Key Assumptions CheckExampleKey Assumption

Assessment

China is developing good

access to US scientists.

Supported. In the postCold War

environment, the United Stateswas emphasizing the value ofdeveloping strategic partnershipswith former adversaries.

China had an aggressive

program to collectinformation from USscientists, targeting ChineseAmericans in particular.

Supported. The Chinese have

A Taiwanese American would

With caveats. Taiwan and China

are rivals, and which country tospy for would be influenced bypast loyalties and where onesclose relatives resided.

Step 4: Review the critical evidence that provides the

foundation for the argument. Is the analysis based on anycritical item of information? On a particular stream ofreporting? If any of this evidence or the source of thereporting turned out to be incorrect, how much would thisaffect the analysis?In the Wen Ho Lee case, the forensic evidence generatedfrom a review of LANL computer files and Wen Ho Leesown computer can be assumed to be reliable. Reportingfrom most other sources is subject to challenge. Forexample, investigators differed as to whether theinformation on the tapes was highly sensitive (the crownjewels) or could be found by searching diligently on theInternet.Step 5: Is there any contradictory or anomalousinformation? Was any information overlooked that isinconsistent with the lead hypothesis?Several key pieces of evidence are inconsistent or at leastanomalous with the hypothesis that Wen Ho Lee is a spy,including the following: Lee was an informant for the FBI. Wen Ho Lees wife was an informant for the FBI. Wen Ho Lee agreed to have his home computersearched.

Wen Ho Lee passed secret

information.

With caveats. The information

was not classified at the time;it was marked Protect asRestricted Data. Only later didinvestigators decide that some ofthe information was classified.

On the other hand, the fact that Wen Ho Lee did notdownload computer manuals is inconsistent with thealternative hypothesis that he was only archiving nuclear datahe worked on.

Wen Ho Lee is the spy.

Unsupported. Lee did not have

access to the actual informationallegedly passed. In fact, theinformation included revisionsmade to the design after he lostaccess to it.

China could have made

rapid advances only withthe help of stolen secrets;the Chinese could not havepieced together informationfrom open sources or throughsanctioned scientific contacts.

Unsupported. Almost all the

information was in the publicdomain. The Chinese design wasnearly, but not exactly, the same asthe US W-88.

The stolen data were unique

to Los Alamos NuclearLaboratory; individualsat other locations wereunlikely to have provided theinformation.

Unsupported. The information

could have been obtained fromother labs. It also could have comefrom the thirty-six other Chineseemployees working in the labs orfrom Russian scientists.

Step 6: Is there a potential for deception? Does anyone

have motive, opportunity, and means to deceive you, eitherintentionally or unintentionally?The available information indicates that the possibility ofChinese deception cannot be discounted. The Chinesecertainly had the motive, opportunity, and means to deceivethe United States. They also had a deeply rooted tradition ofconducting deception operations. Their ability tomanipulate the walk-in was restricted because it would havebeen challenging to maintain communication with thewalk-in after he delivered the information. However, theprimary value of the walk-in was to provide the initialdocumentation; the Chinese could have used otherchannels, including double agents, to continue thedeception operation. The quantity of evidence and the levelof detail in the evidence provided by the walk-in are

36 Chapter 4consistent with both hypotheses: that the walk-in waslegitimate or that the Chinese decided to provide detailedinformation to make the walk-in look credible in the eyes ofUS government officials.Step 7: Is there an absence of evidence, and does itinfluence the key judgment? Table 4.9 shows an exampleresponse.

Table 4.9 Wen Ho Lee Absence of Evidence

Assessment ExampleAbsence of Evidence

Table 4.10 Wen Ho Lee Common Analytic Pitfalls

ExampleAnalytic PitfallMindset

The mindset that the Chinese could not develop

the W-88 without stealing nuclear secrets from theUnited States.The mindset that LANL and Wen Ho Lee wouldbe the logical source of the leak. But what ifthis is untrue in this case? Are there alternativehypotheses? Once a mindset is identified, it mustbe challenged.

Confirmationbias

We tend to see what we expect to see, and we

tend to look for evidence that confirms our mindset. In this case, it is easy to accept assumptionsmasquerading as fact because they conform toour mindset. For example, when Wen Ho Leewithdrew $700 in Hong Kong, analysts observedthat this would be enough money to pay for aflight to Shanghai. There was no evidence tosuggest that such a flight ever occurred.

Satisficing

It is easy to jump to the first, most plausible

explanation in the presence of firmly held mindsets. In this case, given the substantial pressureon the FBI to pursue vigorously any reports ofChinese scientific espionage and the existence ofa DOE study that nuclear secrets probably werestolen from LANL and most likely by Wen Ho Lee,an FBI investigation of Wen Ho Lee was likely tosatisfy most critics.

Historicalanalogy

In the presence of a long history of Chinese

espionage targeting Chinese American scientistsin the United States, it is easy to conclude thatan investigation of Wen Ho Lee is a priority.This assumes that what has happened before ishappening again.

Assessment

No evidence of Wen Ho Lee

ever passing documents tothe Chinese.

Although Wen Ho Lee was

suspected of providing nuclearsecrets to the Chinese, noevidence was ever provided thatdocuments were physically passed.

No evidence that Wen Ho Lee

had communicated secretsorally to the Chinese.

The FBI never presented any

evidence that Wen Ho Leeprovided classified information tothe Chinese in any of his meetingsor conversations.

Step 8: Have you considered the presence of common

analytic pitfalls such as confirmation bias, satisficing, andhistorical analogy? (Use Table 1.2 in chapter 1 as your guideto do so.) Table 4.10 shows an example response.Step 9: Based on the answers to the themes of inquiryoutlined, list the potential deficiencies in the argument inorder of potential impact on the analysis.Analysts should recognize that there are potentialdeficiencies in each element of the Premortem Analysis,including the following: Unsupported assumptions. Presence of credible alternative hypotheses. Absence of evidence. Presence of analytic pitfalls. Potential for deception.Analytic Value Added: As a result of your analysis,would you retain, add a caveat to, or dismiss the mainlinejudgment, and why? Students should seek to add caveatsto their analysis in order to reflect the uncertaintyintroduced by unsupported assumptions, the possibility

Assessment

that alternative hypotheses could explain Wen Ho Lees

behavior, the absence of hard evidence that anything wasactually passed to the Chinese, the potential for deception,and the presence of analytic pitfalls. They should also citethe gaps in their information base and consider what wouldbe the most profitable avenues for new research andinvestigation.In this case, the case for Wen Ho Lees guilt is at least asstrong as the case for his innocence. Perhaps the moreproductive strategy would be to focus on which alternativehypotheses are most consistent with his actual behavior andwhat implications these hypotheses might have for federalinvestigators. If, for example, the fact that Wen Ho Lee ispart of an informal network of informants is deemedcredible, then attention should turn to who comprised that

Is Wen Ho Lee a Spy? 37

network and whether the other members of the network are

doing greater damage to US national security interests thanWen Ho Lee.In dealing with the potential for deception, it isimportant to keep in mind that often the issue is not Wassomeone being deceptive? but Is there sufficient evidenceor argumentation to justify opening a major investigationand dedicating significant resources to find out?Task 5.Rewrite the lead judgment of the case so that it reflects anychanges you would incorporate as a result of the PremortemAnalysis.CONCLUSIONWen Ho Lee is retired and living in Albuquerque, NewMexico. At the conclusion of his trial, the presiding judgetook the unusual step of issuing an apology from the bench,saying, I sincerely apologize to you, Dr. Lee, for the unfairmanner you were held in custody by the Executive Branch.2After the trial concluded, Lee filed a lawsuit against the LosAngeles Times, the Washington Post, ABC, the AssociatedPress, and the New York Times for invasion of his privacy.3He ultimately won the lawsuit. Lee subsequently wrote abook titled My Country versus Me: The First-Hand Accountby the Los Alamos Scientist Who Was Falsely Accused of

Being a Spy. He also completed a textbook on applied physics, which he began writing while he was in prison.4KEY TAKEAWAYSApplication of structured analytic techniques to the WenHo Lee case underscores the need to: Always challenge inherited assumptions. TheDepartment of Energy presented the FBI with thefindings of an administrative inquiry that was basedon several keyand unchallengedassumptions.Before launching the investigation of Wen HoLee, it is important to critically examine the keyassumptions upon which the DOE case was based. Be open to alternative hypotheses. When data areinconsistent with the lead hypothesis, stop and askyourself if there are alternative and more compellingexplanations for the behavior being observed. Make time to reflect, especially at the start of a newproject or investigation. When operating undermajor time constraints and substantial pressurefrom above to produce, avoid the temptation toplunge in. The need to employ structured analytictechniques, like a Key Assumptions Check, is greatestwhen the stakes are high. A quick answer will satisfyyour customer for the moment, but you will have tolive with a wrong answer for the rest of your life.

NOTES 1. The steps as outlined in this case combine the processesfor a Premortem Analysis and Structured Self-Critique. This combination is particularly helpful in cases that require analysts tothink broadly, imaginatively, and exhaustively about how theymight have been wrong. The Premortem Analysis taps into thecreative brainstorming process, and the Structured Self-Critiqueprovides a step-by-step assessment of each analytic element. To aidstudents learning process, the questions in this case have alreadybeen narrowed from the fuller set of Structured Self-Critiquequestions found in Richards J. Heuer Jr. and Randolph H. Pherson,Structured Analytic Techniques for Intelligence Analysis, 2nd ed.(Washington, DC: CQ Press, 2015).

Hypothesis Generation and Testing

Analysis of Competing Hypotheses

Hypothesis Generation and Testing

5 Jousting with Cuba over Radio Marti

he US government jousted with Cuba for four years

over radio broadcasts to Cuba from Florida. Cubanpresident Fidel Castro saw the plan as one more deliberateAmerican challenge to the legitimacy of the CubanRevolution. Both countries engaged in threats andcounterthreats, and the full range of intelligence collectionand analysis capabilities was employed, including opensource, human, and technical collection efforts. Analystswere called in to help the Reagan administration assess howCastro would respond if Radio Marti started broadcasting.In this situation, use of Chronologies and Timelines wouldhelp analysts evaluate Castros behavior and determinewhether he was prompting the United States to respond to hisinitiatives or simply reacting to US actions. Part of this process of evaluation involves using the Deception Detectiontechnique to explore whether some of the information orreporting could be deliberate deception meant to intimidateWashington and persuade the US Congress or the executivebranch that broadcasts to Cuba would be too risky. Manyspeculated about what Castro might do, but a technique suchas Quadrant Hypothesis Generation would help structure thisprocess, generating a more rigorous set of hypotheses. Use ofhypothesis-testing techniques such as Analysis of CompetingHypotheses would help analysts assess which actions Castrowould be most likely to take, further illuminating whetherevents could be leading up to a radio war with Cuba.TECHNIQUE 1: CHRONOLOGIES AND TIMELINESChronologies and Timelines are simple but useful toolsthat help order events sequentially; display the informationgraphically; and identify possible gaps, anomalies, or

correlations. In addition, these techniques pull the analyst

out of the evidentiary weeds to view a data set from a morestrategic vantage point. The complex and contradictorydata in this case make an annotated Timeline particularlyuseful in identifying key pieces of evidence, confidence levels in the reporting, and gaps in the information.Task 1.Create a Chronology and Timeline of relevant events leading up to President Reagans decision to sign the RadioMarti legislation on 4 October 1983 (see Table 5.5).Step 1: Identify all the key events and arrange themchronologically in a table with one column for the date andone column for the event.

Table 5.5 Chronology of the Radio Marti Case

1981

Ronald Reagan inaugurated President of the United

States on 20 January.In August, during technical discussions concerning radiointerference, Cuba says it will move forward with plansfor two 500 kW stations and shift to frequency 1040kHzthe frequency designated for Radio Marti in Floridabut also used by clear channel station WHO in Iowa.1On 22 September, US president Reagan announcesExecutive Order 12323, setting up the PresidentialCommission on Broadcasting to Cuba.2

1982

The Board of Directors of the Florida Association of

Broadcasters adopts a resolution urging the UnitedStates to jam Cuban radio broadcasts until illegalinterference from Cuba ends.3

(Continued)

39

40 Chapter 5Step 2: Select the most relevant information from the casenarrative. Consider how best to array the data along theTimeline. Can the information be organized by category?Construct a Timeline of the Radio Marti case.A Timeline that contrasts US actions with Cuban actionsis provided in Figure 5.3.

Table 5.5 Chronology of the Radio Marti Case

(Continued)The US House of Representatives passes H.R. 5427 on 10August, authorizing Radio Marti.Cuba on 30 August disrupts broadcasts of radio stationWHO in Des Moines, Iowa, and several other stationsacross the United States.

Step 3: Review the Timeline by asking the following questions: Should any underlying assumptions about the evidence be taken into consideration? Do the duration andsequence of events suggested by the data make sense? Arethere data gaps? Could any events outside the Timeline haveinfluenced the activities?A review of the Timeline suggests four majorobservations:

Committee on Foreign Relations on 9 September

approves Radio Marti legislation.The US Senate on 21 December declines to take up RadioMarti legislation.1983

Commercial broadcasters are informed in May that US

countermeasures include destruction of offending Cubantransmitters if Cuba interferes with US radio stations.Amended version of Radio Marti legislation passes theUS Senate on 13 September. Revised legislation requiresRadio Marti to adopt Voice of America (VOA) standardsand broadcast on 1180 kHz.

The issue was very contentious for the political

system in the United States, both in terms ofcongressional infighting and within the broaderpopulation.

Radio Marti legislation passes the US House of

Representatives on 29 September with a legislativehistory that enables Radio Marti to become a surrogatehome broadcasting service for Cuba.

Cuban actions were both proactive and reactive and

tended to keep Washington off balance.

President Reagan signs the legislation on 4 October.

Figure 5.3 Radio Marti: Timeline of US and Cuban Actions

RonaldReaganelectedUSPresident

Radio Marti Timeline

USEstablishesPresidentialCommission

USGovernmentActions

Timeline

CubanGovernmentActions

Jan

Jan

1979

1980

Cubaannouncesplans for two500 kWtransmitters.

Struggling aseconomiccrisis spawnspopulardiscontent.

US urgedby Floridabroadcastersto jamCubanradios

Jousting with Cuba over Radio Marti 41

The launch of Radio Marti probably was delayed by

at least one year. Castro did not carry out his threat of massiveradio interference. We do not know whether itwas because he never intended to do so and wastransmitting false and deceptive informationthrough public as well as intelligence channels, or,alternatively, that he intended to do so and changedhis mind at the last minute for reasons unknownor because he did not want to suffer the costs of USretaliation on this issue.A major gap in this record is the lack of information fromclandestine sources and to what extent this influenced USgovernment actions. Cuba has a long and persistent recordof attempting to influence the perceptions of US executiveand legislative branch officials. More important, we nowknow that during this time the Cubans controlled US assetsreporting from Cuba and, according to a State Departmentofficer, used them for passing information through intelligence channels. More information about these activitieswould help in assessing the effectiveness of Cuban perception management/deception efforts.

Congress lobbied by NAB to delay

USHouseauthorizesRadioMarti

USSenateCommitteeapproveslegislation

USSenateopts notto takeup bill

Aug Sep

1981

Analytic Value Added: How confident are you in the

sources of information? What does the sequence of eventstell you? Are there any gaps in the information that shouldbe addressed? Should you seek any additional information?We would have high confidence in the sources of information on US government actions because they are mostly amatter of public record. Information on Cuban actions isderived from both first- and second-hand sources, whichwould give us a medium level of confidence. A key gap inthe information is what US and Cuban officials were thinking and doing in late 1984 and early 1985 before RadioMarti went on the air.TECHNIQUE 2: DECEPTION DETECTIONThe Radio Marti case presented several significant analyticchallenges. One of the principal challenges was whether theCastro regime was engaging in perceptions managementand/or strategic deception to support its opposition toRadio Marti. Analysts should routinely consider the possibility that adversaries are attempting to mislead them or tohide important information. The possibility of deception

42 Chapter 5cannot be rejected simply because there is no evidence of it;if deception is well done, one should not expect to see evidence of it. There are, however, some indicators that shouldalert analysts that they may be targets of deception, such asthe timing of reporting, the bona fides of a source, or whenbelieving what a source says could have known and potentially serious consequences.Cuba had been engaged in adversarial relations with theUnited States for two decades before the Reagan administration came into office. Both sides had employed the full rangeof diplomatic and military tactics, including the threat posedby nuclear missiles on Cuban soil. The Soviet Union and itsexternal intelligence service (the KGB) had mentored andsupported the Cuban service. The KGB had a long history ofusing perceptions management and deception. Given thesebackground circumstances, analysts need to be alert to thepossibility that the opposition would employ perceptionsmanagement and/or deception to help achieve its goals.Task 2.Using Deception Detection techniques, determine whetherCuba might be employing perceptions management and/ordeception against the United States.

Step 1: Using Table 5.2 in the book as your guide, assess

whether a good case can be made to employ Deception Detection techniques. If a case can be made that Cuba has a motiveto deceive, state this as a hypothesis to be proved or disproved.As discussed in Table 5.6, most Cuba-watchers wouldsay that a strong case could be made that Havana wouldconsider using deception to thwart US efforts to broadcastinto Cuba with Radio Marti.Step 2: One method of structuring analysis to help analysts evaluate their data for possible deception by the opposition can be found in four checklists identified by theiracronyms: Motive, Opportunity, and Means (MOM); PastOpposition Practices (POP); Manipulability of Sources(MOSES); and Evaluation of Evidence (EVE). Use the templates and questions in Table 5.3 in the book as your guide.As noted in Table 5.7, a strong case can be made that theCuban government employed perceptions management anddeception techniques in the case of Radio Marti.Analytic Value Added: Summarize the results of allfour checklists in terms of whether they tend to prove ordisprove the deception hypothesis. Did the technique

Table 5.6 Radio Marti: Likelihood That Cuba Is Employing Deception

Analysts should be concerned about the possibility of deception when:The potential deceiver has a history of conductingdeception.

The Cuban governmentas well as its Soviet allyhas a long history of employingdeception.

Key information is received at a critical timethat

is, when either the recipient or the potential deceiverhas a great deal to gain or to lose.

Cuban threats and actions were often received in response to critical congressionalactions on Radio Marti. Both public and private statements suggested that the Cubangovernment believed it had much to lose if the United States began broadcasting toCuba. It was concerned that Radio Marti programming would publicize the failures of therevolutionary government and help foment discontent with the regime.

Information is received from a source whose bona

fides are questionable.Analysis hinges on a single critical piece ofinformation or reporting.Accepting new information would require theanalyst to alter a key assumption or key judgment.

Accepting reports that Cuba was preparing to jam or otherwise interfere with USradio broadcasting could prompt the US Congress to decide not to initiate broadcasts,anticipating the commotion this might generate in the business community.

Accepting the new information would cause the

Intelligence Community, the US government, or theclient to expend or divert significant resources.

Accepting reports that Cuba was preparing to jam or otherwise interfere with US radiobroadcasting prompted Washington to develop costly countermeasures.

The potential deceiver may have a feedback channel

that illuminates whether and how the deceptiveinformation is being processed, and to what effect.

The Cubans had a timely, accurate feedback channel throughout this period in theform of congressional reaction to its various threats and the access to questions aboutRadio Marti received by its double agents. In addition, its own penetrations of the USgovernment, discovered or undiscovered, may have been able to provide additionalreporting.

Jousting with Cuba over Radio Marti 43

Table 5.7 Radio Marti: Assessing the Likelihood of Cuban Deception with MOM, POP, MOSES, and EVEMotive, Opportunity, and Means (MOM):

Motive: What are the goals and motives of

the potential deceiver?

In the case of Radio Marti, the Cuban goal was clear: prevent Radio Marti from broadcasting toCuba as a surrogate radio service providing a source of internal news not controlled by the Castroregime. To thwart the US administrations plan, Cubas best tactic was to prevent passage of thelegislation in the US Congress, or cause Congress to modify the broadcast content of Radio Martiso that it would not cause internal problems for the Cuban government. Threats to disrupt USbroadcasts if Radio Marti began broadcasting were a tactic designed to encourage opposition ofpowerful US commercial interests and their representatives in Congress to oppose Radio Marti.

Channels: What means are available to the

potential deceiver to feed information to us?

The United States was receiving information about Cubas intentions through multiple channels.Open sources included public statements by Cuban diplomats and other officials. Diplomaticexchanges in multiple forums provided additional information. Cubas demonstration of thepower of its transmitters to disrupt US broadcasts provided both open information and data fortechnical analysis of the capabilities of the transmitters. In addition, if Cuba could control someor all of the oppositions clandestine collection of intelligence about Cuban intentions, it couldinfluence US perceptions of its intentions.

Risks: What consequences would the

adversary suffer if such a deception wererevealed?

Given the Cubans objective of thwarting the Reagan administrations plans for Radio Marti,if the deception failed or was detected and failed, the worst that could happen would be thatRadio Marti would start up, probably sooner rather than later because the administration wouldnot need to prepare countermeasures and would not be running the political risks involved withCuba disrupting US radio broadcasting. Detection of a deception operation also runs the riskthat the opposition will identify the means by which the deception is being conducted. The riskto the Cubans would be calculated in terms of the value of those means.

Costs: Would the potential deceiver need to

sacrifice sensitive information to establish thecredibility of the deception channel?

Castros intentions were the critical information in this case. If Castro were providing thatinformation as part of the deception or perceptions management campaign, no sensitiveinformation would be lost and there would be no cost.

Feedback: Does the potential deceiver have

a feedback mechanism to monitor the impactof the deception operation?

The Cubans had rich sources of feedback on a potential deception. The response of the maintarget, the US Congress, and various interest groups provided an excellent means of monitoringthe impact of a deception and its continuing credibility. If the Cubans controlled some or all ofthe clandestine information, they could gain some insights about how the opposition assessedthe information and its impact on their analysis by evaluating the follow-up questions asked oftheir controlled sources.Past Opposition Practices (POP):

Does the adversary have a history of

engaging in deception?

The clandestine introduction of Soviet nuclear missiles into Cuba represented one of the greatstrategic deceptions of the 20th century. The Cubans were partners and enablers in that deception.4

Does the current circumstance fit the pattern

of past deceptions?

Deception is often used by a weak or weaker power against a stronger adversary. In that sense,the possibility of Cuban deception would fit a well-established universal pattern of deception.The specifics of this case indicate that Cuba would have a motive for deceiving the UnitedStates about its intentions to disrupt radio broadcasting. However, no specific information wasavailable at the time to indicate whether or not they would disrupt broadcasts.

If not, are there changed circumstances

The generalized history of deception is the guiding principle in this case.

Manipulability of Sources (MOSES):

Is the source vulnerable to control ormanipulation by the potential deceiver?

The Cubans had the potential to manipulate all of the open sources providing information abouttheir position on Radio Marti. Furthermore, they had the ability to coordinate their open sourceinformation with any controlled clandestine collection.

What is the basis for judging the source to be

reliable?

Open sources could be manipulated at will. Technical information derived from open sources wouldbe much more difficult to manipulate. Specifically, the capabilities of the Cuban transmitters todisrupt US radio broadcasts were subject to standard technical analytic techniques. Clandestinehuman sources can always be manipulated if controlled. In addition to standard counterintelligencetradecraft used to vet sources, the specific sources reporting on Radio Marti could be evaluated, inpart, by the consistency of their reporting with other sources of information.

(Continued)

44 Chapter 5

Table 5.7 Radio Marti: Assessing the Likelihood of Cuban Deception with MOM, POP, MOSES, and EVE(Continued)Does the source have direct access or onlyindirect access to the information?

In this case, whether sources had direct access to the information or not would not provide theanalysts with any means to judge whether Castro knew what he would do at the end of the day,was telling the truth to the source, or was manipulating the source.5

How good is the sources track record of

reporting?

Even if the source had been reporting for a substantial period of time, the question is whetherthe source was controlled, and, if so, at what point was he controlled.

Does the source have personal reasons for

providing faulty informationfor example,to please the collector, promote a personalagenda, or gain more revenue? Or could awell-meaning source just be naive?

Not applicable.

Evaluation of Evidence (EVE):

How accurate is the sources reporting?Has the whole chain of evidence, includingtranslations, been checked?

In this case, analysts had a substantial body of sources derived from open, clandestine, human,and technical means of collection.

Does the critical evidence check out?

Remember, the subsource can be more criticalthan the source.

The critical unknown was how Fidel Castro would respond when and if Radio Marti began tobroadcast to Cuba; that could only be determined at the last minute. The United States wouldlikely learn of that final decision by listening to US radio stations.

Does evidence from one source of reporting

(e.g., human intelligence) conflict with thatcoming from another source (e.g., signalsintelligence or open source reporting)?

No. But analytically, this could be a sign of deception. Conflicts and inconsistencies are the normin intelligence collection.

Is any evidence one would expect to see

noteworthy by its absence?

Yes. See above.

Do other sources of information provide

corroborating evidence?

No. However, as noted, no evidence could answer the ultimate questionwhat would Fidel dowhen he heard Radio Marti in Havana?

expose any embedded assumptions or critical gaps that

need to be examined more critically? The analysis contained in all four checklists makes a strong case for the likelihood of deception: Cuba had strong motivation to engage in deception.Havana believed Radio Marti broadcasts couldquickly fan the flames of popular discontent withthe Castro regime, lacked the wherewithal to resistsuch an initiative with military force or economicsanctions, and dared not give the United States areason for taking direct action against the island. Cuba and its Soviet benefactor both had a strongtradition of conducting deception operations. The Cuban regime controlled all public informationsources on the island, andas was learned in lateryearsit also was manipulating US perceptionsthrough a network of double agents. More important,it had a network of spies that had penetrated muchof official Washington as well as Florida, which gaveit an excellent feedback loop with which to calibrateany deception operation.

The lack of open source or classified reporting on

Cuban internal dynamics and strategizing makes itharder to make a case for deception based on theEvaluation of Evidence.The technique exposed several assumptions and gaps ininformation: A key assumption was that Cubas only strategy foropposing the startup of Radio Marti was to disruptUS commercial AM radio broadcasts. Severalother options were available to Havana, includingsabotaging the facility, jamming the broadcasts,and terminating bilateral agreements that woulddo harm to the interests of the Cuban Americancommunity. Little was known about what Fidel Castro andhis core leadership were actually thinking andplanning. Little also was known about the sophistication ofCuban espionage and perception managementoperations in the United States.

Jousting with Cuba over Radio Marti 45

TECHNIQUE 3: MULTIPLEHYPOTHSIS GENERATION:QUADRANT HYPOTHESIS GENERATIONMany techniques can be used to help generate a set ofhypotheses, including basic brainstorming, SimpleHypothesis Generation using the Structured Brainstormingtechnique, Quadrant Hypothesis Generation using a 2 2matrix to structure the process, and the Multiple HypothesesGeneratorTM. The Multiple Hypotheses GeneratorTM is asoftware tool that applies the journalists classic set of questions (Who? What? How? When? Where? and Why?) todevelop a set of mutually exclusive hypotheses by generatingpermutations of the lead hypothesis.6Of the four techniques just mentioned, basic brainstorming is the least rigorous because it simply involves listingwhat first comes to mind. Such an unstructured processusually fails the key test of hypothesis generation: that theset of hypotheses generated should be comprehensive andmutually exclusive. The other three techniques are morelikely to pass this test if performed correctly.In this case study, Quadrant Hypothesis Generationwould be a good choice because the analytic challenge canbe defined along two key dimensions: what range of optionsthe Cubans might consider and how serious the impactmight be on the United States. By creating four mutuallyexclusive quadrants, each defined by different endpoints ofthe two key dimensions, the Quadrant Hypothesis Generation process reframes the question in four different ways,spurring more creativity and ensuring a more comprehensive analytic approach.Task 3.Use the Quadrant Hypothesis Generation technique todevelop a set of three to five hypotheses that address thequestion: How will Cuba respond to the launch of RadioMarti broadcasts?Step 1: Identify two key dimensions or drivers influencing Cubas decision making about how to respondusing Structured Brainstorming or drawing from expertanalysis.The two primary actors in this case study are Cuba andthe United States. In determining a set of key drivers or keydimensions of the issue, this is the best place to start. Withregard to Cuba, the key question is: What is Castros underlying objective? Is he determined to prevent Radio Martifrom broadcasting regardless of the consequences, or would

he be satisfied with partial success by delaying the launch

date or modifying the programming so that it posed lessdanger to the regime? From the perspective of the UnitedStates, the key concern would be how much damage Cubaintended to inflict on the United States. Would it go so faras to disrupt all US commercial AM broadcasting and evenattack Radio Marti facilities in Florida, or would it settle fora milder response by only jamming US broadcasts or evennot responding at all?Step 2: Construct a 2 2 matrix using the two driversor primary dimensions of the issue. Use Figure 5.2 as atemplate.Step 3: Think of each key dimension or driver as a continuum from one extreme to another. Write the extremesof each of the drivers at the end of the vertical and horizontalaxes.In this instance, the two key dimensions would beCuban Objectives in trying to counter US broadcasting toCuba on Radio Marti and the potential Impact on theUnited States of any Cuban actions. In terms of CubanObjectives, the extremes would be either to Prevent any USbroadcasting by Radio Marti or, at the other end of thespectrum, to accept a more moderate response by seekingto Delay or Modify the content of the broadcasts, as shownin Figure 5.4.Step 4: In each quadrant, describe a likely endstatethat would be shaped by the two dimensions or drivers.Some quadrants may have more than one endstatedefined.Potential endstates are described below for eachquadrant (see Table 5.8) and summarized graphically inFigure 5.5.The following two steps (5 and 6) form part of the technique but will not be used in this case study:Step 5: Develop signposts or indicators that show whetherdevelopments are moving toward one of the endstates.Step 6: Use the signposts to develop intelligence collectionstrategies to determine the direction in which events aremoving.Analytic Value Added: Did the Quadrant Hypothesis Generation technique help you generate alternativehypotheses that you might not have thought of usingtraditional brainstorming techniques? Was your resulting s et of hy p othe s es mutual ly exclusive and

46 Chapter 5

Figure 5.4 Radio Marti: Quadrant Hypothesis Drivers

Impact on theUnited States

SEVERE

DELAYorMODIFY

Cuban Objectives

PREVENT

MILD or NONE

Table 5.8 Radio Marti: Quadrant Hypotheses Generation Endstates

Hypothesis

Description

Comment

1. Prevent Radio Marti

broadcasts in a way that wouldhave Severe Impact on theUnited States

Use threats and then proceed to disrupt US radio

broadcasting across most, if not all, of the UnitedStates to force the US administration to shut downRadio Marti.

The Cubans have demonstrated the capability

to disrupt US radio broadcasts and could do soindefinitely or until the United States agreed to shutdown Radio Marti. The Cubans, however, would berisking US retaliation.

2. Delay or Modify Radio Marti

broadcasts in a way that wouldhave Severe Impact on theUnited States

Damage or destroy Radio Marti broadcast facilities,

especially the antennas in Florida, to delayorrepeatedly delayits broadcasts.

The Cubans have, or could develop, a clandestine

infrastructure in Florida to damage the Radio Martitransmitters on Marathon Key. This highly riskyresponse would more likely delay rather than endRadio Marti broadcasts.

3. Prevent Radio Marti

broadcasts in a way that wouldhave Mild or No Impact on theUnited States

Jam Radio Marti broadcasts but do not use

sufficient power to interfere with US commercialbroadcasting and do nothing else.

Jamming is a traditional response to unwelcome

foreign radio broadcasts, widely employed by the SovietUnion and other Communist states. The challenge forCuba would be to jam the signal but avoid disruptingUS broadcasts using the same frequencies.

4a. Delay or Modify Radio

Marti broadcasts in a way thatwould have Mild or No Impacton the United States

Threaten to disrupt US radio broadcasts and

conduct some disruption as a bluff to deter theUnited States from initiating broadcasts, but do notactually engage in disruption if Radio Marti startsbroadcasting.

With the transmitters in place, Cuba would incur little

incremental cost to threaten to use them to disruptUS broadcasts as a ploy to prevent or delay RadioMarti broadcasts. However, if the United States choseto begin broadcasting, the Cubans might calculatethe risk of US reprisals would outweigh any benefitsfrom actually disrupting US AM broadcasts.

4b. Delay or Modify Radio

Marti broadcasts in a way thatwould have Mild or No Impacton the United States

Threaten to disrupt US radio broadcasts and

conduct some disruption as a bluff to cause theUnited States to modify the content of Radio Martiprogramming to conform to VOA standards moreacceptable to Havana.

Threatened disruption designed to cause changes

in content would be more politically palatable inWashington and more likely to succeed.

4c. Delay or Modify Radio

Marti broadcasts in a way thatwould have Mild or No Impacton the United States

Take actions to negatively affect the interests of

Radio Martis main proponent, the Cuban Americancommunity, by not allowing family members to visitthe island or permit their relatives to leave Cuba.

If the Cubans believe that Radio Marti will continue

broadcasting and will not change its content, theycould try to punish the Cuban American communityfor supporting Radio Marti.

Jousting with Cuba over Radio Marti 47

Figure 5.5 Radio Marti: Quadrant Hypotheses Generation Endstates

SEVERE

DELAYorMODIFY

Disrupt US AMbroadcasting toprevent launch ofRadio Marti

Impact onUnited States

Damage or destroyRadio Marti by sabotaging its facilitiesin Florida

Cuban Objectives

Threaten to disrupt USbroadcasts or punishthe Cuban Americancommunity in Florida

PREVENT

Jam Radio Marti broadcasts

but avoid disruptingUS radio stations that usethe same frequency

MILD or NONE

comprehensive? Did you generate more than one

hypothesis or endstate for any of the quadrants? TheQuadrant Hypothesis Generation technique drives theanalyst to think about potential hypotheses from four different perspectives. This not only prompts analysts to generate a broader set of hypotheses but also to explorepossibilities they would not have otherwise considered.Another advantage is that each quadrant in the 2 2matrix is defined by a different set of drivers or dimensions, thus ensuring that most, if not all, of the hypothesesare mutually exclusive. Obviously, this rule does not holdif two hypotheses are generated for a single quadrant ofthe 2 2 matrix.This raises a legitimate question as to whether more thanone hypothesis should be entered into any quadrant. Theargument for a one hypothesis per quadrant rule is thatthis ensures mutual exclusivity. The argument for allowingmore than one hypothesis per quadrant is that it spurs analysts to get out of the box and generate a more robust set ofhypothesessome of which often are counterintuitiveandin that sense highly valuable.In this case study, three hypotheses were generated forthe Delay or Modify Radio Marti broadcasting with Modest or No Impact on the United States. The value in generating more than one hypothesis for this category is that itsparked some new ideas on what actions Havana mightundertakeone of which actually came to pass when Cubaterminated the USCuba Emigration Agreement, thereby

cancelling provisions for Cuban American families to visit

their relatives in Cuba.TECHNIQUE 4: ANALYSIS OFCOMPETING HYPOTHESESThe principles of social science research and decades ofexperiments on cognition and decision making have established that analysts considering complex issues benefit fromstructuring their analytic process in order to ensure that allrelevant data are collected and evaluated as objectively aspossible.7 Analysts face a perennial challenge of workingwith incomplete, ambiguous, anomalous, and sometimesdeceptive data. In addition, strict time constraints on analysis and the need to make a call often conspire with anumber of natural human cognitive tendencies to result ininaccurate or incomplete judgments.One approach to structured analysis, Analysis of Competing Hypotheses (ACH), was developed for the IntelligenceCommunity and, particularly, for analysts working on issuesin which deception may be employed. ACH improves theanalysts chances of overcoming these challenges by requiringthe analyst to identify and refute possible hypotheses usingthe full range of data, assumptions, and gaps that are pertinent to the problem at hand. According to Heuer and Pherson, ACH involves identifying a set of mutually exclusivealternative explanations or outcomes (presented as hypotheses), and selecting the hypothesis that best fits the evidence.8

48 Chapter 5Task 4.Use the ACH software to identify which hypotheses providethe most credible explanation in answering this question:How will Cuba seek to delay or prevent Radio Marti frombroadcasting? The basic ACH software is available at http://www.globalytica.com or from the Palo Alto ResearchCenter at http://www2.parc.com. A collaborative version ofACH called Te@mACH can be accessed at http://www.globalytica.com.Step 1: Select three to five hypotheses based on the resultsof Quadrant Hypothesis Generation exercise, striving formutual exclusivity.The principal concern of the US stakeholders was thatCuba would disrupt commercial radio broadcasts across thecountry. However, posing the intelligence question in abroader form, How will Cuba seek to delay or preventRadio Marti from broadcasting? includes other possibleresponses by the Cubans. So the first step in structuring theanalysis is to pose the question properly to ensure that thefull range of possible outcomes is considered.A hypothesis is essentially a persons best guess to answera question. According to Heuer and Pherson, in an ACHexercise, Hypotheses should be mutually exclusive; that is, ifone hypothesis is true, all others must be false. The list ofhypotheses should include all reasonable possibilities. Includea deception hypothesis if that is appropriate.9 In the case ofhypotheses related to Radio Marti, some of the hypotheseswould be mutually exclusive only because of the intent of theCubans, not their capabilities to disrupt US broadcasts. A setof hypotheses to consider is provided in Table 5.9.

Table 5.9 Radio Marti: Selected Hypotheses for

ACH AnalysisNo.

Hypothesis

1.

Cuba Disrupts US radio broadcasting to prevent Radio

Marti broadcasts

2.

Cuba Sabotages Radio Marti facilities to delay or prevent Radio Marti broadcasts

3.

Cuba Jams Radio Marti broadcasts without disrupting US

broadcasts and does nothing else

4.

Cuba Deceives with threats and some disruption to delay

or modify Radio Marti broadcasts

5.

Cuba Punishes the Cuban American community to delay

or modify Radio Marti broadcasts

Step 2: Make a list of all relevant information, including

significant evidence, arguments, gaps, and assumptions.See Table 5.10, which identifies fourteen distinct items ofrelevant information.Step 3: Assess the relevant information againsteach hypothesis by asking, Is this information highlyconsistent, consistent, highly inconsistent, inconsistent,neutral, or not applicable vis--vis the hypothesis?(The Te@mACH software does not include the neutralcategory.)The five hypotheses and fourteen items of relevant information can be entered into the Te@mACH software tool,and each cell can be rated as shown in Figure 5.6.Step 4: Refine the matrix by reconsidering the hypotheses. Does it make sense to combine two hypotheses, add anew hypothesis, or disaggregate an existing one?The Deceive and the Punish hypotheses might be combined because they seek similar goalsto delay or modifythe content of Radio Marti broadcastsand would risk lessretaliation against Cuba by the United States.Step 5: Draw tentative conclusions about the relative likelihood of each hypothesis. An inconsistency score will becalculated by the software; the hypothesis with the lowestinconsistency score is tentatively the most likely hypothesis.The one with the most inconsistencies is the least likely. Thehypotheses with the lowest inconsistency scores appear onthe left of the matrix, and those with the highest inconsistency scores appear on the right.The two hypotheses with the most Inconsistent items ofrelevant information are the Sabotage and Jam hypotheses.The Jamand nothing elsehypothesis is inconsistentwith much of Cubas past behavior; it would be highlyunlikely for Cuba to decide to stop pressing the US administration to stand down on launching Radio Marti. TheSabotage hypothesis had a large number of ratings showing that past Cuban activity to build transmitters anddevelop a capacity to disrupt broadcasts was inconsistentwith a sabotage strategy. Implementing either strategywould not require Cuba to construct a major radio broadcasting capability or demonstrate its ability to disrupt USradio broadcasts.Two hypothesesDisrupt US radio broadcasting andPunish the Cuban American communityhad a smallernumber of Inconsistent ratings, none of which were compelling, suggesting that they should not be discarded. The

Jousting with Cuba over Radio Marti 49

Table 5.10 Radio Marti: Relevant Information for ACH Analysis

1.

Despite Cubas signing of the North American Radio Broadcasting (NARB) Agreement in 1950, Cuban interference on the AM band beginsto grow in the 1960s after Castro comes to power; by the 1970s, it is a serious problem.

2.

In 1979, Cuba submits an inventory to ITU that includes plans for two radio stations transmitting with 500 kW of powera volume tentimes the limit permitted to any US radio station.

3.

The collapse of the Soviet Union and its economic subsidies severely damages the Cuban economy, resulting in an explosion of populardiscontent.

4.

In August 1981, Cuba says it intends to shift the frequencies of its 500 kW stations to 1040 kHz and 1160 kHz.

5.

In 1982, the Board of Directors of the Florida Association of Broadcasters adopts a resolution urging the United States to jam Cuban radiobroadcasts until illegal interference from Cuba ends.

The Federal Communications Commission (FCC) estimates that, at full power, the two 500 kW transmitters could be heard as far away asAlaska and Hawaii.

8.

On 30 August, the Cuban transmitter broadcasts on 1040 kHz for several hours at 150 kW (three times the US legal maximum), causingsignificant interference with WHOs broadcasting and several other US radio stations.

9.

The National Association of Broadcasters, citing the broadcasts, lobbies Congress on behalf of farmers and truckers to delay implementation of Radio Marti, and the Senate decides not to take up the legislation.

10.

The New York Times reports in May 1983 that senior US officials have told commercial broadcasters that a list of some forty US countermeasures are being considered if Cuba interferes with US radio stations, including destruction of offending Cuban transmitters.

11.

An amended version of Radio Marti legislation passes the US House of Representatives, stating that Radio Marti must adopt Voice ofAmerica (VOA) standards.

12.

Congress finally passes Radio Marti legislation in September 1983, with a legislative history that enables Radio Marti to become a surrogate home broadcasting service for Cuba.

13.

The president signs legislation establishing Radio Marti on 4 October 1983.

14.

Radio Marti is set to broadcast from Florida at 50 kW on 1040 kHz, which will not interfere with the signal of radio station WHO in DesMoines, Iowa.

most likely hypothesis to emerge from the analysis

was the Deceive hypothesis, which had only two Inconsistent ratings.Step 6: Analyze the sensitivity of your tentative conclusion to a change in the interpretation of a few critical itemsof information. If using the basic ACH software, sort theevidence by diagnosticity, and the most diagnostic information will appear at the top of the matrix. The Te@mACHsoftware will automatically display the most diagnost icinformation at the top of the matrix.The analysis would be most sensitive to any crediblereporting on what Castro and his key advisors were actuallythinking or intending to do as the confrontation played out.Discriminating between whether an observed action isintended to manage US perceptions or signal true intent toretaliate is difficult, if not impossible, lacking any information on or access to the actual decision-making process. Thevalue of ACH, in part, is that it helps the analyst thinkthrough all possible strategies in a rigorous manner, thereby

increasing the analysts confidence in his or her ability to

defend a final judgment.Step 7: Report the conclusions by considering the relativelikelihood of all the hypotheses.In this case, the Deceive hypotheses appear to emerge asCastros most likely course of action, but caveats would berequired. For example, it would be prudent to note thatCastro has been known to act precipitously in the past ifsufficiently provoked (as he did in shooting down the USU-2 aircraft during the Cuban Missile Crisis).Step 8: Identify indicators or milestones for futureobservation.A good analyst would be on the lookout for informationthat was inconsistent with any of the lead hypotheses. Forexample, key indicators to seek that would disprove theDeceive hypothesis would include: Renewed Cuban efforts to disrupt US commercialbroadcasting

50 Chapter 5

Figure 5.6 Radio Marti: Te@mACH Group Matrix with Ratings

Jousting with Cuba over Radio Marti 51

A public speech by Castro threatening specific

retaliatory action by Cuba Reports of Cuban plans to sabotage Radio MartifacilitiesSimilarly, key indicators that would tend to disprove theDisrupt hypothesis that Castro intended to defeat RadioMarti through a program of disrupting US radio broadcastswould include: Private assurances from senior Cuban officials toFlorida (or other) broadcasters that disruption wouldnot occur Relatively moderate statements, made publicly orprivately, that Castro was seeking a way to avoid amajor confrontation by striking a deal of some sortwith the United StatesAnalytic Value Added: As a result of your analysis,what are the most and least likely hypotheses? What arethe most diagnostic items of information? What, if any,assumptions underlie the data? Are there any gaps in therelevant information that could affect your confidence?How confident are you in your assessment of the mostlikely hypothesis? The analysis suggested that Castros mostlikely course of action would be to employ deception andmoderate disruption to press the United States to delay ormitigate the effects of Radio Marti by adopting VOA standards. The possibility of taking more serious retaliatory steps,however, could not be ruled out. Much would depend onCastros state of mind at the time Radio Marti was turnedon; his perception of how seriously the United States wouldretaliate; and his level of confidence that he could jam orotherwise interfere with the signal, making it less politicallydangerous for his regime. A key assumption throughout allthe analysis is that Castro would act rationally in responseto both US and any domestic Cuban stimuli. The biggestgap in information would be Castros intent. Because so littleis known about the intent of Castroor of any of his keyadvisorsthe level of confidence in the analysis would bemedium at best.CONCLUSIONAbout two weeks after President Reagan signed the legislation in October 1983 to initiate AM radio broadcasts toCuba, Havana announced its withdrawal from radio interference talks, citing its opposition to planned broadcasting

by Radio Marti to Cuba.10 Havana also continued to threaten

to disrupt US AM commercial radio broadcasting.11Analysts cautioned that regardless of what Castro saidpubliclyor was predicted to do in intelligence reportinghe could always change his mind at the last minute. Fromthe available facts, analysts could infer that Cuba could disrupt US broadcasting, but they could not infer that Cubawould disrupt US broadcasting when Radio Marti startedbroadcasting.On 20 May 1985, more than a year and a half after theRadio Marti legislation was signed, Radio Marti beganbroadcasting to Cuba.12 Cuba did not retaliate by disruptingUS commercial AM radio broadcasting. It chose instead toimmediately terminate the USCuba Emigration Agreement, thereby cancelling provisions for family visits.VALUE OF USING STRUCTUREDANALYTIC TECHNIQUESIn this case study, the use of Structured Analytic Techniqueswould have benefited the analytic process in two ways. Theywould have: Encouraged analysts to develop a full range ofpossible outcomesor testable hypothesesincluding a deception hypothesis. In this situation,the analysts focused mostly on only two outcomessignificant disruption or no significant disruption.To this extent, Skoug was correct when he observedthat no one had thought about Cuba striking back atthe Cuban American supporters of Radio Marti bycancelling the family visit agreement. By encouragingthe development of the full range of hypotheses,Structured Analytic Techniques would have helpedanalysts inform policy makers about alternativepossible outcomes, spurring them in turn to seekmore information about those outcomes. Prompted analysts to focus on the data most criticalin examining which course of action Castro wasmost likely to take. The use of analytic techniquescould have spurred analysts to examine clandestinereporting with special care because it offered thebest insights into Castros true intentions. However,the analysts would have been extremely unlikely tohave recognized at the time that Castro controlledvirtually all human sources reporting on Cubacollected by the US Intelligence Community and wasusing that stream of reporting to transmit deceptiveinformation about his plans to respond to Radio

52 Chapter 5Marti. That said, after Castro did not disrupt USAM broadcasting, some hard questions about thereliability of the key sources could have been asked.13KEY TAKEAWAYS Structured analytic techniques provides one ofthe best mechanisms for overcomingor, at least,mitigating the effects ofcognitive traps andmental mindsets that lead to making poor analytic

judgments. Always develop a full range of credible

hypotheses when beginning an analysis. This alsohelps ensure that policy makers will not be surprisedby what actually transpires. When working with reportingparticularly fromclandestine sourcesthat is critical to the analysis,always ask if the reporting might be intentionallydeceptive. In this case, it was used to reinforce opensource reporting that Cuba had the means and theintent to disrupt US AM broadcasting.

NOTES 1. Kenneth N. Skoug Jr., The United States and Cuba UnderReagan and Shultz: A Foreign Service Officer Reports (Westport,CT: Praeger, 1996), 17. 2. E.O. 12323. The Federal Register. 3. Skoug, The United States and Cuba Under Reagan andShultz, 19. 4. For a detailed treatment of the Cuban Missile Crisis case,see Graham Allison and Philip Zelikow, Essence of Decision:Explaining the Cuban Missile Crisis (New York: Longman, 1999). 5. Skoug, The United States and Cuba Under Reagan andShultz, 27; Michael Wines and Ronald J. Ostrow, Cuba ExultsThat CIAs Men in Havana Were Double Agents; In a TelevisionSeries, Alleged Spies-Turned-Heroes Tell How They DupedAmerican Agency, LA Times, August 12, 1987. 6. For more information on the Multiple HypothesesGeneratorTM, go to http:// www.globalytica.com. 7. See Gary King, Robert O. Keohane, and Sidney Verba,Designing Social Inquiry (Princeton, NJ: Princeton University

6 The Road to Tarin Kowt

his case asks students to grapple not only with hard

tactical and operational choices but also with implicitbeliefs about economic and political development and theirsuitability for the regions culture. At the tactical andoperational levels, the case presents several potential tradeoffs: to build the road quickly might compromise theprojects security; to proceed more deliberately could reduceits potential political impact. It also highlights somecomplex realities that demand a carefully consideredapproach. The people in the region are not only the villagerswith whom relationships must be built to facilitateconstruction and generate support for central government;they are also the very insurgents with which the UnitedStates must contend, and it is unclear how many might beopen to changing sides. The cultural code of Pashtunwalimeans that many locals will outwardly embrace and evenaid US plans, but they will inwardly reject the incursion intotheir way of life; people who are assisting the project by daymay very well be planting improvised explosive devices(IEDs) along the construction route by night.At the strategic level, the case presents a contrastbetween local cultural norms and the transformationalgoals of the United States andostensiblythe Kabulgovernment. One of the goals of this case is to teachstudents techniques that help them to uncover hiddenassumptions underpinning policy options in order totroubleshoot policy plans and improve the odds of success.The techniques in this case help students to assess implicitbeliefs about the operating environment, anticipated enemyresponse, and the potential impact on broader US goals forAfghanistan. Students should focus their efforts not onbuilding the specific steps in a course of action but onidentifying those issues that could not only undermine the

immediate missioncompleting the roadbut also subvert

the broader US goals in the region.TECHNIQUE 1: KEY ASSUMPTIONS CHECKThe Key Assumptions Check is a systematic effort to makeexplicit and question the assumptions that guide an analysts interpretation of evidence and reasoning about anyparticular problem. Assumptions are usually a necessaryand unavoidable means of filling gaps in the incomplete,ambiguous, and somet imes deceptive information withwhich the analyst must work. They are driven by the analysts education, training, and experience, including thecultural and organizational contexts in which the analystlives and works. It can be difficult to identify assumptions,because many are sociocultural beliefs that are unconsciously or so firmly held that they are assumed to be truthand not subject to challenge. Nonetheless, identifying keyassumptions and assessing the overall impact should theybe invalid are critical parts of a robust analytic process.Task 1.Conduct a Key Assumptions Check of the following issue:The United States is leaning toward making a decision tocomplete the road from Kandahar to Tarin Kowt in time forthe 18 September National Assembly elections as part of itsbroader goals to spur economic development, promotecentral governance, and improve security.Step 1: Gather a small group of individuals who areworking on the issue along with a few outsiders. Theprimary analytic unit already is working from an establishedmental model, so the outsiders are needed to bring otherperspectives.

53

54 Chapter 6Step 2: Ideally, participants should be asked to bring alist of assumptions when they come to the meeting. If not,start the meeting with a silent brainstorming session. Askeach participant to write down several assumptions on 3 5cards.Step 3: Collect the cards and list the assumptions on awhiteboard for all to see. A simple template can be used, asin Table 6.4 in the book.An initial list of brainstormed Key Assumptions for thiscase might include several higher-order assumptions suchas the following:

Asking these questions allows analysts to disaggregate

and refine the initial brainstorming list. In this case, doingso reveals new, more nuanced assumptions and underlyingassumptions. For example, an assumption about theTalibans willingness to allow the road to be built underpinsthe key assumption that the road will benefit the locals,Afghan government, and US/NATO operations. Theseotherwise hidden assumptions bear consideration as well,and they should be captured in the Key Assumptions table.Step 5: After identifying a full set of assumptions,critically examine each assumption. Ask: Why am I confident that this assumption is correct?

The local populace wants/needs the road.

In what circumstances might this assumption be

untrue?

The Afghan government wants/needs the road.

The US military wants/needs the road.

Could it have been true in the past but no longer true

today?

The US military has the capacity to construct the road.

The road will benefit the locals, the Afghangovernment, and US/NATO operations far morethan it will benefit the Taliban.Step 4: Elicit additional assumptions. Work from theprevailing analytic line back to the key arguments thatsupport it. Use various devices to help prod participantsthinking. Ask the standard journalistic questions: Who?What? How? When? Where? and Why?Phrases such as will always, will never, or wouldhave to be suggest that an idea is not being challenged andperhaps should be. Phrases such as based on or generallythe case usually suggest that a challengeable assumption isbeing made.

How much confidence do I have that this assumption

is valid? If the assumption turns out to be invalid, how muchimpact would this have on the analysis?Step 6: Using Table 6.4, place each assumption in one ofthree categories:1. Basically supported2. Correct with some caveats3. Unsupported or questionablethe keyuncertaintiesTable 6.7 shows an example classification of assumptions.

Table 6.7 Key Assumptions Check Example

Key AssumptionThe local population wants the road.

The local population needs the road.

The local population will be able to use the

road if it is built.

Commentary

Supported

With Caveat

They may not want the asphalt road. Deep suspicions

about outsiders may color local perceptions about theroads true purpose and likely impact on the region.

The assumption is that they currently are limited by the

absence of a road. They experience long travel timesfor commerce, goods, services, political participation,and security. Underlying assumption that a road wouldimprove all of these. (See below for these assumptions.)

Will they feel safe using the road? Perhaps while the USmilitary is there, but Soviet history suggests an ongoingsecurity presence will be necessary.

Unsupported

The Road to Tarin Kowt 55

Table 6.7 (Continued)

Key Assumption

Commentary

Supported

With Caveat

Unsupported

The code of Pashtunwali means that the locals

will embrace and aid the project.

Hospitality and hostility go hand-in-hand in the code

of Pashtunwali. The locals may embrace and even aidthe project when interacting with the US Army butundermine it in the absence of US forces.

Completion of the road in time for the

Unsupported. It cannot be assumed that a local culture

that is inherently suspicious of outsiders and centralgovernment will be grateful that these outsiders haveconstructed a highway through its midst.

The United States and its foreign contractors

are the only ones who can build the road intime.

The key factor is the compressed schedule, which does

not allow adequate time for the Army to hire and traina local construction crew.

The road will improve security in the region.

The road will improve voter turnout in the

parliamentary election.

56 Chapter 6Step 7: Refine the list, deleting those assumptions that donot hold up to scrutiny and adding new assumptions thatemerge from the discussion.This process reveals that it is important to amendassumptions to capture important nuances, such as bydisaggregating the assumption that the local populace wantsand needs the road. This process also reveals newassumptions that underpin initial assumptions. Oneexample is the assumption that the road will improvecommerce in the region and, in turn, that the Afghangovernment has the capacity to use it to promote commerce.Step 8: Consider whether key uncertainties should beconverted into collection requirements or research topics.In this case, several key uncertainties stem from theassumption that the road will improve voter participation,security, commerce, and the central governments reach.Other key uncertainties are that a functioning road willbenefit the Afghan government, locals, and US/NATOforces more than the Taliban and that the Taliban willcontinue to oppose US/NATO presence at its current,manageable level. Both of these warrant additionalresearch into how much permanent security presence(US, NATO, or Afghan) will be required for the roadscontinued use.Analytic Value Added: What impact couldunsupported assumptions have on the decision to build theroad? How confident should military decision makers bethat the benefits of building the road will outweigh therisks? Much of the strategy is premised on assumptions thatmay be valid in the Western context but are questionablewhen applied to Pashtun culture. As a result, it cannot beassumed that the locals will be grateful for the road and willexpress that gratitude through participation in a democraticprocess. Neither can it be assumed that the localsincluding the Talibanintend to use the road in the waysenvisioned by the United States.Another key factor in this analysis is the behavior of theTaliban forces in the region. If the Taliban increases themagnitude of its campaign against the United States andcooperative locals, it could significantly affect the ability ofthe United States to build the road in a timely and securemanner and the roads impact on local opinion. Thedecision to pursue construction is based in part on theassumption that Taliban operations will remain at theircurrent level and that the United States can suppress anychange in that level.

TECHNIQUE 2: DEVILS ADVOCACY

Devils Advocacy can be used to critique a proposed analytic judgment, plan, or decision. Devils Advocacy is oftenused before a final decision is made, when a military commander or policy maker asks for an analysis of what couldgo wrong. The Devils Advocate builds the strongest possible case against the proposed decision and its prospect forachieving its broader goals, often by examining criticalassumptions and sources of uncertainty, among otherissues.Task 2.Build the strongest possible case against the United Statespending decision to build the road from Kandahar to TarinKowt before the election.Steps: Although there is no prescribed procedure for aDevils Advocacy, begin with the strategic goals of theproject, assumptions, and gaps. These can serve as a usefulstarting point from which to build the case against the roadproject. Next, build a logical argument that undermineseach goal.The best Devils Advocate will identify the goals of USstrategy and disassemble them, drawing from andaugmenting the key assumptions and gaps identified in theprevious exercise. Beginning with the strategic goals of theUnited States allows students to address the fundamentaldifficulties surrounding the broader security, economic, andpolitical situation and then work downward to the moretactical issues facing the engineers as they embark on theirmission. The argument might proceed as follows:The USACE project will undermine the broader USgoals of economic development, improved governance, andenhanced security in the region. The project is premised onthe overarching assumption that the local population willwelcome a highway constructed by outsiders and willexpress its gratitude by supporting the Karzai governmentin the September election and beyond. This assumptionflies in the face of Pashtun culture, which is deeplydistrustful of foreigners and central government. Throughlocal eyes, the road is likely to be seen as a symbol ofintrusion by invaders and would-be Kabul-basedhegemons. Commerce. The project assumes that the road willspur licit local trade, but there is no indication thatformal studies of its potential commercial impacthave been done. Historical precedents provide

The Road to Tarin Kowt 57

little basis for confidence that the road will have

the intended commercial impact. Other Afghanroads have served as moneymakers for warlords,who extract tolls on truckers in return for allowingpassage, and as transportation links for drug andarms traffickers.

impediments. When these potential impediments are

exposed, decision makers can address them.

Governance. The project assumes that a compressed

timeline will have a more salutary effect on localopinion than a slower and more patient approach.The case for an accelerated schedule is based onthe belief that the locals will be impressed by theUS engineering feat, will recognize its benefits fortheir daily lives, and will translate their gratitudeinto support for progressive forces in the Septemberelections. A more likely outcome, however, is thatlocals will recoil at the rapidity with which outsidersintrude on their region. Most Pashtuns have littledesire for links to Kabul and are unlikely to begrateful for construction of those links. By contrast,a slower timeline would allow the US Army to play afacilitating rather than a performing role, hiring andtraining a local construction force to build the road.This would have the best chance of investing thelocal population with ownership of the highway andavoiding the perception that the road is an externallyimposed project.

Strengths-Weaknesses-Opportunities-Threats (SWOT) can

be used to evaluate a goal or objective by providing a framework for organizing and collecting data for strategic planning. SWOT is designed to illuminate areas for furtherexploration and more detailed planning, and therefore it istypically an early step in a robust policy process. SWOTanalysis can also be an important part of troubleshooting apolicy option and identifying specific actions that mayimprove the chances of success.

Security. Although the Army is equipped with

many of the needed resources, the 864th EngineerBattalion cannot by itself provide sufficient securityfor the mission, given the threat along the road.Furthermore, the project assumes that once built,the road can function with little or no requirementfor an ongoing US/NATO or Afghan governmentsecurity presence. The Soviet experience was telling.Securing roads required massive deployments offorces, which proved impossible. In the absence of anongoing Soviet security presence, mujahidin fighterstook advantage of roads to ambush Soviet convoyswith devastating effect. As a result, the roads did littleto spur commerce, and Soviet forces never managedto extend control beyond major highways andpopulation centers.

Step 2: Fill in Table 6.5 in the book by listing the Strengths,

Weaknesses, Opportunities, and Threats that are expected tofacilitate or hinder achievement of the objective. Table 6.8shows an example SWOT analysis.

Analytic Value Added: Which issues could

undermine the goals of the project, and why? Somestudents may be uncomfortable with a process that theyperceive as second-guessing an order or task. It should bestressed to students that the goal of the exercise is toimprove the chances of mission success by thinking asbroadly and exhaustively as possible about potential

TECHNIQUE 3: STRENGTHS-WEAKNESSESOPPORTUNITIES-THREATS

Task 3.Conduct a SWOT analysis of the pending decision to spureconomic development, promote central governance, andimprove security in the region by building a road connecting Kandahar City to Tarin Kowt prior to the Septemberelection.Step 1: Clearly define the objective.

Step 3: Identify possible strategies for achieving the

objective by asking: How can we use each Strength? How can we improve each Weakness? How can we exploit each Opportunity? How can we mitigate each Threat?Fill in Table 6.6 in the book with your strategies. Table 6.9shows an example.Analytic Value Added: What steps should the USArmy take to prepare for road construction? The greatestbenefits of the SWOT are that it encourages exhaustive andexplicit thinking about each category and, in doing so, helpsanalysts to identify a number of practical steps that theUnited States should take to prepare for road construction.

58 Chapter 6

Table 6.8 SWOT Example

US Strengths

US Weaknesses

Knowledge, skills, equipment, logistics.

US soldiers and equipment are challenged by the extreme

environment (heat/altitude/desert).

Ability to secure immediate area around job site.

United States faces cultural and linguistic barriers.

Sufficient funding.

The road is remote and far from the nearest base.

Support of Afghan government.

Not enough security forces (infantry) are attached to the engineering

battalion.No established network of local informers exists.Ephemeral presence in the region prevents establishment ofrelationships and fuels perception of US troops as outsiders.Threats to the US

Opportunities for the US

Engagement with a range of local villagers.

Easy target for Taliban harassment/ambush; Taliban could step up

targeting.

Hiring and training of local construction force.

Use of road for US logistics and lines of communication.Use of road to establish and maintain relations with a local networkof informants.Research on potential commercial impact of road on local andregional economies.

Taliban could exploit finished road to finance and support its ownoperations at the expense of the United States.Taliban could use the road for propaganda purposes to turn localsagainst the project.The US engineers will be blamed for any errors or accidents duringconstruction.Supply line is threatened by the remote environment and byinsurgents.Successful construction could saddle Afghan government withexpensive upkeep.

Table 6.9 SWOT Second-Stage Analysis

Use StrengthsThe United States is positioned to build the base road quicklywith US Army assets and USAID assistance.

Improve WeaknessesConstruct logistic bases along road route and preposition neededsupplies.Use local national interpreters and cultural advisors to identify triballeaders.Establish small civil affairs units to work with local population.Request infantry and air assets in support of the mission.Rotate in new equipment or work at less hot times of the day.

Exploit OpportunitiesUse early outreach to discuss and vet the route with local villageelders.Use air superiority to deliver supplies.Use local construction forces when possible.

Mitigate ThreatsEmpower the village elders so that they see the benefits of the road andwill be more inclined to accept any unforeseen problems that arise inconstruction.Use locals to deliver supplies and augment this with air supply.Use US Infantry units to flush out Taliban forces from surrounding mountains.Use of locals on construction teams could slow the process, but couldredound to US advantage if it helps establish a workforce knowledgeableabout road upkeep and capable of providing needed information aboutsurrounding local and insurgent positions.

The Road to Tarin Kowt 59

A robust SWOT analysis would delve deeper into these

areas to develop plans to address each requirement: Conduct outreach with the local Afghan leaders toobtain buy-in for the roads route and locate adequatewater supply and local logistics support and resupply. Identify interpreters and cultural advisors who havespecific local knowledge. Coordinate with other US Army elements forsecurity and resupply.CONCLUSIONThe United States ultimately committed to a compressedtimeline to build the road. On 18 August 2005, Army engineers concluded road construction with a symbolic meeting of the blades at the midway point. The constructionteam, led by Task Force Pacemaker, included the US Army,the Afghan National Army, USAID, and international contractors, all of whom played important roles in meeting thedeadline. The engineers spent over four months on overdrive to complete the road and credited success to carefuland innovative planning and execution that drew onefficient use of equipment crew rotations, establishing andworking from Forward Operating Bases, using materialalong the route, and relying on soldiers to adopt rolesoutside of their military occupational specialties...tostreamline the process.1

The 864th Engineering Brigade arrived in Afghanistan

organically equipped with heavy equipment, constructionpersonnel, combat engineers trained to clear minefieldsand find hidden IEDs, and additional maintenancepersonnel and repair assets to assist with the vehicles andequipment. They also collaborated with other Army unitsin the area for infantry support. These units assisted withsecurity missions on the road itself and patrols meant toflush out Taliban in the area. Logistical units ensured theflow of supplies, parts, and mail, in addition to providingsappers for route clearance operations and armoredpersonnel carriers to safely transport the sappers. USAIDcontractors and subcontractors worked with the Armyto pave the road. They provided supplementary heavyequipment, material testing services and laboratories,additional observation post support security for theforward operating bases, water wells, subsoil materials,and additional funding.2, 3

Instead of simply picking up where the 528th left off,

working from south to north, the Pacemakers also beganconstruction at the city of Tarin Kowt and worked south,establishing Forward Operating Base (FOB) Pacemakerat the midway point to support operations. At FOBPacemaker, which was secured with a dirt bermperimeter and guard towers, the construction crewscould safely store and maintain their equipment, eat,sleep, occasionally shower, and sometimes be able to callhome.The construction of the road to Tarin Kowt predates theUnited States official adoption of the counterinsurgencydoctrine (COIN). Although not a new concept, COINdefeats the goals of the enemy not primarily through kineticoperations against insurgents but by winning over the localpopulation. As David Galula explained in his classic text oncounterinsurgency warfare,if the insurgent manages to dissociate the populationfrom the counterinsurgent, to control it physically, to getits active support, he will win the war because, in thefinal analysis, the exercise of political power depends onthe tacit or explicit agreement of the population or, atworst, on its submissiveness.4

Task Force Pacemaker used local interpreters to ensure

that the villages along the road were supported and friendly.The United States provided everything from security tostandard infrastructure, with the hope that doing so wouldcause the insurgents to lose credibility among the localpopulace. Task Force Pacemaker built working relationshipswith the locals during the mission, but with the completionof the road the Army Engineers moved elsewhere, and theresponsibility of maintaining partnerships with the communities fell on the local government officials and securityforces.5The tactical and operational success of Task ForcePacemaker is clear, but determining the extent to whichthis engineering feat advanced strategic US goals to spureconomic development, promote governance, andimprove security is difficult.6 Between 2002 and 2007,the US government invested approximately $1.7 billion inroad construction projects in Afghanistan. A 2008 studyby the US Government Accountability Office (USGAO)found thatthe United States and other international donors havecommitted billions of dollars toward road reconstruction in Afghanistan to promote economic and social

60 Chapter 6development as well as security and stability. Whilesome have noted that reconstructed roads contribute positively to economic and social conditions inAfghanistan, there is currently little evidence basedon sound impact assessments that these projectshave resulted in expected benefits. . . . 7

Figure 6.1 V oter Turnout by Election in Afghanistan, 20042010

308,896

The USGAO also stated that

[USAID] agency officials and others have reportedsome examples of projects positive impact, such asincreased commerce and decreased transportationcosts. However, these results are based on a limitedqualitative assessment or anecdotal information andtherefore cannot be generalized.8

171,470

71,78361,043

34,283

23,646

85,835

13,611

USGAO found that between 2004 and 2007, the

2005200420092010Presidential Parliamentary Presidential ParliamentaryDepartment of Defense (DOD) spent nearly $15ElectionElectionElectionElectionmillion on Commanders Emergency ResponseProgram (CERP) projects in Kandahar and UruzganKandahar ProvinceUruzgan Provinceprovinces, and USAID spent $25 million on theKandahar City to Tarin Kowt road.9 The US ArmySource: Compiled by the authors based on final election results released byCorp of Engineers (USACE) reported to USGAOthe International Election Commission (IEC) of Afghanistan. The raw datathat general impact indicators it observedare found at http://www.iec.org.af/.included increased traffic when a new road isbuilt and more gas stations.10 For the DOD, theseUruzgan provinces between the 2004 presidential anddevelopments underscored how the roads have improved2005 p arli ament ar y ele c t ions (s e e Figure 6.1).governance by opening up lines of communication amongCountrywide voter turnout for the 2004 election wasdistricts, provinces, and the central government.11 A seniorapproximately 73 percent, while for the 2005 election itAfghan security force leader working with Task Forceapproached only 50 percent. The drop continued with thePacemaker, however, said he was afraid to travel to his2009 election, with turnout falling to 31 percent. Forhome, only forty-five minutes away, noting that the TalibanUruzgan and Kandahar provinces, voter turnout fell fromdo not like the Tarin Kowt Road, and terrorize those whojust over 40 percent to just over 20 percent combined.do; he also predicted that if the Americans pulled out,Isolating the precise impact of the road on voter turnout isNo one would travel down that road. 12 Upon completionimpossible.14 At best, it can be said that the road couldof the road, the engineers no longer secured any areashave mitigated what otherwise would have been a morealong the route from Kandahar City to Tarin Kowt. The jobprecipitous decline in voter turnout. At a minimum, theof ensuring its safe accessibility fell to the Afghan securityfigures suggest the road did not have the catalytic effect onforces.13electoral participation that it was intended to have.Assessing the impact of the road on the election isThe road to Tarin Kowt has become a testimony to thefurther complicated by events surrounding election daygap between hope and reality in Afghanistan. When theitself and the inherent difficulty of isolating the roadUS Army Engineers began to build the road in 2004,construction as an independent factor. One month aftertravelling the route along the dirt path linking the twothe Army completed the road, on 18 September 2005,cities took fifteen hours; immediately after the ArmyAfghans headed to the polls in the first democraticcompleted its work in 2005, the journey along the newlyparliamentary election since 1969. Voting took place amidpaved road took the engineers only three.15 But within aTaliban threats of violence. The election results indicate afew years, the road to Tarin Kowt had become one of theprecipitous drop in voter turnout in both Kandahar andmost dangerous roads in the world. Neither foreigners nor

The Road to Tarin Kowt 61

Afghans could freely travel it for fear of attack by Taliban

insurgents, and traffic was largely restricted to slowmoving biweekly convoys of 100 to 200 trucks. 16 Thetrucks were escorted by a local policeman who ran a forceof about 300 uniformed police and another 1,700 militia.17In 2009, an Australian journalist chronicled a trip alongthe road, leaving Kandahar with an Afghan convoy atdawn and arriving in Tarin Kowt over twenty-four hourslater. This journey along the modern road took nearly tenhours longer than travel along the centuries-old dirt pathhad taken.18

KEY TAKEAWAYS An effective Red Team approach can include a rangeof techniques and is an essential part of any processaimed at uncovering hidden weaknesses in a courseof action. In this case, the approach helps to identify amisalignment of strategic, operational, and tactical goals. Even without an abundance of time or specializedknowledge, analysts can use these structured analytictechniques to identify the right questions to ask and tooutline an approach that can mitigate weaknesses beforethey have deleterious effects on mission outcome.

12. Laura M. Walker, Up Close . . . Task Force Pacemakers

Solders: Impressive Dedication and Professionalism, ArmyEngineer, SeptemberOctober 2005, 26. 13. Crossland, interview. 14. The author compiled the raw voting data based on finalelection results released by the International Election Commission(IEC) of Afghanistan, which is the official election body. The rawdata are found at http://www.iec.org.af. The mission of the IEC,which is a constitutional body . . . and a professional Electionmanagement body is to conduct free and fair elections and referendums in an efficient and impartial way. 15. Walker, Task Force Pacemaker Constructing a Road toDemocracy, 19. 16. Bette Dam, Danger on the Road to Uruzgan, RadioNetherlands Worldwide (RNW) News, July 10, 2009, http://hunaamsterdam.nl/english/article/danger-road-uruzgan. 17. Jeremy Kelly, Long Road to Tarin Kowt, The Australian,April 28, 2008, http://www.theaustralian.com.au/news/world/longroad-to-tarin-kowt-story-e6frg6so-1225704435431. 18. Ibid.

Table 7.1 Case Snapshot: Who Murdered Jonathan Luna?

Structured Analytic Technique Used

Heuer and Pherson Page Number

Analytic Family

Chronologies and Timelines

p. 56

Decomposition and Visualization

Simple Hypotheses

p. 171

Hypothesis Generation and Testing

Multiple Hypotheses Generator

p. 173

Hypothesis Generation and Testing

Analysis of Competing Hypotheses

p. 181

Hypothesis Generation and Testing

7 Who Murdered Jonathan Luna?

he Luna case has never been solved. It is not a puzzle

for which there is a correct and final answer thatpoints to a killer, whether it is Luna himself or someoneelse. When confronting a case in which so much significantinformation is unknown, the analyst should focus first ondevising and executing a solid analytic process that framesthe problem and brings order to the jumble of data points,assumptions, and gaps that form the case. In short, thefocus is on defining an analytic process now that willincrease the chances that the analyst will identify andincorporate emerging information to help solve the puzzlein the future.The controversy surrounding this case as well as thedetailed information that is already publicly available makesit a particularly good tool for teaching how analytictechniques such as Timelines, Chronologies, HypothesisGeneration, and Analysis of Competing Hypotheses canhelp analysts systematically sort, array, and analyze a dataset in a way that brings a complex group of events intobetter, if not complete, focus. It also drives home howgeospatial visualization tools such as mapping software canilluminate analytic points that otherwise may be overlooked,such as anomalies in distance, timing, and locationinformation. Lastly, as with all cases in which human,electronic, and press reporting are used, the case highlightsthe importance of both sourcing and confidence levels inanalysis, particularly when dealing with eyewitnesses,secondhand reporting, and after-the-fact recollections.TECHNIQUE 1: CHRONOLOGIES AND TIMELINESChronologies and Timelines are simple but useful tools thathelp order events sequentially; display the information

graphically; and identify possible gaps, anomalies, and correlations. In addition, these techniques pull the analyst outof the evidentiary weeds to view a data set from a more strategic vantage point. Chronologies and Timelines can bepaired with mapping software to create geospatial productsthat display multiple layers of information such as time,location, terrain, weather, and other travel conditions.The details of this case make an annotated Timeline andMap particularly useful in identifying key pieces ofevidence, confidence levels in the reporting, and gaps in theinformation.Task 1.Create a Timeline of Lunas last hours.Step 1: Identify the relevant information from the casenarrative with the date and order in which it occurred.Consider how best to array the data along the Timeline.Can any of the information be categorized?There are many ways to present the data in this case in atimeline. A full timeline of the case will reflect a periodfrom Lunas youth in New York through his death and intothe present day. It will include all references in the case toLunas activities prior to his death and new informationuncovered in the investigation. This new informationshould be reflected on the timeline at the time it allegedlyoccurred. A more sophisticated timeline would also includea separate line for when the information was reported.Doing so not only helps an analyst see events as theyunfolded but also understand when information becameavailable. This allows analysts to look for any anomalies inthe pattern of the reporting that might be associated with adeception hypothesis.

63

64 Chapter 7The timeline in Figure 7.1 is excerpted from a longertimeline of the case and illustrates how relevant informationcan be displayed along a two-sided timeline in order toreflect evidence and analysis, including assumptions andgaps. It also shows how color coding can be used to reflectcategories of activities. In this timeline, the evidence isbroken into three categories: Lunas known movements, thecars movements, and his bank card activities.

have assumed that these sources as reported are accurate,

and we have included them on the timeline. When thereare questions about the reliability of reporting, or thereare anomalies, these can be listed on the timeline as ananalytic comment. In this timeline, analytic comments arereflected in italics above the timeline.

Step 2: Review the Timeline by asking the following

questions:

Create an annotated Map of events based on your Timeline.

Are there any missing pieces of data?

There is a lack of information about Lunas activitiesbetween 1730 and his return to the office after 2300 thatnight. This gap raises a number of important questions.For instance, what time did he arrive at home? Did he godirectly home? When exactly did he leave for the office laterthat night? Where was he when he called opposing counsel? Do any of the events appear to occur too rapidly orslowly to have reasonably occurred in the order ortiming suggested by the data?At the time of the investigation, authorities said thatthey could not account for a two-hour period beginningat 0057, when Lunas ATM card was used at a rest stop inDelaware, and ending at 0247, when his car passed throughthe Delaware River Bridge toll plaza on Interstate 276.1The earliest, judging by driving times, that he could haveentered the Pennsylvania Turnpike would have been 0145,but the E-ZPass record indicates that the car did not enterthe Turnpike until 0247. In addition, the timing of the Kingof Prussia and Elverson Roy Rogers stops seems too close. Itseems unlikely that Luna would have been able to travel thatfar in such a short period of time. Could any events outside the timeline haveinfluenced the activities?Possibly. Given the unexplained gaps outlined above,events could have occurred during these gaps that havedirect bearing on the timeline. Are there any underlying assumptions about theevidence that should be taken into consideration?The sources of information include eyewitnesses andconfidential sources. For the purposes of the timeline, we

Task 2.

Step 1: Use publicly available software of your choosing to

create a Map of the area.Step 2: Overlay the route.Step 3: Annotate the Map with appropriate times andlocations presented in the case (see Map 7.2).For those seeking to employ a more sophisticated geospatial presentation, geographic coordinates are includedwith key data points in Table 7.2.

Analytic Value Added: What does the sequence of

events tell you? From the time Luna left his home until thetime his body was found in Pennsylvania on the morning of4 December 2010, we have only information about his carand bank card. From Map 7.2, it appears that Luna took aroundabout route from his Baltimore office to Lancaster,Pennsylvania. He drove northeastward on I-95 fromBaltimore to Delaware and then toward the Philadelphiaarea, but then veered westward on the PennsylvaniaTurnpike.Are there any gaps in the information that should beaddressed? There are gaps between 1730 and 2100, 0057and 0237, and 0404 and 0530. There are conflicting reportsabout his whereabouts between 0300 and 0400. The 0057to 0237 period is most perplexing, because is unclear whatroute he took from the JFK rest stop to New Jersey Turnpikeinterchange 6A from New Jersey Route 130. Did he makeany stops during that period?What additional information should you seek? Thereis a lack of information that would determine whether hewas alone or with someone, whether he was the driverfor the entire trip, or whether he was the user of the debitcard. A second driver, for example, could have used a paperticket, not realizing that the car was equipped with E-ZPass.

At 0237 Lunas car enters

Route unknown from 0057

until 0237.

76

MiddletownMdletl town

Pottstown

47

Centreville

Annapolis

Additional information should be sought about his route

and activities from 0057 until 0237.How confident are you in the sources of information?Much of the reporting comes from unnamed law enforcement sources, eyewitness reports, or character witnesses.As a result, the analysis should reflect the reliability of thesesources, particularly when there are conflicting or anomalous aspects to the reporting. Also, for electronic evidence,such as building records, E-ZPass, and bank records, confidence levels and underlying assumptions should be noted;while the reporting probably reflects accurate time stamps,it is unknown if Luna himself was the user of the car anddebit cards at all times.

TECHNIQUE 2: MULTIPLE HYPOTHESIS

GENERATION: SIMPLE HYPOTHESESMultiple Hypothesis Generation is part of any rigorous analytic process because it helps the analyst avoid common pitfalls such as coming to premature closure or being overly

13

0113

1010

20 miles20 kilometers

influenced by first impressions. Instead, it helps the analyst

think broadly and creatively about a range of possibilities.The goal is to develop an exhaustive list of hypotheses thatcan be scrutinized and tested over time against both existing evidence and new data that may become available in thefuture.This case is well suited to Simple Hypotheses, whichemploys a group process that can be used to think creativelyabout a range of possible explanations that go beyond thoseraised by authorities in the case. Using a group helps togenerate a large list of possible hypotheses; group the lists;and refine the groupings to arrive at a set of plausible,clearly stated hypotheses for further investigation.Task 3.Use Simple Hypotheses to create a list of alternative hypotheses that explain Jonathan Lunas death.Step 1: Ask each member of the group to write down onseparate 3 5 cards or sticky notes up to three plausible

Who Murdered Jonathan Luna? 67

Table 7.2 Jonathan Lunas Route with Geographic Coordinates

DateWednesday3 December

Thursday4 December

Time

Location

Activity

2338

Court House, Baltimore, MD

Lunas car leaves parking garage at US District Court Building.

391713.21N76372.43W

2349

Baltimore, MD

Lunas car passes Fort McHenry Tunnel toll plaza, northbound

on I-95.

391539.12N763438.87W

0028

Perryville, MD

Lunas car passes through Perryville toll plaza, northbound.

393515.68N76 424.15W

0046

Delaware Line toll plaza

Lunas car passes through toll plaza, northbound.

393842.39N754552.56W

0057

I-95 Exit 3, Newark, DE

Lunas debit card was used for a $200 ATM withdrawal fromExxon at Travel Plaza.

393945.30N754125.71W

0237

New Jersey Turnpike

Lunas car enters Turnpike at interchange 6A from NJ Route

130.

40 65.78N744721.25W

0247

Delaware River Bridge, PA

Lunas car enters Pennsylvania Turnpike at interchange 359,

the Delaware River Bridge.

40 718.18N745046.90W

0320

King of Prussia, PA

Lunas debit card was used at a Sunoco Station to buy gas andpossibly for another ATM withdrawal.

40 522.03N752215.61W

0330

PA Turnpike, Elverson, PA

A Roy Rogers restaurant manager at a rest stop says she saw

Luna. FBI investigators doubt this.

40 858.46N754959.85W

0404

PA Turnpike, the Reading/

Lancaster interchange

Lunas car exited PA Turnpike at exit 286. Paper ticket (with

blood spot) was turned in to toll collector even though Lunascar has E-ZPass.

401258.97N76 429.27W

After0530

Denver, PA

Sensening & Weaver employee finds Lunas car on company

property, hood down in a creek.

401237.45N76 330.58W

alternative hypotheses or explanations. Think broadly and

creatively but strive to incorporate the elements of a goodhypothesis: It is written as a definite statement. It is based on observations and knowledge. It is testable and falsifiable. It contains a dependent and an independent variable.

Geo-coordinates

Table 7.3 Luna Simple Hypothesis Generation:

Example of Consolidated HypothesesLuna was murdered by those he was negotiating a plea bargainfor; they did not like the deal.Luna committed suicide.Luna was killed by someone associated with another case he hadworked.Luna was murdered by a female or male lover in an establishedrelationship.Luna was murdered by the established lovers spouse.

Step 2: Collect the cards and display the results.

Consolidate the hypotheses to avoid duplication. Aconsolidated set of hypotheses might look like Table 7.3.

Luna was abducted and murdered by creditors for his failure to

pay off bad debts.

Step 3: Aggregate the hypotheses into affinity groups and

label each group.Consider multiple ways to display the affinity groups. Inthis case, the hypotheses may be grouped by perpetrator ofthe crime, which includes Luna himself (the suicide

His wife had him killed because she found out he was cheating.

Luna had a liaison with someone he had just met on an Internet

sex site, and the affair went bad, resulting in his stabbing. He fellinto a creek and died.Lunas attorney colleagues were jealous of him and had him killed/killed him.Luna was being blackmailed and the operation went bad and theykilled him.

68 Chapter 7hypothesis), a lover, a hit man, Lunas colleagues, etc.Alternatively, grouping by Why (debt, work-related issues,jealousy/envy, and random violence), for example, can helpconsiderably with achieving mutual exclusivity and can helpconsolidate the Who list later.Step 4: Use problem restatement and consideration of theopposite to develop new ideas.Problem Restatement: Why did Jonathan Luna take sucha circuitous and late-night trip toward Philadelphia?Opposite: Luna was not suicidal; he was a victim ofsomeone elses rage. This could include a random act ofviolence or a murder by a lover, colleague, criminal he hadpreviously prosecuted, or creditor.This process illuminates the possibility of a random actof violence. Luna had allegedly traveled to Philadelphianumerous times. His circuitous route that night took himfirst directly toward Philadelphia. Only after the anomaloustwo-hour period from the 0057 ATM withdrawal to 0247did his car take a turn westward. Could he have beenheaded to Philadelphia and fallen victim to a random actof violence on his trip? Lunas key witness in the case hehad been prosecuting that day, who had reversed himselfon the stand, had been in custody in Philadelphia. CouldLuna have been returning to Philadelphia for work-relatedpurposes?Step 5: Update the list of alternative hypotheses.Problem restatement augments the list of hypotheses byincluding the possibility of a random act of violence.Step 6: Clarify each hypothesis by asking, Who? What?How? When? Where? and Why?Make a list of each of the categories above. Step back andconsider how each list could be augmented. The Who listincludes colleagues, stranger, lover, creditors, criminal hehad prosecuted in the past. Refine this list to make thecategories more mutually exclusive. This helps clarify thehypotheses. For example, creditors, criminals, andcolleagues could all have employed a hit man.Step 7: Select the most promising hypotheses for furtherexploration.Luna was murdered by those he was negotiating a pleabargain for, his creditors, or his lover; Luna committedsuicide; Luna was killed in a random act of violence.

TECHNIQUE 3: MULTIPLE HYPOTHESIS

GENERATION: MULTIPLE HYPOTHESESGENERATORTMThe Multiple Hypotheses GeneratorTM is a useful tool forbroadening the spectrum of plausible hypotheses. It is particularly helpful when there is a reigning lead hypothesisin this case, the hypothesis that Luna was alone the night hedied and therefore must have committed suicide.The most important aspect of the tool is the discussion itgenerates among analysts about the range of plausiblehypotheses, especially about the credibility score for eachpermutation. It is important to remember that thecredibility score is meant to illuminate new, crediblehypotheses for further examination. And while the processdoes encourage analysts to focus on the hypotheses withhigher credibility scores, hypotheses with low credibilityscores should not be entirely discarded because newevidence may emerge that changes their status.Task 4.Use the Multiple Hypotheses GeneratorTM to create andassess alternative hypotheses that explain Jonathan Lunasdeath. Contact Globalytica, LLC at THINKSuite@globalytica.com or go to http://www.globalytica.com to obtainaccess to the Multiple Hypotheses GeneratorTM software if itis not available on your system.Step 1: Identify the lead hypothesis and its componentparts using Who? What? How? When? Where? and Why?Jonathan Luna committed suicide as a result of personalproblems, including debt and a possible investigation ofpersonal wrongdoing.Steps 2 & 3: Identify plausible alternatives for each keycomponent and strive to keep them mutually exclusive.Discard any given factors.Discard How (drowning), Where (Pennsylvania), What(killed), When (4 December 2003), which will be the samefor all hypotheses. Brainstorm possible alternatives for eachof the remaining components, which in this case are Whoand Why. Consolidate the lists into alternatives that are asmutually exclusive as possible. For example, adversary isused in the example in Table 7.4 to reflect Lunas enemies orsomeone who is hired by or is associated with those whowould want to kill Luna. A random attacker could reflect arobbery or hate crime.

Who Murdered Jonathan Luna? 69

Table 7.4 Luna Multiple Hypotheses GeneratorTM:

Examples of Brainstormed AlternativesLead Hypothesis: Jonathan Luna committed suicide as a result ofpersonal problems he was facing.Components

Who?

Why?

Lead Hypothesis

Suicide (Luna)

Debt

BrainstormedAlternatives

Adversary/Hit ManLoverRandom Attacker

Work-RelatedProblemJealousy/EnvyAccident

Steps 4, 5, & 6: Generate a list of possible permutations,

discard any permutations that simply make no sense, andevaluate the credibility of the remaining hypotheses on ascale of 1 to 5, where 1 is low credibility and 5 is highcredibility.Table 7.5 shows an example response.Step 7: Re-sort the remaining hypotheses, listing themfrom most to least credible.Table 7.6 shows an example.Step 8: Restate the permutations as hypotheses.

The permutations in Table 7.6 are stated as hypotheses.

Step 9: Select from the top of the list those alternativehypotheses most deserving of attention and note why thesehypotheses are most interesting.For this case, this includes hypotheses with a credibilityscore of 3 or higher (see Table 7.7). While the credibilityscore is subjective in nature, it should reflect reasoning thatcan be used to weed out nonsensical or highly unlikelyhypotheses. The unused hypotheses should not bediscarded. They should be reserved, and the list should bereconsidered as new information becomes available.Analytic Value Added: Which hypotheses should beexplored further? For this case, the lead hypothesis, thatLuna committed suicide, should certainly be furtherexplored, as should the new random act of violencehypothesis.What motives should be considered, and why?A full set of motives, including jealousy, envy, his debt, hiswork, or accident should also be explored.Which hypotheses from the original list were setaside, and why? It is up to the analyst to decide how manyand which hypotheses should be considered for furtherexploration. A general rule of thumb is that more than

Adversary killed Luna out of envy.

Adversary killed Luna accidentally.

A lover killed Luna because of Lunas debt.

A lover killed Luna because of his performance on a case at work.

A random attacker killed Luna because of his indebtedness.

A random attacker killed Luna because of his performance on a case at work.

Table 7.7 Luna Multiple Hypotheses GeneratorTM: Example of Hypotheses for Further ExplorationHypotheses for Further Exploration

Reasoning

Luna committed suicide because he was

having problems at work.

Suicidewhether intentional or unintentionalis authorities lead hypothesis; authorities have

heretofore undisclosed reasons to believe Luna was alone the night of his death.

Luna committed suicide accidentally.

The main motivation for such an accidental suicide has been reported as being an effortto garner sympathy and/or stave off taking a polygraph in connection with an ongoinginvestigation.

Adversary killed Luna because of his

performance on a case at work.

His profession makes him a possible target of many individuals. Whether the death was ahit or an attack by a known acquaintance, the work-related adversary hypothesis should beexplored further.

Adversary killed Luna because of his

indebtedness.

Luna had credit card debt. Were there any other debts that could have prompted an adversary tointentionally or unintentionally take his life?

A lover killed Luna out of jealousy.

The so called personal nature of the attack, including wounds to the genitals, could point to alovers involvement.

A random attacker killed Luna out of envy.

Given stops along the roundabout route and gaps in information concerning the route itselfafter the 0057 withdrawal, must consider a random attacker.

five hypotheses becomes cumbersome and should signal

possible problems with mutual exclusivity. In such cases,analysts should be encouraged to aggregate hypothesesor review the basis for the credibility scoring. Also, analysts often will include hypotheses for which there is no

evidence in the original list. In this case, students may

raise the possibility that Luna was murdered by his spouse.This kind of creative thinking should not be discouragedin the initial brainstorming phase, but hypotheses thatare not based on observations or knowledge should not

Who Murdered Jonathan Luna? 71

constitute the lead hypotheses for further exploration.

Analysts should, however, explicitly discuss why certainhypotheses do not make the final list and how that couldchange in the future should new information come to light.TECHNIQUE 4: ANALYSIS OFCOMPETING HYPOTHESESAnalysts face a perennial challenge of working with incomplete, ambiguous, anomalous, and sometimes deceptivedata. In addition, strict time constraints on analysis and theneed to make a call often conspire with a number of natural human cognitive tendencies to zero in on a singlehypothesis too early in the analytic process. The result isoften inaccurate or incomplete judgments. Analysis ofCompeting Hypotheses (ACH) improves the analystschances of overcoming these challenges by requiring theanalyst to identify and refute possible hypotheses using thefull range of data, assumptions, and gaps that are pertinentto the problem at hand.Task 5.Use the top hypotheses compiled with the MultipleHypotheses GeneratorTM to conduct an Analysis of CompetingHypotheses of the Luna case. Contact Globalytica, LLC atTHINKSuite@globalytica.com or go to http://www.globalytica.com to obtain access to the basic software, or the collaborative version called Te@mACH, if it is not available on yoursystem.Step 1: List the hypotheses to be considered, striving formutual exclusivity.The Multiple Hypotheses Generator TM and SimpleHypotheses techniques help to ensure mutual exclusivityand an exhaustive set of hypotheses, which greatly aids theutility of ACH.ACH matrices can include as many hypotheses as theanalyst requires. However, more than five hypothesesusually become cumbersome and reflect a problem withmutual exclusivity. In this case, there is some overlap withthe suicide, but the motivations (accidental versusintentional suicide) are sufficiently exclusive of one anotherto retain both hypotheses in the matrix. As a result, anotional list might include: Luna committed suicide becauseof problems at work; Luna accidentally committed suicide;an adversary killed Luna because of his performance on acase at work; a lover killed Luna out of jealousy; a randomattacker killed Luna out of envy.

Step 2: Make a list of all relevant information, including

significant evidence, arguments, gaps, and assumptions.Figure 7.2 shows an example of list of information.Step 3: Assess the relevant information againsteach hypothesis by asking, Is this information highlyinconsistent, inconsistent, neutral, not applicable, consistent,or highly consistent vis--vis the hypothesis? (TheTe@mACH software does not include the neutral category.)Analysts using the basic ACH software will have theoption of choosing highly consistent (CC), consistent (C),inconsistent (I), highly inconsistent (II), not applicable (NA),or neutral (N). When using basic ACH or My Matrix withthe Te@mACH tool, it is important that analysts code theevidence line by line, in other words horizontally across thematrix, not hypothesis by hypothesis, or vertically downthe matrix. Doing so helps the analyst consider each piece ofevidence fully against each hypothesis before moving on tothe next piece of evidence. This process keeps the analystfocused on the evidence rather than on proving a pethypothesis. The Survey option in Te@mACH generatesthe cells randomly, avoiding this problem.When entering and coding the data, the credibility scoreof all evidence is set at a default of medium. Analysts canalso include a credibility score of low or high. Doing sowhen using the basic ACH tool will allow the ACH softwareto calculate a weighted inconsistency score that reflects theanalysts judgment about credibility of the data. For thiscase, the credibility of evidence is particularly important.Direct, expert evidence from coroner Dr. Barry Walp, forexample, could be coded as highly credible, while indirectevidence from anonymous law enforcement sources maysimply remain medium. DiBagios contradictory reportingcould be coded as low. Any credibility issues incorporatedinto the matrix should be included in the final, writtenanalysis, because they are assumptions embedded in theanalysis. With Te@mACH, you can check a special KeyAssumptions box to record and explain any keyassumptions relating to a particular item of relevantinformation. Figure 7.3 shows coding matrices for two ACHsoftware packages.Step 4: Rate the credibility of each item of relevantinformation.Step 5: Refine the matrix by reconsidering the hypotheses.Does it make sense to combine two hypotheses, add a newhypothesis, or disaggregate an existing one?

Walp classifies as homicide.

Brought up in rough neighborhood.

Body discovered off Dry Tavern Road.

Plea agreement because of problem with FBI witness.

Coroner Walp says no sign of defensive wounds.

Coroner (Kirchner) classifies as homicide.

Luna showed signs of defensive wounds.

Pool of blood in back seat.

Signs of restraint.

Traumatic neck wound.

Injuries to genitals.

Allegations that FBI mishandled informant.

S ource says Luna came into $10K just as $36K inevidence went missing. D iBiagio privately admitted to coworkers that he had liedabout Lunas job being in jeopardy. Internal FBI inquiry into FBIs handling of allegations ofagents affair with Luna. Roy Rogers at 0330, timing odd.

If the hypotheses are not mutually exclusive, this fact will

become apparent at this stage in the process if it has notalready become so during the coding process. Analystsshould consider disaggregating hypotheses whenever theyfind themselves clarifying the hypothesis as they code.Such is the case if one only considers a basic suicidehypothesis. As evidence is coded, it will become apparentthat a separate, accidental/staged suicide hypothesis isnecessary. The trigger, or indicator, that this is necessaryoccurs during the coding process. If a piece of evidence thatis inconsistent with intentional suicide is often clarified by

But it could be consistent if he was trying to stage the attack

and it went wrong, then another hypothesis is needed.Step 6: Draw tentative conclusions about the relativelikelihood of each hypothesis. An inconsistency score willbe calculated by the software; the hypothesis with the lowestinconsistency score is tentatively the most likely hypothesis.The one with the most inconsistencies is the least likely. Thehypotheses with the lowest scores appear on the left of thematrix, and those with the highest inconsistency scoresappear on the right.

Who Murdered Jonathan Luna? 73

It is important to address the likelihood of every

hypothesis, not simply the most and least likely. Based uponthe above hypotheses and relevant information, sometentative conclusions about the relative likelihood of eachhypothesis would include the following observations. Itappears that an intentional, work-related suicide is by farthe least likely hypothesis because it has the mostinconsistent evidence. Another less likely hypothesis is theaccidental suicide hypothesisthat Luna killed himselfwhile attempting to stage an attack on himself. For example,it makes little sense that he would inflict injury to his owngenitals or that blood of a second person would be present.Likewise, a random attack is nearly as unlikely as accidentalsuicide; a case can be made that a random attacker wouldnot use the victims own penknife. And finally, although ajealous lover hypothesis is the least inconsistent with thedata, a work-related attack is a very close second. It is just asimportant to critically examine the inconsistent for themost likely hypotheses as well. If there are manyinconsistencies associated with the most likely hypotheses,this could signal that there is a missing hypothesis.However, if the inconsistent evidence can be refuted, then itcan be regarded as squishily inconsistent, and thehypothesis probably is the most likely explanation.

Step 7: Analyze the sensitivity of your tentative conclusion

to a change in the interpretation of a few critical items ofevidence by using the software to sort the evidence bydiagnosticity.All of the hypotheses will include at least someinconsistent data. The goal of this step is to understandwhich pieces of evidence have the most overall effect on therelative likelihood of the hypotheses and what could happenif those pieces of evidence change.When sorted by diagnosticity, it becomes apparent thatsome of the most potentially diagnostic pieces of evidenceare already sources of controversy. For example, Walp saidthat he saw no signs of defensive wounds. By itself, this is ahighly diagnostic piece of evidence because it is consistentwith suicide, but it is inconsistent with the other hypotheses.While we should have fairly high confidence in thisfirsthand reporting, several law enforcement sources havereported that Luna did suffer defensive wounds as well assigns of restraint. As a result, this critical piece of evidencedeserves further scrutiny.Thomas DiBiagios public comment that Luna was notin danger of losing his job is another diagnostic piece ofevidence because it is highly inconsistent with both suicidehypotheses and fairly inconsistent with a work-related

74 Chapter 7attack by an adversary. However, separate reporting citesinside sources saying that DiBiagio had lied about Lunaswork status to protect Lunas family. If, however, DiBiagiospublic and alleged private comments are removed from thematrix, the suicide hypotheses remain the mostinconsistent with the data. As a result, this piece ofevidence is not as crucial as initially thought, becausewhile DiBiagios comments are highly applicable to thesuicide hypotheses, they are not applicable to the other,more likely hypotheses.Another piece of highly diagnostic evidence is the FBIsstatement that Luna was alone all night. For the purposes ofthe ACH matrix, this evidence can be treated as anassumption. If it is assumed that this is true, it becomes acritical piece of evidence because it is highly inconsistent withall of the hypotheses except suicide. As a result, it is importantto track down the underlying evidence that would supportthis assumption. The FBI did not make this evidence public,so analysts should consider what indicators would raise orlower their confidence in the veracity of this assumption.Continue this process until all diagnostic evidence isreviewed.Step 8: Report the conclusions by considering the relativelikelihood of all the hypotheses.The sensitivity analysis reveals areas for further scrutiny,but in the absence of additional information, the tentativeconclusions about the relative likelihood of the hypotheseshold. However, any written analysis should include a fullaccounting of conflicting information, gaps, andassumptions upon which the analysis is based and what newinformation might change the likelihood of the hypotheses.Step 9: Identify indicators or milestones for futureobservation.The ACH process suggests that analysts should paycareful attention to new information that eithercorroborates or discredits Coronor Walps assessment, theFBIs assertion that Luna was alone, or information aboutblood from a second person in the car. These pieces ofinformation would differentiate further between the suicideand other hypotheses. Information about possible workrelated problems, adversaries, recent contacts, extramaritalactivities, and previous threats could serve as importantevidence that would discriminate between the lover andwork-related hypotheses. These pieces of information couldsignificantly affect the likelihood of the hypotheses and

should therefore be targeted as key areas for further

investigation in any future collection plan.Analytic Value Added: As a result of your analysis,what are the most and least likely hypotheses? Workrelated suicide and accidental suicide are the least likelyhypotheses. A random attack is as unlikely as accidentalsuicide. The hypotheses that are least inconsistent with therelevant information are the jealous lover and work relatedattack.What are the most diagnostic pieces of information?In addition to the diagnostic evidence discussed above,the alleged injuries to Lunas genitals, allegations that FBImishandled a key informant, the possibility that there wasblood of a second person in the car, and the fact that Lunawas killed by his own penknife are most diagnostic.What, if any, assumptions underlie the data? There is animplicit assumption that Walp and the FBIs public statementsare highly credible sources of information and that they aremore credible than the numerous law enforcement sourcescited in the press reports.Are there any gaps in the relevant information thatcould affect your confidence? Lack of information aboutthe coroners report, the basis for the FBIs assertion thatLuna was alone, any known Luna adversaries or extramarital relationships, and the details of his financial situationconstitutes important gaps that could affect overall confidence levels.How confident are you in your assessment of the mostlikely hypothesis? Given the extensive gaps and contradictions in the evidentiary base, any assessment should includea low overall confidence level. However, analysts shouldhave higher confidence that their analytic process has illuminated key areas for future research and collection.Why do you think that the case remains unsolved?While it is impossible to know with certainty why the caseremains unsolved, significant evidentiary gaps, anomalies,and uncertainties as captured in the public record mostlikely have played a role.KEY TAKEAWAYS Write it down! When contradictory evidence ispresent, it is essential to review key assumptions andthe reliability of all the data. Stand back and ask,Why? Consider a full range of hypotheses against all the

Who Murdered Jonathan Luna? 75

evidence and return to this analysis over time. There

could be several, intertwined explanations, or thehypothesis could change over time as more information

Decomposition and Visualization

Analysis of Competing Hypotheses

Hypothesis Generation and Testing

8 The Assassination of Benazir Bhutto

ome controversy still surrounds the question of who

was responsible for Benazir Bhuttos death. Manypeople had motives, and more than one person or groupcould easily have been plotting to kill her. Whenconfronting a case in which a significant amount ofinformation is unknown, the analyst should focus first ondevising and executing a solid analytic process that framesthe problem and brings order to the jumble of data points,assumptions, and gaps that form the case. In short, theanalyst should focus first on defining an analytic process atthe outset that will increase the chances that he or she willidentify and incorporate emerging information to solve thepuzzle in the future.The initial controversy surrounding this case as well as thedetailed information that is publicly available make the case aparticularly good vehicle for showing how analytictechniques such as Timelines, Chronologies, Mind Maps,Hypothesis Generation, and Analysis of CompetingHypotheses can help analysts systematically sort, array, andanalyze a dataset to bring a complex set of events into better,if not complete, focus. Lastly, as with all cases in whichhuman, technical, and press reporting are used, the casehighlights the importance of both sourcing and confidencelevels in analysis, particularly when dealing witheyewitnesses, secondhand reporting, and statements that maybe intended to obscure the truth or misguide the analyst.TECHNIQUE 1: CHRONOLOGIES AND TIMELINESChronologies and Timelines are simple but useful tools thathelp order events sequentially; display the informationgraphically; and identify possible gaps, anomalies, or

correlations. In addition, these techniques pull the analyst

out of the evidentiary weeds to view a data set from a morestrategic vantage point. The complex and contradictory dataregarding this case make an annotated Timeline particularlyuseful in identifying key pieces of evidence, confidencelevels in the reporting, and gaps in the information.Task 1.Create a Timeline of events surrounding Benazir Bhuttosdeath.Step 1: Label the relevant information from the casenarrative with the date and order in which it reportedlyoccurred. Consider how best to array the data along theTimeline. Can the information be organized by category?There are many ways to construct a Timeline for this casestudy. A complete Timeline of the case should go back to atleast 1977, when General Zia al Haq overthrew Zulfikar AliBhutto, began Islamicizing Pakistan, and started nurturingmilitant groups to advance the states perceived interests inAfghanistan and India and inside Pakistan. It should includeall of the events leading up to her assassination on 27December 2007 as well as all subsequent reporting thatfocused on the cause of death. It would include all referencesto key policy positions taken by Benazir Bhutto, her family,and her close associates as well as the statements andactivities of all her political rivals and enemies.For the purposes of this exercise, however, it is morepractical to confine the Timeline exercise to the day she waskilled and the information that surfaced subsequent to herdeath that shed light on how she died. A key objective increating the Timeline is to capture all the critical information

77

78 Chapter 8uncovered in the investigations. This new information shouldbe reflected on the Timeline at the time it allegedly occurred.In some cases, it might be preferable to include a separatecitation for when the information was reported. Doing so notonly helps an analyst see events as they unfolded but also tounderstand when information became available. This allowsanalysts to look for any anomalies in the pattern of thereporting that might support a deception hypothesis.The Timeline in Figure 8.2, excerpted from a longerTimeline of the case, illustrates how relevant informationcan be displayed along several parallel tracks illustratingfour dimensions of the event: Bhuttos activities, thegovernments actions and statements, the actions of theattackers and the Taliban, and the role of the media.Step 2: Review the Timeline by asking the followingquestions: Are there data gaps?The key issue that emerges from the Timeline is theapparent dispute over what actually caused Bhuttos death.The Timeline helps analysts sort through this issue byallowing them to compare known facts with the variousstatements of government officials and others cited by themedia. Most of the initial reporting stated that she diedof gunshot wounds. In subsequent days, the governmentdeclared that the actual cause of death was a head traumacaused by a major explosion that went off near Bhuttos SUV.Many have argued that the government was too quick toclean up the crime site and that a more methodical searchmight have revealed additional critical items of evidence.Some controversy also erupted over whether one or moreassassins were involved in the plot. The only reference toa second bomber was the speculation prompted by therelease of a grainy video that showed a man with a whitescarf standing just behind the purported gunman. No otherreference to this man appears in the case, and the ScotlandYard investigators contended that only one gunman wasinvolved, who detonated his explosive vest after firingseveral shots. In contrast, the intercepted communicationindicates that the purported perpetrators, the PakistaniTaliban, had intended to engage up to five assassins in theplot. Lastly, some would question the husbands decisionnot to demand an autopsy, expecting that a proper autopsycould have revealed more information. Do the duration and sequence of events suggested bythe data make sense?

Some might question whether the governments

seemingly premature statements were intended to coverup its failure to provide adequate security or, possibly, evensome connivance in the plot to kill Bhutto. Many cite thequick decision to hose down the crime scene as indicative ofpossible government complicity in the crime. Could any events outside the Timeline haveinfluenced the activities?Little is known about the activities and whereabouts ofseveral of the potential assailants, especially those tied to theTaliban or al-Qaeda. Should any underlying assumptions about theevidence be taken into consideration?The sources of information include eyewitnesses andconfidential sources. For the purposes of the Timeline, wehave segregated all the press reports as a separate streamof data. The government reporting also is presented as aseparate stream of data because of the potential for biasin how it would cover the event. Sometimes when thereare questions about the reliability of reporting or thereare anomalies in the reports, analytic comment can beannotated on the report or the reports can be set off by ashaded box.Analytic Value Added: What does the sequence ofevents tell you? The timeline helps the analyst distinguishbetween the various streams of information emanatingfrom press sources, the government, and family friends. Byisolating each stream of reporting, the analyst can betterevaluate each. The timeline also illuminates the discrepancybetween press reports that Bhutto died of a gunshot woundand subsequent government statements that the cause ofdeath was a head trauma resulting from a nearby explosion.In addition, it calls out key data points for furtherinvestigation, such as the exact sequence of events justbefore the blast and the various accounts of whattranspired.Are there any gaps in the information that should beaddressed? Several major gaps emerge, including the lackof information about the alleged attackers, confusion overwhether just one or several attackers were involved, theidentity or relevance of the man with a white scarf on thegrainy video of the crowd, and the failure to learn morefrom an autopsy.

80 Chapter 8What additional information should you seek? Keytopics to pursue would include information on any plotting prior to the incident, any indications of government orISID collusion with Baitullah Mehsud or other individualswho might target Bhutto, and any concrete evidence that thepolice were ordered to clean up the site prematurely.How confident are you in the sources of information?The timeline suggests that careful scrutiny should be givento press reporting and eyewitness reports. In addition, themotives of all reporting sources should be evaluated with aneye toward determining if there was intent to deceive investigators or the public.TECHNIQUE 2: MIND MAPSMind Maps are visual representations of how an individualor a group thinks about a topic of interest. A Mind Mapdiagram has two basic elements: the ideas that are judgedrelevant to whatever topic one is thinking about and thelines that show and briefly describe the connectionsbetween these ideas. Whenever you try to put a series ofthoughts together, that series of thoughts can be representedvisually with words or images connected by lines that represent the nature of the relationships between them. Anythinking for any purp ose, whether about a personaldecision or analysis of an intelligence issue, can bediagrammed in this manner. In fact, Mind Mapping wasoriginally developed as a fast and efficient way for studentsto take notes during briefings and lectures.In cases such as this, where initially there is little solidevidence and much speculation, it is particularlyimportant to cast the net wide to make sure that nothing isexcluded. This is especially so because the Pakistanigovernment immediately leaped to a conclusion, blamingthe so-called Pakistani Taliban operating in Pakistanstribal belt. Although the hypothesis offered by thePakistani government appears credible, the moreimportant question is whether it is the only hypothesisworth considering.

Task 2.Generate a Mind Map to explore who could have beenbehind Benazir Bhuttos assassination.Step 1: Identify the focal question or the logical startingpoint for an investigation. Write the focal question down inthe center of the page and draw a circle around it.

The focal question for this exercise is Who was behind

Benazir Bhuttos assassination? The question Who killedBenazir Bhutto? would be inappropriate because the keyquestion is who is the mastermind behind the killing, notwho specifically pulled the trigger or exploded the bomb.With one possible exceptiona lone-wolf scenariotheperpetrator(s) almost certainly was operating as an agent ofa higher power.Step 2: Brainstorm a list of possible explanations thatmight answer the focal question.Step 3: Sort these ideas into groupings. These groups maybe based on things they have in common or on their statusas either direct or indirect causes of the matter beinganalyzed.Step 4: Give each grouping a label and distribute theselabels around the focal question. Draw lines from the focalquestion to each label.Five groupings usually emerge in classroom discussions: The Pakistani government, including PresidentPervez Musharraf and senior officials in hisgovernment. Rival politicians. Islamic militants. Family members. Nation-states.Step 5: For each label, draw a line to an issue or conceptrelated to that label. A single label could have several spokesradiating from it, and each issue related to the label couldhave multiple spokes radiating from it as well.Step 6: Continue to expand the diagram until all aspectsof the issue or case have been captured.As shown in Figure 8.3, the Mind Map is easier to read ifdifferent shapes and colors or shadings are used to show thevarious levels of hierarchy. In this case, the focal question isrepresented by a circle, categories by boxes, and specificentities and individuals by ovals. Different colors orshadings are also used to distinguish entities such as nationstates or organizations from individuals.The focal question is presented in the circle as, Who wasbehind Bhuttos assassination? Five categories are depicted:Pakistani Government, Political Rivals, Nation-States,Family Members, and Islamic Militants. Each category has

(Husband)

Asif AliZardari

HindiNationalists

Family Members

Qari SaifullahAkhtar

IslamicMilitants

PakistaniTaliban

India*

*Denotes entities suspected based on little or no evidence.

Shaded ovals represent entities such as nation-states or organizations.

(Niece)

FatimaBhutto

LoneWolf

al-Qaeda

AitezazShah

BaitullahMehsud

Figure 8.3 Mind Map of Who Was Behind Bhuttos Assassination

United States*

Nation-States

Who was behind

Bhuttosassassination?

(President)

PervezMusharraf

RogueElements

(former ISID)

Ijaz Shah

(IntelligenceBureau Chief)

China*

Senior Officials

(Minister ofReligious Affairs)

Ejaj ul-Haq

Imran Khan

(former cricketer/politician)

Political Rivals

PakistaniGovernment

Intel Services

Gen.Hamid Gul

(former PrimeMinister)

Nawaz Sharif

(former ChiefMinister ofPunjab)

ChaudhryPervez Elahi*

(former ChiefMinister ofSindh)

ArbabGhulam Rahim*

(Pakistani MuslimLeague)

ChaudhryHussein

The Assassination of Benazir Bhutto 81

82 Chapter 8several entities and/or individuals associated with it. Forexample, two of Bhuttos relatives (her niece and husband)are connected to the Family Members category. ThePakistani government category is more complex, with oneindividual (President Musharraf ) linked to it as well as twoentitiesIntelligence Services and Senior Officials. Each ofthese entities has several names associated with it, whichcan be extracted from the case study.Step 7: While building the Mind Map, consider thepossibility of cross-links from one issue to another. Showdirectionality with arrows pointing in one or bothdirections.Several connections may be worth noting on the MindMap, especially the link between President Musharraf andthe Pakistani Taliban headed by Mehsud. The link betweenPakistani Intelligence Chief Hamid Gul and the Taliban isalso worth noting. These connections suggest that Mehsudcould have acted either alone or with the support of thePakistani government. Mehsuds links to al-Qaeda shouldbe depicted as well, suggesting that this link could provideanother reason for suspecting Mehsud. Lastly, AitezazShahs reported links to the Pakistani Taliban require notingand possible further discussion.Step 8: While building the Mind Map, consider thepossibility of conflicting evidence or conflicting concepts. Ifthey appear, label them differently by color, written name,or shape, or by putting an asterisk or other icon inside thecircle or box.In this case, it would be useful to color code linkages orhypotheses that could have been surfaced based on weakdata or information that may have been provided withintent to deceive. Benazir Bhuttos message accusing fourcurrent and former Pakistani officials of having motive tokill her is not substantiated by any other information inthe case. Similarly, a case can be made for nation-statessuch as India, China, or the United States being possiblesuspects given histories of past tensions, but suchallegations are not substantiated by any informationpresented in the case study. It is a good idea to includesuch potential suspects in the Mind Map in order togenerate a comprehensive list of suspects, but it is alsohelpful to indicate with color coding or an icon that theevidence supporting these suspects is weak.Step 9: Reposition, refine, and expand the Mind Mapstructure as appropriate.

Once you have completed the Mind Map, take a final

look to consider whether all the boxes and circles arearranged in the most effective way. For example, boxesconnected by dotted lines should be in close proximity toeach other. Sometimes, it is important to show the mostimportant categories at the top of the Mind Map, where thereaders attention is most likely to focus first. In this MindMap, both objectives were achieved by putting IslamicMilitants and Pakistani Government at the top of the MindMap.Once the Mind Map is completed, the next task is toreview all the options that have been generated and developa list of alternative answers to the question, Who was behindthe assassination of Benazir Bhutto? This is most efficientlyaccomplished by creating a table listing each branch of theMind Map and assigning a motive to that person or group.Step 10: List all the individuals or entities who may bebehind the assassination as well as their most likelymotivations.See Table 8.2 for a list of potential masterminds and theirmotives. As a result of the Mind Map exercise, twenty-oneindividuals or groups have been identified.Step 11: Identify the most likely people or entities thatwould have wanted to kill Benazir Bhutto.Review the list of potential masterminds and select thosewith the strongest motives and the capability to orchestrateher assassination. A candidate list of five suspects providedin Table 8.3 includes the following: Pakistani Taliban leader Baitullah Mehsud, whoallegedly authored the incriminating interceptedmessage praising one of his operatives for asuccessful attack. Pakistani President Pervez Musharraf, who couldhave viewed Bhuttos return and popularity as athreat to his regime. Former Prime Minister Nawaz Sharif, who was oneof Bhuttos primary political challengers. Rogue elements of the ISID, who could have decidedto take it upon themselves to remove a potentialchallenge to how they ran their business and howthey related to other Islamic militant groups. Bhuttos niece, Fatima Bhutto, who held BenazirBhutto responsible for her fathers death and calledBhutto the most dangerous thing to happen toPakistan.

The Assassination of Benazir Bhutto 83

Table 8.2 List of Potential Masterminds and

Motives for the Bhutto AssassinationIndividual or Entity

Possible Motive

Table 8.3 List of Most Likely Masterminds of the

Bhutto AssassinationMost Likely Candidates

Possible Motive

Pakistani President Pervez

Musharraf

Bhutto was a political rival

who threatened his rule.

Pakistani Taliban leader

Baitullah Mehsud

Saw Bhutto as too pro-American,

too secular, and anti-Taliban.

Rogue elements of the ISID

Bhuttos return to power would

threaten their power andpositions.

Pakistani President Pervez

Musharraf

Bhutto was a political rival who

threatened his rule.

Former ISID Chief Hamid Gul

Bhutto believed he was

plotting to kill her.

Former Prime Minister

Nawaz Sharif

Bhutto was competing with him in

the upcoming election.

Intelligence Bureau Chief Ijaz

Shah

Bhutto believed he was

plotting to kill her.

Rogue elements of the ISID

Bhuttos return to power would

threaten their power and positions.

Minister of Religious Affairs

Ejaj ul-Haq

Saw Bhuttos return as

unnecessarily destabilizingPakistan.

Fatima Bhutto (Bhuttos

niece)

Fatima holds Bhutto responsible

for her fathers death.

Pakistani Muslim League

leader Chaudhry Hussein

Strongly opposed any

compromise with Bhutto.

Islamic militant lone wolf

She was viewed as too secular and

female; an unacceptable Muslim.

Former Chief Minister of Sindh

Arbab Ghulam Rahim

Bhutto believed he was

plotting to kill her.

al-Qaeda

Former Chief Minister of

Punjab Chaudhry Pervez Elahi

Bhutto believed he was

plotting to kill her.

She was viewed as too secular and

too pro-American.

Qari Saifullah Akhtar

Former Prime Minister Nawaz

Sharif

Bhutto was competing with

him in the upcoming election.

Attempted a coup against her

previously; suspect in Octoberbombing.

Former politician Imran Khan

Had lambasted Bhutto in the

press as a kleptocrat.

Former ISID Chief Hamid Gul

Bhutto believed he was plotting

to kill her.

China

A Bhutto government could

lead to a less-stable border andless-reliable partner.

Intelligence Bureau Chief

Ijaz Shah

Bhutto believed he was plotting

to kill her.

Mininster of Religious Affairs

Ejaj ul-Haq

Saw Bhuttos return as

unnecessarily destabilizingPakistan.

Pakistani Muslim League

leader Chaudhry Hussein

Strongly opposed any compromise

with Bhutto.

Former Chief Minister of

Sindh Arbab Ghulam Rahim

Bhutto believed he was plotting

to kill her.

Former Chief Minister of

Punjab Chaudhry Pervez Elahi

Bhutto believed he was plotting

to kill her.

Former politician Imran Khan

Had lambasted Bhutto in the press

as a kleptocrat.

Hindu Nationalist extremists

Her return posed a threat to all

Hindus and to India.

United States

India

She was viewed as too antiAmerican or an unreliable

future ally.The return of a Bhuttogovernment would resurfaceold tensions.

Less Likely Candidates

Possible Motive

Hindu Nationalist Extremists

Her return posed a threat to all

Hindus and to India.

Asif Ali Zardari (Bhuttos

husband)

Her death could open political

doors and protect him fromcorruption charges.

Fatima Bhutto (Bhuttos niece)

Fatima holds Bhutto responsible

for her fathers death.

Qari Saifullah Akhtar

Attempted a coup against her

previously; suspect in Octoberbombing.

Asif Ali Zardari (Bhuttos

husband)

Her death could open political

doors and protect him fromcorruption charges.

Islamic militant lone wolf

She was viewed as too secular

and female; an unacceptableMuslim.

India

The return of a Bhutto government

would resurface old tensions.

China

al-Qaeda

She was viewed as too secular

and too pro-American.

A Bhutto government could lead

to a less-stable border and lessreliable ally.

Pakistani Taliban leader

Baitullah Mehsud

Saw Bhutto as too proAmerican, too secular, and

anti-Taliban.

United States

She was viewed as too antiAmerican or an unreliable future

ally.

84 Chapter 8Analytic Value Added: Does the creation of the MindMap prompt you to consider a much broader array ofpotential explanations or hypotheses? The act of drawingthe Mind Map prompts analysts to think about a largerrange of alternatives at the outset of a project. For example,once the analyst decides to list Fatima Bhutto as a potentialmastermind, the question that immediately comes to mindis whether other family members, such as the husband,should be added to the Mind Map. The Mind Map approachalso makes it easier to array a large number of alternativesin a simple display that is easy to embellish and refine.Does it help you drill down for each hypothesis toconsider second- and third-level questions? In this exercise, the Mind Map approach prompts the analyst to consider possible linkages between the groups and individualsdepicted and to come up with the names of specific peoplewho could have been the mastermind behind the operation.In considering the Islamic Militants category, for example,creating the Mind Map prompts one to explore several questions such as these: Which key Pakistani militant groups, such as theHarkat-ul-Jihad-al-Islami (HUJI), deserve attention,apart from the Pakistani Taliban? How are these various actors linked? Would they combine forces in an attempt toassassinate Bhutto? Did they have the capability to launch the attack thatkilled Bhutto?Does it help you identify potential gaps in knowledge? The Mind Map approach not only reveals key gaps inknowledge but helps open the door to considering the possibility that several entities might simultaneously have beenattempting to kill Bhutto and that more than one plot mayhave been playing out at the time of her death.

TECHNIQUE 3: ANALYSIS OFCOMPETING HYPOTHESESAnalysts face a perennial challenge of working withincomplete, ambiguous, anomalous, and sometimesdeceptive data. In addition, strict time constraints and theneed to make a call often conspire with a number ofnatural human cognitive tendencies to result in inaccurateor incomplete judgments. Analysis of CompetingHypotheses (ACH) improves the analysts chances of

overcoming these challenges by requiring the analyst to

identify and refute possible hypotheses using the full rangeof data, assumptions, and gaps that are pertinent to theproblem at hand.Task 3.Use the most credible hypotheses compiled with the MindMap or other hypothesis generation techniques to conductan Analysis of Competing Hypotheses of the Bhutto case.Contact Globalytica, LLC at THINKSuite@globalytica.com or go to http://www.globalytica.com to obtain accessto the basic software, or the collaborative version calledTe@mACH, if it is not available on your system.Step 1: List the hypotheses to be considered, striving formutual exclusivity.The Mind Map technique can provide a useful startingpoint for generating a set of hypotheses. In the Mind Map,almost twenty groups or individuals were identified assuspects who may have given the order to have BenazirBhutto killed. Lead the class in a discussion of all thepossible motives for each entity and then choose thosehypotheses that appear to be the most compelling andworthy of serious consideration. In this case study, the leadhypotheses that usually emerge are as follows: The Pakistani government (to include PresidentMusharraf and other senior officials). The Pakistani Taliban (to include its leader, BaitullahMehsud). Political rivals (specifically Nawaz Sharif, Bhuttoschief rival on the campaign trail). Rogue elements of ISID (who may not be acting onthe specific orders of their leaders).In class exercises, it usually is effective to include atleast one other, less compelling hypothesis, such as one ofBhuttos family members, in order to illustrate the powerof the ACH tool. Including a less likely suspect usually willresult in generating a large number of inconsistent scores forthat hypothesis, thereby showing how ACH illuminates theweakness of a poorly substantiated hypothesis.Step 2: Make a list of all relevant information, includingsignificant evidence, arguments, gaps, and assumptions.Step 3: Assess the relevant information against eachhypothesis by asking, Is this information highly inconsistent,

The Assassination of Benazir Bhutto 85

inconsistent, neutral, not applicable, consistent, or highly

consistent vis--vis the hypothesis? The Te@mACH software does not include the neutral category.Step 4: Rate the credibility of each item of relevantinformation.Figure 8.4 provides a partial list of fifty items of relevantinformation culled from the case study that could be helpfulin conducting an ACH. Each of the items was assessed on a5-point scale as Highly Consistent, Consistent, Inconsistent,Highly Inconsistent, or Not Applicable for each of the fivecandidate hypotheses.In reviewing the completed matrix, it is noteworthy thatalmost half of the items of relevant information have littlediagnostic value: they were rated as consistent or notapplicable for all five hypotheses. Five, however, emerged as

highly diagnostic because they were consistent with one

hypothesis and inconsistent or highly inconsistent with theother four hypotheses. Two of the five items of relevantinformation were deemed highly diagnostic primarilybecause it was assumed that the other masterminds wouldbe unlikely to utilize a suicide bomber to kill Bhutto. Aword of caution is appropriate in that all but one of themost diagnostic items of evidence were rated as havingmedium credibility. For example, the intercept wasdeemed highly diagnostic but should not overly influencethe analysis until the authenticity of the intercept can beestablished.Step 5: Refine the matrix by reconsidering the hypotheses.Does it make sense to combine two hypotheses, add a newhypothesis, or disaggregate an existing one?

Figure 8.4 Bhutto Analysis of Competing Hypotheses Sample Matrix

86 Chapter 8The current set of five hypotheses are sufficiently distinctfrom each other to argue against combining any into asingle hypothesis. Given the strength of the Talibanhypothesis, thought should be given to exploring whetherother hypotheses from the Islamic Militants category shouldbe considered, such as a lone wolf, HUJI, or an al-Qaedaoperative.Step 6: Draw tentative conclusions about the relativelikelihood of each hypothesis. An inconsistency score willbe calculated by the software; the hypothesis with thelowest inconsistency score is tentatively the most likelyhypothesis. The one with the most inconsistencies is theleast likely.The two hypotheses with the highest inconsistencyscores are Rogue ISID elements and Musharraf and hisgovernment. Some of the most compelling arguments fordiscarding these hypotheses are the fact that a suicidebomber was employed, the government had providedheavy security, Bhutto had stopped short of attackingMusharraf directly, and up to this point most of thesuicide bombings had been targeted at the ISID and themilitary. The primary reason for dismissing PoliticalRival Sharif and Bhuttos Niece Fatima is the findingthat Bhutto was killed by a suicide bombing, not bulletsfrom a gun. Neither Sharif nor Fatima are likely candidatesto have used a suicide bomber.Step 7: Analyze the sensitivity of your tentative conclusionto a change in the interpretation of a few critical items ofevidence by using the software to sort the evidence bydiagnosticity.The analysis would change dramatically if it weredetermined that the intercepted communication or theteenagers confession was not authentic or if new evidenceemerged that one of the other suspects was involved in aplot to assassinate Bhutto that day. Also of concern wouldbe a finding that the Scotland Yard report included thecaveat that restrictions placed on its investigation by thePakistani government may have precluded it fromconducting a thorough inquiry.Step 8: Report the conclusions by considering the relativelikelihood of all the hypotheses.The ACH software automatically moves the hypothesisor hypotheses that are the most credible to the left side ofthe matrix. The least likely hypothesis will appear on thefar right. The most credible hypotheses are those with the

fewest items of relevant information that are inconsistent

with that hypothesis. Hypotheses with a large number ofinconsistent items of relevant information that appearcompelling can be discarded, unless some of the items ofinformation are later found to be deceptive or inaccurate.In this case study, Taliban leader Mehsud appears asthe most likely mastermind behind the assassination ofBenazir Bhutto. Only six items of relevant information werenoted as being inconsistent with this hypothesis, and threeof those were given a credibility rating of low. Forexample, former ISID Chief Guls complaint that authoritieshosed down the crime scene could be interpreted as selfserving and an attempt to make the Taliban look innocent.Of more concern is the fact that Scotland Yard concludedthere was only one attacker and no other suspiciousindividuals in the crowd. This seems to contradict what wassaid in the purported intercepted communication in whichMehsud was told that three men were involved in theassassination. One possibility is that three men wereinvolved in the planning but only one suicide bomber wassent to the rally.Step 9: Identify indicators or milestones for futureobservation.The case for proving that Mehsud was the mastermind ofthe Bhutto assassination would be strengthened if additionalinformation surfaced over the course of the investigationshowing the following: Detailed planning by the Taliban to use a suicidebomber to kill Bhutto. Evidence that Mehsud or the Taliban were planningan attack on 27 December. More convincing evidence linking Mehsud to theteenager. Evidence that Musharraf or ISID was committedto protecting Bhutto and making an extra effort toensure her safety.

Analytic Value Added: As a result of your analysis,

what are the most and least likely hypotheses? Based onthe ACH analysis, the most credible hypothesis is thatMehsud was the mastermind behind the assassination ofBenazir Bhutto. All the other hypotheses had a significantlylarger number of inconsistent items of relevant information,making them much less likely. Although Mehsud emerges

The Assassination of Benazir Bhutto 87

as the most likely suspect, a case can be made that he

represents a family of likely suspectsIslamic militantsand that other individuals and groups in this category alsomerit close scrutiny. This would suggest that a second ACHexercise be conducted to apply the evidence to al-Qaeda,Qari Suifullah Akhtar, and a possible lone-wolf Muslimextremist.The hypotheses Musharraf and his government andRogue ISID elements both had a large number of inconsistencies, making them the least likely hypotheses. In theMind Map exercise, however, historical links were citedconnecting the intelligence services and the Taliban leadership. While the ACH methodology makes a strong case todismiss the theory of Pakistani officials orchestrating a suicide bombing to eliminate Bhutto, the case to dismiss themas suspects becomes weaker if an argument is made thatPakistani officials were either colluding with or encouragingIslamic extremists to kill Bhutto.What are the most diagnostic pieces of information?The most diagnostic evidence is the intercepted communication and subsequent arrest of the teenager who claimedto be part of a group tasked with assassinating Bhutto. Themost compelling logic for discounting the other hypotheses was the use of a suicide bomb; other suspects wouldhave lacked the capability to recruit a suicide bomberand almost certainly would have opted to use a sniper orgunman.What, if any, assumptions underlie the data? The mostimportant assumption was that only Islamic militants wouldresort to using a suicide bomber to kill Bhutto. Another keyassumption is that only one assassination scenario was inplay. Bhutto was regarded as a serious threat by a wide arrayof actors, and it is possible more than one was trying to killher on that day.Are there any gaps in the relevant information thatcould affect your confidence? How confident are you inyour assessment of the most likely hypothesis? The keygap is not knowing if the intercepted communication andthe statements made by the teenager are authentic. Anothergap is whether more than one attacker was present in thecrowd at the time of the bombing.CONCLUSION: THE UN REPORTContinued interest in the assassination of Benazir Bhuttoled the Pakistani government and the United NationsSecurity Council to ask the UN Secretary-General toappoint a Commission of Inquiry to look into the events

surrounding the killing and its aftermath. The threemember commission conducted more than 250 interviewsin Pakistan with government officials and private citizenswho had knowledge of the assassination. The commissionsinvestigative team also examined the Scotland Yard reportand reviewed hundreds of documents, photographs, andother documentary material provided by Pakistani andBritish officials. Following are some of the key findings ofthe report, published on 30 March 2010:Ms. Bhuttos assassination could have been prevented ifadequate security measures had been taken....Thefederal government under General Musharraf...[was]not proactive in neutralizing [threats] and/or ensuringthat the security provided was commensurate to thosethreats.1She died when a 15 and a half year-old suicide bomberdetonated his explosives near her vehicle, [but] no onebelieves that this boy acted alone.2Ms. Naheed Khan recalled that immediately after shehad heard the three gunshots, Ms. Bhutto fell down intothe vehicle onto her lap. Ms. Khan said that she felt theimpact of the explosion immediately thereafter....Ms.Khan saw that Ms. Bhutto was not moving and saw thatblood was also trickling from the ear.3Five persons were arrested by [Pakistani officials]:Aitezaz Shah, Sher Zehman, Husnain Gul, MohamadRafaqat, and Rasheed Ahmed. In addition, [Pakistaniofficials] charged Nasrullah, Abdullah, BaitullahMe h s u d , a n d Mau l v i S a h i b a s pro c l a i m e doffenders....The accused are alleged to have served ashandlers and logistics supporters of the suicide bomber,or as persons who were knowledgeable about the plansto assassinate Ms. Bhutto.4The investigation into Ms. Bhuttos assassination, andthose who died with her, lacked direction, wasineffective, and suffered from a lack of commitment toidentify and bring all of the perpetrators to justice.5The [Joint Investigation Team]...did nothing to build acase against Mr. Mehsud, treating the contents of theintercept presented to the public by Brigadier Cheema asdeterminative of his culpability. AIG Majeed told theCommission that he saw no need to establish theauthenticity of the intercept or the basis for its analysis,including the voice identification and the interpretationof the conversation as a reference to Ms. Bhuttosassassination.6

88 Chapter 8The UN report shed light on several key aspects of theinvestigation. It noted that no blood or tissue was foundon the trucks escape hatch lever, drawing into questionwhether Bhutto had hit her head on the lever when she fellinto the cab.7 The report also dismissed reports that doctorshad deliberately altered their initial findings that Bhutto hadsuffered gunshot injuries. More significant, the commissionsaid it had not found any credible, new information showingthat Bhutto had received bullet wounds.8The report noted that numerous people may have wishedBhutto harm, including local jihadi groups, the PakistanTaliban, al-Qaeda, and members of the Pakistanigovernment and political elite.9 After the Karachi attack,Bhuttos attorney said that he had received a handwrittenletter from someone claiming to be the head of suicidebombers and a friend of al-Qaeda who threatened toassassinate Bhutto in a gruesome manner. An al-Qaedaspokesperson, Mustafa Abu al Yazid, had also claimedresponsibility for her assassination in an interview with theAsia Times Online.10According to the UN report, many senior Pakistaniofficials believed Baitullah Mehsud was part of a largerconspiracy to assassinate Bhutto, but the report observesthat many of these same officials would have had a motiveto eliminate Bhutto because they were threatened by thepossibility of her regaining power.11,12 The true story ofMehsuds involvement may never be known because he waskilled in a drone attack in August 2009.13The commission took the police to task for focusing theinvestigation on lower-level operatives and not exploringwhether any higher-level officials may have been involvedin the planning, financing, or execution of theassassination.14 It attributed police reluctance in part to aconcern that Pakistani intelligence services may have had arole in the assassination.15

KEY TAKEAWAYS The tendency to plunge in should always be temperedby a process designed to identify all the relevantinformation and evaluate all possible explanations. Chronologies and Timelines are invariably some ofthe best ways to begin an analysis; they not only helpthe analyst organize the data but can reveal key gaps,inconsistencies, and correlations in the data. Employing a more systematic process, such as aMind Map, at the start of the investigation helpsframe the issue. It also helps analysts identify a morecomprehensive set of hypotheses early on. Consider a full range of hypotheses against all therelevant information and return to this analysisover time. There could be several, intertwinedexplanations, or the hypotheses could changeover time as more information comes to light. Beprepared to evaluate each piece of new informationagainst all the possibilities.

Hypothesis Generation and Testing

Analysis of Competing Hypotheses

Hypothesis Generation and Testing

9 Death in the Southwest

his case study puts students in the shoes of Centers for

Disease Control (CDC) investigators and local medicalauthorities who are under extreme pressure to determinewhy seemingly healthy people are suddenly dying. Althoughthe instructional materials provide a detailed conclusionoutlining how the case was actually resolved, much of thisinformation was excluded from the narrative to give thestudents a better appreciation of how often analysts mustmake difficult judgments with relatively little solid data inhand. The Structured Brainstorming exercise is designed toprompt the students to consider all possible alternatives atthe outset of a case, no matter how unrealistic they mightappear at the time. The Starbursting exercise helps themtransition from a divergent mode of analysis to a convergentmode by organizing and structuring the results of theirbrainstorming. The Multiple Hypotheses Generator TMprovides a more systematic way to generate alternativehypotheses. Of the three techniques, the MultipleHypotheses GeneratorTM probably does the best job ofensuring that the alternative hypotheses are mutuallyexclusive.After reading the narrative, students usually are quick toarticulate what they think is the most likely solution. TheKey Assumptions Check and the Analysis of CompetingHypotheses (ACH) both prompt the analyst to subject theirviews to more critical scrutiny. The Key Assumptions Checkforces the analysts explicitly to list their assumptions, someof which almost always turn out to be unfounded. Analysisof Competing Hypotheses requires analysts to consider anarray of possible alternative hypotheses and thensystematically evaluate which is the most likely based onwhether the relevant information presented in the narrativeis consistent or inconsistent with each hypothesis.

TECHNIQUE 1: STRUCTURED BRAINSTORMING

Brainstorming is a group process that follows specific rulesand procedures designed to generate new ideas and concepts.The stimulus for creativity comes from two or more analystsbouncing ideas off each other. A brainstorming session usuallyexposes an analyst to a greater range of ideas and perspectivesthan the analyst could generate alone, and this broadening ofviews typically results in a better analytic product.Structured Brainstorming is a systematic twelve-stepprocess (described following) for conducting groupbrainstorming. It requires a facilitator, in part becauseparticipants are not allowed to talk during thebrainstorming session. Structured Brainstorming is mostoften used to identify key drivers or all the forces andfactors that may come into play in a given situation.Task 1.Conduct a Structured Brainstorming exercise to explorewhy a healthy young Navajo couple died suddenly.Step 1: Gather a group of analysts with some knowledgeof medicine and the Four Corners region.It is helpful to include in the brainstorming group bothexperts on the topic and generalists who can provide morediverse perspectives. When only those directly involvedwith the issue are included, often the group tends to focuson the most current information gathered or the mostreadily available data; as a result, key assumptions remainunchallenged, and historical analogies can be ignored. Inthis case, having someone who understands Navajo cultureand is familiar with both basic medical practice and theFour Corners area would be a major benefit.

89

90 Chapter 9

Box 9.1 EIGHT RULES FOR SUCCESSFUL

BRAINSTORMING1. Be specific about the purpose and the topic of thebrainstorming session.2. Never criticize an idea, no matter how weird, unconventional,or improbable it might sound. Instead, try to figure out howthe idea might be applied to the task at hand.3. Allow only one conversation at a time and ensure thateveryone has an opportunity to speak.4. Allocate enough time to complete the brainstorming session.5. Engage all participants in the discussion; sometimes this mightrequire silent brainstorming techniques such as askingeveryone to be quiet for five minutes and write down their keyideas on 3 5 cards and then discussing what everyone wrotedown on their cards.6. Try to include one or more outsiders in the group to avoidgroupthink and stimulate divergent thinking. Recruit astutethinkers who do not share the same body of knowledge orperspective as other group members but have somefamiliarity with the topic.7. Write it down! Track the discussion by using a whiteboard, aneasel, or sticky notes.8. Summarize key findings at the end of the session. Ask theparticipants to write down their key takeaway or the mostimportant thing they learned on a 3 5 card as they departthe session. Then, prepare a short summary and distribute thelist to the participants (who may add items to the list) and toothers interested in the topic (including those who could notattend).

Step 2: Pass out sticky notes and marker-type pens or

markers to all participants. Inform the team that there is notalking during the sticky notes portion of the brainstormingexercise.Use different color sticky notes and encourage theparticipants to write down short phrases consisting of threeto five words, not long sentences.Step 3: Present the team with the following question:What are all the forces and factors that might explain why ayoung Navajo couple died suddenly?Keep the question as general as possible so as not toinadvertently restrict the creative brainstorming process. Italso helps to ask the group if they understand the questionand whether they believe it should be worded differently.Spending a few minutes to ensure that ever yoneunderstands what the question means is always a good

investment. Students should have the case study at hand for

quick reference.Step 4: Ask the group to write down responses to thequestion with a few key words that will fit on a sticky note.After a response is written down, the participant gives it tothe facilitator, who then reads it aloud. Marker-type orfelt-tip pens are used so that people can easily see what iswritten on the sticky notes later in the exercise.Go around the room and collect the sticky notes. Givethe students a few minutes to think about the issue and jotdown a few ideas before you start reading out theresponses. Read the responses slowly and stick them onthe wall or the whiteboard in random order as you readthem. Some sample sticky notes might read or addresstopics such as these: Is the disease contagious? Who else isgetting sick? Have these symptoms been observedpreviously? Did the couple engage in patterns of activitythat are common to other victims? Did the couple andother known victims visit the same location? Are therereports of toxic chemical dumps in the region? Arefarmers using any new herbicides or other newlyintroduced chemicals? Did terrorists do it? Was it a hatecrime? Who might want this to happen?Step 5: Place all the sticky notes on a wall randomly asthey are called out. Treat all ideas the same. Encourageparticipants to build on one anothers ideas.Step 6: Usually an initial spurt of ideas is followed bypauses as participants contemplate the question. After fiveor ten minutes there is often a long pause of a minute or so.This slowing down suggests that the group has emptied thebarrel of the obvious and is now on the verge of coming upwith some fresh insights and ideas. Do not talk during thispause, even if the silence is uncomfortable.Remind the group not to talk during this part of theexercise. It is important for them to hear what others aresuggesting, as this might stimulate new ideas for them to jotdown. Also take care not to talk too much yourself. Theparticipants need quiet time to think, and it is veryimportant for the instructor not to interrupt their thoughtprocesses. Often when it is the quietest, the best thinking istaking place.Step 7: After two or three long pauses, conclude thisdivergent thinking phase of the brainstorming session.

Death in the Southwest 91

Step 8: Ask all participants (or a small group) to go up to

the wall and rearrange the sticky notes by affinity groups(groups that have some common characteristics). Somesticky notes may be moved several times, and some may becopied if the idea applies to more than one affinity group.If only a subset of the group goes to the wall to rearrangethe sticky notes, then ask those who are remaining in theirseats to form into small groups and come up with a list ofkey dimensions of the problem or key areas for moreresearch based on the themes they heard emerge when theinstructor was reading out the sticky notes. This keepseveryone busy and provides a useful check on what isgenerated by those working at the wall.Step 9: When all sticky notes have been arranged, ask thegroup to select a word or phrase that best describes eachgrouping.Four or five themes usually emerge from this part of theexercise. Exposure. The couple (and other victims) cameinto contact with a toxic substance that causedtheir illness. Exposure could have been accidentalor intentional, a one-time occurrence or overa prolonged period of time. For example, thevictims may have worked at Fort Wingate andbeen exposed to a lethal chemical or biologicalsubstance. Identity. The couple became ill because they wereNavajos, belonged to a particular tribal group, livedon a particular compound, or were members orassociates of a criminal gang. Victims. The two young Navajos were victims ofa plot launched by international terrorists, whitesupremacists, or some other extremist group. Theymight have been targeted personally or simply beenat the wrong place at the wrong time. Natural causes. The couple succumbed to a naturallyoccurring pathogen or virus that was particularlylethal. A visitor might have recently brought thepathogen to the area from some other part of theworld, or something in the local environment mighthave caused it to surface.Step 10: Look for sticky notes that do not fit neatly intoany of the groups. Consider whether such an outlier is

useless noise or the germ of an idea that deserves further

attention.Often one or two outlier sticky notes are worth pointingout to the class because they provide a fresh perspective orsuggest a potentially valuable new line of inquiry. Here aresome examples: A sticky note that said Fort Wingate could prompta robust discussion of ways that Fort Wingate couldbe relevant. Were biological or chemical weaponsbeing built or stored at the fort? Were there anyknown toxic waste sites at the fort? Did the coupleor their associates work at the fort? Were any knownwhite supremacist groups active at the fort? If so,did they have a website? Did it contain informationcritical of the Navajo Nation? A sticky note that said rats could promptquestions such as, What types of rats wereindigenous to Four Corners? What types of diseaseswere such rats known to carry? How do diseasesget transmitted from rats to humans? Under whatconditions do rats pose a greater threat to thehuman population?Step 11: Assess what the group has accomplished. Canyou identify four or five key factors or forces that mightexplain why the young Navajo couple died?Work with the group to develop a consensus on three orfour themes that emerge as the most important dimensionsof this problem or potential explanations for why the coupledied. Write the candidate explanations on the board. Thethemes that most often are generated by this stage of theexercise are the following: Exposure to a toxic substance. The couple came intocontact with a toxic chemical or biological substancein their surroundings that made them ill. Natural causes. The couple was exposed to a newpathogen that had recently manifested itself in theirenvironment, or they died of a particularly virulenttype of flu. Victims of an attack. Terrorists or domesticextremists introduced a particularly virulentbiological substance into the environment withthe intent to terrorize the population, to causedeaths among Navajos, or to draw attention to FortWingate.

92 Chapter 9Step 12: Present the results, describing the key themes ordimensions of the problem that deserve investigation.The group should end up with a set of three to fivehypotheses that best explain why the young Navajo coupledied suddenly. At this stage of the exercise, the hypothesescan be fairly general so as not to rule out a viable alternative.Some sample hypotheses include these: The couple came in contact with a highly toxicchemical or biological substance. The two young Navajos were the victims of adeliberate hate crime targeting the Navajo Nation. The two young Navajos were collateral damage in aterrorist plot that for the first time involved the use ofbiological weapons. The couple succumbed to a particularly virulent,naturally occurring pathogen. The two young people had other health problemsthat made them more susceptible to the commonflu.Analytic Value Added: Did we explore all thepossible forces and factors that could explain why theyoung Navajo couple died? Did our ideas groupthemselves into coherent affinity groups? StructuredBrainstorming is a powerful tool for generating a diversenumber of ideas; it taps the expertise and past experiencesof everyone in the group and gives them equal opportunityto provide their input. The requirement to place all theideas into affinity groups forces the group to criticallyexamine the underlying forces and factors that might havecaused the deaths while avoiding the cognitive trap ofsatisficing, wherein one generates a short list of readyanswers to the question without any underlying rigor tothe process.The silent, structured brainstorming approach is a powerful technique to pull out new and often never previouslyconsidered ideas and concepts. It avoids the trap of deferring to the most knowledgeable person in the room bygiving all participants an equal, but silent, opportunity tosurface their ideas.Did our ideas group themselves into coherent affinity groups? How did we treat outliersthat is, the stickynotes that seemed to belong in a group all by themselves?Did the outliers spark new lines of inquiry? Did thelabels we generated for each group accurately capturethe essence of that set of sticky notes? While conducting

the structured brainstorming exercise, it is useful to note

whether particularly useful and creative ideas are generatedafter long pauses when everyone is thinking; if this doesoccur, it is important to alert the entire group to the phenomenon. Placing like ideas into affinity groups can be achallenging task; asking those not at the whiteboard to comeup with their own categories often provides a useful sanitycheck. Always be careful to give outlier ideas their due attention; they often will point to new lines of inquiry or dimensions not previously considered.

TECHNIQUE 2: STARBURSTINGStarbursting is a form of structured brainstorming thathelps analysts generate as many questions as possible. It isparticularly useful in developing a research project, but itcan also help to elicit many questions and ideas to challengeconventional wisdom. This process allows the analyst toconsider the issue at hand from many different perspectives,thereby increasing the chances that the analyst will uncovera heretofore unconsidered question or idea that will yieldnew analytic insights.Task 2.Construct a Starbursting diagram to explore the Who?What? How? When? Where? and Why? questions relatingto the untimely death of a healthy young Navajo couple.Step 1: Use the template in Figure 9.1 in the book or drawa six-pointed star and write one of the following words ateach point of the star: Who? What? How? When? Where?and Why?Step 2: Start the brainstorming session, using one of thewords at a time to generate questions about the topic. Donot try to answer the questions during the brainstormingsession; just focus on generating as many questions aspossible.Students should be able to develop at least two to fourquestions per point in the star, as reflected in exampleFigure 9.2.Step 3: After generating questions that start with each ofthe six words, the group should either prioritize thequestions to be answered or sort the questions into logicalcategories.

Death in the Southwest 93

Figure 9.2 Death in the Southwest Starbursting Example

Why would someone want to killl

Navajos? Was it an act of nature or adeliberate decision to kill them? If a new disease, why would itsuddenly manifest itself?

WHO?

Did White Supremacists kill them?

Are international terrorists to blame? Could it have been a criminal group or a gagang?

WH

?AT

Y?

WH

HOW?

Where did the couple live?

W Where did they travel? Where did others who became illll live?xposed to toxins? Where would they have been exposed

WHEN?

RE

HE

WWhat was the cause of death?WWhat toxins have they been exposed to?WWhat chemical toxins could cause these symptoms?WWhat natural pathogens could cause these symptoms?WWhat has changed in the environment?

HHow did they become ill?DDid they inhale harmful fumes?DDid they experiment with illegal substances?

When did they become ill; how quickly did they die? When did others show the same symptoms; when did they die? Does time of year matter?

Depending on the specific questions they develop,

students may choose to categorize the questions on thebasis of the affinity groups they developed in the StructuredBrainstorming exercise. In this case, possible pairings couldinclude these: What? Can their deaths be attributed to exposureto a known transmitted disease; a new, naturallyoccurring pathogen; or a chemical toxin such as anew herbicide? Who? Might international terrorists, domesticextremists, or criminal elements have beenresponsible for their deaths? Why? Did they die because they were membersof the Navajo Nation? Or because they belongedto some other group? Did they die as the result ofnatural causes or due to deliberate human acts? Where? Did where they live cause their death? Didthey and other victims travel to the same place beforebecoming ill? Did something in the region make themill or something at a specific location at Fort Wingate?Another approach would be to organize the questions onthe basis of a known factor, such as supporting evidence. For

instance, they could form three groups of questions: one group

of questions that have evidence to support the answer, anotherfor which there is only indirect evidence or assumptions,and another for which there is no supporting evidence at all.Alternatively, students could prioritize the questions on thebasis of known unknowns or gaps they seek to fill.Analytic Value Added: As a result of your analysis,which questions or categor ies deser ve furtherinvestigation? Analysts could focus their assessment onthose questions that are most likely to move theinvestigation forward quickly either by eliminating potentialhypotheses or further substantiating a lead hypothesis. Forthe example above, these might include the following: Are people who do not belong to the Navajo Nationdying as well? Are there any indications on the Internet that certaingroups are targeting the Navajo Nation? What are the indications that the illness iscontagious? What similarities can we detect among those whohave become ill?

94 Chapter 9 Are there known toxic waste sites that all the victimsmight have visited? Are the symptoms consistent with any other viruses ordiseases that are more lethal than the common flu?TECHNIQUE 3: KEY ASSUMPTIONS CHECKThe Key Assumptions Check is a systematic effort to makeexplicit and question the assumptions that guide an analystsinterpretation of evidence and reas oning about anyparticular problem. Such assumptions are usually necessaryand unavoidable as a means of filling gaps in the incomplete,ambiguous, and sometimes deceptive information withwhich the analyst must work. They are driven by the analysts education, training, and experience, including theorganizational context in which the analyst works. It can bedifficult to identify assumptions, because many aresociocultural beliefs that are held unconsciously or so firmlythat they are assumed to be truth and not subject tochallenge. Nonetheless, identifying key assumptions andassessing the overall impact should conditions change arecritical parts of a robust analytic process.Task 3.Conduct a Key Assumptions Check of the initial theory thatthe young Navajo couple died from a particularly virulentcommon flu virus.Step 1: Gather a small group of individuals who areworking the issue along with a few outsiders. The primaryanalytic unit already is working from an established mentalmodel, so the outsiders are needed to bring otherperspectives.In this instance, the Navajo tribal healers and expertsfrom CDC in essence played the role of outsiders. Thehistorical perspective provided by the tribal healers turnedout to be critical to solving the case.

Step 2: Ideally, participants should be asked to bring their

list of assumptions when they come to the meeting. If not,start the meeting with a silent brainstorming session. Ask eachparticipant to write down several assumptions on 3 5 cards.Step 3: Collect the cards and list the assumptions on awhiteboard for all to see. A simple template can be used, asin Table 9.3.In the early days of the investigation, much of theattention focused on the fact that almost all the victimswere Navajos. Were they targeted because of their identity,did they frequent the same places, or did the illness have todo with where they lived? A keyand unwarrantedassumption early on was that the disease was contagiousand might spread rapidly to other populations.Step 4: Elicit additional assumptions. Work from theprevailing analytic line back to the key arguments thatsupport it. Use various devices to prod particip antsthinking. Ask the standard journalist questions: Who?What? How? When? Where? and Why? Phrases such aswill always, will never, or would have to be suggest thatan idea is not being challenged and perhaps should be.Phrases such as based on or generally the case usuallysuggest that a challengeable assumption is being made.In this case, a key assumption deserving furtherinvestigation is that Fort Wingate may be the source of theproblem because of its assumed involvement with thedevelopment of chemical and biological weapons.The challenge would be to establish a credible link betweenthe facilities at Fort Wingate and the dead and sick people.Additional research also would be warranted to explorewhether the recorded increase in the rodent populationcould be linked to the surge in sudden deaths. What diseasesare rodents known to carry that would cause the symptomsreported of those who died? What would be required totransmit the disease from rodents to humans?

Table 9.3 Key Assumptions Check Template

Key Assumption1.2.3.4.

Commentary

Supported

With Caveat

Unsupported

Death in the Southwest 95

Unsupported or questionablethe key

uncertainties

Step 5: After identifying a full set of assumptions,

critically examine each assumption. Ask: Why am I confident that this assumption is correct? In what circumstances might this assumption beuntrue? Could this assumption have been true in the past butno longer be true today? How much confidence do I have that this assumptionis valid? If this assumption turns out to be invalid, how muchimpact would it have on the analysis?Step 6: Using Table 9.3, place each assumption in one ofthree categories: Basically supported Correct with some caveats

Step 7: Refine the list, deleting those assumptions that do

not hold up to scrutiny and adding new assumptions thatemerge from the discussion.In this instance, a final list of twelve key assumptions wasgenerated. A critical examination of the list would placefour assumptions in the Supported category, four in theWith Caveats category, and four in the Unsupportedcategory, as shown in Table 9.5. The Supported assumptionsare supported by evidence reported by reputable sourceseither doctors working the case or reports from wellrespected research organizations. The assumptions WithCaveats may well turn out to be correct, but there isinsufficient evidence to prove they are true at this time. Theassumption that the disease could spread quickly may bewarranted at the outset of the investigation when publicsafety is a priority concern, but should not be used to justify

Table 9.5 Death in the Southwest Key Assumptions Check Example

Key Assumption

Commentary

Supported

1. Cause of death is a highly potent flu virus.

Symptoms are similar to those of flu, but

flu strain would have to be unique to area.

2. Disease could spread quickly.

This is a genuine concern, but no evidence

of spread beyond Four Corners.

3. Disease has unusually high mortality rate.

Most of those who contract disease die

within a few days.

4. The rapid deaths suggest a terrorist act.

There is no evidence that terrorists were

targeting the Four Corners area.

5. Illness can be treated with antibiotics.

Some treated did recover, but there is no

proof recovery was due to antibiotics.

6. Most of the victims are Navajos.

The preponderance of those dying are

members of the Navajo nation.

7. Navajos are being targeted.

There is no evidence that someone is

intentionally targeting Navajos.

8. E xposure to a toxic substance caused the

deaths.

Many of the symptoms correlate with

exposure to a toxic substance.

9. D ead Navajos were victims of a hatecrime.

There is no evidence to support this.

10. The disease is not contagious.

To date, no medical personnel have fallen

ill from the disease.

11. Rodents are known carriers of disease.

Rodents are known carriers of many

diseases with similar symptoms.

12. Rodent population grew tenfold 199293.

This fact has been documented by

ecological researchers.

With Caveats

Unsupported

96 Chapter 9major resource decisions given the fact that caregivers arenot coming down with the illness. The assumption thatNavajos are deliberate targets is mere speculationunjustified by any known data.Step 8: Consider whether key uncertainties should beconverted into collection requirements or research topics.The Key Assumptions Check should inspire the analyststo focus their attention on the Unsupported assumptionsthat have emerged as Key Uncertainties. Analysts couldfocus their assessment on those questions that are mostlikely to move the investigation forward. These mightinclude the following: Are people who do not belong to the Navajo Nationdying as well? What are the indications that the illness iscontagious? Are the symptoms consistent with any other virusesor diseases that are far more virulent than thecommon flu? Are there any reports of tourists contracting thedisease or spreading it to other parts of the countrywhen they return home? Are any Internet sites or blogs posting informationcritical of the Navajo Nation? What similarities can we detect among those whohave become ill? Are there known toxic waste sites that all the victimsmight have visited? Can any link be established between Fort Wingateand those who have fallen ill or died of this disease? Can a link be established between a mushroomingrodent population and Navajos suddenly becomingill? What would the tribal healers and history tell usabout a potential link?Analytic Value Added: When CDC investigatorsarrived on the scene and interviewed doctors, did theyinherit any key assumptions that would have had animpact on how effectively they organized theirinvestigation? CDC investigators were careful to review allthe information provided by the on-site caregivers and toinitiate new research to establish patterns and look forsimilarities. More important, they reached outside theirnormal circles to seek input from Navajo tribal healers inhopes of gaining additional perspectives on the case. This

opened their minds to the possibility that they were dealing

with a phenomenon that might have historical precedents;to wit, that the dramatic increase in the rodent populationresulted in far greater rodent/human contact, allowing aparticularly virulent disease to be transmitted to humansliving in the area, most of whom were Navajos.TECHNIQUE 4: MULTIPLE HYPOTHESISGENERATION: MULTIPLE HYPOTHESESGENERATORTMMultiple Hypothesis Generation is part of any rigorousanalytic process because it helps the analyst avoid commonpitfalls, such as coming to premature closure or being overlyinfluenced by first impressions. Instead, it helps the analystthink broadly and creatively about a range of possibilities.The goal is to develop an exhaustive list of hypotheses,which can be scrutinized and tested over time againstexisting evidence and new data that may become availablein the future.The Multiple Hypotheses GeneratorTM is a useful tool forbroadening the spectrum of plausible hypotheses. It isparticularly helpful when there is a prev ailing, butincreasingly unconvincing, lead hypothesisin this case,that healthy, young Navajos are dying from exposure to avirulent form of the common flu virus.Task 4.Use the Multiple Hypotheses GeneratorTM to create andassess alternative hypotheses that explain why the youngNavajo couple died. Contact Globalytica, LLC atTHINKSuite@globalytica.com or go to http://www.globalytica.com to obtain access to the Multiple HypothesesGeneratorTM software if it is not available on your system.Step 1: Identify the lead hypothesis and its componentparts using Who? What? How? When? Where? and Why?The lead hypothesis is this: Healthy young Navajos aredying from exposure to a virulent form of the common fluvirus. The key component parts are, Who (just Navajos orthe population in general)? What caused them to becomeill? How did they get ill? and possibly Where (was becomingill associated with any particular facility or location)?Steps 2 & 3: Identify plausible alternatives for the two orthree most relevant key component parts and strive to keepthem mutually exclusive. Discard any key componentquestions that one would consider to be given factors.

Death in the Southwest 97

Two hypotheses could be generated in response to the

Who question: just Navajos (because of shared identity,genetics, or specific Navajo Nation cultural practices) oranyone in the general population. Options for the Whatcomponent could be the common flu, some other disease ornatural pathogen, or a chemical toxin. The How componentcould be that the disease or toxin was present in the naturalenvironment or that it was present because of humanactivity. In the latter case, someone could have deliberatelyexposed the victims to a biological or chemical agent, or thevictims could have been exposed accidentally to a containeror a location where chemical or biological toxins werepresent. In the former case, possible perpetrators couldinclude domestic extremists, such as a white supremacistgroup, that deliberately wanted to target members of theNavajo Nation or international terrorists who wanted toincite terror among the general population. Accidentalexposure could occur during the conduct of a tribalceremony or because chemical or biological agents presentat Fort Wingate were not being stored or handled properly.The component When can be discarded because it is agiven. The time frame is established as spring of 1993. Somestudents might choose to break down Why into categoriessuch as to incite terror or to kill Navajos, but suchcategories generally overlap with both How or What. Wewould recommend not using this component.Table 9.6 shows the example output from the MultipleHypotheses GeneratorTM for this lead hypothesis.

Table 9.6 Multiple Hypotheses GeneratorTM:

Death in the Southwest Alternative HypothesesLead Hypothesis: Healthy young Navajos are dying fromexposure to a virulent form of the common flu virus.Components

Who?

What?

How?

Lead Hypothesis Navajo

Components

Virulent Form of the

Common Flu

Act ofNature

BrainstormedAlternativeComponents

Unknown Disease(Natural Pathogen)

IntentionalAct of Man

Chemical Toxin

AccidentalExposure

Anyone

Step 4 & 5: Generate a list of possible permutations.

Discard any permutations that simply make no sense.The best way to array the various permutations is tocreate a permutation tree with multiple branches, asillustrated in Table 9.7. Once all the permutations are listed,

it quickly becomes evident that several permutations can be

dropped because they make no sense. For example, it makesno sense that only a subset of the population (e.g., membersof the Navajo Nation) would be susceptible to the commonflu. Similarly, if someone was intent on killing or terrorizingpeople, they would not pick the common flu as a weapon.Step 6: Evaluate the credibility of the remainingpermutations on a scale of 1 to 5, where 1 is low credibilityand 5 is high credibility.Two permutations that state that only Navajos are dyingfrom a new pathogen or chemical toxin were not very likelybut could not be ruled out entirely and thus received a ratingof 1. For example, tribal healers could have unintentionallyintroduced a new and highly toxic substance into tribalceremonies. Permutations that are slightly more crediblewere given a rating of 2. For example, it is possiblebut notlikelythat a naturally occurring chemical toxin hadrecently been exposed or had become present in some morevirulent form, causing some people to die.Permutations given ratings of 3 or above were deemed tohave a more persuasive internal logic; if it turns out that theywere correct, no one would be surprised. In this case, none ofthe permutations is so compelling that it received a rating of5. It is important to note, however, that as more informationbecomes available, any of these ratings might be raised orlowered depending on what the new information reveals.Step 7: Re-sort the remaining permutations, listing themfrom most to least credible, as shown in Table 9.8.In this case study, the three permutations that received arating of 4 and the three permutations that received a ratingof 3 all deserve serious consideration. Several reasons canbe given for assigning these permutations high ratings: The common flu kills thousands of people eachyear in the United States, and there have been pastinstances where a variant of the virus has caused anunusually high number of deaths. It is just as possible that some new form of a naturallyoccurring virus other than the common flu hasbroken out in the region and that a new pathogen iscausing normally healthy people to die. There are multiple examples of radical extremistsgroups using biological agents to cause illness inthe United States, as well as the celebrated case of aJapanese terrorist group, Aum Shinrikyo, dispersingsarin gas in the Tokyo subway system on 20 March1995, causing hundreds of casualties.

Someone is using a new, unknown pathogen to kill people.

People are dying from accidental exposure to a new, unknown

People are dying from a naturally occurring chemical toxin.

Intentional Act of Man

Someone is using a chemical toxin to kill people.

Accidental Exposure

People are dying from accidental exposure to a chemical toxin.

Slightly less credible would be these three possibilities:

The history of the United States is replete with storiesof hate crimes targeting minority populations. Theuse of a biological agent to target such people wouldnot be surprising, particularly given recent historyof a scientist sending anthrax through the mail tomembers of the US Congress and the media. The Four Corners region is largely rural, and it ispossible that a new chemical substance or herbicidewas recently introduced by farmers and is causingpeople to become ill and some to die. People in certain locations, possibly at Fort Wingate,have been accidentally exposed to a new and, forsome, lethal form of a natural pathogen that is beingdeveloped or processed as part of a weaponizationprogram.

discard4

Step 8: Restate the permutations as hypotheses.

The top six permutations could be restated as hypothesesin the following way: People in the Four Corners region are dying from aparticularly virulent form of the common flu. People in the Four Corners region are dying from anaturally occurring, new, and still unknown naturalpathogen. Someone (most likely international terrorists) isspreading a lethal biological pathogen to terrorizethe population; similar attacks in other parts of theUnited States may be imminent. Someone (most likely a white supremacist group) isusing a lethal biological agent like ricin or anthrax tokill members of the Navajo Nation.

Death in the Southwest 99

Table 9.8 Multiple Hypotheses GeneratorTM: Death in the Southwest Hypotheses Re-sorted by CredibilityPermutations

Credibility Score

People are dying from a virulent form of the common flu.

People are dying from a naturally occurring new, unknown pathogen.

Someone is using a new, unknown natural pathogen to kill people.

Someone is using a new, unknown natural pathogen to kill Navajos.

People are dying from accidental exposure to a new, unknown natural pathogen.

People are dying from accidental exposure to a chemical toxin.

Someone is using a chemical toxin to kill Navajos.

People are dying from a naturally occurring chemical toxin.

Someone is using a chemical toxin to kill people.

Only Navajos are dying from a naturally occurring chemical toxin.

Only Navajos are dying from a new, unknown natural pathogen.

Only Navajos are dying from a virulent form of the common flu.

discard

Someone is using a virulent form of the common flu to kill Navajos.

discard

Only Navajos are dying from accidental exposure to a virulent form of the common flu.

discard

Only Navajos are dying from accidental exposure to a new, unknown natural pathogen.

discard

Only Navajos are dying from accidental exposure to a chemical toxin.

discard

Someone is using a virulent form of the common flu to kill people.

discard

People are dying from accidental exposure to a virulent form of the common flu.

discard

People who work at Fort Wingate have been

accidentally exposed to a new, unknown naturalpathogen. People living in the Navajo Nation have beenaccidentally exposed to a toxic chemical substance.Step 9: Select from the top of the list those alternativehypotheses most deserving of attention and note why thesehypotheses are most interesting (see Table 9.9).Most of the symptoms manifested by those becomingsick or dying point to a naturally occurring disease as themost likely culprit. Although most of the victims aremembers of the Navajo Nation, other members of thegeneral population also are dying. At this stage in theinvestigation, a key question is, What could have causedthis new, natural pathogen to emerge? Is it a naturallyoccurring phenomenon, or was it intentionally introducedby someone to cause terror or to kill members of the NavajoNation? The presence of Fort Wingate in the region alsoraises the possibility that people working there are being

Table 9.9 Multiple Hypotheses GeneratorTM:

Death in the Southwest Top HypothesesTop Hypotheses

Credibility Score

1. People are dying from a virulent form of

the common flu.

2. People are dying from a naturally occurring

new, unknown natural pathogen.

3. Someone is using a new, unknown natural

pathogen to kill people.

4. Someone is using a new, unknown natural

pathogen to kill Navajos.

5. People are dying from accidental exposure

to a new, unknown natural pathogen.

6. People are dying from accidental exposure

to a chemical toxin.

accidentally exposed to a lethal chemical or biological

substance used in a weapons program at that facility.

100 Chapter 9Analytic Value Added: Which hypotheses should beexplored further? Additional medical tests should beconducted to help determine if a new virus might be thecause of the problem. Researchers also need to investigatehow t he vic t ims acquire d t he p at hogen. Whatcommonalities exist in terms of where the victims worked,where they played, what locations they all might havefrequented, or what work practices they might all share? Ifdomestic radical extremists or terrorists were to blame,then research is needed to investigate why they would betargeting the Four Corners region or, more specifically,members of the Navajo Nation. For example, are there anyrecent postings on the Internet by such groups that wouldsuggest that an attack on members of the Navajo Nationwas justified? The chances that Fort Wingate is the sourceof the problem would be greatly increased if most of thosewho became ill worked at the fort or had relatives oracquaintances who worked there. Almost certainly, therewould be press reports and a major buzz in the localcommunity if Fort Wingate were the actual source of theproblem.Which of the six key components (Who? What? How?When? Where? and Why?) can be set aside because theyare givens, and why? The case study is challenging becausemany of the answers to these questions overlap. For example, the answer to Where? would indicate a natural cause ifthe Where turned out to be pastureland or farmland and,alternatively, an act of man if a specific location was identified that all the victims have frequented in recent weeks. TheWhy component poses similar challenges; at a minimum itfocuses attention on what specific groups would have motiveto launch an attack aimed at the Navajo Nation or the FourCorners region.Which hypotheses from the original list were discarded, and why? Most of the hypotheses that were discarded were dropped because the internal logic of thepermutation did not stand up to scrutiny. For example, aterrorist is not likely to use the common flu to cause a largescale panic, nor would the use of the common flu be likelyto generate large numbers of casualties.TECHNIQUE 5: ANALYSIS OFCOMPETING HYPOTHESESAnalysts face a perennial challenge of working withincomplete, ambiguous, anomalous, and sometimesdeceptive data. In addition, strict time constraints on analysisand the need to make a call often conspire with a numberof natural human cognitive tendencies to result in inaccurate

or incomplete judgments. Analysis of Competing Hypotheses

(ACH) improves the analysts chances of overcoming thesechallenges by requiring the analyst to identify and refutepossible hypotheses using the full range of data, assumptions,and gaps that are pertinent to the problem at hand.Task 5.Develop a set of hypotheses and use the Analysis ofCompeting Hypotheses software to identify which hypothesesprovide the most credible explanation for the deaths in thiscase. Contact Globalytica, LLC at THINKSuite@globalytica.com or go to http://www.globalytica.com to obtain access tothe basic software, or the collaborative version calledTe@mACH, if it is not available on your system.Step 1: Generate a set of hypotheses to be consideredbased on what was learned from the StructuredBrainstorming exercise, the Starbursting exercise, or theMultiple Hypotheses GeneratorTM exercise, striving formutual exclusivity.For the purposes of this illustration, the following fourhypotheses were selected based on work done in previousexercises. It is recommended to include the initial leadhypothesis or the accepted common wisdom. Deaths are due to exposure to a particularly virulentcommon flu. (Common Flu) Deaths are due to accidental exposure to a toxicsubstance such as a chemical herbicide. (ToxicSubstance) Navajos are the deliberate target of a hate crime.(Hate Crime) People are succumbing to a new pathogenamystery disease. (New Pathogen)Step 2: Make a list of all relevant information, includingsignificant evidence, arguments, gaps, and assumptions.A careful reading of the narrative should generatefifteen to twenty items of evidence or relevant informationthat can be loaded on the software tool. Sixteen of themost important items of relevant information are listed inFigure 9.3.Step 3: Assess the relevant information against eachhypothesis by asking, Is this information highly consistent,consistent, highly inconsistent, inconsistent, neutral, or not

Death in the Southwest 101

Figure 9.3 Death in the Southwest ACH Evidence List

the item Some people treated with antibiotics recovered,

doctors could not prove that patients recovery was directlyconnected to the use of antibiotics. The entry Fort Wingatemunitions storage and demo facility is nearby, also includesan implicit assumption that biological or chemical weaponsare or were being processed at the fort and anyone workingthere could be exposed to toxic substances.Step 4: Rate the credibility of each item of relevantinformation.

applicable vis--vis the hypothesis? (The Te@mACH

software does not include the neutral category.)Analysts using the basic ACH software will have theoption of choosing highly consistent (CC), consistent (C),inconsistent (I), highly inconsistent (II), not applicable (NA),or neutral (N). When using basic ACH or My Matrix withTe@mACH tool, it is important that analysts code theevidence line by line, in other words horizontally acrossthe matrix, not hypothesis by hypothesis, or vertically downthe matrix. Doing so helps the analyst consider each piece ofevidence fully against each hypothesis before moving on tothe next piece of evidence. This process keeps the analystfocused on the evidence rather than on proving a pethypothesis. The Survey option in Te@mACH randomlygenerates the cells to be coded, thus avoiding this problem.When entering and coding the data, the credibility scoreof all evidence or relevant information is set at a default ofmedium. Analysts can also choose a credibility score of lowor high. The software in the basic ACH tool will calculate aweighted inconsistency score that reflects the analystsjudgment about credibility of the data.With Te@mACH, there is a special Key Assumptionsbox you can check to record and explain any keyassumptions relating to a particular item of relevantinformation. In this case, one might want to note that for

Step 5: Refine the matrix by reconsidering the hypotheses.

Does it make sense to combine two hypotheses, add a newhypothesis, or disaggregate an existing one?If the hypotheses are not mutually exclusive, this willbecome apparent at this stage in the process if the problemdid not already surface during the coding process. Analystsshould consider disaggregating hypotheses whenever theyfind themselves clarifying the hypothesis as they code.The trigger, or indicator, that disaggregation is necessaryoccurs during the coding process. For example, thehypothesis Deliberate act by extremists, should bedisaggregated to include one hypothesis for terrorists, whomight want to target the general population, and a secondhypothesis for white supremacists, who would only want totarget Navajos or non-Caucasians.Sometimes hypotheses can be disaggregated into a familyof hypotheses. For example, exposure to a toxic substancecould involve either a chemical or a biological substance. Itcould also involve an herbicide or some previously benignsubstance. It usually is more efficient to first address theoverarching hypothesis. If this hypothesis seems likely, thena second ACH analysis can be created breaking thehypothesis into several mutually exclusive components.Similarly, if the hate-crime hypothesis emerges as a viableexplanation, then serious consideration should be given toadding a terrorism hypothesis or a gang-warfare hypothesis.Step 6: Draw tentative conclusions about the relativelikelihood of each hypothesis. An inconsistency score willbe calculated by the software; the hypothesis with the lowestinconsistency score is tentatively the most likely hypothesis.The one with the most inconsistencies is the least likely. Thehypotheses with the lowest inconsistency scores appear onthe left of the matrix, and those with the highest inconsistency scores appear on the right.It is important to address the likelihood of everyhypothesis, not simply the most and least likely. Based uponthe above hypotheses and relevant information, some

102 Chapter 9tentative conclusions about the relative likelihood of eachhypothesis would include the following observations: The Common Flu hypothesis is likely to have themost Inconsistents and is the easiest to dismiss. The Hate Crime hypothesis also has severalInconsistents and is not likely to be correct. The remaining two hypotheses have the fewestInconsistents and appear worthy of seriousconsideration and further investigation.It is just as important to critically examine the Inconsistentitems of relevant information for the most likely hypothesesas well. If many Inconsistents are associated with all themost likely hypotheses, this could signal that there is amissing hypothesis. However, if the inconsistent evidencecan be described at best as a squishy Inconsistent, then thehypothesis probably is the most likely explanation.

Figure 9.4 Death in the Southwest ACH Sorted by Diagnosticity

Step 7: Analyze the sensitivity of your tentative conclusion

to a change in the interpretation of a few critical items ofinformation, as shown in Figure 9.4. If using the basic ACHsoftware, sort the evidence by diagnosticity, and the mostdiagnostic information will appear at the top of the matrix.The Te@mACH software will automatically display themost diagnostic information at the top of the matrix.All of the hypotheses will include at least someinconsistent data. The goal of this step is to understandwhich pieces of relevant information have the most overalleffect on the relative likelihood of the hypotheses and whatcould happen if those pieces of evidence change.Step 8: Report the conclusions by considering the relativelikelihood of all the hypotheses.The sensitivity analysis reveals areas for furtherinvestigation, but in the absence of additional information,the tentative conclusions about the relative likelihood of thehypotheses hold. However, any written analysis should

Death in the Southwest 103

include a full accounting of conflicting information, gaps,

and assumptions upon which the analysis is based and whatnew information might change the likelihood of thehypotheses.Step 9: Identify indicators or milestones for futureobservation.The ACH process suggests that analysts should paycareful attention to new information that eithercorroborates or discredits the two lead hypotheses: NewPathogen or Toxic Substance. Critical questions for furtherinvestigation for the New Pathogen hypothesis include thefollowing: What pathogens best match the symptoms that arebeing reported? Why do Navajos seem particularly susceptible tothis new pathogen? What has changed in theirenvironment to make them more susceptible or moreexposed to a new pathogen? Do some rodents pose a particular threat? Are someknown to carry a pathogen that could produce thesesymptoms? Are these rodents indigenous to areaspopulated by Navajos?Critical questions for further investigation of the ToxicSubstance hypothesis include the following: Have any new herbicides been introduced recently byfarmers in the Four Corners area? Are there any toxic sites on the lands of the NavajoNation that could be the cause of the problem? Did any of the victims work at Fort Wingate? Arethere toxic dump sites at the fort, or are biologicaland/or chemical weapons being manufactured orstored there?

Analytic Value Added: As a result of your analysis,

what are the most and least likely hypotheses? The twomost likely hypotheses are that the people living in the FourCorners area were struck down by a new pathogen orrecently exposed to a toxic substance.What are the most diagnostic pieces of information?The most diagnostic items of information were the negative tests for flu, the specific symptoms of abdominal/backpain and low blood platelet counts, the lack of reporting of

anti-Navajo rhetoric on the Internet, and the failure of care

providers to come down with the same illness.What, if any, assumptions underlie the data? At thestart of the investigation, the CDC investigators were working from two key assumptions: that the cause of the sicknessand deaths was either an unknown pathogen or a bioterrorist act. A corollary to the second assumption was that residents had been exposed to an unannounced or undetectedbiochemical spill at nearby Fort Wingate.Are there any gaps in the relevant information thatcould affect your confidence? Many gaps remain in the evidence, as surfaced in the Starbursting and Key Assumptionsexercises.How confident are you in your assessment of the mostlikely hypotheses? We can be fairly certain that the cause ofdeaths was not the common flu and moderately confidentthat Navajos were not deliberately targeted for attack byterrorists or domestic extremists. More research is needed,however, before we can be confident that the cause of deathwas the introduction of a new pathogen or a recent, suddenexposure to a lethal chemical toxin.

CONCLUSION: THE ANSWER FROM ATLANTA

After a week of intense work, medical investigatorsconcluded that the disease was not spreading throughperson-to-person contact, but they still had not yetidentified its cause. On 4 June, CDC called with the resultsof tests they had run on the blood of the victims. They saidthe deaths were due to a never-before-seen strain ofhantavirus. The hantavirus is named after the HantaanRiver, which flows through North and South Korea, becauseit caused the illness and deaths of thousands of UnitedNations troops during the Korean War. Previously identifiedhantaviruses had caused kidney failure, but this newlyidentified strain was causing respiratory failure, and it wasmuch more deadly.1,2 A new viral hemorrhagic fever hadbeen discovered in America.Once medical investigators knew the cause of the illness,they turned to identifying the carrier of the virus andstopping its spread. CDC investigators immediatelysuspected, as with other hantaviruses, that the likely carrierwas a rodent. Each hantavirus appears to prefer differentrodents; the key question in this case was, What rodent?CDC provided the answer ten days later: the deer mouse.3Even with the culprit identified, there were still manyunanswered questions: How was the virus transmitted?

104 Chapter 9How long had the virus been present in the area? Tribalelders knew the presence of rodents in tribal homes putpeople at risk because it potentially exposed them to rodentfeces and urine.4 To avoid sickness, the elders recommendedburning affected clothing and isolating food supplies. Testson tissue samples collected and preserved by SevilletaWildlife Refuge ecologists showed that the now-termed SinNombre or Without a Name virus had been present inthe rodent population for at least ten years before the 1993epidemic. Based on the Navajo tribal healers oral histories,epidemiologists suspected that rodent-transmitted diseasehad been present in the Four Corners Region since the earlypart of the twentieth century.5In 1993, when precipitation plummetedactuallyreturned to normaland available vegetative foodsources were depleted, the increased rodent populationbegan searching for food in new environments, such asbarns and peoples homes. The virus, which does notcause illness in the rodent host, was transferred fromrodents to humans via saliva, urine, or fecal matter.Human infection occurs when the materials are inhaledas aerosols or introduced onto broken skin, similar to ananthrax infection. The disease was concentrated in theNavajo population simply because environmentalconditions in the local area and agricultural cultivationincreased contact between man and infected rodents.Visitors who had hiked or camped in the Navajo Nationarea also became victims because of their exposure to thedeer mouse.6,7Research on the outbreak later determined that 50percent of the infections were acquired in or around thehome, 10 percent at the workplace, 5 percent duringrecreation, and the remainder for mixed or unknownreasons. A frequent antecedent of contracting the virus wasopening and inhabiting a long unused cabin. This may berelated to several factors: entry disturbs deer mice, whichoften urinate as they flee; the closed cabin lacks ventilation;and the roof prevents inactivation of the virus by theultraviolet component of sunlight.8Hantaviruses often bring death quickly. Usually 30 to 40percent of patients die within twenty-four to forty-eighthours after admission to a hospital, even in well-runintensive care units (ICUs). The best indicator that ahantavirus is present is a finding of decreasing orabnormally low platelet counts. Approximately 40 percentof patients do not require the placement of a plastic tubeinto the trachea to protect the patients airway and provide a

means of mechanical ventilation. Treatment of the

remainder of patients can be very challenging. Patients whosurvive, however, are often released in two to three weeksand usually show no major effects.9THE FOLLOW-UPOnce the disease and the carrier were identified, publichealth officials advised local residents and visitors to thearea to avoid activities that resulted in contact with wildrodents and to avoid disturbing rodent burrows to minimizethe possibility of inhaling dried excreta. Homeowners whosaw evidence of rodent infestation in their homes wereencouraged to set traps; wash bedding; and don rubbergloves to wipe down countertops, cabinets, and walls withdiluted bleach or disinfectant.Since 1993, there have been a total of 560 cases of thevirus in 32 states. About three-quarters of the infectedpeople came from rural areas, with 63 percent of thereported cases being males. There is no treatment oreffective cure.10, 11KEY TAKEAWAYS It always pays to consider a broad range ofalternatives before launching into a project orinvestigation. One of the first questions to ask at the start of aproject or investigation is, What external expertise orexternal resources might I need to tap to perform mymission successfully? Consider a full range of hypotheses against all therelevant information and return to this analysisover time. There could be several, intertwinedexplanations, or the hypothesis could change asmore information comes to light. Be prepared toevaluate each piece of new information against all thepossibilities.

Multiple Hypotheses Generator

Hypothesis Generation and Testing

10 The Atlanta Olympics Bombing

olice investigators were under severe pressure to discover who placed the bomb in Centennial Park and tobring that person or persons to justice. One person hadbeen killed by the bomb and over a hundred were injured,and the public was justifiably concerned about safety at theOlympic Games. In such circumstances, the investigatingteam is under extreme pressure to come to closure quicklyand to identify a prime suspect. Such dynamics make analysts and investigators vulnerable to groupthink and morelikely to adopt satisficing strategies that will please all keystakeholders.The best way to cope with such pressure is to employstructured techniques that allow investigators and analystssupporting them to take a few moments to reflect on whatthey know and what they need to know before plunging in toresolve the case. In this case study, we explore how threestructured analytic techniquesthe Key Assumptions Check,Pros-Cons-Faults-and-Fixes, and the Multiple HypothesesGeneratorTMcan be employed to better frame the problemand avoid going down unnecessarily time-consuming investigative blind alleys. Each technique takes relatively little timeto employusually only an hour or twobut can save investigators much time over the long run by avoiding nonproductive leads. The techniques also can make the investigationmore efficient by focusing attention on key information gapsand what types of additional information could prove themost compelling in helping to solve the case.TECHNIQUE 1: KEY ASSUMPTIONS CHECKThe Key Assumptions Check is a systematic effort to makeexplicit and question the assumptions that guide an analysts

interpretation of evidence and reasoning about any particular problem. Such assumptions are usually necessary andunavoidable as a means of filling gaps in the incomplete,ambiguous, and sometimes deceptive information withwhich the analyst must work. They are driven by the analysts education, training, and experience, including theorganizational context in which the analyst works. It can bedifficult to identify assumptions because many are sociocultural beliefs that are held unconsciously or so firmly thatthey are assumed to be true and not subject to challenge.Nonetheless, identifying key assumptions and assessing theoverall impact should conditions change are critical parts ofa robust analytic process.Task 1.Assume you are a member of the FBI team investigating thebombing. Piedmont College President Cleere has called theFBI office in Atlanta to present his rationale for makingRichard Jewell a prime suspect in the case. Following consultations with Washington, D.C., your team has decided todo just that. To help kick off the investigation, you havebeen asked to conduct a Key Assumptions Check with yourteammates to go over what assumptions the team is makingabout Jewell and the bombing in Centennial Park. Your taskis to guide the team through the following eight steps forconducting a Key Assumptions Check.Step 1: Gather a small group of individuals who areworking the issue along with a few outsiders. The primaryanalytic unit already is working from an established mentalmodel, so the outsiders are needed to bring otherperspectives.

107

108 Chapter 10In this case, the FBI team of investigators would benefitfrom including some local or state law enforcement officialsin the brainstorming process.Step 2: Ideally, participants should be asked to bring theirlists of assumptions when they come to the meeting. If not,start the meeting with a silent brainstorming session. Ask eachparticipant to write down several assumptions on a 3 5 card.

support it. Use various devices to prod participants

thinking. Ask the standard journalist questions: Who?What? How? When? Where? And Why? Phrases such aswill always, will never, or would have to be suggest thatan idea is not being challenged and perhaps should be.Phrases such as based on or generally the case usuallysuggest that a challengeable assumption is being made. Alist of possible key assumptions is provided in Table 10.5.

Step 3: Collect the cards and list the assumptions on a

whiteboard for all to see. A simple template can be used,like the one shown in Table 10.2 in the book.

Step 5: After identifying a full set of assumptions,

critically examine each assumption. Ask:

Step 4: Elicit additional assumptions. Work from the

prevailing analytic line back to the key arguments that

In what circumstances might this assumption be

untrue?

Why am I confident that this assumption is correct?

Table 10.5 Atlanta Olympics Bombing Key Assumptions Example

Key Assumption

Supported

1. The attack was a single incident involving one bomb.

2. M any more people would have died or been injured if Richard Jewell had notalerted authorities to the knapsack.

With Caveats

3. Jewell placed the 911 call.

4. The bomb materials were readily available.

5. Jewell could have constructed the bomb.

6. Jewell would have known how to place the bomb without being seen.

7. The bomb was intended to kill large numbers of people indiscriminately.

8. The bombing was not a political act.

9. J ewell intended the bomb to explode in fewer than 30 minutes because hisintent was to clear the area of people and ambush police and security officers.

10. R ay Cleeres statements were truthful and not motivated by his holding agrudge against Jewell.

11. Jewell had law enforcement or military training in bomb making.

12. Jewell wanted a job with the Atlanta police.

Unsupported

13. Jewell placed the bomb so he could become a hero.

14. J ewells personality fit the profile of someone who would create an incident sohe could emerge a hero.

15. J ewells personality fit the profile because he sought out publicity after thebombing.

16. J ewell might be the bomber because he appeared uncomfortable talking aboutthe victims out of guilt.

17. J ewells statement that he wanted to get a position on the Atlanta policedepartment was inappropriate and could indicate he had a motive for plantingthe bomb.

18. Law enforcement officials were receiving daily bomb threats.

The Atlanta Olympics Bombing 109

Could this assumption have been true in the past but

no longer be true today? How much confidence do I have that this assumptionis valid? If this assumption turns out to be invalid, how muchimpact would it have on the analysis?Many of the assumptions make sense when taken at facevalue but quickly fall apart when examined more closely. Forexample, several assumptions suggesting that Jewells statements after the bombing indicated he might be the bomberare totally unsupported. Jewell had a legitimate reason tobe looking for a job because he expected to be unemployedafter the Olympics ended, and most of the press sought himout because he had a seemingly powerful story to tell ofhelping save many lives. The assumptions that he plantedthe bomb to create an incident to make him look like a herocant be totally dismissed, however, given Jewells rockyemployment history and problems in previous law enforcement positions.The assumption that Jewell placed the 911 call isunfounded because Jewell would have needed more time toget from Centennial Park to the Days Inn. While this arguesconvincingly against assuming Jewell made the phone call,it raises a different question: What if Jewell had an accomplice? The accomplice could have made the call, and thetwo perpetrators could have communicated with each otherover cell phones.Step 6: Using Table 10.2, place each assumption in one ofthree categories: Basically supported Correct with some caveats Unsupported or questionablethe keyuncertaintiesOne technique you can employ to decide which categoryto assign to an assumption is to ask the questions: Can I makedecisions about moving resources or people based on thisassumption? If the answer is yes then the assumption canbe rated as supported. If the answer is it depends, then theassumption merits a rating of with caveats, and the caveat(s)needs to be recorded. If it would be inappropriate or hard tojustify the movement of people or resources on the basis ofthis assumption, then the assumption is unsupported.In this case study, five of the assumptions appear solid,seven require caveats, and six of the key assumptions are

unfounded. The assumption that the bomb was intended

to kill large numbers of people is supported by the use ofnails and shrapnel in the bomb construction; however, acredible alternative hypothesis is that Jewells real intent wasto minimize casualties and limit deaths to a small numberof law enforcement and security officials because he madethe warning call to 911. Other assumptions requiring caveats relate to whether Jewell was creating an incident in orderto become a hero and to get a good job. While there is nodirect evidence to support this assumption, Jewells pastproblems working in law enforcement would argue thatsuch a hypothesis is worthy of investigation.A key question that usually arises from the exercise is,What motivated Cleere to make the call? If he had notcalled the FBI Atlanta Field Office to offer his theory, Jewellmay have never risen to the status of a prime suspect. Cleerecould have held a grudge against Jewell and made the callsimply to get him in trouble with the authorities. At a minimum and pending further investigation, the assumptionthat Cleere was truthful should be considered with caveats.Finally, the assumption that Jewell had military or lawenforcement training in bomb making is correct but shouldbe considered with caveats because we do not know if thetraining was sufficient to teach him how to make the actualbomb that was used.Step 7: Refine the list, deleting those assumptions that donot hold up to scrutiny and adding new assumptions thatemerge from the discussion.The assumption Jewell placed the 911 call, would haveto be dropped, given the time differences, or replaced by anew assumption that An accomplice of Jewell placed thecall. At a minimum, the discrepancy would argue for carefully reviewing and validating key segments of the chronology of events.Step 8: Consider whether key uncertainties should beconverted into investigative leads, collection requirements,or research topics.The Key Assumptions Check suggests several new avenues for investigation. For example, an effort should bemade to determine if Cleere could have had any ulteriormotives in calling the FBI Atlanta Field Office to present histheory. Moreover, should we assume that Jewell acted alone,or could there have been several perpetrators? If the timingsuggests that Jewell was primarily interested in killing policeand security personnel, would the placement of the bombsupport this theory as well? Would Jewell have known that

110 Chapter 10a large group of law enforcement officers would convergeon the site fairly quickly? How would Jewell have acquiredthis information? Would this suggest that Jewell might havebeen surveilling the site for several days? If so, would suchactivity show up on the security video cameras? If so,wouldnt Jewell be concerned that the cameras would catchhim planting the bomb? Would Jewell have known aboutthe security cameras?Analytic Value Added: What assumptions, if any, didlaw enforcement analysts and officials make as they beganthe investigation? Law enforcement officials fairly quicklyfocused on a single, lead hypothesis that Jewell had plantedthe bomb with the intent of revealing it to the authoritiesand taking credit for minimizing the number of casualties.They assumed motive and capability and, as new information surfaced, decided how it could be made to fit the leadhypothesis. Information inconsistent with this lead hypothesis, such as the impossibility of both making the 911 calland alerting authorities in Centennial Park to its presenceone minute later, was ignored.Were they influenced by key assumptions of others,including the press and the experts they interviewed, whowanted to assist their work? FBI investigators initiallyresponded to the call from Piedmont College PresidentCleere, appropriately treating this hypothesis as worthy offurther investigation, but nothing in the public recordshows that they challenged the assumption that Cleere wastruthful and not carrying a grudge against Jewell.As colleagues generated other examples of the wannabehero syndrome, however, they fell into the trap of satisficing, whereby a proposed explanation or theory of the casequickly gains acceptance because it fits with most of the keyfacts and the explanation satisfies the needs of ones supervisors and the public.Did the investigators fall into the trap of groupthink,or did they have sufficient cause to focus on Jewell as asuspect? The investigators quickly fell into the trap ofgroupthink, allowing a tip from President Cleere and a fewanecdotesof people having taken credit for incidents tomake themselves appear as heroesto dominate theirthinking. In reviewing Jewells past history in law enforcement, they were quick to confuse correlation with causality.Moreover, the case study notes that Jewell was charged withimpersonating a police officer but does not reveal if he wasactually convicted. Although Jewell had a history of employment problems, there was nothing in his case history tosuggest that he would go to the extreme of constructing an

antipersonnel bomb and exploding it at the Olympic

Games.What impact did key assumptions have on how effectively the FBI organized its investigation? If the investigators had critically examined all their key assumptions,asking themselves under what circumstances each assumption could turn out to be incorrect, they would have beenless prone to jump to the conclusion that Jewell was thebomber. Conducting the Key Assumptions Check raisesseveral additional questions that merit more serious attention: (1) Should Jewell be considered the prime suspect ifhe could not have placed the phone call? (2) WouldntJewell have had more prospects of success if he discovereda bomb that was yet to explode? and, more generally,(3) Was the bomber acting alone?TECHNIQUE 2: PROS-CONS-FAULTS-AND-FIXESPros-Cons-Faults-and-Fixes (PCFF) is a simple strategyfor evaluating many types of decisions, including thedecision to launch a police investigation. In this case, lawenforcement officials are under substantial pressure todecide whether Richard Jewell was responsible for planting the bomb. PCFF is part icularly well suited to situations in which decision makers must act quickly, becausethe technique helps to explicate and troubleshoot a decision in a quick and organized manner so that the decision can be shared and discussed by all decision-makingparticipants.Task 2.Use PCFF to help you decide whether Richard Jewell wasresponsible for planting the bomb in Centennial Park, asshown in Table 10.6.Step 1: Clearly define the proposed action or choice.The question to address is Did Richard Jewell plant thebomb in Centennial Park?Step 2: List all the Pros in favor of the decision. Thinkbroadly and creatively and list as many benefits, advantages,or other positives as possible. Merge any overlapping Pros.Step 3: List all the Cons or arguments against what isproposed. Review and consolidate the Cons. If two Cons aresimilar or overlapping, merge them to eliminateredundancy.

The Atlanta Olympics Bombing 111

Table 10.6 Atlanta Olympics Bombing Pros and Cons Example

Question: Did Richard Jewell plant the bomb in Centennial Park?Pros

Cons

1. He alerted the police to the knapsack containing the bomb.

1. He could not have made 911 call and alerted police to the presence ofthe knapsack.

2. He enjoyed getting publicity.

2. He would not have treated other police officers as his prime target.

3. He had problems in past jobs and needed a future job.

3. He would not have constructed an antipersonnel bomb.

4. He had previous bomb training.

4. He had no reason to detonate the bomb early, before 30 minutes.

5. The bomb was crude.

5. There were no witnesses or any forensics linking him to the attack.

Step 4: Determine Fixes to neutralize as many Cons as

possible. To do so, propose a modification of the Con thatwould significantly lower the risk of the Con being a problem, identify a preventive measure that would significantlyreduce the chances of the Con being a problem, conductcontingency planning that includes a change of course ifcertain indicators are observed, or identify a need for further research or to collect information to confirm or refutethe assumption that the Con is a problem.Fixes can be generated for several of the Cons: He could not have made the 911 call and alertedpolice to the presence of the knapsackJewell had anaccomplice. He would not have treated other police officers as hisprime targetthe more damage that was done, themore he could be portrayed as a hero. He would not have constructed an antipersonnelbombthe more damage that was done, the more hecould be portrayed as a hero. He had no reason to detonate the bomb early, before30 minutesit went off unintentionally. There were no witnesses or forensics linking himto the attackhe knew he might become a suspectand so was careful to avoid leaving any fingerprintsbehind.Step 5: Fault the Pros. Identify a reason the Pro would notwork or the benefit would not be received, pinpoint anundesirable side effect that might accompany the benefit, ornote a need for further research to confirm or refute theassumption that the Pro will work or be beneficial.

Faults can also be generated for all of the Pros:

He alerted the police to the knapsack containing thebombhe was just doing his job as he was trained todo it. He enjoyed getting publicitythis did not becomeapparent until several interviews had been done andhe realized how much fun it was to be an instantcelebrity. He had problems in past jobs and needed a futurejobthere is no past history of him being involvedin making bombs, espousing extreme views, orthreatening to do violence. He had previous bomb trainingthis is frequentlythe case for most police officers. The bomb was crudelots of people would havebeen just as capable as Jewell at making such abomb.Step 6: Compare the Pros, including any Faults, againstthe Cons and Fixes, as shown in Table 10.7.On balance, the Cons appear to make a strongerstatement than the Pros. Similarly, the Fixes for the Consare relatively weak, and the Faults for the Pros present moreconvincing counterarguments. The fact that Jewell couldnot have made the 911 call and alerted police, given thetiming of both events, is the most compelling factor. Onfurther inspection, one could question whether a wannabehero would have even bothered to make a phone callespecially one that would require using an accomplice andthereby forfeit personal control over a key part of thescenario. Similarly, the choice of an antipersonnel device is

Most police officers do.

He had previous bomb training.

He had no reason to detonate the

It went off accidentally.

Many people could have made

The bomb was crude.

There were no witnesses or

He took care to leave noforensics linking him to the attack. fingerprints, assuming he wouldbe a suspect.

hard to explain if Jewells primary motive was just to keep

himself employed.

Is there any evidence of Jewell making radical

statements justifying the use of violence orthreatening violent acts?

Analytic Value Added: Based upon your assessment

of the Pros and Cons, can you make a strong case thatRichard Jewell planted the bomb in Centennial Park? Theanalysis generated by using the Pros-Cons-Faults-and-Fixestechnique argues that the case against Jewell is highlycircumstantial and that Jewell should not be treated as aprimeand particularly not as the onlytarget of theinvestigation. At this stage of the investigation, however, italso would appear imprudent to remove him from the list ofpossible suspects until further avenues of investigation arepursued. Key avenues for additional investigation wouldinclude these:

Did the 911 call fit a pattern of any previous bomb

threats; did it stand out from the crowd of dailythreats received by the police?

Did the video surveillance cameras show anyone

placing the knapsack under the bench? Did the surveillance cameras show any suspiciousperson or persons appearing to surveil the site in thedays before the bombing? What actual experience did Jewell have in bombmaking? Is there any forensic evidence in Jewells car, on hisclothes, or in his apartment indicating that he was inpossession of bomb-making materials? Can we determine if Jewell was in CentennialPark when the phone call was made from theDays Inn?

TECHNIQUE 3: MULTIPLE HYPOTHESIS

GENERATION: MULTIPLE HYPOTHESESGENERATORTMMultiple Hypothesis Generation is part of any rigorousanalytic process because it helps the analyst avoid commonpitfalls such as coming to premature closure or beingoverly influenced by first impressions. Instead, it helps theanalyst think broadly and creatively about a range of possibilities. The goal is to develop an exhaustive list ofhypotheses that can be scrutinized and tested over timeagainst existing evidence and new data that may becomeavailable in the future.The Multiple Hypotheses GeneratorTM is one of severaltools that can be used to broaden the spectrum of plausible hypotheses. It is particularly helpful when there is areigning lead hypothesisin this case, the lead hypothesisthat Richard Jewell planted the bomb in Centennial Parkas part of a scheme to make himself a hero and obtain aposition in law enforcement after the Olympic Gamesconcluded.The most important aspect of the tool is the discussion itgenerates among analysts about the range of plausible

Domestic violent extremists

Disgruntled contractors

To inflict harm

To promote a political agenda

To protest losing a job

Centennial ParkPrepositioned explosive

hypotheses, especially about the credibility score for each

permutation. It is important to remember that the credibility score is meant to illuminate new, credible hypotheses forfurther examination. And while the process does encourageanalysts to focus on the hypotheses with higher credibilityscores, hypotheses with low credibility scores should not beentirely discarded because new evidence may emerge thatchanges their status.

same for all hypotheses. Brainstorm possible alternatives for

each of the remaining components, which in this case areWho and Why. Consolidate the lists into alternatives thatare as mutually exclusive as possible. For example,al-Qaeda would have different motives than a radicaldomestic extremist group.

Task 3.

Step 5: Discard any permutations that simply make no

sense.

Use the Multiple Hypotheses GeneratorTM to create and

assess alternative hypotheses for the bombing in CentennialPark (see Table 10.8). Contact Globalytica, LLC atTHINKSuite@globalytica.com or go to http://www.globalytica.com to obtain access to the Multiple HypothesesGeneratorTM software if it is not available on your system.Step 1: Identify the lead hypothesis and its componentparts using Who? What? How? When? Where? and Why?using Table 10.4 in the book.Richard Jewell placed the bomb under a bench inCentennial Park, alerted authorities to the bomb, andhelped clear the area before the bomb exploded becausehe thought people would never know he placed the bomband would consider him a hero for saving so many lives.With his reputation so enhanced, it would be easier forhim to get a fulltime job as a police officer.Steps 2 & 3: Identify plausible alternatives for each keycomponent and strive to keep them mutually exclusive.Discard any given factors.The given factors here include What (antipersonnelbomb), Where (Centennial Park), When (at 0120 on 27 July1996), and How (prepositioned explosive); these will be the

Step 4: Generate a list of possible permutations.

Step 6: Evaluate the credibility of the remaining hypotheses on a scale of 1 to 5, where 1 is low credibility and 5 ishigh credibility.The three hypotheses rated 0 in Table 10.9 can be discardedbecause they make little sense. For example, it makes no sensethat terrorists would plant bombs to protest being laid off.Step 7: Re-sort the remaining hypotheses, listing themfrom most to least credible, as shown in Table 10.10.Step 8: Restate the permutations as hypotheses.The permutations above are stated as hypotheses.Step 9: Select from the top of the list those alternativehypotheses most deserving of attention and note why thesehypotheses are most interesting (see Table 10.11).The four most plausible hypotheses with a credibilityscore of 3 or higher are these: Richard Jewell planted the bomb to make himself ahero and obtain a job. International terrorists planted the bomb to inflictharm on America.

International terrorists planted the bomb to inflict

Domestic violent extremists planted the bomb to

Disgruntled workers planted the bomb to protest

Domestic violent extremists planted the bomb to

inflict harm.

International terrorists planted the bomb to promote

a political agenda.

Disgruntled workers planted the bomb to inflict

harm.

Domestic violent extremists planted the bomb to

promote a political agenda. Disgruntled workers planted the bomb to protestlosing a job.If none of these top four hypotheses generates seriousinvestigative leads, then less highly rated hypotheses shouldreceive increased attention.

It is possible that disgruntled workers might have

planted a bomb out of a general sense of anger over losingtheir jobs but unlikely that they would target their anger atpeople attending the Olympics. A more likely target forthem would be the nearby AT&T facility. International terrorists generally have not used terrorism to promote someone elses domestic political agenda, but it is possible theywould collaborate in attacking the Olympic Games becauseit is an appropriate iconic target.While the credibility score is subjective in nature, itshould reflect reasoning that can be used to weed out nonsensical or highly unlikely hypotheses. The unused hypotheses should not be discarded. They should be reserved, andthe list should be referred to and reconsidered as new information becomes available.Analytic Value Added: Which hypotheses should beexplored further? Use of the Multiple HypothesesGeneratorTM flagged several new hypotheses that appear atleast as credible as the lead hypothesis. Given the recentdestruction of TWA 800, it would be imprudent not toconsider international terrorists as a possible perpetrator.Domestic violent extremists might possess even strongermotives and capabilities to conduct such a bombing. Thedisgruntled workers hypothesis is probably less likelygiven the type of bomb used and its location, but it shouldnot be dismissed at the onset of the investigation.What motives should be considered, and why? Some ofthe more likely motives to emerge from the exercise would

The Atlanta Olympics Bombing 115

Richard Jewell planted the bomb to make himself a hero and obtain a job. Jewells past employment history makes him a candidate for awannabe attack.International terrorists planted the bomb to inflict harm.

International terrorists had struck several times at America, and the

Olympics would be an iconic target.

Domestic violent extremists planted the bomb to promote a political

agenda.

White supremacists, for example, could be protesting the multiethnic

character of the Olympics, or anarchists could be targeting theOlympics to send out their nihilist message.

Disgruntled workers planted the bomb to protest losing a job.

Security guards who had recently been laid off were angry aboutlosing their jobs.

be that the bomber has a personal agenda (to look like a

hero); has an ideological agenda (to make a political statement or to promote an extremist cause such as whitesupremacy, the primacy of sovereign rights, anti-abortion,or anti-internationalism); or wants to do harm against people or institutions (perpetrators could range from localanarchists to al-Qaeda).Which hypotheses from the original list were set aside,and why? It is up to the analyst to decide how many andwhich hypotheses should be considered for further exploration. A general rule of thumb is that more than five hypotheses become cumbersome and signal possible problemswith mutual exclusivity. In such cases, analysts should beencouraged to aggregate hypotheses when taking a firstlook at the available evidence. Also, analysts should beencouraged initially to include hypotheses in the originallist for which there is little or no evidence in the hope thatnew information might be obtained later that would support an initially outlier hypothesis. Hypotheses that are notbased on observations, logic, or supportable assumptions,however, should not constitute a lead hypothesis. Analystsshould state explicitly why certain hypotheses do not makethe final list and record what new information could changethat status in the future.CONCLUSIONTwo days after the bombing, President Bill Clinton told theAmerican public that the Games should carry on as plannedto show that the United States would not be cowed by actsof terrorism. He said: An act of terrorism like this is clearly

directed at the spirit of our own democracy. We must not let

these attacks stop us from going forward. We cannot let terror win. That is not the American way.1On 26 October 1996, Jewell was informed that heno longer was a target of the Atlanta Olympics bombinvestigation. An internal investigation was launchedinside the FBI focusing on whether Jewells status as aprime suspect had been leaked to the media, but ultimatelythe Bureau never identified or disciplined anyone for thealleged leak.2Following his ordeal, Jewell filed slander and libel lawsuits against several media organizations.3 NBC, CNN, andthe New York Post all settled their cases with Jewell forundisclosed amounts. Piedmont College, the school whereJewell was once employed, also settled for an undisclosedamount. Several school employees, including Cleere, hadsaid unfavorable things about Jewell when they were interviewed by the FBI.Months later, Jewells attorney, Lin Wood, said that therole the media played in his clients status as a suspect wascrucial. We know, Wood said, that the FBI was interestedin Richard, but had really not decided whether Richard Jewell was a possible suspect or a potentially valuable witness.But before they could execute their plan, the banner headline gets published, and now all of a sudden, the FBIs got tocome to grips with Richard Jewell in a public investigation,and that changed, I think, the whole approach that the FBItook.4Jewell died on 29 August 2007 from natural causes at theage of 44. He was suffering from severe heart disease,kidney disease, and diabetes.5

116 Chapter 10THE HUNT FOR ERIC RUDOLPHOver a two-year period after the bombing, special agents onthe Southeast Bomb Task Force interviewed thousands ofwitnesses and traced nearly every component of the bomb.The task force was comprised of the FBI; Bureau of Alcohol,Tobacco, and Firearms (ATF); Georgia Bureau ofInvestigation; Alabama Bureau of Investigation;Birmingham Police Department; and prosecutors from theJustice Department. In addition, many local and state lawenforcement units supported the task force.6On 14 October 1998, federal authorities charged EricRudolph with conducting the fatal bombing at Atlantas Centennial Park on 27 July 1996. Rudolph became a serious targetof investigation in part because a Tennessee couple identifiedhim as the man to whom they sold the smokeless powderbelieved to have been used in the Atlanta bomb device.7Federal authorities also charged Rudolph with a doublebombing at a health clinic in the Sandy Springs ProfessionalBuilding in North Atlanta on 16 January 1977 and with thebombing of a gay night club, the Otherside Lounge, in Atlantaon 21 February 1997.8 In the Sandy Springs bombing, the firstbomb caused significant damage at the back of the building.The second bomb was designed to kill and maim rescuers,paramedics, firefighters, and police officers who rushed to thescene to help, according to the Director of the ATF.9 A secondbomb was also found at the scene of the Otherside Loungebombing, but the area was cleared before it exploded.In addition, Rudolph was charged with the bombing at theNew Woman All Woman Health Care Clinic in Birmingham,Alabama, on 29 January 1998, which killed Birminghampolice officer Robert Sanderson and severely injured the clinics head nurse, Emily Lyons. In announcing the chargesagainst Rudolph, the government said it would pay a rewardof $500,000 for information leading to a conviction ofRudolph and a reward of up to $1,000,000 for informationleading to Rudolphs arrest.10Rudolph became one of Americas top ten most wantedfugitives from justice. 11 A sizeable law enforcementcontingent, supported by infrared-equipped helicopters andtracking dogs, was dispatched to comb the 517,000-acreNantahala Forest in western North Carolina to look for anysign of Rudolph.12,13After more than five years on the run, Rudolph wascaptured in May 2003 when police spotted him near a trashbin in Murphy, North Carolina, apparently scavenging forfood.14 He was brought to trial in July 2004 and chargedwith the bombings of the health clinic and the Otherside

Lounge in Atlanta, the bombing of the abortion clinic in

Alabama, and the Centennial Park bombing.15 Rudolph toldfederal investigators that his motive for planting the bombin Centennial Park was to bring down the Olympic Gamesand embarrass the US government for legalizing abortion.16In April 2005, Rudolph admitted to the crimes and, as partof a plea bargain, was spared the death penalty, receiving fourconsecutive life sentences without parole.17 Deborah Rudolph,Rudolphs sister-in-law, said her brother-in-law accepted thegovernments offer of life without parole in exchange for admitting guilt in order to protect his family from further scrutiny.18 Rudolph characterized his decision as purely a tacticalchoice, leaving open the question as to whether his confessionfor having conducted all four bombings was legitimate.19

KEY TAKEAWAYS When under severe pressure to find a culpritor generate an analytic conclusion quickly, analarm should go off telling you that these are thecircumstances where the use of structured analytictechniques is most justified. The use of techniques like the Key AssumptionsCheck or Pros-Cons-Faults-and-Fixes only take a fewhours but can save investigators days, if not weeks,of energy they would otherwise waste tracking downlow-priority leads or working from assumptions thatupon close inspection prove invalid. Considering multiple credible hypotheses (orsuspects) at the start of an investigation often provesmuch more efficient and less time-consumingoverall than conducting the investigation in a serialfashion by first going after a prime suspect, andthen a second suspect if the first does not pan out,and then a third suspect, etc. Considering multiplesuspects also helps focus attention on the mostdiagnostic evidence.INSTRUCTORS READING LISTFederal Bureau of Investigation, Counterterrorism Division,Counterterrorism Threat Assessment and Warning Unit,National Security Division. Terrorism in the United States:1996. http://www.fbi.gov/stats-services/publications/terror_96.pdf.Ostrow, Ron. Richard Jewell and the Olympic Bombing:Case Study. Pew Research Centers Project for Excellencein Journalism. February 15, 2003. http://www.journalism.org/node/1791.

Multiple Hypotheses Generator

Hypothesis Generation and Testing

Classic Quadrant Crunching

n a crisis, it is easy to allow the pace of breaking events

to lead to the first, most obvious answers. This casehighlights the importance of using a systematic processearly in a project to avoid this temptation. The techniqueshelp analysts to frame the issue effectively by challengingfaulty mental models and generating a full array of possibleexplanations. The Key Assumptions Check does this byhelping analysts explicate and challenge implicit assumptions about the sniper. The Multiple HypothesesGeneratorTM and Classic Quadrant CrunchingTM exercisesare two prisms through which analysts can systematicallydevelop and begin to assess a range of possible explanations.In this case, the Multiple Hypotheses GeneratorTM highlights the need to consider a broader range of suspects, andClassic Quadrant CrunchingTM helps uncover new dimensions for consideration, many of which had direct bearingon the true outcome of the case.TECHNIQUE 1: KEY ASSUMPTIONS CHECKThe Key Assumptions Check is a systematic effort to makeexplicit and question the assumptions that guide an analystsinterpretation of evidence and reasoning about any particular problem. Such assumptions are usually necessary andunavoidable as a means of filling gaps in the incomplete,ambiguous, and sometimes deceptive information withwhich the analyst must work. They are driven by the analysts education, training, and experience, including theorganizational context in which the analyst works. It can bedifficult to identify assumptions, because many are sociocultural beliefs that are held unconsciously or so firmly thatthey are assumed to be truth and not subject to challenge.

Nonetheless, identifying key assumptions and assessing the

overall impact should conditions change are critical parts ofa robust analytic process.Task 1.Conduct a Key Assumptions Check of the initial theory thatthe shooter most likely fits the profile of a classic serialkillera lone, white male with some military experience.Step 1: Gather a small group of individuals who areworking the issue along with a few outsiders. The primary analytic unit already is working from an establishedmental model, so the outsiders are needed to bring otherperspectives.In this instance, expert commentators interviewed onthe various TV networksand the public in generalplayed the role of outsiders. As it turned out, the expertcommentators perspectives tracked closely with the FBIsregarding the most likely criteria, focusing on the theory ofa serial killer. This tended to reinforce the theory of a lone,white male shooter when other options deserved more serious consideration.Step 2: Ideally, participants should be asked to bring theirlists of assumptions when they come to the meeting. If not,start the meeting with a silent brainstorming session. Askeach participant to write down several assumptions on 3 5 cards.Step 3: Collect the cards and list the assumptions on awhiteboard for all to see. A simple template can be used,like the one shown in Table 11.2.

119

120 Chapter 11In the early days of the investigation, the lead hypothesishad four key components: LoneOnly one shooter was involved in themultiple shootings. WhiteSerial killers are almost always Caucasian. MaleSerial killers are almost always male. Military experienceThe shooter must have hadmilitary experience in order to shoot so well and mayhave even been a sharpshooter.Step 4: Elicit additional assumptions. Work from the prevailing analytic line back to the key arguments that supportit. Use various devices to help prod participants thinking.Ask the standard journalist questions: Who? What? How?When? Where? and Why? Phrases such as will always,will never, or would have to be suggest that an idea isnot being challenged and perhaps should be. Phrases suchas based on or generally the case usually suggest that achallengeable assumption is being made.For the purposes of this case study, it works best tofocus the conversation on the lone, white male theory. Atthe time, other explanations were considered, including thepossibility that the shooter was a foreign terrorist; a domestic extremist, and possibly a white supremacist because

several persons of color were killed; or a disgruntled

employee of Michaels, Home Depot, or gas stations wherethe shootings took place.Step 5: After identifying a full set of assumptions, critically examine each assumption. Ask: Why am I confident that this assumption is correct? In what circumstances might this assumption beuntrue? Could this assumption have been true in the past butno longer be true today? How much confidence do I have that this assumptionis valid? If this assumption turns out to be invalid, how muchimpact would it have on the analysis?Step 6: Using Table 11.2, place each assumption in one ofthree categories: Basically supported Supported with some caveats Unsupported or questionablethe keyuncertainties

Table 11.6 Key Assumptions Check: DC Sniper as a Serial Killer

Key Assumption

Commentary

Supported

With Caveats

1. Lone

Empirical studies show that 80 percent of serial killers

operate alone and only 12 percent with partners. Thisis a fairly good assumption for planning purposes, butanalysts should be alert to the possibility of a partnerbeing involved.

2. White

Empirical studies show that about 80 percent of all

serial killers are Caucasians. If we were to assumethat the shooter is a Caucasian, we would be rulingout 20 percent of the potential targetsan evenbigger mistake.

3. Male

Empirical studies show that about 85 percent of all

serial killers are male. Again, this is a good operatingassumption, but we should be alert to any indicationsthis case could prove to be an exception.

4. Military experience

The weapon used was a high-caliber Bushmaster rifle.

Most people require only a few hours of training tolearn how to use a Bushmaster with some accuracy,particularly if it has a scope and a tripod or somethingelse to stabilize the shooting platform.

Unsupported

The DC Sniper 121

A critical review of the assumptions would place three

assumptions in the With Caveats category and one assumption in the Unsupported category, as shown in Table 11.6.Step 7: Refine the list, deleting those assumptions that donot hold up to scrutiny and adding new assumptions thatemerge from the discussion. The assumption that a serial killer would beoperating alone is rated as With Caveats, giventhat 12 percent of serial killers have partners.1 Giventhe spectacular nature of this case and how little isknown about the shootings, it would be premature todiscount the possibility of the killer operating witha confederate. In fact, the students might point outthat one characteristic of the casethat the shootingsoccurred with neither the shooter nor anyonedeparting the scene observedwould argue that theshooter was using a mobile shooting platform andwould need a driver to ensure a quick getaway. Assuming the shooter must be a Caucasian wouldbe a major mistake, as this would rule out 20 percentof all possible suspects despite no case evidencesuggesting the shooter is a Caucasian.2 In fact, one ofthe police reports relating to the first shooting intoa Michaels craft store noted that two black maleswere seen departing the parking lot in a suspiciousmanner. Knowing that 85 percent of all serial killers are malessuggests that this would be a solid assumption formounting an investigation.3 However, given thespectacular nature of the crimes, the urgency of theproblem, and the lack of evidence at this stage of theinvestigation, it would be make more sense not torule out any options and list this assumption as WithCaveats. The assumption that the shooter must have militaryexperience is reasonable but certainly not conclusive.Most people could learn to shoot a Bushmasterwith little training. More important, a discussionof this assumption should prompt a much moreproductive exploration of what is needed to shootpeople with such accuracy. When asked thisquestion, most students immediately respond bysuggesting the value of having a scope on the rifle.Usually with a little more time they suggest a tripodor something that can be used to stabilize the rifle.Since the shooter has not been seen yet, this begstwo questions: Where is the shooter shooting from?and How would he be able to stabilize the shooting

platform? One answer is that he might be shooting

from a van or some other vehicle with a built-inshooting platform.Step 8: Consider whether key uncertainties should beconverted into collection requirements or research topics.Analytic Value Added: Did the FBI investigatorsinherit any key assumptions when they took over the casethat had an impact on how effectively they pursued thecase? What is the value of conducting a Key AssumptionsCheck at the beginning of a major investigation? Whatimpact did key assumptions have on how the investigation was conducted? In this case, a Key Assumptions Checkexercise, if conducted, would have reinforced MontgomeryCounty Police Chief Mooses views that the investigationshould not prematurely focus only on whites but shouldconsider persons of all races as suspect. It might also havewarned investigators not to give military experience undueweight in conducting the investigation. In addition, a KeyAssumptions Check could have sparked a discussion of howthe shooter was taking shots, what kinds of vehicles mightbe involved, and whether the perpetrator would need anaccomplice. Lastly, it would have sensitized the investigatorsto several wild-card possibilities that the shooter could be anon-Caucasian, a female, or operating with a partner.Although historically the chances of these possibilities beingtrue were remote, if evidence surfaced later in the investigation pointing to any of these three possibilities, it wouldhave been helpful to have a bin to place that evidence in.In fact, from the outset of the case there was evidence,mostly in the form of eyewitness accounts, that black maleswere seen acting suspiciously in the vicinity of the crime,and about halfway through the investigation evidence beganto surface that more than one shooter was involved.

TECHNIQUE 2: MULTIPLE HYPOTHESIS

GENERATION: MULTIPLE HYPOTHESESGENERATORTMThe Multiple Hypotheses GeneratorTM is a useful tool forbroadening the spectrum of plausible hypotheses. It is particularly useful when there is a reigning lead hypothesisinthis case, the FBI profileand there are few facts to prove ordisprove it. The most important aspect of the tool is the discussion it generates among analysts about the range of plausible hypotheses, especially about the relative credibility ofeach permutation. It is important to remember that the

122 Chapter 11credibility score is meant to illuminate new, credible hypotheses for further examination. And although the processencourages analysts to focus on the hypotheses with thehighest credibility scores, hypotheses with low credibilityscores should not be entirely discarded because new evidencecould emerge that could make a hypothesis more credible.Task 2.Use the Multiple Hypotheses GeneratorTM (see Table 11.3)to create and assess alternative hypotheses. ContactGlobalytica, LLC at THINKSuite@globalytica.com or go tohttp://www.globalytica.com to obtain access to the softwareif it is not available on your system.Step 1: Identify the lead hypothesis and its componentparts.In this example, the Who, Why, and What have beenexplored. The lead hypothesis could best be articulated asfollows: A white male is driving a white van and killing toextort money. The key components are white male, whitevan, and killing to extort money. Since it is a fact thatshootings are happening and that the ballistic tests haveresulted in the identification of the type of weapon used,these aspects can be considered to be static and need not beincluded in the permutations.Steps 2 & 3: Identify plausible alternatives for each keycomponent and strive to keep them mutually exclusive.Discard any given factors such as the How (shooting) thatwill be the same for all hypotheses. Table 11.7 shows theresults of a brainstorming session on alternatives.The students are likely to suggest additional alternatives,but the two alternatives listed above have generally provenmost effective in illustrating the technique. For example,other alternatives to White Male could be Hispanic or

Middle Easterner. Similarly, possible alternatives to

White Van are Public Transportation, Motorcycle, orBicycle. Any of these could be substituted for On Foot.The Why? question usually prompts a robust discussion,and almost any alternative is worthy of consideration,including Hate Crime, Corporate Grievance, GangInitiation, or Political Protest. At the time, some citedHate Crime as the motive because of the number of persons of color killed, maintaining that the shooting of whiteswas intended to disguise the shooters true motive. Similarly,some analysts suggested that the killers were aggrievedemployees of Michaels Arts & Crafts store, Home Depot, orgas stations because of the locations of the shootings.Steps 4, 5, & 6: Generate a list of possible permutations,discard any permutations that simply make no sense, andevaluate the credibility of the remaining hypotheses on ascale of 1 to 5, where 1 is low credibility and 5 is high credibility.Table 11.8 contains the list of all the permutations alongwith their respective credibility score. All permutationsmade sense, and therefore none has been discarded.When evaluating the credibility of the hypotheses, it isimportant to consider each element separately and workacross the permutation table. The discussion points belowdescribe this process and list the underlying facts andassumptions that contributed to the credibility scores in thefigure. All permutations with On Foot received acredibility score of 1 because it is highly unlikely thatthe shooter could successfully travel by foot with aconcealed rifle of the caliber used in the shootingsand not be detected. Permutations for a White Female sniper received acredibility score of 2 because snipers are historicallyless likely to be female. Nonetheless, the credibilityscore is higher than the scores above becausefemales have engaged in terrorist attacks, andwe cannot rule out hypotheses on the absence ofevidence alone. Of the remaining permutations for White Male,it seems equally plausible that the sniper couldbe working from a White Van or Sedan, andtherefore the scores are the same for these twoelements. The sniper activities were very successful in instillingterror, so this alternative received a credibility score of 5.

A black male is killing to extort money and is driving a sedan.

A black male is killing to cause terror and is driving a sedan.

A black male is killing to seek fame and is driving a sedan.

A black male is killing to extort money and is on foot.

Terrorize

A black male is killing to cause terror and is on foot.

Seek Fame

A black male is killing to seek fame and is on foot.

Given the difficulty the sniper had in making

arrangements to extort money from theauthorities, Extort Money received a slightlylower score of 4. It is possible the sniper is acting out of a desire toseek fame, but there is less evidence in the case tosupport this alternative, so Seek Fame received acredibility score of 3. For the remaining Black permutations, as withWhite, there is no variation in credibility score

between White Van and Sedan. Also like White,

Seek Fame received a score of 3. For the White permutations, Extort Moneyand Terrorize received scores of 4 and 5 to reflectthe fact that historically, similar attacks have beencommitted by white males. Although this case maychallenge this historical precedent, there is not yet astrong reason to lower this score.Step 7: Re-sort the remaining hypotheses from most toleast credible, as shown in Table 11.9.

124 Chapter 11

Table 11.9 DC Sniper Hypotheses Re-sorted by

CredibilityPermutations

CredibilityScore

Step 8: Restate the permutations as hypotheses.

The permutations above are stated as hypotheses.Step 9: Select from the top of the list those alternativehypotheses most deserving of attention and note why thesehypotheses are most interesting.For this example, we have selected those permutationswith a credibility score of 3 or higher as deserving the mostattention based on the reasoning detailed in step 6 (seeTable 11.10).

A white male is killing to cause terror and is driving

a white van.

A white male is killing to cause terror and is driving

a sedan.

A black male is killing to cause terror and is driving

a white van.

A black male is killing to cause terror and is driving

a sedan.

A white male is killing to extort money and is driving

a white van.

A white male is killing to extort money and is driving

a sedan.

A black male is killing to extort money and is driving

a white van.

A black male is killing to extort money and is driving

a sedan.

A white male is killing to seek fame and is driving a

white van.

A white male is killing to seek fame and is driving a

sedan.

A black male is killing to seek fame and is driving a

white van.

A black male is killing to seek fame and is driving a

sedan.

Permutations

Credibility Score

A white female is killing to extort money and is

driving a white van.

A white male is killing to cause terror and is

driving a white van.

A white female is killing to cause terror and is driving

a white van.

A white female is killing to seek fame and is driving

a white van.

A white male is killing to cause terror and is

driving a sedan.

A white female is killing to extort money and is

driving a sedan.

A black male is killing to cause terror and is

driving a white van.

A white female is killing to cause terror and is driving

a sedan.

A black male is killing to cause terror and is

driving a sedan.

A white female is killing to seek fame and is driving

a sedan.

A white male is killing to extort money and

is driving a white van.

A white male is killing to extort money and is on

foot.

A white male is killing to extort money and

is driving a sedan.

A white male is killing to cause terror and is on foot.

A black male is killing to extort money and

is driving a white van.

A white male is killing to seek fame and is on foot.

A white female is killing to extort money and is on

foot.

A black male is killing to extort money and

is driving a sedan.

A white female is killing to cause terror and is on

foot.

A white male is killing to seek fame and is

driving a white van.

A white female is killing to seek fame and is on foot.

A white male is killing to seek fame and is

driving a sedan.

A black male is killing to extort money and is on foot.

A black male is killing to seek fame and is

driving a white van.

A black male is killing to cause terror and is on foot.

11

A black male is killing to seek fame and is

driving a sedan.

A black male is killing to seek fame and is on foot.

Analytic Value Added: In light of your findings,

how should investigators in the DC Sniper case haveused this information? What new suspects should theyhave pursued? When the permutations with a credibilityscore of 3 or higher are listed together, it quickly becomesapparent that the task force might need to consider abroader range of suspects. Credibility scores suggest that it isjust as plausible for the sniper to be working from a whitevan as it is from a sedan. It also becomes apparent that thetask force might consider looking for both black males and

Table 11.10 DC Sniper Multiple Hypotheses

GeneratorTM: Top Hypotheses

The DC Sniper 125

white males. The exact motive is less important than knowing the Who and What, but examining the potential reasonsmay assist investigators in how they approach the investigation and potential future communication with the sniper.Using the Multiple Hypotheses GeneratorTM allowed eachaspect of the alternative hypotheses to be evaluated in arobust manner that explicitly detailed the facts and assumptions underlying each credibility score. These conversationsare often enlightening and may not happen if the techniqueis not used.TECHNIQUE 3: CLASSIC QUADRANTCRUNCHINGTMClassic Quadrant CrunchingTM combines the methodologyof a Key Assumptions Check with Multiple ScenariosGeneration to generate an array of alternative scenarios orstories. This process is particularly helpful in the DC Snipercase because of embedded assumptions in the FBI profile,witness reports of white vans, and the contents of thedemand note. This technique allows the user to look at andchallenge those key assumptions. When combined with theMultiple Hypotheses GeneratorTM, this technique providesa strong basis for developing and considering alternativeexplanations and scenarios.Task 3.Use Classic Quadrant Crunching to challenge the keyassumptions in the case that is listed below.TM

Step 1 & 2: State your lead hypothesis or key assumption

and break it down into its component parts. For the purposes of this exercise: A lone white male is conducting theshootings from a white van to extort money.The words lone, white, white van, and to extortmoney are the component parts to be explored. Since it is afact that shootings are happening and that the ballistic tests

have identified the type of rifle, neither of these aspects is

included.Step 3: Identify contrary assumptions and two contrarydimensions in a template like that shown in Table 11.4.Table 11.11 details the brainstormed contrary assumptions and two contrary dimensions.The students are likely to suggest additional contrarydimensions, but the pairs listed in Table 11.11 are effectivein illustrating the technique. For example, other possibilitiesin the Other Transportation Method category are PublicTransportation, Motorcycle, or Bicycle. Any of thesecould be substituted for On Foot. Similarly, in the MultipleAttackers category, some might suggest independentshooters, and in the Other Race category, some might suggest Middle Easterners. The Other Motivation categoryusually prompts a robust discussion, and almost any alternative is worthy of consideration, including Hate Crimeand Corporate Grievance. At the time, some cited HateCrime as the motive because of the number of persons ofcolor killed, maintaining that the shooting of whites wasintended to disguise the shooters true motive. Similarly,some analysts suggested that the killers were aggrievedemployees of Michaels Arts and Crafts, Home Depot, or gasstations because of the locations of the shootings.Step 4: Array combinations of these contrary assumptionsin a set of 2 2 matrices.From the contrary dimensions, 6 matrices are possible fora total of 24 cells, as shown in Table 11.12. For ease of discussion, each 2 2 matrix and quadrant have been given a letterand number identifier. For example, in the first matrix,A/B-1 refers to the quadrant with a team of black shooters.Step 5: Generate scenarios for each quadrant.For each cell in each matrix, generate one to three examples of how this scenario might happen. For example,

Table 11.11 DC Sniper Classic Quadrant CrunchingTM Dimensions

Key Assumptions

Contrary Assumption

Contrary Dimensions

A. Lone Attacker

Multiple Attackers

Team

Copycat Killers

B. White

Other Race

Black

Hispanic

C. White Van

Other Transportation Method

Sedan

On Foot

D. To Extort Money

Other Motivation

Seek Fame

Cause Terror

126 Chapter 11

Table 11.12 DC Sniper Classic Quadrant

CrunchingTM: 2 2 MatricesA/B12

Multiple Attackers/RaceTeam

Team

Black

Hispanic

Copycat Killers

Copycat Killers

Black

Hispanic

A/C12

Multiple Attackers/TransportTeam

Team

Sedan

On Foot

Copycat Killers

Copycat Killers

Sedan

On Foot

A/D12

Multiple Attackers/MotivationTeam

Team

Seek Fame

Cause Terror

Copycat Killers

Copycat Killers

Cause Terror

Seek FameB/C12

Race/TransportBlack

Black

Sedan

On Foot

Hispanic

Hispanic

Sedan

On Foot

B/D12

Race/MotivationBlack

Black

Seek Fame

Cause Terror

Hispanic

Hispanic

Cause Terror

Seek FameC/D12

Transport/MotivationSedan

Sedan

Seek Fame

Cause Terror

On Foot

On Foot

Seek Fame

Cause Terror

Quadrant A/B-1 is a team of black snipers that is conducting attacks in multiple locations across the metropolitanWashington, D.C., area. The snipers formed a team sometime over the past year and set their well-practiced plan inmotion after several months of planning and training. Thecircumstances surrounding the formulation of their groupand the exact number of members in the cell are unknown.As a result, if this team is quite small, they could be conducting the attacks one at a time. If the team is larger anddispersed, they could be conducting coordinated attacks atpreappointed times.

In some cases, such a scenario might already have been

imagined. In other quadrants, it will be difficult to come upwith a credible scenario. But several of the quadrants willusually stretch the analysts thinking, forcing them to thinkabout the dynamic in new and different ways.Step 6: Select those scenarios (cells) deserving the mostattention.Review all the scenarios generated in Step 5 and selectthose most deserving of attention based on a pre-established set of criteria. In this example, possible criteria mightinclude those scenarios that would be the hardest to detector prevent. This would include those scenarios in which ateam operates on foot and would have difficulty exiting thescene of the crime undetected. Similarly, copycat killersmight have difficulty making arrangements to extort formoney.Another way to narrow the list of cells in this case is toremove those cells that are less likely either because ofknown facts in the case or due to strong historical precedent. As a result, the following scenarios were excluded: Cells with Copycat Killers were given low prioritybecause ballistic tests indicated only one type of rifle,a Bushmaster .223, was used and it seems highlyimprobable that imitative snipers would be using thesame weapon. On Foot cells have been excluded because it seemshighly improbable that the shooter, carrying a rifle,would go unnoticed at the scene of the crime. Whilesome rifles disassemble quickly, it would be easyto further refute this by examining those weaponscapable of firing the .223 round to determine if theyare capable of easily being disassembled. In addition,a review of public transportation available near theshooting sites could further discount such a scenario.This process results in dropping 11 of the 24 scenariosfrom our list of priority combinations. In this case, all thescenarios could be defined as nightmare scenarios becausethey all have an unknown probability but high impact: themetropolitan Washington, D.C., area is being terrorized bya sniper who is killing at a high rate. The main elements thatare shared by all the remaining scenarios and that appearmost deserving of further attention are these: Team cells could explain how the shooter gets awayso quickly. One person shoots, and one acts as thedriver/lookout.

The DC Sniper 127

Sedan cells could explain why the dragnets that

have been looking for a white van have failed to catchthe sniper. Cells with either race option seem equally probableand are both worth considering in addition to thelead hypothesis, which is white. Cells with Cause Terror seem realistic since theattacks were causing severe and widespread fear.It is important to remember that although we have identified some cells as deserving of the most attention, we do notdelete or discard the other cells. New information could bediscovered that would increase the plausibility of those cells.Step 7: Develop indicators for the selected scenarios.The goal of developing indicators for each scenario is tohelp investigators look for and be aware of a broad rangeof scenarios and indications that one or another scenariomay be emerging. For example, indicators of scenario B/C1, a black sniper using a sedan, would encourage investigators not to disregard additional reports of sedans leavingthe area and to review previous reporting and contact witnesses who previously reported the presence of a sedan.Reports that the shooter had a Hispanic accent when talking on the telephone provide strong justification for considering Hispanics in addition to whites. The discussion ofmatrix B/D that focuses on race and motivation, however,should surface the fact that blacks, whites, and Hispanicscan have a Hispanic accent, as is often the case in theCaribbean. Without this analytic process forcing a criticalexamination of all credible alternatives, authorities mightprematurelyand incorrectlyfocus their investigationon Hispanics and ignore other credible suspects.Analytic Value Added: Which alternative scenariosshould investigators have pursued, and why? By criticallyexamining each assumption and how a contrary assumption might play out, analysts can better assess their level ofconfidence in their predictions, the strength of their leadhypothesis, and the likelihood of their lead scenario. In theDC Sniper case, the use of this technique revealed someinteresting possibilities that may not have otherwise beenconsidered. This is of particular note because some of thecells in gray are what actually was happeningspecificallyA/B-1, A/C-1, and B/C-1. The hypotheses that containedBlack, Team, and Sedan were accurate. While themotive of the snipers remains a bit confused to this day,and money certainly was a factor, terror and fame also

played a role. In fact, the only erroneous cells were those

with On Foot, Copycat Killers, and Hispanic. Out of24 cells, 13 were identified as deserving serious attention,and of those 13, 9 contained accurate elements.CONCLUSIONThe terror finally ended on 24 October 2002. One blackman, John Allen Muhammad, formerly in the US Army,and one black teen, John Lee Malvo, of Jamaican decent,were caught sleeping at a rest stop off I-70 in Marylandwhen the authorities arrested them. 4 Malvos Jamaicanaccent had been misinterpreted as Hispanic. The vehiclethey were sleeping in was a blue 1990 Chevy Caprice.5 Thesnipers had modified the vehicle by removing the metaldivider between the backseat and the trunk and by makinga hole above the license plate so that Muhammad andMalvo could fire from inside the car.6 Authorities also foundin the car a Bushmaster rifle, considered to be easy to use,7along with a scope and tripod.8The note left at the Ponderosa did in fact use a pluralpronoun, we, and a note left after the Johnson shootingused us.9 Muhammad and Malvo had also attempted tocontact the police multiple times. In fact, it was duringone of their attempts to contact the police that they gaveaway crucial information. The snipers referred to a crimein Montgomery, Alabama, that would prove invaluable inidentifying the suspects.10 At that crime, fingerprint andballistics had been obtained that pointed the task forcedirectly at Malvo and, through him, to Muhammad.11 Inaddition, a former army buddy of Muhammads calledthe police on 17 October and was interviewed on 22October.The exact motive for the killing spree remains unclear.Malvo reportedly gave at least two reasons. The first wasthat whites had tried to hurt Louis Farrakhan.12 Whenasked directly if money was the reason for the killings,Malvo indicated yes and said that Montgomery County waschosen because thats where the rich people lived.13 AtMuhammads trial, the motive argued by the prosecutorwas revenge over a lost custody battle with Muhammadswife.14 Specifically, Malvo testified that the plan was to create havoc to cover for Mr. Muhammads plans to kidnap histhree children.The longer-term goal . . . was to extort law enforcement tostop the killing, after which Mr. Muhammad would takethe money and move to Canada with Mr. Malvo and the

128 Chapter 11three children. There...Mr. Muhammad planned to create a training ground for 140 young homeless men whomhe would send out to wreak similar havoc and to shutthings down in cities across the United States.15

At Malvos trial, the financial motive was further expanded

on by a claim that Muhammad intended to create a blackutopia in Canada populated by 70 boys and 70 girls who hadbeen unexposed to racism.16On 4 May 2004, Muhammad was sentenced to death inVirginia, and on 1 June 2006, he was sentenced to six lifeterms without parole in Maryland.17,18 On 7 August 2009,the death sentence was upheld by the Fourth US CircuitCourt of Appeals, and he was executed in Virginia on 10November 2009.19,20On 19 December 2003, Malvo was sentenced in Virginiato life imprisonment without the possibility of parole, andon 8 November 2006, he received six more years inMaryland in addition to the life sentence, all to be servedconsecutively.21,22

KEY TAKEAWAYS Decision making based on faulty assumptions canimpede an investigation. Always explicitly identifyand assess the effect implicit assumptions may haveon an investigation. The tendency to plunge in should always betempered by a process designed to identify allevidence and evaluate all possible explanations. Failure to consider alternative explanations from thestart can slow an investigation and let the real killeravoid prosecution. Employing a more systematic process at the startof the investigation to better frame the issue helpsanalysts identify unproductive blind alleys early onand avoid them.INSTRUCTORS READING LISTHorwitz, Sari, and Michael E. Ruane. Sniper: Inside the Huntfor the Killers Who Terrorized the Nation. New York: Random House, 2003.

Scenarios and Indicators

Scenarios and Indicators

12 Colombias FARC Attacks the US Homeland

he challenge for analysts in this case is to convert a very

generalized threat warning (The FARC intends tolaunch an attack on the US homeland) into an analyticframework that field operators and policy makers can use toprotect the nation from a possible terrorist attack. The following exercises walk students through an analytic processthat uses Red Hat Analysis, Structured Brainstorming,Multiple Scenarios Generation, Indicators, and the IndicatorsValidatorTM to anticipate how the adversaries are most likelyto behave, outline a set of the most likely terrorist courses ofaction, recognize the signs that the enemy is beginning toimplement a particular course of action, and tailor a set ofcollection requirements for specific field elements.This case puts students in the shoes of FBI, law enforcement, or Homeland Security analysts who would work thistype of case. Students should be advised that the case itselfis rooted in factthe history and tactics described in thetext are true. Also, while the threat posited in the case isfictitious, it mimics reality in which specific warningnotices are rare and analysts under tight time constraintsmust work rapidly to direct collection assets and providedecision makers with timely, actionable analysis that canmean the difference between averting disaster or not.

in the internal affairs of Colombia, and its leaders have concluded that the time has come. In this fictitious scenario,members of the Secretariat and top military commandersgather in the Amazon jungle to formulate a strategy for aretaliatory strike in the United States.The challenge for US analysts is to forecast how an attackis most likely to be launched and, in so doing, help federal,state, local, and tribal officials prevent or mitigate the damage of such an attack. When confronted with this challenge,the first reaction of many students is to propose that the USgovernment issue a general alert to all state, local, and tribalofficials that a FARC attack on the homeland may be imminent, and ask them to look out for any suspicious activitythat would indicate a FARC attack is being planned orimplemented. Unfortunately, such guidance is so unspecificas to lack value for law enforcement officials. The purposeof this exercise is to show that with the use of structuredanalytic techniques, analysts can generate a plausible set ofattention-deserving scenarios and create tailored lists of collection requirements that provide operational value toheadquarters, FBI field offices, and fusion centers.Task 1.

TECHNIQUE 1: RED HAT ANALYSIS AND

STRUCTURED BRAINSTORMING

Conduct a Red Hat/Structured Brainstorming exercise to

identify the forces and factors that would most influence aFARC decision to attack the US homeland.1

The major victory of the Colombian army and its US military supporters in Colombia against the FARC has created anew situation wherein the FARC sees itself substantiallyweakened, increasingly desperate, and determined to demonstrate that it is not a spent force. The FARC had threatened toretaliate against the United States in the past for interfering

Step 1: Gather a group of analysts with knowledge of the

FARC Secretariat; operating environment; and senior decision makers personality, motives, and style of thinking.It is helpful to include in the brainstorming group bothexperts on the topic and generalists who can provide morediverse perspectives. When only those working the issue are

129

130 Chapter 12included, often the groups perspective is limited to thestream of reporting it reads every day; as a result, keyassumptions may remain unchallenged, and historical analogies may be ignored.Step 2: Pass out sticky notes and marker-type pens to allparticipants. Inform the team that there is no talking duringthe sticky-notes portion of the brainstorming exercise.Use different color sticky notes and encourage the participants to write down short phrases consisting of three tofive words, not long sentences.Step 3: Present the team with the following question: Ifyou were in the FARC Secretariat, what are all the thingsyou personally would think about when planning an attackon the US homeland? The reason for first asking groupmembers how they would react is to establish a baseline forassessing whether the adversary is likely to react differently.Keep the question as general as possible so as not to inadvertently restrict the creative brainstorming process. It alsohelps to ask the group if they understand the question andwhether they believe it should be worded differently. Spending a few minutes to ensure that everyone understands whatthe question means is always a good investment.Ask them to put themselves in the FARCs shoes andsimulate how its leaders would respond. Emphasize theneed to avoid mirror imaging. The question is not Whatwould you do if you were in their shoes? but How wouldthe FARC leadership approach this problem, given theirbackground, past experience, and the current situation? Itis important to emphasize the importance of avoiding mirror imaging. In a classroom situation, many students maynot know much about the FARC; this is why it is importantto ensure that all participants read the case study with therelevant background material carefully. They should alsohave the case study at hand for quick reference.Step 4: Ask the group to write down responses to thequestion using a few key words that will fit on a sticky note.After a response is written down, the participant gives it tothe facilitator, who then reads it out loud. Marker-type pensare used so that people can easily see what is written on thesticky notes when they are posted on a wall or whiteboard.Give the students a few minutes to think about the issueand jot down a few ideas. Then go around the room andcollect the sticky notes. Read the responses slowly and stickthem on the wall or the whiteboard as you read them. Somesample sticky notes might address topics such as financing,type of weapon, target, deniability, need for contacts in the

United States, escape plan, motive, logistic support, infiltration, partners, and access to technology.Step 5: Post all the sticky notes on a wall in the order inwhich they are called out. Treat all ideas the same.Encourage participants to build on one anothers ideas.Usually there is an initial spurt of ideas followed by pausesas participants contemplate the question. After five or tenminutes there is often a long pause of a minute or so. Thisslowing down suggests that the group has emptied the barrel of the obvious and is now on the verge of coming upwith some fresh insights and ideas. Do not talk during thispause, even if the silence is uncomfortable.Remind the group not to talk during this part of theexercise. It is important for them to hear what others aresuggesting, as this might stimulate new ideas for them to jotdown. Also take care not to spend too much time talkingyourself. The participants need quiet time to think, and it isvery important for the instructor not to interrupt theirthought processes. Often when it is the quietest, the bestthinking is taking place.Step 6: After two or three long pauses, conclude thisdivergent thinking phase of the brainstorming session.Step 7: Ask all participants (or a small group) to go up tothe wall and rearrange the sticky notes by affinity groups(groups that have some common characteristics). Somesticky notes may be moved several times; some may also becopied if the idea applies to more than one affinity group.If only a subset of the group goes to the wall to rearrangethe sticky notes, then ask those who are remaining in theirseats to form into small groups and come up with a list of keydrivers or dimensions of the problem based on the themesthey heard emerge when the instructor was reading out thesticky notes. This keeps everyone busy and provides a usefulcheck on what is generated by those working at the wall.Step 8: When all sticky notes have been arranged, ask thegroup to select a word or phrase that best describes eachgrouping.Four or five themes usually emerge from this part of theexercise. A variety of potential targets, including US militaryinstallations and particularly USSOUTHCOMin Miami; FBI and DEA facilities, mostly inWashington, D.C., and along the US southernborder; and senior US officials, who could be targetsof assassinations or kidnappings.

Colombias FARC Attacks the US Homeland 131

The type of weapons that might be employed,

including the rompas that the FARC uses inColombia, rifles or other small arms, far moresophisticated weapons of mass destruction, andeven impure drugs such as cocaine adulterated withpoison or some other toxic substance. Motives for the attack and the intendedconsequences, including direct military retaliation; adesire to terrorize the broader US population; a hopethat creating major economic damage could divertUS attention from Colombia; or pure revenge, whichcould be satisfied by assassinating a senior official. Logistic considerations, including how to fund anoperation, infiltrate operatives into the United States,identify support networks within the United States,create appropriate documents, and devise effectiveescape plans once an operation has been completed. Whether FARC will seek the assistance of othersin designing and implementing the attack. If asophisticated attack is under consideration, thenFARC might require experts in chemical, biological,radiological, or nuclear warfare (CBRN). It mightalso look to known past partners such as the IRAor Spains ETA for expertise in planning a terroristattack against a sophisticated Western nation.Lastly, FARC could reach out to established drugdistribution networks already operating within theUnited States.Step 9: Ask the group to articulate how, taking all thesefactors into consideration, they would have orchestrated anattack and to explain why they think they would behavethat way. Ask them to list what core values or core assumptions were motivating their behavior or actions. Again, thisstep establishes a baseline for assessing why the FARCSecretariat is likely to react differently from you and theother members of your group.Step 10: Once the group can explain in a convincing waywhy it chose to act the way it did, ask the group members toput themselves in the shoes of the FARC Secretariat andsimulate how it would respond, repeating Steps 4 to 8.Emphasize the need to avoid mirror imaging. The questionis not What would you do if you were in their shoes? butHow would the FARC leadership approach this problem,given their background, past experience, and the currentsituation?Step 11: Once all the sticky notes have been arranged onthe board, look for sticky notes that do not fit neatly into

any of the groups. Consider whether such an outlier is useless noise or the germ of an idea that deserves furtherattention.Often one or two outlier sticky notes are worth pointing out to the class because they provide a fresh perspectiveor suggest a potentially valuable new line of inquiry. Hereare some examples: A note that says heroin could open the door toa discussion of whether the FARC would consideroperations to corrupt heroin currently being suppliedin the United States to force drug addicts to switch tococaine as a safer drug of choice. A note that says attack the US embassy in Bogotmight be initially rejected as outside the scope ofthe original question, but the instructor shouldnote that by raising the question of an attack onthe US embassy, the participant has, in effect,challenged a key assumption of the exercise (thatthe attack would take place on US soil), andperhaps in the real world this might prompt thegroup to conduct a key assumptions check andsubject this particular assumption to more carefulscrutiny.Step 12: Assess what the group has accomplished. Canyou identify four or five key factors, forces, themes, ordimensions that are most likely to influence how the FARCleadership would mount an attack?Work with the group to develop a consensus on fourthemes that emerge as the most important drivers for thistopic. Write the candidate drivers on the board and draw aline under each driver. The line represents the spectrum forthat driver. Label the end points of the spectrum for eachdimension or driver being considered. For example, if onedriver is sophistication of the weapon, then at the rightend of the line you would write CBRN or WMD and atthe left end of the line you would write small arms orsimple weapons or rifle.The themes that most often are generated by this stage ofthe exercise are as follows: Sophistication of weapons (simple such as a rifle oran assassination to highly sophisticated such as aCBRN-type attack). Motive (straightforward revenge to terrorizing USpopulation). Target (tactical such as a US military base to strategicsuch as the Pentagon or senior Washington officials).

132 Chapter 12 Partners (a do it alone operation to partnering withother terrorist groups such as the IRA or ETA orobtaining the support of drug distribution networksin the United States).Other themes that might emerge but usually do not workas well when conducting a Multiple Scenarios Generationexercise include these: Cost/benefit (minimal or major commitment ofresources and personnel). Infiltration/exfiltration (whether to infiltrate FARCoperatives or contract out to drug networks orradical extremists already operating in the UnitedStates). Willingness to accept risk (Are FARC leaders willingto consider a spectacular operation that could spurthe United States to launch a major retaliatorystrike in Colombia, or would they opt for a moremodest attack that sends a message but reducesthe prospects of a retaliatory strike against theirforces?). Timing (Will the attack be a quick response easilytied to recent events in Colombia or a much betterplanned and more sophisticated attack that couldtake months or even years to pull off?). Target security (Will the FARC go after hard or softtargets?).Step 13: At this point, the group should ask, Does theFARC Secretariat share our values or motives or methods ofoperation? If not, then how do those differences lead themto act in ways we might not have anticipated before engaging in this exercise?Step 14: Present the results, describing the alternativesthat were considered and the rationale for selecting the paththe group believes the FARC Secretariat is most likely totake. Consider less conventional means of presenting theresults of the analysis, such as the following: Describing a hypothetical conversation in which theSecretariat leaders would discuss the issue in the firstperson. Drafting a document (set of instructions, militaryorders, or directives) that the FARC Secretariatwould likely generate.

In most cases, the group should end up with a presentation that defines some version of the following four keydrivers and associated spectrums: type of weapon, motivefor the attack, target of the attack, and whether any outsideassistance is sought.Students should be encouraged to present their key findings by speaking in the first person, as if they were actualFARC members planning the attack.Analytic Value Added: The silent structuredbrainstorming approach is a powerful technique to pull outnew and often never previously considered ideas andconcepts. It avoids the trap of deferring to the mostknowledgeable person in the room by giving everyone anequal, but silent, opportunity to surface ideas. Whileconducting the structured brainstorming exercise, it isuseful to note whether particularly useful and creative ideasare generated after long pauses when everyone is thinking;if this does occur, it is important to alert the entire group tothe phenomenon.Were we careful to avoid mirror imaging when weput ourselves in the shoes of the FARC Secretariat?By putting themselves in the shoes of the FARC, analysts are more likely to focus on attack scenarios theFARC would be best positioned to implement successfullyand thus be the most likely. By conducting a Red HatAnalysis, they usually focus not only on how to launch anattack but the extent to which the plan they choosecould make them vulnerable to retaliation. Often exfiltrating forces is as important as infiltrating them into theUnited States.Did we explore all the possible forces and factors thatcould influence how the FARC might launch an attack onthe US homeland? The sticky notes should capture a broadspectrum of forces and factors, including logistical preparations, financing, preferred target, type of weapon to employ,ability to maintain operational security, mechanisms forinfiltrating and exfiltrating forces, and whether to seek theassistance of or partner with other groups.Did our ideas group themselves into coherent affinity groups? How did we treat outliers or sticky notesthat seemed to belong in a group all by themselves? Didthe outliers spark new lines of inquiry? Placing like ideasinto affinity groups can be a challenging task; asking thosenot at the wall to come up with their own categories oftenprovides a useful sanity check. Always take time to giveoutlier ideas their due attention. Invariably a structured

Colombias FARC Attacks the US Homeland 133

brainstorming exercise will stimulate ideas that at first

appear to be off-the-wall or not directly related to thetask. It is useful in the group discussion to ask whatprompted the person to prepare that note. Sometimes theexplanation will surface an idea or a concept that no oneelse in the group would have considered. For example, anote that said submarines might at first appear odd, butsubmarines or submersibles are used increasingly to movedrugs from Colombia to the United States and it is possible they could be adapted to infiltrate a FARC assassination team.Did the labels we generated for each group accuratelycapture the essence of that set of sticky notes? Groupsoften have difficulty avoiding the trap of assigning obviouslabels such as political, economic, social or foreign,domestic. Encourage the students to think beyond theseobvious categories by asking a series of Why? or Because?questions.

TECHNIQUE 2: MULTIPLE SCENARIOS

GENERATIONIn the complex, evolving, uncertain situations that intelligence analysts and decision makers must deal with, thefuture is not easily predicable. The best an analyst can do isto identify the driving forces that may determine futureoutc omes and monitor those forces as they interact tobecome the future. Scenarios are a principal vehicle fordoing this. Scenarios are plausible and sometimes provocative stories about how the future might unfold. Whenalternative futures have been clearly outlined, decisionmakers can mentally rehearse these futures and askthemselves, What should I be doing now to prepare forthese futures?Scenarios Analysis provides a framework for consideringvarious plausible futures. Trying to divine or predict a singleoutcome typically is a disservice to senior officials and decision makers. Generating several scenarios helps focus attention on the key underlying forces and factors most likely toinfluence how a situation develops. Multiple Scenarios Generation creates a large number of possible scenarios. This isdesirable to make sure nothing has been overlooked. Oncegenerated, the scenarios can be screened quickly, withoutdetailed analysis of each one. Once sensitized to these different scenarios, analysts are more likely to pay attention tooutlying data that would suggest that events are playing outin a way not previously imagined.

Task 2.Use Multiple Scenarios Generation to identify the mostplausible attack scenarios the FARC would consider inlaunching a retaliatory attack on the US homeland.Step 1: Clearly define the focal issue and the specific goalsof the futures exercise.When you have little intelligence on a specific threat butsubstantial information on the potential perpetrator, Multiple Scenarios Generation is a useful tool to scope theproblem, think creatively about potential attack scenarios,and generate actionable intelligence. In this case, the focalquestion is What are the most plausible ways the FARCwould mount an attack on the US homeland? The goal ofthe exercise is to use the four key drivers selected in the RedHat/Structured Brainstorming Exercise first to generate amultitude of possible attack scenarios and then to select thescenarios that seem the most plausible, thus deserving theattention of those responsible for thwarting or mitigatingthe consequences of such an attack.Step 2: Brainstorm to identify the key forces, factors, orevents that are most likely to influence how the issue willdevelop over a specified time period. In this case, use thefour or five key drivers, themes, or dimensions that emergedfrom Task 1, the Red Hat/Structured Brainstorming exercise.In Task 1, four key drivers emerged: the type of weapon,the motive for the attack, the most likely target of an attack,and whether outside assistance will be sought.Step 3: For each of these key drivers, define the two endsof the spectrum.For the purposes of illustration, the spectrums can bedefined as follows:A. Weapon (simple weapon such as a rifle to a highlysophisticated CBRN attack).B. Motive (retaliation for recent military operation inColombia to much broader aim to terrorize the USpopulation).C. Target (tactical attack on a US military base to thestrategic targeting of a senior Washington official).D. Partners (a do it alone operation or partnering withthe IRA).Step 4: Pair the drivers in a series of 2 2 matrices. Ifyou have four drivers, they can be combined into six pairs,

134 Chapter 12generating six different matrices. Five drivers would generate ten different matrices.In this case study, the pairs used to form the six matriceswould be: AB (weapon/motive), AC (weapon/target), AD(weapon/partner), BC (motive/target), BD (motive/partner), and CD (target/partner). The class usually is brokeninto smaller groups to work each 2 2 matrix. With sixmatrices, it usually works best to assign two matrices toeach of three groups. Be careful in assigning the matrices togive each group the opportunity to think about all of thedrivers. This can be accomplished by assigning the matricesas follows: Group 1 (AB and CD), Group 2 (AC and BD),and Group 3 (AD and BC).Step 5: Develop a story or two for each quadrant of each2 2 matrix.For example, Group 2 was asked to come up withfour stories (one story for each quadrant of the matrix)for AC (weapon/target). Their work might look likeFigure 9.2, in which the x-axis represents a tactical versus a strategic target and the y-axis represents the spectrum of simple to sophisticated weapons. In each matrix,the students have brainstormed a potential attack scenario. For example, a tactical attack using weapons ofmass destruction could involve a biological attack on thewater supply of a military base that was supporting USmilitary operations in Colombia. In another quadrant, asimple attack designed to terrorize the US populationcould be the kidnapping of the son or daughter of a chiefof police of a major metropolitan area such as Miami.The students opted to propose the kidnapping of a childbecause it was assumed a child would be a soft targetunlikely to have security protection.If one group works more quickly than the others, theinstructor can ask the group to start putting together lists ofindicators for their favorite scenarios.Students should present similar matrices for all six combinations of drivers. Once all the matrices have been presented and discussed, the class should look for themes thatemerge or seem to repeat in several of the matrices. Thesemay be more deserving of attention if similar ideas weregenerated by different groups independently. Studentsshould also discuss which of the scenarios are most deserving of the attention of US policy makers and law enforcement officials and provide reasons to support their choices.Step 6: From all the scenarios generated, select three orfour that are the most deserving of attention because they

best illustrate the range of attacks the FARC is most likely to

contemplate.After some discussion, the class can either reach consensus on the top four scenarios to consider, or it can voteto identify the most attention-deserving scenarios. Thegroup should endeavor to select a set of scenarios thatbest defines the most likely attack space. When two scenarios appear to be very similar, then they should becombined.The standard rule is to give participants one vote forevery three things being considered. In this instance, iftwenty-four different scenarios were generated, each participant would be allowed to vote for the eight scenarios heor she deemed most deserving of attention. The scenarioswith the most votes would be the lead candidates to presentto the customer.Some sample scenarios that might be generated includethese: Use rompas to attack USSOUTHCOMs headquartersin Miami. Conduct a sniper attack on US counterdrug officialsor military officers associated with operations inColombia. Contaminate the food supply or water supply of a USmilitary base supporting anti-FARC operations inColombia. Enlist the support of the IRA to conduct a targetedbombing aimed at the Colombian ambassador tothe United Nations or the Colombian ambassadorin Washington, D.C. The FARC assassins could bedressed as Colombian military officers with IRAoperatives providing logistic support. Kill as many American drug users as possible toterrorize the US population and send a clear messagenot to fool with the FARC and Colombia.Step 7: Consider whether one of the final scenarios youselect might be described as a wild card (low-probability/high-impact) or nightmare scenario.Although plausibility is a major criterion for selectingthe most attention-deserving scenarios, there are timeswhen a highly unlikely scenario still should be included inthe final set of four because albeit unlikely, the consequences for the United States would be severe and seniorpolicy makers should be alerted to the possibility, howeverremote. An illustration of how four scenarios might beselected is provided in Figure 12.4.

Figure 12.4 Multiple Scenarios Generation: Selecting the Most Attention-Deserving Scenarios of a FARC Attack on the USHomeland

Selecting ScenariosWeapon/MotiveScenario A

Story 3

Story 2

Scenario B

Target/Partner

Weapon/TargetStory 5

Story 6

Scenario B

Story 8

Motive Partner

Story 9

Story 10

Story 13

Story 14

NightmareScenario

Story 12

Story 15

Scenario C

Scenarios deserving the most attention

Nightmare Scenario

136 Chapter 12Some possible wildcard or nightmare scenarios thatmight be generated from this exercise would be these: A decision by the FARC leadership to pay drugdistributors within the United States to spike illegaldrugs with a highly toxic substance and distributethem in communities that surround US militarybases that have deployed troops to Colombia. An attempt by FARC members to assassinate theadministrator or assistant administrator of the DrugEnforcement Administration.Analytic Value Added: Did the technique help usgenerate a robust set of potential scenarios to consider?The Multiple Scenarios Generation technique can be apowerful tool to generate new ideas and attack scenariosthat might never have been considered as part of atraditional analysis.Did we discover new scenarios that we probably wouldnot have imagined if we had not used this particulartechnique? The technique forces analysts to reframe thequestion in many different ways; often the combinationsprompt totally new ways of defining the threat environment. The approach should give analysts more confidencethat they have captured the entire threat space and someassurance that they are less likely to be surprised by howevents actually play out.Did similar themes emerge from different matriceseven though different pairs of drivers were being considered? When similar themes emerge from more than onematrix, analysts can be more confident that a key dimensionhas been captured that may require the attention of the decision makers.Were the final scenarios selected both plausible and themost deserving of attention? The exercise helps analystsavoid the frequent trap of coming to premature closure andfocusing on the one or two plausible scenarios that first cometo mind. In selecting the most attention-deserving scenarios, itis always helpful to work from a previously agreed upon set ofkey criteria.TECHNIQUE 3: INDICATORSIndicators are observable or deduced phenomena that can beperiodically reviewed to help track events, distinguishbetween competing hypotheses, spot emerging trends, andwarn of unanticipated change. An indicators list is a preestablished set of actions, conditions, facts, or events whosesimultaneous occurrence would argue strongly that aphenomenon is present or a hypothesis is correct. The

identification and monitoring of indicators are fundamental

tasks of intelligence analysis because they are the principalmeans of avoiding surprise. In intelligence analysis, indicatorsare often described as predictive indicators that look forward.In the law enforcement community, indicators are used toassess whether a targets activities or behavior are consistentwith an established pattern or lead hypothesis. These are oftendescribed as descriptive indicators that look backward.Preparation of a detailed indicator list by a group ofknowledgeable analysts is usually a good learning experiencefor all participants. It can be a useful medium for anexchange of knowledge between analysts from differentorganizations or those with different types of expertiseforexample, counterterrorism or counterdrug analysis, infrastructure protection, and country expertise. The indicatorlist can become the basis for conducting an investigation ordirecting collection efforts and routing relevant informationto all interested parties. Identification and monitoring ofindicators or signposts that a scenario is emerging can provide early warning of the direction in which the future isheading, but these early signs are not obvious. The humanmind tends to see what it expects to see and to overlook theunexpected. Indicators take on meaning only in the contextof a specific scenario with which they have been identified.The prior identification of a scenario and associated indicators can create an awareness that prepares the mind to recognize and prevent a bad scenario from unfolding or help agood scenario to come about.Task 3.Create separate sets of indicators for each alternative scenario that was generated in Task 2.Step 1: Work alone, or preferably with a small group, tobrainstorm a list of indicators for each scenario.For the purposes of illustrating this case study, we havegenerated indicators for the following four scenarios:A. Kill as many American drug users as possible toterrorize the US population and send a clear messagenot to fool with the FARC and Colombia.B. Use rompas to attack USSOUTHCOMs headquartersin Miami.C. Enlist the support of the IRA to conduct a targetedbombing aimed at the Colombian ambassador to theUN or the Colombian ambassador in Washington,D.C. The FARC assassins could be dressed asColombian military officers with IRA operativesproviding logistic support.

Colombias FARC Attacks the US Homeland 137

Table 12.5 FARC Attack on the US Homeland: Indicators List

Number

IndicatorScenario A: FARC poisons cocaine to terrorize US population.

A-1

DEA chemists see increase in reports of cocaine laced with toxic substance in several major cities.

A-2

Border police report fewer seizures of bulk cash heading south.

A-3

Informants report a buzz on the street to avoid purchases of cocaine.

A-4

There is an unusual spike in reported drug overdoses in several cities.

A-5

Drug informants talk of special payoffs to local drug distributors.

A-6

The FARC posts statements on the Internet saying it will retaliate against the United States for supporting Colombian military strikesagainst FARC guerrillas.

A-7

Urban drug treatment centers receive queries about what substances are most often mixed with cocaine to increase volume andprofits.

A-8

Drug mules are carrying smaller amounts of cash back to Colombia.

A-9

Communications increase between US drug distributors and Latin America.

A-10

Local US law enforcement reports increased bulk purchases of poisonous substances such as arsenic.Scenario B: FARC uses rompas to launch mortar attack on USSOUTHCOM headquarters in Miami.

B-1

USSOUTHCOM security reports suspicious cars seen loitering on streets in vicinity of headquarters.

B-2

Analysts looking at FARC Internet site report claims that FARC will make the US military pay for its misdeeds.

B-3

Hispanic males are observed taking photos of USSOUTHCOM headquarters from a distance.

US government sources report that Venezuela has provided documents and passports to FARC operatives to facilitate theirinternational travel.

B-6

Recent FARC guerrilla defectors mention a mock-up building in the Amazon is being used for target practice with rompas.

B-7

USSOUTHCOM employees tell their supervisors that they are being approached by strangers and asked who works where in the complex.

B-8

An increased number of mortar attacks using rompas is reported in Colombia.

Scenario C: FARC assassinates Colombian ambassadors with IRA support.

C-1

There are reports of FARC meetings and communications with the IRA.

C-2

FARC publishes open letter to the US president stating that FARC will not be intimidated by actions of the US military.

C-3

Kidnappings of field-grade Colombian military officers in Colombia surge.

C-4

There are intelligence reports of IRA hit squads being dispatched to North America.

C-5

Defecting FARC guerrillas report talk of a big operation up north.

C-6

Colombians in New York report suspicious persons loitering outside the mission offices.

C-7

FARC Internet site claims that FARC will make the US military pay for its misdeeds.

C-8

Suspected FARC members entering the United States are found in possession of Colombian military uniforms.

C-9

A FARC informant reports that a special squad is being formed for a major operation.Scenario D: Marijuana laced with poison kills many in the vicinity of US military bases.

D-1

Street informants report a buzz in the Hispanic community that the FARC is planning a special operation in the United States.

D-2

Local drug dealers say they are being surveyed by people up their distribution chain asking for details on their user populations.

D-3

Local health officials report an increase in drug-related deaths among teenagers.

D-4

DEA chemists report an increase in marijuana laced with arsenic and other toxic substances.

D-5

Street informants report that their suppliers are talking about making easy money.

D-6

A new theme emerges on Facebook that marijuana consumption may be more dangerous than most suspect.

D-7

Analysts note postings by FARC on its Internet site stating that the United States will pay dearly for violating Colombian sovereignty.

D-8

Drug users become increasingly anxious that the drugs they might purchase could be contaminated.

138 Chapter 12D. Pay drug distributors within the United States tolace marijuana sold mostly to teenagers with ahighly toxic, lethal substance and distribute it tocommunities that surround US military bases thathave deployed troops to Colombia.A brainstorming session generated the indicators shownin Table 12.5 for each scenario.Step 2: Review and refine each set of indicators, discarding any that are duplicative within any given scenario andcombining those that are similar.In this example, C-5 and C-9 are similar and merit combination into a new indicator: FARC informants or defectors report that a special squad is being formed for a majoroperation up north. Similarly, C-2 and C-7 should be combined to state: FARC warns the United States publicly thatit will no longer tolerate American interference in Colombias internal affairs, particularly with its military forces.Step 3: Examine each indicator to determine whether itmeets the following five criteria. Discard those that arefound wanting.1. Observable and collectible. There must be somereasonable expectation that, if present, the indicatorwill be observed and reported by a reliable source.If an indicator will be used to monitor change overtime, it must be collectible over time.2. Valid. An indicator must be clearly relevant to theendstate the analyst is trying to predict or assess, andit must be inconsistent with all or at least some of thealternative explanations or outcomes. It must accuratelymeasure the concept or phenomenon at issue.3. Reliable. Data collection must be consistent whencomparable methods are used. Those observingand collecting data must observe the same things.Reliability requires precise definition of the indicators.4. Stable. An indicator must be useful over time toallow comparisons and to track events. Ideally, theindicator should be observable early in the evolutionof a development so that analysts and decisionmakers have time to react accordingly.5. Unique. An indicator should measure only onething and, in combination with other indicators,should point only to the phenomenon being studied.Valuable indicators are those that are not onlyconsistent with a specified scenario or hypothesisbut are also inconsistent with all other alternativescenarios.

In this case study:

A-8 should be dropped from the list because it failsthe test as an observable and collectible indicator.Few mules are intercepted taking money backto Colombia, and it would be very difficult toknow if the total volume of cash moving from theUnited States to the drug lords in Colombia wasdiminishing. A-9 fails two tests: it is neither unique norvalid. It needs to be rewritten as follows: Newcommunications are identified between FARCleaders and drug distributors in the United States. B-4 is not valid because it lacks specificity. It shouldbe rewritten to state: Known FARC sympathizers arereported purchasing suspicious quantities of liquidpetroleum gas canisters. D-8 fails the test of an observable and collectibleindicator. It should be rewritten to state: Informantsreport that drug users are complaining that the drugsthey are purchasing may be contaminated.A revised list of indicators is presented in Table 12.6.Analytic Value Added: What new or otherwiseimplicit criteria did the indicators process expose?Students answers will vary according to the specifics of theirindicator sets. However, a good indicator set should help theanalyst identify explicit criteria for tracking and judging thecourse of events. Often it is useful to note that it is easyto generate indicators for some scenarios, such as amortar attack on USSOUTHCOM headquarters thatinvolves surveillance activity and the acquisition orimportation of weaponry, and difficult for others, such as anassassination plot.Do the indicators prompt additional areas for collection? This will vary according to the students indicatorsets. However, a well-conceived set of indicators shouldbecome the basis for directing collection efforts and forrouting relevant information to all interested parties in several US government agencies.TECHNIQUE 4: INDICATORS VALIDATORTMThe Indicators ValidatorTM is a simple tool for assessing thediagnostic power of indicators. Once an analyst has developeda set of attention-deserving alternative scenarios or competinghypotheses, the next step is to generate indicators for eachscenario or hypothesis that would appear if that particular

Colombias FARC Attacks the US Homeland 139

Table 12.6 FARC Attack on the US Homeland: Revised Indicators

Number

IndicatorScenario A: FARC poisons cocaine to terrorize US population.

A-1

DEA chemists see increase in reports of cocaine laced with toxic substance in several major cities.

A-2

Border police report fewer seizures of bulk cash heading south.

A-3

Informants report a buzz on the street to avoid purchases of cocaine.

A-4

There is an unusual spike in reported drug overdoses in several cities.

A-5

Drug informants talk of special payoffs to local drug distributors.

A-6

The FARC posts statements on the Internet saying it will retaliate against the United States for supporting Colombian militarystrikes against FARC guerrillas.

A-7

Urban drug treatment centers receive queries about what substances are most often mixed with cocaine to increase volumeand profits.

A-8

New communications are identified between FARC leaders and drug distributors in the United States.

A-9

Local US law enforcement reports increased bulk purchases of poisonous substances such as arsenic.Scenario B: FARC uses rompas to launch mortar attack on USSOUTHCOM headquarters in Miami.

B-1

USSOUTHCOM security reports suspicious cars seen loitering on streets in vicinity of headquarters.

B-2

Analysts looking at FARC Internet site report claims that FARC will make the US military pay for its misdeeds.

B-3

Hispanic males are observed taking photos of USSOUTHCOM headquarters from a distance.

US government sources report that Venezuela has provided documents and passports to FARC operatives to facilitate theirinternational travel.

B-6

Recent FARC guerrilla defectors mention a mock-up building in the Amazon is being used for target practice with rompas.

B-7

USSOUTHCOM employees tell their supervisors that they are being approached by strangers and asked who works where in thecomplex.

B-8

An increased number of mortar attacks using rompas is reported in Colombia.

Scenario C: FARC assassinates Colombian ambassadors with IRA support.

C-1

There are reports of FARC meetings and communications with the IRA.

C-2

FARC warns the United States publicly that it will no longer tolerate American interference in Colombias internal affairs,particularly with its military forces.

C-3

Kidnappings of field-grade Colombian military officers surge.

C-4

There are intelligence reports of IRA hit squads being dispatched to North America.

C-5

FARC informants or defectors report that a special squad is being formed for a major operation up north.

C-6

Colombians in New York report suspicious persons loitering outside the mission offices.

C-7

Suspected FARC members entering the United States are found in possession of Colombian military uniforms.Scenario D: Marijuana laced with poison kills many in the vicinity of US military bases.

D-1

Street informants report a buzz in the Hispanic community that the FARC is planning a special operation in the United States.

D-2

Local drug dealers say they are being surveyed by people up their distribution chain asking for details on their user populations.

D-3

Local health officials report an increase in drug-related deaths among teenagers.

D-4

DEA chemists report an increase in marijuana laced with arsenic and other toxic substances.

D-5

Street informants report that their suppliers are talking about making easy money.

D-6

A new theme emerges on Facebook that marijuana consumption may be more dangerous than most suspect.

D-7

Analysts note postings by FARC on its Internet site stating that the United States will pay dearly for violating Colombiansovereignty.

D-8

Informants report that drug users are complaining that the drugs they are purchasing are contaminated.

140 Chapter 12scenario were beginning to emerge or that particular hypothesis were true. A critical question that is not often asked iswhether a given indicator would appear only for the scenarioor hypothesis to which it is assigned or also in one or morealternative scenarios or hypotheses. Indicators that couldappear under several are not considered diagnostic, suggestingthat they are not particularly useful in determining whether aspecific scenario is beginning to emerge or a particular hypothesis is true. The ideal indicator is highly likely for the scenarioto which it is assigned and highly unlikely for all others.

Could appear Is unlikely to appear Is highly unlikely to appearIndicators developed for their particular scenario, thehome scenario, should be either highly likely or likely.If the software is unavailable, you can do your own scoring. If the indicator is highly likely in the home scenario,then in the other scenarios, Highly likely is 0 points.

Task 4.Use the Indicators ValidatorTM to assess the diagnosticity ofyour indicators.Step 1: Create a matrix similar to that used for Analysis ofCompeting Hypotheses. This can be done manually or byusing the Indicators Validator TM software. ContactGlobalytica, LLC at THINKSuite@globalytica.com or go tohttp://www.globalytica.com to obtain access to theIndicators ValidatorTM software if it is not available on yoursystem. List the alternative scenarios along the top of thematrix and the indicators that have been generated for eachof the scenarios down the left side of the matrix.Step 2: Moving across the indicator rows, assess whetherthe indicator for each scenario Is highly likely to appear

Likely is 1 point. Could appear is 2 points. Unlikely is 4 points. Highly unlikely is 6 points.If the indicator is likely in the home scenario, then in theother scenarios, Highly likely is 0 points. Likely is 0 points. Could appear is 1 point. Unlikely is 3 points. Highly unlikely is 5 points.Step 3: Tally up the scores across each row, as shown inTable 12.7, and then rank order all the indicators.

144 Chapter 12highly likely to appear in all scenarios. Most indicatorswill fall somewhere in between.Step 5: The indicators with the most highly unlikely andunlikely ratings are the most discriminating and should beretained.Step 6: Indicators with no highly unlikely or unlikely ratingsshould be discarded.Step 7: Use your judgment as to whether you shouldretain or discard indic ators that score fewer points.

Generally, you should discard all indicators that have highly

unlikely or unlikely ratings. In some cases, an indicator maybe worth keeping if it is useful when viewed in combinationwith several other indicators.In this illustration, the following indicators would be discarded: B-5 (4 points), B-1 (2), C-2 (2), B-2 (1), A-6 (0), andD-7 (0). Although D-1 has a score of only 5 points, it is notdiscarded because it had an unlikely rating in the row.Step 8: Once nondiscriminating indicators have been eliminated, regroup the indicators under their home scenario(Table 12.9).

Table 12.9 FARC Attack on the US Homeland: Rank Ordering of the Indicators on the Basis of Diagnosticity byScenarioNumber

Indicator

Scenario A

Scenario B

Scenario C

Scenario D

Score

Scenario A: FARC poisons cocaine to terrorize US population.

A-1

DEA chemists see increase in reports of cocaine

laced with toxic substance in several major cities.

HL

HU (6)

HU (6)

C (2)

14

A-3

Informants report a buzz on the street to avoid

purchases of cocaine.

HL

HU (6)

HU (6)

C (2)

14

A-4

There is an unusual spike in reported drug overdoses

in several cities.

HL

HU (6)

HU (6)

HL (0)

12

A-5

Drug informants talk of special payoffs to local

drug distributors.

HU (5)

HU (5)

C (1)

11

A-7

Urban drug treatment centers receive queries about

what substances are most often mixed with cocaineto increase volume and profits.

Step 9: If a large number of indicators for a particular

scenario have been eliminated, develop additionalandmore diagnosticindicators for that scenario.

scenario. In this instance, two more indicators have been

generated and their diagnosticity examined, as shown inTable 12.10.

Step 10: Check the diagnostic value of any new indicators

by applying the Indicators ValidatorTM to them as well.In this illustration, Scenario B has only five indicatorsremaining, suggesting that at least two more indicatorsare needed to ensure an adequate number for that

The Colombian government finds maps of Miami and

Do these indicator lists provide useful leads for alerting FBI field offices and state and local fusion centers ofplausible, potential emerging threats? Yes, the indicatorsare sufficiently specific to provide operationally useful guidance to field offices or fusion centers.Are they focused enough to generate specific collection requirements, giving federal, state, local, and tribalofficials a more concrete idea of what to look for? Yes, thetechnique has generated a robust set of concrete indicatorsthat provide effective guidance to the field.KEY TAKEAWAYS When analysts have little data and a mandate toanticipate a potential terrorist attack, often the

best approach is to use imagination techniques to

generate a large number of possible outcomes.Then pare this list down by identifying the mostplausible or attention-deserving options. Over thelong run, this is likely to be a much more efficientway to approach problem solving, especially if thekey goal is to avoid surprise. Analysts should always assess the diagnosticityof their indicators and immediately discard thosethat fail the test. Failure to do so can give ananalyst a false sense of validation. It can alsoresult in tasking collectors to invest valuableresources in acquiring information that in thelong run does not aid in analysis or help solve theproblem.

NOTES 1. The description of Red Hat Analysis in this case was takenfrom the first edition of Structured Analytic Techniques forIntelligence Analysis. A more robust approach for conducting Red

Foresight Quadrant Crunching

13 Understanding Revolutionary Organization 17 November

nalysts often deal with ambiguous situations in which

information is limited or unconfirmed, as was thecase with the investigation of 17 November (17N). In thesesituations, diagnostic techniques such as Simple Hypothesescan help explore alternative views and hypotheses systematically. Challenge techniques such as What If? Analysis(with the corollary technique of Indicators) helps analyststhink through the viability of the analysis and its implications. Imagination techniques such as Foresight QuadrantCrunchingTM can help challenge assumptions and explorethe implications of specific hypotheses.TECHNIQUE 1: MULTIPLE HYPOTHESISGENERATION: SIMPLE HYPOTHESESHypothesis Generation is a category of techniques fordeveloping alternative potential explanations for events,trends, or activities. Hypothesis Generation is part of anyrigorous analytic process because it helps the analyst avoidcommon pitfalls such as coming to premature closure orbeing overly influenced by first impressions. Instead, ithelps the analyst think creatively about a range of possibilities. The goal is to develop an exhaustive list of hypothesesthat can be scrutinized and tested over time against bothexisting evidence and new data that may become availablein the future.This case is well suited to Simple Hypotheses, whichemploys a group process for thinking creatively about arange of possible explanations for 17Ns motives and identity. These explanations, in turn, help expand the thinkingof investigators who are working to apprehend and counter the group, as well as security officers working to protect US officials in Athens. Engaging a small group helps

to generate a large list of possible hypotheses for further

investigation. Simple Hypotheses is a method best used bya diverse group that includes expertise from multiple perspectives and stakeholders. This technique includes anexercise in Structured Brainstorming.In a classroom or workplace setting, this technique canbe used by breaking participants into groups to work inseparate breakout sessions or by conducting a simpler classor conference roombased version. For the breakoutgroupbased version, simply assign groups the task below.For the classroom-based version, have participants silentlywrite down possible hypotheses, list those hypotheses on awhiteboard, group the hypotheses, and then refine thehypotheses.Task 1.Use Simple Hypotheses to explore all possible explanationsfor what kind of group 17 November is.Step 1: Ask each member of the group to write down onseparate 3 5 cards or sticky notes up to three plausiblealternative hypotheses or explanations. Think broadly andcreatively, but strive to incorporate the elements of a goodhypothesis that is Written as a definite statement Based on observations and knowledge Testable and falsifiable Composed of a dependent and an independentvariableStep 2: Collect the cards and display the results.Consolidate the hypotheses to avoid duplication.

147

148 Chapter 13A consolidated set of hypotheses might look like Table 13.4.Step 3: Aggregate the hypotheses into affinity groups andlabel each group.Consider multiple ways to display the affinity groups. Inthis case, the hypotheses may be grouped by the issue ofautonomy, addressing the question of whether 17N workedalone or in collaboration with other violent groups active inGreece and Europe. Another important consideration ismotive, and whether 17N was truly a manifestation of radical politics or whether it was alsoor insteada criminalenterprise.Step 4: Use problem restatement and consideration of theopposite to develop new ideas. Problem Restatement: Why did it take twenty-sevenyears to capture the members of 17N? Consideration of the Opposite: 17N benefittedfrom official protection. 17N benefitted from thelimitations of Greek police and security services.17N evaded detection because its attacks wereso low-tech. All of these ideas have implicationsabout 17Ns identity and motive and help expandexplanations for what the group might havebeen. Also consider whether 17Ns longevitymight be due to its evolutionary nature. Was 17Nconsistently the same thing for the length of itsperiod of activity? Might its motives, composition,and objectives have changed over time?Step 5: Update the list of alternative hypotheses.Problem restatement augments the list of hypotheses byincluding the possibility of government collusion or protection. It also raises the possibility that the groups motive,objectives, and identity evolved over time.Step 6: Clarify each hypothesis by asking Who? What?How? When? Where? and Why?

Table 13.4 Simple Hypotheses Generation:

Examples of Consolidated 17N Hypotheses 17N started out as a far-left Greek terrorist group and thenbecame a criminal enterprise. 17N was always a criminal enterprise masquerading as aterrorist group. 17N was part of a larger pan-European violent extremistmovement.

Make a list of each of the categories. Step back and consider how each list could be augmented. Who and Whatsuggest possible identities: an autonomous group of Greekviolent extremists, a criminal enterprise, or a subgroup of alarger regional violent extremist movement? Whenaddresses the issue of whether 17N had a consistent identity, composition, and objectives over the years, or whetherit evolved. Where addresses the theater of operations: Allclaimed attacks were in Athens, but could there have beenactivity elsewhere not credited to the group? Howaddresses the longevity of the groups success. If it evadeddetection for so many years because of the low-tech natureof its attacks, what does that also say about what it was?Why addresses motive: to inspire political revolution, tomake money, to advance political goals of invested officials?Refine this list to make the categories as mutually exclusiveas possible. This helps clarify the hypotheses.Step 7: Select the most promising hypotheses for furtherexploration. 17N is a Greek violent far-left group that, for a periodof time, worked in collaboration with other violentgroups, Greek and/or foreign, to inspire a Marxistrevolution. 17N is a Greek violent extremist group working inconjunction with criminal enterprises, in Greece andregionally, both for monetary gain and to advance apolitical agenda. 17N is a group manipulated by or influenced byGreek political officials to engage in dirty politics inAthens.Analytic Value Added: Did using the technique helpyou challenge conventional wisdom about the group andits motives? The technique generated several new ways tothink about the group, suggesting different motives inparticular. This is important because the analyst now will belooking for additional indicators that can prove or disproveeach of the hypotheses.Did it reveal ideas or concepts that you might havemissed if you had engaged in conventional brainstormingonly? The technique raised the possibility that 17N mightbe operating entirely or partially for criminal motives andmay have evolved over timeideas that certainly wouldrequire more research.Was it difficult to select those hypotheses thatdeserved the most attention? As themes emerged from theStructured Brainstorming process, it was helpful to use

Understanding Revolutionary Organization 17 November 149

them to develop an expanded set of hypotheses that

reflected the themes. Selecting the most important hypotheses is easier if the analysts work from a specific set of criteria that defines what makes a good hypothesis.TECHNIQUE 2: WHAT IF? ANALYSISWhat If? Analysis posits that an event has occurred with thepotential for a major positive or negative impact and thenexplains how it came about. This technique is best usedwhen analysts are having difficulty getting others to focuson the potential for, or the consequences of, a high-impact/low-probability event to occur. It is also appropriate when acontroversial mindset is well ingrained. In the late 1990s,US security officials continued to be concerned about thepotential for an attack by the group. Because What If ?Analysis shifts the focus from whether an event could occurto how it might happen, the techn ique allows analyststo make more informed judgments about whether suchdevelopmentseven if unlikelymight actually occur.Task 2.Assume you are an analyst working at the US Embassy inAthens in 1999. Use What If? Analysis to explore the viabilityand likely nature of another attack on a US official in Athensby 17N. It had been eight years since 17N had killed a USofficial. The rocket shot at the US Embassys back gate in1996 spoke to intent, but also to limited capabilities. Securityat the US Embassy in Athens was at an all-time high. Not

only did senior officers at the embassy have armored vehicles

and robust protection, but they, and all embassy staff, wereadvised to vary their routes and lower their profiles. What if17N had managed to kill a US official despite this highsecurity? What would it look like? What would it suggest?Step 1: Begin by assuming what could happen has actuallyoccurred. In December 1999, 17N has attacked yet anotherUS official in Athens despite enhanced security.Step 2: Develop a chain of argumentationbased on evidence and logicto explain how this event could havecome about. Create more than one scenario or chain ofargument. In Figure 13.1 we have described how one ofthese scenarios might be portrayed. Scenario A: 17N shoots US military officer Scenario B: 17N bombs US Embassy vehicle inAthens Scenario C: 17N assassinates US political counseloras he leaves for workStep 3: Generate a list of indicators for each scenario thatwould point to the events starting to play out. A sample setof indicators is provided in Table 13.5.Step 4: Assess the level of damage or disruption thatwould result from each scenario and how difficult it wouldbe to overcome.

Figure 13.1 What If? Analysis Scenario: 17N Shoots US Military OfficerIt is 1999, the peak of the NATO campaign in the Balkans. The majority of Greeks feel a religio-ethnic affinity with theSerbs, and vehemently oppose the strikes and any overt support given to the Bosnians and Kosovars by the West. Popularprotests make it clear that this is an issue that resonates with a large swath of the Greek people. 17N sees an opportunityto advance its agenda and decides to target a US military officer with NATO ties. Senior US military officers or defenseattachs affiliated with the embassy and stationed in Athens are afforded careful security protection by both DoD andDiplomatic Security. They have armored vehicles and, sometimes, security escorts, and their drivers carefully vary theirroutes. All vehicles entering the embassy compound are screened for explosives, and the building itself is inaccessible tooutsiders. Their residences and families are similarly protected. Lower-level officers also receive security training and areinstructed to report any signs of surveillance or unusual behavior. All local embassy hires are carefully screened.Despite this high security, 17N is still focused on targeting an American military officer and making a statement aboutwhat the group perceives to be immorality of a US-backed NATO campaign. It decides to monitor the major restaurantsand tourist venues in central Athens, where American Embassy personnel are known to congregate, but finds that thereare too many people and it is too hard to distinguish which Americans might have military affiliations. It surveils all carscoming and going from the embassy compound and finds that some lower-level officers with less security detail are notalways careful about varying their commutes to and from work, especially after several months at post.One young man in particular, who drives an old model Honda, takes the same major thoroughfare to the embassy fromhis residence every day. His short haircut suggests he might have a military affiliation. 17N decides it is their best shotand plots a drive-by shooting timed for the peak morning rush hour. It prepares the proclamation in advance, accusingthe nameless American of being centrally involved in the incursion into Serbian sovereign space.

150 Chapter 13

Table 13.5 What If? Analysis: Indicators of

Military Officer Scenario Starting to Unfold Possible surveillance activity reported by embassy securitypersonnel guarding the embassy compound gates Reports of unidentified or suspicious vehicles being parked invicinity of embassy residences 17N posts statements describing US military involvement inBosnia as inhumane and politically biased Greek police inform the embassy that they have picked up abuzz on the streets that a terrorist attack is being planned Proactive embassy security personnel surveil traditional 17Nambush sites and observe suspicious activity by two men whomay be casing the site

For the military officer scenario, the killing would signal that 17N was still active, and security would be heightened not only for US officials but also other for diplomaticposts in Athens and the Greek government and privatesector.Step 5: Rank the scenarios in terms of which deserves themost attention by taking into consideration the difficulty ofimplementation and the potential severity of the impact.Depending on how the other scenarios are constructed,a likely ranking in descending order of difficulty of implementation would be: Scenario C: 17N assassinates US political counselornear US Embassy Scenario A: 17N shoots US military officer en routeto work Scenario B: 17N bombs US Embassy vehicle in AthensAnalytic Value Added: Did the technique help yougenerate new ways of thinking about the problem? Thetechnique moved the conversation beyond the debate overwhether 17N is still a viable terrorist organization, but it didnot generate new ideas regarding what type of attack mightbe launched. It did, however, provide insight into thelikelihood of a particular type of attack based on degree ofdifficulty.Did it help you assess how difficult each scenariowould be to carry out? By working ones way step by stepthrough each scenario, it is easier to assess how 17N is mostlikely to launch each attack and assess what is required foreach to succeed.

Did the exercise indicate that any new security mea

sures should be implemented? By describing in somedetail how an attack would be launchedworking from theplanning stages to the actual attackit made it easier toanticipate what types of security measures would be neededto forewarn officials that planning for such an attack may beunderway. Generating indicators for a scenario can be adaunting task, particularly when so little is known about thegroup or its key membersbut the process helps stimulatea useful list of things that might be observed and reported.TECHNIQUE 3: FORESIGHTQUADRANT CRUNCHINGTMQuadrant CrunchingTM combines the methodology of aKey Assumptions Check with Multiple ScenariosGeneration to generate an array of alternative scenarios orstories. Two versions of Quadrant Crunching TM haveevolved in recent years; each technique serves a differentanalytic function:In Classic Quadrant CrunchingTM, the analyst beginswith a lead hypothesis (an example of a lead hypothesiswould be, A criminal group has penetrated a large corporate database to steal Personal Identity Information[PII]), breaks the lead hypothesis into its componentparts (criminal group/steal PII); flips the assumptioninherent in each segment (noncriminal group/alternativemotive); and brainstorms contrary dimensions or explanations (usually one to three) consistent with each flippedassumption (business competitor or foreign country, todownload corporate data or to alter corporate information).The analyst then arrays the contrary dimensions or explanations in a 2 2 matrix, generating new and uniqueattack scenarios in each quadrant (Business competitorpenetrates database to download corporate data, Businesscompetitor penetrates database to alter corporate information, Foreign country penetrates database to download corporate data, and Foreign country penetrates database toalter information.) As more dimensions of the problem areconsidered, the number of potential scenarios increasesrapidly and the chances of being surprised by a new andunanticipated development diminish.Classic Quadrant CrunchingTM differs from multiple scenarios analysis in two ways: (1) the focus is on ways thingscould happen other than what is generally expected, and(2) the technique relies on contrary dimensions versusspectrums to define the endpoints of the x- and y-axes.

Understanding Revolutionary Organization 17 November 151

The Foresight Quadrant CrunchingTM technique differs from Classic Quadrant CrunchingTM in that the focus ison all of the ways something could happen, not just whatmight be different. In this version of the technique, the leadhypothesis dimensions are included in the analysis. Foresight Quadrant CrunchingTM is similar to Classic QuadrantCrunchingTM, however, in that both use contrary dimensions versus spectrums to define the endpoints of the xand y-axes.To use our previous example again, the analyst beginswith a lead hypothesis (A criminal group has penetrated alarge corporate database to steal Personal Identity Information [PII]), breaks the lead hypothesis into its componentparts (criminal group/to steal PII); flips the assumptioninherent in each segment (noncriminal group/alternativemotives); brainstorms contrary dimensions (usually fromone to three) consistent with the flipped assumption (business competitor or foreign country, to download corporatedata or to alter corporate information); and then lists allpossible combinations, comprising nine different attackscenarios:

1. Criminal group penetrates database to steal PII.

2. Criminal group penetrates database to download

corporate data.

3. Criminal group penetrates database to alter

corporate information.

4. Business competitor penetrates database to steal

PII.

5. Business competitor penetrates database to

download corporate data.

6. Business competitor penetrates database to alter

corporate information.

7. Foreign government penetrates database to steal

PII.

8. Foreign government penetrates database to

download corporate data.

9. Foreign government penetrates database to alter

corporate information.

The Foresight Quadrant CrunchingTM technique is particularly applicable to the 17N case because (1) little wasknown about the identity of the group members or theirplans while they were active, and (2) in several cases onlyone credible alternative dimension merited the analysts

attention. Foresight Quadrant CrunchingTM helps the

analyst identify and challenge key assumptions that mayunderpin the analysis while generating a comprehensiveand mutually exclusive array of credible scenarios to helpinvestigators focus on the most likely types of attacks toanticipate.Task 3.It is now 2001, and you are an analyst based in the USEmbassy in Athens, supporting the ongoing investigation of17N. The embassy is beginning to focus its attention on preparing for the Olympic Games in Greece in 2004. UseForesight Quadrant CrunchingTM to brainstorm all possibleways 17N might pose a serious threat to the American community.Step 1: State your lead hypothesis.This hypothesis should reflect either the analytic consensus regarding the most likely means of attack or the currentconventional wisdom, which usually reflects how suchattacks have been launched in the past. 17Ns attacks againstAmerican targets traditionally were assassinations of USgovernment or military officials using a signature 17Nhandgun. For this exercise, we will use the following as ourlead hypothesis: a 17 November operative will shoot a USofficial in Athens prior to the Olympic Games in 2004.Step 2: Break the lead hypothesis down into its component parts based on the journalists list of Who? What?How? When? Where? and Why?Step 3: Identify which of these components are most critical to the analysis.Step 4: For each of the critical components, identify eitherone or three contrary dimensions in a table, as shown inTable 13.6.Six key components were identified in this exerciseonefor each of the five Ws and H questions. Three of the keycomponents (not shaded in Table 13. 6) deserve serious discussion and analysis because the contrary dimensions couldpose significant new challenges for how best to protect USofficials from a 17N attack before and during the 2004Olympics. Who? Historically, 17N has only targeted individualsdeemed guilty of crimes against the Greek peopleor nation: US, Greek, European, and Turkish military

152 Chapter 13

Table 13.6 Foresight Quadrant CrunchingTM: Contrary Dimensions

Key Assumptions

Lead Hypothesis

Contrary Dimension

Who? (target)

US official

Tourists attending the Olympics

What? (tactics)

Assassination

Hostage taking or kidnapping

How? (weapon)

Shooting with signature weapon

Remote-control bomb

When? (timing)

Before the August 2004 Olympics

During an Olympic event

Where? (location)

In metropolitan Athens

Outside Athens (including other Olympic

venues)

Why? (motives)

To advance extreme political ideology

Protest holding the Olympics in Greece

officers and diplomats, as well as members of the

Greek wealthy elite. With the scheduling of theOlympics in Greece, however, 17N might decideto change tactics and target those attending theOlympics in order to gain more publicity for itsmovement. 17N might also conclude that it would bemore likely to succeed if it shifted to new tactics thatwould require a different type of security mitigationstrategy than what had been previously practiced bythe police. What? 17N has operated with different modioperandi over the years. The nature of 17N attackshas evolved over time, increasing in sophisticationand daring, from shootings on abandoned streetslate at night to makeshift rockets launched on busyintersections in downtown Athens in broad daylight.There is no reason not to explore the possibilitythat its tactics may continue to change, advancingto kidnappings or hostage taking, especially if thegroup sees an Olympics attack as helping them gaininternational publicity. Where? The 2004 Olympics involves venues acrossGreece; 17N could conclude that sites outsideAthens could be more vulnerable targets. Although17N would be launching an attack outside of itshistorical comfort zonegreater metropolitanAthensit might conclude that the benefitsoutweighed the risks.The remaining questions are poorer candidates for aForesight Quadrant CrunchingTM exercise because (1) thealternatives to the lead hypothesis are not likely to have significant impact on how the analysis is conducted, or (2) thealternatives would not require different security strategies tomitigate the threat.

Rockets

Protest Greek ties to the

United States

How? The primary concern is whether a lethal attack

might occur, not the type of weapon that would beused to kill people. 17N only carried out three typesof attacks during its twenty-seven years of activity:shootings with its signature handguns, bombings,and rocket attacks. This speaks both to the groupscapabilities and to its intent. 17N focused on targetingselect individuals, not on carrying out attacks thatresulted in mass casualties. The group learned overtime that its makeshift rockets were often hard tomanipulate and control. In one instance, a rocketmissed its target (Vardinoyiannis 1990), and inanother, it inadvertently killed an innocent bystander(Paliokrassas 1992). This would suggest that thegroup is unlikely to use this tactic again. When? This is important, but whether an attackwould be launched before or during the Olympicswould have little impact on how the analysis isconducted, although it may have larger implicationsfor those charged with managing the crowds. Theexercise raises a good question, however: Would17Ns avoidance of injuring innocent civilians affectits choice of timing? Why? This question explores multiple motivesfor launching an attack. Whether 17N attacked toadvance an extremist ideology, to protest Greecesparticipation in or hosting of the Olympic Games, orto protest Greeces close ties with the United Statesmore generically, it would probably not change thenature of the attack.Step 5: Array combinations of these contrary assumptionsin sets of 2 2 matrices.For this exercise, 2 x 2 matrices will be constructed based onboth the lead assumption and selected contrary dimensions.

Understanding Revolutionary Organization 17 November 153

Who? (target): US officials or tourists attending the

Olympics What? (tactics): Assassination or hostage taking/kidnapping Where? (location): In metropolitan Athens oroutside Athens (including other Olympic Gamesevents)These pairs of dimensions then must be paired to createthree different matrices with a total of twelve combinations.For ease of discussion, each quadrant has been given a number identifier. For example, in the first matrix, Quadrant1 refers to an attack scenario involving an attack on a USEmbassy official in Athens. The twelve possible combinations are shown in Table 13.7.Step 6: Generate one or two credible scenarios for eachquadrant.For each cell in each matrix, generate one or two examples of how this scenario could play out. In some quadrants, the most likely scenario might be relatively easy toidentify. For example, the scenarios generated for Quadrants 1 and 5 would look like traditional 17N attacks. Theterrorists probably would stay within their comfort zone,selecting an embassy official with an established pattern

Table 13.7 Foresight Quadrant Crunching:

Potential Attack ScenariosTarget/Location1

US official

In metropolitan Athens2

Tourists at Olympics

US officialOutside Athens

In metropolitan Athens

Tourists at OlympicsOutside Athens

Target/Tactics5

US official

Assassination6

US officialHostage taking/kidnapping

Tourists at Olympics

Assassination

Tourists at OlympicsHostage taking/kidnapping

Location/Tactics9

In metropolitan Athens

11

Assassination10

Outside AthensAssassination

In metropolitan AthensHostage taking/kidnapping

12

Outside AthensHostage taking/kidnapping

who would offer an easy target in Athensa city whose

chaos and crowds afford a certain level of camouflage forthe operatives.The scenario for Quadrant 10 would require 17N tocarry out a shooting outside of downtown Athens, its usualdomain. Staging an attack in a less-populated location suchas Olympia or Marathon, where some of the Olympicsevents will be held, might mean that the drivers would optfor the motorcycle approach, and limit their exposurebefore the attack. The scenario for Quadrant 11 and wouldrequire consideration of the risk of hurting innocentbystanders, something 17N had avoided in the past.In other quadrants, it could prove difficult to come upwith a credible scenario, but generating scenarios for allthe quadrants will usually stretch the analysts thinking,forcing them to reframe the problem in a variety of ways.In so doing, they are almost certain to gain new insightsand come up with a more creative set of potential attackscenarios.Step 7: Arrange all the scenarios generated in a single listwith the most credible scenario at the top of the list and theleast credible at the bottom using preestablished criteria.In this example, possible criteria might include thosescenarios that are targeting lower-level officers with lesssecurity protection or multiple attacks designed to heightenthe perception of the groups capabilities. After establishinga solid set of criteria, rate each scenario on a 1 to 5 scale,with 5 indicating the scenario that is highly deserving ofattention and 1 indicating that officials should give this scenario a relatively low priority. Place the scenario deservingthe most attention at the top of the list, and the least credible scenario at the bottom.If a scenario makes little sense or is highly unlikely, placean x in the box and eliminate it from further consideration. For example, a scenario involving a hostage takingoutside Athens during the Olympic Games (Quadrant 12)would be well outside the scope of 17Ns practice, difficultto organize, and probably could be dropped from the list.Once the unlikely scenarios are dropped, the next task isto prioritize the remaining scenarios. A useful template isprovided in Table 13.8. Different analysts might rate eachscenario depending on its vantage point. For example, werethey primarily concerned about security for the OlympicGames or the security of the embassy staff ? Had theyworked on previous cases involving the taking of hostagesand believed this was a viable threat too often discounted byother analysts?

154 Chapter 13

Table 13.8 Foresight Quadrant Crunching:

Rating the Attack ScenariosQuadrant

Alternative Scenario

Rating

US official assassinated in Athens en

route to Olympic event

US official visiting Games assassinated as

he leaves hotel

US official shot when attending Olympic

event in Marathon

Car with US official sprayed with bullets

on Athens street

Several US tourists assassinated at Olympics site by sniper

Bus taking US tourists from hotel to

Athens Olympic event bombed

10

US tourist bus en route to Olympic event

outside Athens bombed

Visiting US official taken hostage en route

to Olympic event

Bus taking Americans to Olympic event

outside Athens bombed

11

Americans at Athens hotel taken hostage

and rooms set afire

Americans dining at an Olympic site

restaurant held hostage

12

Americans staying at hotel outside Athens

taken hostage

Analytic Value Added: Which scenario is the most

deserving of attention? The terrorists have shown a consistent pattern of conducting well-planned, focusedattacks on US government or military officials whileavoiding the killing of innocent civilians. They also aremore practiced at operating in metropolitan Athensand probably would continue to prefer that area ofoperations.Should attention focus on just one scenario, or couldseveral scenarios play out simultaneously? It probablywould be wise to give serious consideration to all scenariosreceiving a rating of three or above. Although 17Ns patternof behavior has been fairly consistent over time, new factorscould always come into play, such as the emergence of anew leader or a faction that advocates expanding beyond itstraditional patterns.Are any key themes present when reviewing the mostlikely set of attention-deserving scenarios? The most

likely themes are the likelihood that 17N will continue to

use small arms or bombs and seek to avoid killing innocentpeople, but may expand its theatre of operations.Does this technique help you determine where todevote the most attention in trying to deter an attack?The technique helps the analyst consider a larger range ofattacks and to develop specific criteria for which attacksare most likely to occur. By forcing analysts to thinkoperationally in terms of how easy or difficult it would beto launch various attacks, the analysts get a better senseof what is most feasible, and therefore more likely tooccur.Does it help you challenge any key assumptionsregarding how an attack might take place? The techniquehelped challenge several assumptions. For example, anattack might not necessarily have to take place in Athens. Itis possible that some members of the group might be just asfamiliar with the city landscape of a surrounding town thatwas also going to play host to some Olympic events. Such alocation might also be more attractive as a setting for anattack if it had less police scrutiny.

CONCLUSIONOn June 29, 2002, a botched attempted bombing by one ofthe core members of 17N led to his arrest, confession, andthe subsequent unraveling of the group. Savvas Xiros, aname new to Greek police, was seriously injured when ahomemade explosive device he had placed behind a FlyingDolphin ferry ticket kiosk in Piraeus exploded prematurely.Xiros, a largely self-taught bomb maker, lost several fingersand suffered permanent damage to his eyes. The portpolice who responded to the blast discovered a secondbomb and, more significantly, a bag containing a gun thatlinked to a 17N bank robbery in 1984 in which a policeofficer had been killed.1 After Savvass photo was placed onGreek television, an anonymous caller provided information connecting him to a safehouse.2 Two apartments werediscovered, chock full of all the materials 17N used to carryout its attacks: stolen license plates, keys, forging materials,pvc pipes, guns, bullets, costumes, proclamations, surveillance notes, and perhaps most interesting of all, a detailedledger that chronicled the members pay and expenses peroperative alias.3Savvas awoke in the hospital under heavy police guard,and spent the next few weeks being interrogated. Policeaggressively pursued all leads stemming from Savvass

Understanding Revolutionary Organization 17 November 155

confession and the safehouses and within days had

arrested three of his brothers, all sons of a Greek Orthodox priest from a small village in Northern Greece. Bymid-July, another eight operatives had been identified andarrested.Savvas Xiross cohorts included a real estate agent, aschoolteacher, a shopkeeper, a telephone operator, and amusician, many connected through familial and village ties.He himself was an icon painter by trade.4 The groups operational leader and account keeper, Dimitris Koufondinas,managed to hide for several weeks on a nude beach on oneof the Greek islands but eventually turned himself in. Taking a taxi to police headquarters in Athens, he identifiedhimself to the police officer on duty as the most wantedman in Greece.5 He and his partner had eked out a living asbeekeepers.Missing from this cadre, however, was the ideologicalleadership. The investigation led police to Lipsi, a remoteDodecanese island where Alexandros Giotopoulos, aFrench-educated radical and former head of the Junta resistance group LEA (Popular Revolutionary Resistance), livedunder an assumed name, Mihalis Economou. Giotopoulossfather had been a well-known Trotskyite,6 and Giotopoulosand his French wife lived in a pink house on Lipsi, where heoften held court at the local tavern on politics and tussledwith local authorities over his right to violate the regulations

for whitewashing his home. Authorities from Athens

arrived in Lipsi just in time to arrest Giotopoulos as he waswaiting to catch the next ferry to Turkey. The earliest crimesof 17N were never tried in court due to a twenty-year statute of limitation on murder in Greece, and Giotopoulosnever admitted to any involvement 7, but he is largelybelieved to have been the man who shot and killed RichardWelch in 1975.The unmasked members of what had become the greatGreek unsolved mystery revealed themselves to be a parochial assortment of men, but for almost three decades, theunidentified members of 17N had assumed an almostmythical role in Greek society. What was revealed was anautonomous and indigenous violent far-left group, whosetime was finally over.KEY TAKEAWAYS When information is limited or ambiguous, itis helpful to explore alternative explanations forwhat appears to be or what might be to help findoverlooked explanations and investigative leads. Multiple Hypotheses Generation helps developmore nuanced explanations, such as the possibilitythat a group may have changed or evolved overtime.

Figure 13.2 Mug Shots of the 17N Suspects

The suspects were apprehended in the summer of 2002. Far right is the operational mastermind, Koufondinas, and to his left is the ideologicalleader, Giotopoulos.

Scenarios and Indicators

Scenarios and Indicators

14 Defending Mumbai from Terrorist Attack

t is mid-October 2008. You are an analyst working in the

Mumbai Police Department, and you just received the USwarning about the threat to Mumbai from the IntelligenceBureau in New Delhi. Analysis of the threat has to be donequickly in order to develop guidance to help authoritiesanticipate and detect the type of attack that is being planned.Although no analyst has a crystal ball, it is incumbent uponanalysts to help law enforcement officials and policy makersanticipate how adversaries will behave, outline the range ofpossible futures that could develop, and recognize the signsthat a particular future is beginning to take shape. The techniques in this caseStructured Brainstorming, Red HatAnalysis, Classic Quadrant CrunchingTM, Indicators, andthe Indicators ValidatorTMcan help analysts tackle eachpart of this task.The challenge for law enforcement analysts in this case isto forecast how the anticipated attack is most likely to belaunched and, in so doing, help local officials and businesspeople prevent or mitigate the damage of such an attack.When confronted with this challenge, the first reaction ofmany students is to propose that the Indian governmentincrease its vigilance, issue an alert to local officials that aterrorist attack on Mumbai is imminent, and ask them tolook out for any suspicious activity that would indicate thatsuch an attack is being planned or is underway. Unfortunately, such guidance lacks sufficient specificity to be ofmuch value to Mumbai law enforcement officials and businesspeople. The purpose of these exercises is to show thatwith the use of structured analytic techniques, analysts cangenerate a plausible set of attention-deserving scenarios andcreate tailored lists of collection requirements that provideoperational value to local officials and businesspeople.

These instructor materials are built around what actually

occurred, but a successful student analysis need not mirrorthe events on the day of the attack. Instead, instructors andthe students should judge the resulting analyses on the basisof how well the students apply the analytic process and theextent to which they identify well-considered and actionable steps that intelligence operators, law enforcement officials, and collection agencies can use to counter the threat.TECHNIQUE 1: STRUCTURED BRAINSTORMINGBrainstorming is a group process that follows specific rulesand procedures designed for generating new ideas and concepts. The stimulus for creativity comes from two or moreanalysts bouncing ideas off each other. A brainstorming session usually exposes an analyst to a greater range of ideasand perspectives than the analyst could generate alone, andthis broadening of views typically results in a better analyticproduct. (See eight rules for successful brainstorming inBox 14.2.)Structured Brainstorming is a more systematic twelvestep process for cond ucting group brainstorming. Itrequires a facilitator, in part because participants are notallowed to talk during the brainstorming session. Structured Brainstorming is most often used to identify key drivers or all the forces and factors that may come into play in agiven situation.Task 1.Conduct a Structured Brainstorming exercise to identify allthe various modes of transport the assailants might use toenter Mumbai.

157

158 Chapter 14

Box 14.2 EIGHT RULES FOR SUCCESSFUL BRAINSTORMING

1. Be specific about the purpose and the topic of thebrainstorming session.2. Never criticize an idea, no matter how weird, unconventional,or improbable it might sound. Instead, try to figure out howthe idea might be applied to the task at hand.3. Allow only one conversation at a time and ensure thateveryone has an opportunity to speak.4. Allocate enough time to complete the brainstormingsession.5. Engage all participants in the discussion; sometimes this mightrequire silent brainstorming techniques such as askingeveryone to be quiet for five minutes and write down their keyideas on 3 5 cards and then discussing what everyone wrotedown on their cards.6. Try to include one or more outsiders in the group to avoidgroupthink and stimulate divergent thinking. Recruit astutethinkers who do not share the same body of knowledge orperspective as other group members but have somefamiliarity with the topic.

imaging. The question is not What would you do if you

were in their shoes? but How would the assailants thinkabout this problem?Step 5: Ask the group to write down responses to thequestion with a few key words that will fit on a sticky note.After a response is written down, the participant gives it tothe facilitator, who then reads it out loud. Marker-type pensare used so that people can easily see what is written on thesticky notes when they are posted on the wall.Step 6: Post all the sticky notes on a wall in the order inwhich they are called out. Treat all ideas the same.Encourage participants to build on one anothers ideas.Usually an initial spurt of ideas is followed by pauses as participants contemplate the question. After five or ten minutesthere is often a long pause of a minute or so. This slowingdown suggests that the group has emptied the barrel of theobvious and is now on the verge of coming up with somefresh insights and ideas. Do not talk during this pause, evenif the silence is uncomfortable.

7. Write it down! Track the discussion by using a whiteboard, an

easel, or sticky notes.

Step 7: After two or three long pauses, conclude this

divergent-thinking phase of the brainstorming session.

8. Summarize key findings at the end of the session. Ask the

participants to write down their key takeaways or the mostimportant things they learned on 3 5 cards as they departthe session. Then, prepare a short summary and distribute thelist to the participants (who may add items to the list) and toothers interested in the topic (including those who could notattend).

Step 8: Ask all participants (or a small group) to go up to

the wall and rearrange the sticky notes by affinity groups(groups that have some common characteristics). Somesticky notes may be moved several times; some may also becopied if an idea applies to more than one affinity group.Step 9: When all sticky notes have been arranged, ask thegroup to select a word or phrase that best describes eachgrouping.

Step 1: Gather a group of analysts with knowledge of the

target and its operating culture and environment.Step 2: Pass out sticky notes and marker-type pens toall participants. Inform the team that there is no talkingduring the sticky-notes portion of the brainstormingexercise.Step 3: Present the team with the following question:What are all the various modes of transport the assailantsmight use to enter Mumbai?Step 4: Ask them to pretend they are Muslim terroristsand simulate how they would expect the assailants to thinkabout the problem. Emphasize the need to avoid mirror

Step 10: Look for sticky notes that do not fit neatly intoany of the groups. Consider whether such an outlier is useless noise or the germ of an idea that deserves furtherattention.Step 11: Assess what the group has accomplished. Howmany different ways have you identified that the assailantscould transport a team to Mumbai?Step 12: Present the results, describing the key themes ordimensions of the problem that were identified. Considerless conventional means of presenting the results by engaging in a hypothetical conversation in which terrorist leadersdiscuss the issue in the first person.

Defending Mumbai from Terrorist Attack 159

Did we explore all the possible forces and factors that

could influence how the terrorists might gain access toMumbai to launch their attack? The list appears to becomprehensive, covering all potential forms of transit.Did we cluster the ideas into coherent affinity groups?The ideas easily fell into three categories: land, sea, and air. Akey consideration was whether the same mode of transportwould be used for the entire transit or a two-stage processwould be more effective, particularly if the assailants come bysea from Pakistan. Other groupings that one could considerwould be based on how the form of transit was acquired, forexample, by purchase, rental, hijacking, or buying tickets.How did we treat outliers or sticky notes that seemed tobelong in a group all by themselves? Did the outliers sparkany new lines of inquiry? The brainstorming exerciseshould generate several outliers, such as the use of a touristhelicopter to launch an attack or the use of taxis. Anotheroutlier to consider would be for the terrorists to hide themselves and their supplies in a large cargo container on a planeor a ship and sneak out before passing through customsinspection or bribe the customs inspector to look the otherway. The use of submersibles similar to those used to smuggle drugs from Colombia to the United States would be acreative, albeit potentially more expensive, solution. Theexercise might also prompt students to consider the use ofinsiders, such as residents of Mumbai who have agreed toprovide their vehicles for a price or out of sympathy for themovements objectives.

Over the course of the exercise, students should generate between twenty and fifty ideas. Groups familiar withthe region or with terrorist activity are likely to generatemore ideas. The most obvious ways to group theresponses would be to distinguish efforts to access Mumbai by sea, by land, or by air. If the students are havingtrouble coming up with ideas or their ideas are too general, ask them to drill down on specific ways the terroristswould come to Mumbai using different modes of transport. Table 14.4 provides a sampling of likely responses.Encourage the students to be creative, as this usuallybuilds energy within the group. Some groups, for example, have proposed using gliders, parachutes, and evenSegways. Other seemingly out-of-the-box ideas that couldmerit attention are bicycle tours and the use of humantrafficking networks.Analytic Value Added: Were we careful to avoidmirror imaging when we put ourselves in the shoes ofMuslim terrorist planners? While a regular citizenmight use commercial air or a border crossing to enterIndia, we cannot assume that terrorists would do thesame. The risks of apprehension are too high. Also, someof the ideas generated may not prove practical if the terrorists need to transport weapons and explosives withthem to Mumbai. Crossing the border or transitingthrough an airport might prove impractical, suggestingthat ideas such as using commercial aircraft for transit areunlikely.

Table 14.4 Modes of Transit into Mumbai: Brainstormed Examples

By Sea

By Land

By Air

If departing from Pakistan:

Drive personal vehicles.

Fly commercial air from Pakistan.

Take large boat to Mumbai.

Drive commercial truck.

Fly commercial air from India.

Hide in large container ship.

Rent large truck.

Fly private aircraft from Pakistan.

Take public ferry.

Take train to Mumbai.

Fly private aircraft from India.

If two-staged transit:

Take bus to Mumbai.

Hijack small airplane.

Take large boat to submersible.

If two-staged transit:

Hide in large cargo container in cargo plane.

Take large boat to coast near Mumbai and

transfer to Zodiacs.

Drive large commercial truck and hijack taxis

or bus on outskirts of city.

If two-staged transit:

Take large boat to coast near Mumbai and

transfer to truck, cars, or taxis.

Take train and hijack bus or taxis at train

station.

Fly private aircraft to vicinity of Mumbai and

rent or hijack helicopter to enter city.

160 Chapter 14TECHNIQUE 2: RED HAT ANALYSISAnalysts frequently endeavor to forecast the actions of anadversary or a competitor. In doing so, they need to avoidthe common error of mirror imaging, the natural tendencyto assume that others think and perceive the world in thesame way as they do. Red Hat Analysis is a useful techniquefor trying to perceive threats and opportunities as others seethem, but this technique alone is of limited value withoutsignificant understanding of the cultures of other countries,groups, or people involved. There is a great deal of truth tothe maxim that where you stand depends on where yousit. By imagining the situation as the target perceives it, ananalyst can gain a different and usually more accurate perspective on a problem or issue.Reframing the problem typically changes the analystsperspective from that of an analyst observing and forecasting an adversarys behavior to that of someone who mustmake difficult decisions within that operational culture.This reframing process often introduces new and differentstimuli that might not have been factored into a traditionalanalysis.Task 2.Use Red Hat Analysis to prioritize the list of various modesof transport the terrorists might use to enter Mumbai.1Step 1: Gather a group of experts with in-depth knowledge of the target, operating environment, and the terroristgroups motives and style of thinking. If at all possible, try toinclude people who are well grounded in Mumbais culture,speak the language, share the same ethnic background, orhave lived extensively in the region.Step 2: Ask group members to develop a list of criteriathat they would most likely use when deciding whichmodes of transport they personally would choose to enterMumbai. The reason for first asking the group how it wouldact is to establish a baseline for assessing whether the terrorists are likely to act differently.Key criteria would include the following: Minimizing the chances of detection prior toimplementing the plan. Minimizing the chances of detection while in transit. Minimizing the chances of detection during theattack.

Providing adequate means to transport the terrorists

weapons and ammunition. Maintaining control over the timing and logistics ofthe operation. Opting for the simplest method possible to minimizepotential for miscalculations. Maximizing the chances of escape when theoperation concludes. Minimizing the need to depend on good weather.Step 3: Use this list to prioritize the ideas that were generated for each affinity group in the Structured Brainstormingsession, placing the most likely choice for that group at thetop of the list and the least likely at the bottom.The students need to re-sort the lists they have generated. If the list is short, they can simply rearrange the ideasfrom most to least likely. If the list is long, then the studentsmight first want to assign a rating to each idea, with 5 beingthe most likely and 1 being the least likely. If on furtherinspection some ideas should be dropped, they shouldreceive a 0 and be deleted from the final list.Another mechanism to prioritize the potential modes oftransport is to have the students vote on which modes theybelieve are the most credible. A rule of thumb is to giveeach student one vote for every three possibilities. In thisexample, twenty modes of transport are listed, which meanseach student would have seven votes to distribute. It is recommended that the students be asked to write down theirvotes on 3 x 5 inch cards. The instructor then collects thecards, tallies the responses, and announces the results. If thestudents simply go to the whiteboard to mark their preferences, this could bias the results, as they might be inclinedto vote for options that others have already selected.Finally, they can use paired comparison, which isdetailed in the section on Ranking, Scoring, Prioritizing inHeuer and Pherson (2015).2Step 4: After prioritizing the ideas in each affinity group,generate a master list combining all of the lists. The mostlikely ideas overall should be at the top of the list and theleast likely overall at the bottom.Table 14.5 provides an example of how the final listcould be rearranged. The most likely choices appear at thetop with ratings of 5, 4, or 3. Credible but less likely ideaswere given a score of 2 or 1. Those ideas receiving a 0, as

Defending Mumbai from Terrorist Attack 161

Again, this step establishes a baseline for assessing why the

adversary is likely to react differently.

Table 14.5 Prioritized List of Ways to Enter

Mumbai ExampleWays to Enter Mumbai

Rating

Step 6: Once the group can explain in a convincing way

why it chose to act the way it did, ask the group membersto put themselves in the shoes of the terrorists and simulate how they would respond, repeating Steps 2 to 4.Emphasize the need to avoid mirror imaging. The questionnow is not What would you do if you were in their shoes?but How would the terrorists approach this problem,given their background, past experience, and the currentsituation?

Take large boat to coast near Mumbai and

transfer to small boats or Zodiacs.

Take large boat to coast near Mumbai and

transfer to cars, truck, or taxis.

Conceal weapons in large commercial truck and

accompany in personal cars.

Take large boat and transfer to submersible off

coast of Mumbai.

Fly private aircraft to small airport near Mumbai

and use a helicopter to enter city.

Hide in containers being transported by large

cargo plane and sneak out.

Hide in large container ship and sneak out when

arriving in harbor.

Drive personal vehicles to Mumbai.

Drive large commercial truck to Mumbai.

Take large boat from Pakistan directly to Port of

Mumbai.

Rent large truck for land transport to Mumbai.

Take public ferry directly to Port of Mumbai.

Take private aircraft from India to Mumbai

Airport.

Describing a hypothetical conversation in which the

terrorists would discuss the issue in the first person.

Take bus to Mumbai.

Take train to Mumbai.

Hijack small aircraft to fly to Mumbai Airport.

Drafting a document (set of instructions, military

orders, or directives) that the leader of the terroristgroup would likely generate.

Take private aircraft from Pakistan to Mumbai

Airport.

Take commercial air from India to Mumbai

Airport.

Take commercial air from Pakistan to Mumbai

Airport.

not satisfying the criteria on further inspection, should be

dropped from the final list.Step 5: Once the group has articulated how it would haveacted, ask it to explain why the group members think theywould behave that way. Ask them to list what core values orcore assumptions were motivating their behavior or actions.

Step 7: At this point, after all the terrorists ideas are

gathered and prioritized, the group should ask, Do theterrorists share our values or methods of operation? Ifnot, then how do those differences lead them to act inways we might not have anticipated before engaging in thisexercise?Step 8: Present the results, describing the alternatives thatwere considered and the rationale for selecting the modes oftransit the terrorists are most likely to choose. Consider lessconventional means of presenting the results of the analysis,such as the following:

Analytic Value Added: Was your list of criteria comprehensive? The list provided in Table 14.4 is fairly comprehensive, but challenging the students to come up with afew more ideas is always recommended. Terrorist groupscan be very innovative, and surprise will work to theiradvantage.Did some criteria deserve greater weight than others?Did you reflect this when you rated the various ideas?The process of rating each idea allows the students to reflecton the criteria they have developed. In this case, the conceptof a staged transit appears to have the most utility. If traveling by sea, the assailants would need a larger ship that isocean-worthy but then would have to transfer to some lessvisible mode of transit upon arriving in the vicinity ofMumbai.

162 Chapter 14Usually the students will propose to add criteria to thelist. In this instance, one question would be whether thepossibility of renting trucks (as has been done in the UnitedStates) or stealing them would be a viable option in India orPakistan. Another issue that might arise is what strategy theterrorists have decided to adopt. If the intent is to launch asuicide bombing, then options using aircraft might be ratedhigher.TECHNIQUE 3: CLASSIC QUADRANTCRUNCHINGTMClassic Quadrant CrunchingTM combines the methodologyof a Key Assumptions Check 3 with Multiple ScenariosGeneration4 to generate an array of alternative scenarios orstories. This process is particularly helpful in the Mumbaicase because little is known about the actual plans andintentions of the attackers. This technique helps the analystidentify and challenge key assumptions that may underpinthe analysis while generating an array of credible alternativescenarios to help law enforcement focus on the most likelytypes of attacks to anticipate.Task 3.Use Classic Quadrant CrunchingTM to brainstorm all thepossible ways terrorists might launch an attack on Mumbai.List the scenarios from most to least likely.

Step 1: State your lead hypothesis.

This hypothesis should reflect either the consensus ofthe analytic unit regarding the most likely means of attackor the current conventional wisdom, which usually reflectshow such attacks have been launched in the past. For illustrative purposes, we will use the hypothesis informed by thelimited initial intelligence reporting received prior to theattack: Lakar- e-Taiba (LeT) travels to Mumbai by (inserthighest-ranked option listed in Task 2 or by sea) andattacks the Taj Hotel with small arms and grenades, killingmany people.Step 2: Break the lead hypothesis down into its component parts based on the journalists list of Who? What?How? When? Where? and Why?Step 3: Identify which of these components are most critical to the analysis.Step 4: For each of the critical components, identify twoor four (an even number) contrary dimensions in a table, asshown in Table 14.6.Six key components were identified in this exerciseone for each of the five Ws and H questions. Three of thekey components (not shaded in Table 10.7) deserve seriousdiscussion and analysis because the contrary dimensionscould pose significant new challenges for how best todefend the city.

Indian or Western government

Multiple simultaneous events

To protest India as an enemy

To protest the West or the United States as an

To protest Israel and Jews as

enemies of Islam

When?(timing)

In the near future

On a significant date

A year from now

Defending Mumbai from Terrorist Attack 163

What? Historically, LeT has relied mostly on bombs,

small arms, and grenades to generate large numbers ofcasualties. In several of its more spectacular actions, including its attacks on Indian forces in Kashmir, the strategy wasto launch an assault deep into the target where the assailantsthen killed as many people as possible.5 Since LeT has useda variety of weapons and tactics, a key question is this:What weapons would LeT employ in an attack on Mumbai?Would the use of small arms and grenades allow it to exactenough casualties? Would bombs generate more casualties?Would a large explosion (or several simultaneous explosions) attract more international attention?Where? Would LeT consider attacking targets other thanhotels? The initial intelligence mentions the Taj Mahal PalaceHotel as a primary target of the attack. It is a likely targetbut perhaps not the only one. Indian authorities in February2008 had reported that a suspected terrorist, arrested innorthern India, was found to possess drawings of varioussites in Mumbai, some of which were targets in the November 2008 attack; these included the Taj Hotel and the Bombay Stock Exchange (which had also been a terrorist targetin 1993). The Trident-Oberei Hotel was another prime candidate, as were other large public spaces such as railway stations and restaurants known to be frequented by foreigners.In the past, LeT has attacked Hindu temples. The organizations anti-Western and anti-Jewish rhetoric has also grownmore intense in recent years. Indian and Western government offices and key infrastructure in Mumbai should notbe ruled out as possible targets.How? LeT has operated with different modi operandiover the years, opting for both simultaneous attacks andarmed assaults against high-value targets. Historically,LeT has not conducted extended events or events including the taking of hostages, but this alternative is worthconsidering because an extended event, particularly if itinvolved a hostage taking, would advance several of theorganizations key objectivesgetting more internationalattention and deflecting criticism that it was engaging inindiscriminate violence.The remaining questions are not good candidates for aClassic Quadrant CrunchingTM exercise because either thealternatives to the lead hypotheses are not sufficiently likelyto divert analytic resources or they would not have significant impact on how the analysis is conducted.Who? A strong case can be made that LeT would be theprime candidate to launch the attack on Mumbai. A goodanalyst would challenge this assumption and consider otherpossible perpetrators. For example, another possibility

could be Hindu radicals or a separatist group such as the

Sikhs or the Tamils. For the purposes of illustrating thistechnique, however, we will assume that LeT is planning theattack. If a different group were to launch the attack, itprobably would consider using the same range of weaponsand tactics. The idea that the Pakistani government mightbe responsible for the attack or is providing support to theattackers is worth considering as a wildcard scenario. In thiscase, the key question is what support the attackers mightreceive from the Pakistanis that would significantly changethe key attack scenarios.When? This is important, but whether the attack islaunched next week or next year would have little impact onhow the analysis is conducted. The sense of urgency isalready well established. The exercise raises a good question, however. Are there any particular dates that LeT wouldselect that would further enhance its message?Why? This question explores multiple motives forlaunching an attack. LeT sees India as part of the CrusaderZionist-Hindu alliance and an enemy of Islam. Muslimdominated Kashmir is ruled by the majority Hindupopulation of India, which provides LeT with a specificcause. LeT has increasingly portrayed its struggle in Kashmiras part of an international struggle. This justifies includingforeigners (especially Britons and Americans) as targets aswell as Jewish religious centers.Step 5: Array combinations of these contrary assumptionsin sets of 2 2 matrices.For the purposes of this exercise, 2 2 matrices will beconstructed based on the two What? (weapon) contrarydimensions, the two How? (tactics) contrary dimensions,and two of the four Where? (targets) contrary dimensionsfor a total of six contrary dimensions. These contrarydimensions then must be paired to create three differentmatrices with a total of twelve combinations. For ease ofdiscussion, each quadrant has been given a number identifier. For example, in the first matrix, Quadrant 2 refers to anattack scenario involving large explosives and multipleevents. The twelve possible combinations are shown inTable 14.7.Step 6: Generate one or two credible scenarios for eachquadrant.For each cell in each matrix, generate one or twoexamples of how this scenario could play out. For example, in Quadrant 1, LeT attackers would orchestrate aseries of small bombings. Some might be preplaced to go

164 Chapter 14

Table 14.7 Mumbai Classic Quadrant

CrunchingTM: 2 2 Matrices ExamplesWeapon/Tactics1

Small explosives

Small explosives

Multiple events

Extended event

Large explosives

Large explosives

Multiple events

Extended event

Weapon/Locations5

Small explosives

Small explosives

Transit locations

Religious locations

Large explosives

Large explosives

Transit locations

Religious locations

Tactics/Locations9

10

Multiple events

11

Extended event

Transit locations

Transit locations

Multiple events

12

Extended event

Religious locations

Religious locations

off simultaneously in several hotels and the major train

station, others would be thrown from motorcycles intolarge crowds, and even others would be set to kill policeand other first responders who react to the initial set ofbombings. In Quadrant 2, LeT would place large bombsor possibly suicide car or truck bombs at several iconiclocations. Likely targets would include the Taj Hotel,Oberoi Hotel, train stations, and bus depots. In Quadrant7, LeT assailants might place knapsacks filled with smallexplosives in a Jewish synagogue and time the detonationto go off during services. In Quadrant 10, they mightlaunch multiple attacks at several key religious sites,including temples, synagogues, and Christian churches.In some quadrants, the most likely scenario might be relatively easy to identify. In other quadrants, it could prove difficult to come up with a credible scenario. But several of thequadrants will usually stretch the analysts thinking, forcingthem to reframe the problem in a variety of ways. In sodoing, they are almost certain to gain new insights and comeup with a more creative set of potential attack scenarios.Step 7: Array all the scenarios generated in a single listwith the most credible scenario at the top of the list and theleast credible at the bottom.

Review all the scenarios generated in Step 6 and select

those most deserving of attention based on a preestablishedset of criteria. In this example, possible criteria mightinclude those scenarios that would create the most damage;generate the most publicity, especially on the world stage; orbe the hardest to detect or prevent. This would includethose scenarios most likely to capture the medias attentionby attacking well-known icons or institutions, targeting foreigners, or extending the attack scenario over several daysto give the media time to travel to Mumbai to cover theevent.Another way to narrow the list of scenarios is to removethose that make little or no sense. For example, a scenarioinvolving large explosions as part of an extended event(Quadrant 4) may be beyond the capability of LeT. This scenario has been shaded in Table 14.7 to indicate it probablycan be dropped.Once the illogical scenarios are dropped, the next task isto prioritize the remaining scenarios. An illustrative list isprovided in Table 14.8.Analytic Value Added: Which scenario is the mostdeserving of attention? The scenario that received thehighest score involved a series of simultaneous attacks replicating LeTs traditional reliance on an armed assaultmodel.Should attention focus on just one scenario, or couldseveral scenarios play out simultaneously? Four of theattack scenarios received either a 4 or a 5 rating, suggestingthat LeT might employ a variety of attack options or, atleast, that Mumbai defenders should be prepared to defendagainst a broad array of attack options.Are any key themes present when reviewing the mostlikely set of attention-deserving scenarios? Considerationof the contrary dimension of an extended event raises thepossibility that the terrorists might take hostages as a meansof gaining more publicity. Consideration of the large-explosion contrary dimension introduces the possibility of a largesuicide car bomb or truck bomb. This option is less likely,however, given the logistical challenges of prepositioningsuch a bomb. The idea that insiders might be used to support either the planning of the attack or the actual attackscenario also emerges as a theme worth considering.Does this technique help one determine where todevote the most attention in trying to deter the attack ormitigate the potential damage of the attack? The exercisesuggests that more attention should be given to consideringthe hypotheses that several attack scenarios might be

Defending Mumbai from Terrorist Attack 165

Table 14.8 Mumbai Prioritized List of Alternative Scenarios Examples

Quadrant

Alternative Scenario

Rating

LeT launches simultaneous attacks using small arms and explosives targeting several hotels, the trainstation, and several restaurants.

LeT attacks the Taj Hotel with small arms and grenades and takes hostages; it also uses small explosives toset fire to the hotel.

10

LeT orchestrates a series of simultaneous attacks using small arms and grenades against Hindu temples anda Jewish synagogue, taking hostages at two of the locations.

LeT attacks the main train station, a bus depot, and people congregating at bus stops, throwing smallexplosives from motorcycles and setting small bombs in the train station.

LeT attacks the train station, takes hundreds of hostages, and sets up a defensive perimeter, leading to anextended siege.

LeT explodes several large suicide car bombs at hotels, the train station, and several restaurants.

LeT suicide bombers with vests attack several Hindu temples, a Jewish synagogue, and a Christian church.

12

LeT attacks a Jewish religious center or synagogue and takes hostages, leading to an extended siege.

Large bombs are detonated at a train station and the airport, causing major casualties.

LeT, with the support of insiders, explodes large preset bombs at various religious sites and then ambushesthe first responders.

launched simultaneously instead of trying to predict exactly

which scenario is most likely. Preparing for the possibilityof several different attack scenarios also is a prudentapproach when there is so much uncertainty.TECHNIQUE 4: INDICATORSIndicators are observable or deduced phenomena that canbe periodically reviewed to track events, anticipate anadversarys plan of attack, spot emerging trends, distinguishamong competing hypotheses, and warn of unanticipatedchange. An indicators list is a preestablished set of actions,conditions, facts, or events whose simultaneous occurrencewould argue strongly that a phenomenon is present orabout to be present or that a hypothesis is correct. Theidentification and monitoring of indicators are fundamental tasks of intelligence analysis, because they are the principal means of avoiding surprise. In the law enforcementcommunity, indicators are used to assess whether a targetsactivities or behavior are consistent with an established pattern or lead hypothesis. These are often described as

backward-looking or descriptive indicators. In intelligence

analysis, indicators are often described as forward-lookingor predictive indicators.Preparation of a detailed indicator list by a group ofknowledgeable analysts is usually a good learning experience for all participants. It can be a useful medium for anexchange of knowledge between analysts from differentorganizations or those with different types of expertisefor example, counterterrorism or counterdrug analysis,infrastructure protection, and country expertise. The indicator list can become the basis for conducting an investigation or directing collection efforts and routing relevantinformation to all interested parties. Identification andmonitoring of indicators or signposts that a scenario isemerging can provide early warning of the direction inwhich the future is heading, but these early signs are notobvious. The human mind tends to see what it expects tosee and to overlook the unexpected. Indicators take onmeaning only in the context of a specific scenario withwhich they have been identified. The prior identificationof a scenario and associated indicators can create an

166 Chapter 14awareness that prepares the mind to recognize and preventa bad scenario from unfolding or help a good scenario tocome about.Task 4.Create separate sets of indicators for the most attentiondeserving scenarios, including those that were generated inTask 3, the Classic Quadrant CrunchingTM exercise.Step 1: Create a list of the most attention-deserving scenarios to track for this case.Students should be encouraged to select the most attention-deserving scenarios, realizing that time is of the essenceand the list should be kept short, preferably to no more thanfive scenarios. Usually that will require combining somescenarios that share similar characteristics. Table 14.9 provides an illustrative list of attention-deserving scenarios.

Table 14.9 Mumbai Most Attention-Deserving

Scenarios ExamplesAttention-Deserving Scenarios

QuadrantsRepresented

1. Simple armed assault. LeT conducts an

armed assault with AK-47s and grenadeslaunched from the sea against the Taj Hotel.

2. Simultaneous attacks. LeT launches

3. Suicide attacks. LeT orchestrates several

simultaneous attacks launched from thesea using suicide bombers to target severalpublic places, including hotels, a trainstation, and religious sites.

2, 7

4. Hostage taking. LeT attacks the Taj Hotel

and possibly other sites from the sea,including those frequented by foreigners,with small arms and takes hostages.

3, 10, 11, 12

Step 2: Work alone, or preferably with a small group, to

brainstorm a list of indicators for each scenario.Step 3: Review and refine each set of indicators, as shownin Table 14.10, discarding any that are duplicative and combining those that are similar.

Step 4: Examine each indicator to determine if it meets

the following five criteria. Discard those that are foundwanting.1. Observable and collectible. There must be somereasonable expectation that, if present, the indicatorwill be observed and reported by a reliable source. Ifan indicator is to monitor change over time, it mustbe collectible over time.2. Valid. An indicator must be clearly relevant to theendstate the analyst is trying to predict or assess,and it must be inconsistent with all or at least someof the alternative explanations or outcomes. It mustaccurately measure the concept or phenomenon atissue.3. Reliable. Data collection must be consistent whencomparable methods are used. Those observingand collecting data must observe the same things.Reliability requires precise definition of theindicators.4. Stable. An indicator must be useful over timeto allow comparisons and to track events.Ideally, the indicator should be observableearly in the evolution of a development so thatanalysts and decision makers have time to reactaccordingly.5. Unique. An indicator should measure only onething and, in combination with other indicators,should point only to the phenomenon being studied.Valuable indicators are those that not only areconsistent with a specified scenario or hypothesisbut also are inconsistent with all other alternativescenarios.Several indicators relating to tracking the purchase ofguns, grenades, and ammunition would be very hard toobserve (1-f, 2-d, 3-f, and 4-d). LeT probably has its ownwell-established supply links, and its purchases wouldnot stand out from the ubiquitous trafficking of arms inPakistan.Analytic Value Added: Are the indicators mutuallyexclusive and comprehensive? The indicators focus primarily on preparations for launching an attack and whatlocations might be targeted. Other indicators with meritinclude those indicating how the attackers plan to transportthemselves to Mumbai and those that might prove uniqueto a specific target location.

Defending Mumbai from Terrorist Attack 167

Scenario 1, Simple Armed Assault: LeT conducts an armed assault with AK-47s and grenades launched from the sea against the Taj Hotel.1-a

Sources report LeT is providing small arms/grenades training in Pakistan.

1-b

Suspicious people are only observed surveilling the Taj Mahal Palace.

1-c

People renting rooms at the Taj Mahal Palace for several weeks appear suspicious.

1-d

Sources report that Taj Mahal Palace is a primary target.

1-e

LeT posts anti-Indian rhetoric on its website.

1-f

Reports tell of LeT purchases of assault rifles, grenades, and ammunition in Pakistan.

1-g

Sources report that the attack team is small (five or fewer people).

1-h

Small-arms caches are discovered in or around Mumbai.

1-i

Documents captured in LeT possession show sketches of only the Taj Hotel.Scenario 2, Simultaneous Attacks: LeT launches simultaneous attacks from the sea using small arms andexplosives targeting several hotels, a train station, religious sites, and restaurants.

2-a

Sources report LeT is providing training in small arms, portable bombs, preset bombs, and grenades at camps in Pakistan.

2-b

Suspicious people are observed surveilling a large number of prominent public sites in Mumbai.

2-c

LeT posts anti-Indian rhetoric on its website.

2-d

Reports tell of LeT purchases or acquisition of assault rifles, grenades, and ammunition.

2-e

Reports tell of LeT purchases or acquisition of RDX and other bomb materials.

2-f

Sources report the attackers are formed into several teams and number more than five.

2-g

Possible trial runs are observed in the streets of Mumbai.

Scenario 3, Suicide Attacks: LeT orchestrates several simultaneous attacks launched from the sea using suicidebombers to target several public places, including hotels, the train station, and religious sites.

3-a

Sources report LeT is recruiting suicide bombers.

3-b

Sources report LeT is providing training in the use of suicide vests or it is practicing deploying suicide car or truck bombs.

Have a sufficient number of high-quality indicators

been generated for each scenario to enable an effectiveanalysis? At least nine indicators were developed for eachscenario. Most brainstorming sessions usually generate ahigher number because of the different perspectives beingbrought to the table. However, as the quantity of indicatorsgoes up, their quality often decreases.Can the indicators be used to help detect a plannedattack or deter a possible hostile course of action? Several of the indicators suggest potentially productive avenues for Mumbai police investigators. For example,countersurveillance teams could be dispatched to highvalue targets such as the Taj Hotel, the train station,and other hotels and restaurants often frequented byforeigners.TECHNIQUE 5: INDICATORS VALIDATORTMThe Indicators ValidatorTM is a simple tool for assessing thediagnostic power of indicators. Once an analyst has developed a set of attention-deserving alternative scenarios orcompeting hypotheses, the next step is to generate indicators for each scenario or hypothesis that would appear ifthat particular scenario were beginning to emerge or thatparticular hypothesis were true. A critical question that isnot often asked is whether a given indicator would appearonly for the scenario or hypothesis to which it is assignedor also in one or more alternative scenarios or hypotheses.Indicators that could appear under several scenarios orhypotheses are not considered diagnostic; that is, they arenot particularly useful in determining whether a specificscenario is beginning to emerge or a particular hypothesisis true. The ideal indicator is highly likely for the scenarioto which it is assigned and highly unlikely for all others.

Task 5.Use the Indicators ValidatorTM to assess the diagnosticity ofyour indicators.Step 1: Create a matrix similar to that used for Analysisof Competing Hypotheses.6 This can be done manuallyor by using the Indicators ValidatorTM software. ContactGlobalytica, LLC at THINKSuite@globalytica.com or goto http://www.globalytica.com to obtain access to theIndicators Validator TM software if it is not available onyour system. List the alternative scenarios along the topof the matrix and the indicators that have been generated for each of the scenarios down the left side of thematrix.Step 2: Moving across the indicator rows, assess whetherthe indicator for each scenario Is highly likely to appear Is likely to appear Could appear Is unlikely to appear Is highly unlikely to appearIndicators developed for their particular scenario, thehome scenario, should be either highly likely or likely.If the software is unavailable, you can do your own scoring. If the indicator is highly likely in the home scenario,then in the other scenarios, Highly likely is 0 points. Likely is 1 point.

Defending Mumbai from Terrorist Attack 169

Could is 2 points.

The total score for each indicator is shown in the column

on the far right.

Unlikely is 4 points. Highly unlikely is 6 points.If the indicator is likely in the home scenario, then in theother scenarios, Highly likely is 0 points. Likely is 0 points. Could is 1 point. Unlikely is 3 points. Highly unlikely is 5 points.Step 3: Tally up the scores across each row and then rankorder all the indicators.Table 14.11 shows how each indicator was rated for eachscenario. The number beside the rating is the score. It isimportant to remind the students that the scoring forhome scenario indicators rated likely is different from thescoring for home scenario indicators rated highly likely.

Step 4: Re-sort the indicators, putting those with the

highest total score at the top of the matrix and those withthe lowest score at the bottom. The most discriminatingindicator is highly likely to emerge under the home scenarioand highly unlikely to emerge under all other scenarios.The least discriminating indicator is highly likely to appearin all scenarios. Most indicators will fall somewhere inbetween.Step 5: The indicators with the most highly unlikely andunlikely ratings are the most discriminating and should beretained.Step 6: Indicators with no highly unlikely or unlikely ratings should be discarded.Step 7: Use your judgment as to whether you should retainor discard indicators that score fewer points. Generally, youshould discard all indicators that have no highly unlikely or

Table 14.11 Mumbai Indicators ValidatorTM Scoring Examples

Number

Indicator

Scenario 1

Scenario 2

Scenario 3

Scenario 4

Score

Scenario 1, Simple Armed Assault: LeT conducts an armed assault with AK-47s and grenades launched from the sea against the Taj Hotel.1-a

Reports tell of LeT purchases or acquisition of RDX and

Sources report the attackers are formed into several teams

Target organizations or facilities report receiving threats of

imminent attack.

U (3)

C (1)

C (1)

2-i

Documents captured in LeT possession suggest several

possible targets.

U (3)

HL

C (2)

C (2)

Scenario 3, Suicide Attacks: LeT orchestrates several simultaneous attacks launched from the sea using suicidebombers to target several public places, including hotels, the train station, and religious sites.3-a

Sources report LeT is recruiting suicide bombers.

U (4)

U (4)

HL

HU (6)

14

3-b

Sources report LeT is providing training in the use of suicide

vests or it is practicing deploying suicide car or truckbombs.

HU (6)

HU (6)

HL

HU (6)

18

3-c

Sources report LeT supporters are conducting practice

suicide bombings.

HU (6)

HU (6)

HL

HU (6)

18

3-d

Suspicious people are observed surveilling a large number

of prominent public sites in Mumbai.

U (3)

HL (0)

HL (0)

3-e

LeT posts virulent anti-Indian rhetoric on its website

justifying the use of suicide bombers.

U (4)

L (1)

HL

C (2)

3-h

Sources report little emphasis on small-arms training in LeT

camps.

HU (5)

HU (5)

HU (5)

15

3-i

LeT releases martyrdom videos.

U (3)

U (4)

U (4)

11

Scenario 4, Hostage Taking: LeT attacks the Taj Hotel and possibly other sites from the sea,including those frequented by foreigners, with small arms and takes hostages.4-a

Defending Mumbai from Terrorist Attack 171

unlikely ratings. In some cases, an indicator may be worth

keeping if it is useful when viewed in combination with several other indicators.As shown in Table 14.12, the following indicators shouldbe discarded because of their low point score and lack of

any unlikely or highly unlikely ratings: 1-c (2 points); 1-a,

1-h, 4-a, and 4-c (1 point); and 1-b, 1-d, 1-e, 2-c, and 4-b (0points). Several indicators have scores of 3 (2-b, 3-d, and4-e) but were retained because the indicator was rated asunlikely for at least one scenario.

Suspicious people are observed surveilling sites often

Step 8: Once nondiscriminating indicators have been eliminated, regroup the indicators under their home scenarios.Overall, twenty indicators were deemed diagnostic, andten were discarded as not sufficiently diagnostic to be usefulin the analysis. When these twenty indicators are re-sortedby scenario, as shown in Table 14.13, it is immediatelyapparent that there is an insufficient number of diagnosticindicators for Scenario 1, Simple Armed Assault.Step 9: If a large number of indicators for a particularscenario have been eliminated, develop additionalandmore diagnosticindicators for that scenario.Step 10: Recheck the diagnostic value of any new indicators by applying the Indicators ValidatorTM to them as well.In this case, students should generate a new set ofdiagnostic indicators for Scenario 1. The problem confronted when trying to come up with Scenario 1 indicators is that the scenario is a fairly basic scenario and most

of its elements would be incorporated into the attack

plans in the other scenarios. The indicators that werelisted would help an analyst confirm that, at a minimum,planning was underway for an attack on the Taj Hotel bysea or that LeT was developing a capability to launch suchan attack. Intelligence sources, however, have alreadyindicated that such an attack is being contemplated.Given that circumstance, the indicators would confirmwhat has already been reported but would not distinguishthe type of attack being contemplated. Any new indicators for Scenario 1 should probably focus on activities orstatements indicating that more sophisticated attackshave been ruled out, such as the following: LeT communications indicate that efforts to recruitsuicide bombers have failed. LeT communications underscore the need to keepthe operation as simple as possible to ensure itssuccess.

Defending Mumbai from Terrorist Attack 173

Table 14.13 Mumbai Diagnostic Indicators by Scenario Example

Number

Indicator

Scenario 1

Scenario 2

Scenario 3

Scenario 4

Score

Scenario 1, Simple Armed Assault: LeT conducts an armed assault with AK-47s and grenades launched from the sea against the Taj Hotel.1-i

Documents captured in LeT possession suggest

Sources report LeT is providing training in

Target organizations or facilities report receiving

threats of imminent attack.

U (3)

C (1)

C (1)

2-b

Suspicious people are observed surveilling a large

number of prominent public sites in Mumbai.

U (3)

L (0)

L (0)

Scenario 3, Suicide Attacks: LeT orchestrates several simultaneous attacks launched from the sea usingsuicide bombers to target several public places, including hotels, a train station, and religious sites.3-b

Sources report LeT is providing training in the

use of suicide vests or it is practicing deployingsuicide car or truck bombs.

HU (6)

HU (6)

HL

HU (6)

18

3-c

Sources report LeT supporters are conducting

practice suicide bombings.

HU (6)

HU (6)

HL

HU (6)

18

3-h

Sources report little emphasis on small arms

training in LeT camps.

HU (5)

HU (5)

HU (5)

15

3-a

Sources report LeT is recruiting suicide bombers.

U (4)

U (4)

HL

HU (6)

14

3-i

LeT releases martyrdom videos.

U (3)

U (4)

U (4)

11

3-e

LeT posts virulent anti-Indian rhetoric on its

website justifying the use of suicide bombers.

U (4)

L (1)

HL

C (2)

3-d

Suspicious people are observed surveilling a large

number of prominent public sites in Mumbai.

U (3)

HL (0)

HL (0)

Scenario 4, Hostage Taking: LeT attacks the Taj Hotel and possibly other sites from the sea,including those frequented by foreigners, with small arms and takes hostages.4-g

Sources report the attackers are formed into

174 Chapter 14 Sources report that only small numbers of weaponsand small amounts of ammunition will be used inthe operation.

Additional Intelligence Reporting

During the month between the initial threat reportfrom the United States and the day of the attack, the Indiangovernmentaided by the United Statesdiligentlytracked down additional information about the plot. Inearly November, the Indian Intelligence Bureau interceptedcommunications from a leader of LeT in Pakistan thatreferred to an attack against hotels in Mumbai.9 US intelligence provided additional information about LeTs plansto attack the Taj Hotel and other sites frequented by foreigners and Americans. 10 On 19 November, the Indianintelligence service uncovered information that a suspicious ship might be en route to Mumbai and that an attackon the city was imminent.11, 12

Analytic Value Added: Does each scenario have a

robust set of highly diagnostic indicators? A good starthas been made at developing a set of diagnostic indicators,but additional brainstorming should generate a more robustset. This would suggest that other experts be brought in tohelp brainstorm, especially those familiar with LeT or thesetypes of terrorist operations.Do these indicator lists provide useful leads for alerting local officials and businesspeople, such as hotel andrestaurant owners, of plausible attack scenarios? Are theindicators focused enough to generate specific collection requirements or follow-on tasking by giving localofficials and businesspeople a more concrete idea ofThe Journey to Mumbaiwhat to look for? The indicators provide many usefulleads for law enforcement analysts as well as a good set ofA group of ten men belonging to LeT boarded a ship inquestions analysts can share with the management andKarachi at 0800 on 22 November 2008 and headed out tochiefs of security at likely targetlocations, including the Taj Hotel,Map 14.2 Targets of Mumbai Terrorist Attack, 26 November 2008train stations, various high-visibility Western establishments,and public places often frequented by foreigners.CONCLUSIONA group of Lakar-e-Taiba (LeT)operatives ultimately launched acoordinated attack on multipletargets across Mumbai on 26November 2008 (see Map 14.2).The assailants quietly entered thecountry by sea and used smallarms and explosive devices toattack transportation infrastructure, hotels, other businesses, anda religious site. Sources differ as tohow many casualties occurredduring the attacks, but a survey ofseveral estimates makes it clearthat more than 160 people diedand over 300 suffered woundsover the course of the 60-hourrampage.7 Twenty-six of the deadwere foreigners, including sixAmericans.8

CaCCama&AAlblessb ess HHospitalp

il TerminusTuRail

Ob-T dH tOberoi-TridentHotel

Leopp d CafeCLeopold

Th lPtTaj MahalPalace HotelChabad House

Assailants disembarkation points

00

1000 feet500 meters

Defending Mumbai from Terrorist Attack 175

sea to rendezvous with the Al-Husseini, a vessel owned by

Zaki-ur-Rehman, a LeT commander. 13, 14 The followingday, the Al-Husseini encountered a 45-foot fishing trawlernamed the Kuber. 15 It is unclear whether the meetingbetween the two ships on the Arabian Sea was prearranged or happened by chance. The Kuber was boarded byLeT militants and captured. Four of the Kubers crewmembers were transferred to the Al-Husseini and killed.16Only Amar Singh Solanki, the Kubers captain, was leftaboard the hijacked ship. Indian officials believe thatSolanki helped pilot the trawler to Mumbai, which laysome 550 nautical miles from the point where the twoships met.17It is unknown exactly how many LeT operatives traveledaboard the Kuber to Mumbai, but Indian investigators collected enough personal articles for at least fifteen people.18A satellite phone recovered from the ship revealed that thegroup aboard the fishing vessel kept in close contact withRehman and other senior LeT officials during the voyage toIndia. While on board the trawler, each of the ten men whomet the Al-Husseini off Karachi were given individual bagscontaining a Kalashnikov, a 9 mm pistol, ammunition, grenades, and an improvised explosive device (IED) made witha military-grade explosive known as RDX.19 On 26 November,the Kuber reached the coast of Mumbai, reduced its speed,and idled until darkness fell. In one of the final telephonecalls before the attack began, an unknown LeT official inPakistan instructed the men to kill the ships captain. Afterthe call ended, the militants followed their orders andbeheaded Solanki.20The AssaultIndian officials believe the LeT men came ashore on thenight of 26 November in an inflatable boat that landed nearBadhwar Park in South Mumbai.21 Other sources contendthe attackers used two inflatable boats and arrived separately at Badhwar Park and the Apollo Bunder FishingDocks.22 Upon arrival, the militants divided themselves intofive teams of two gunmen and then proceeded toward theirtargets, all of which appear to have been selected inadvance.23 An interrogation of one of the terrorists conducted after the attacks revealed that extensive surveillanceof the targets had been conducted in the months leading upto the assault. In some cases, LeT operatives had even rentedrooms in the hotels the group was interested in targeting togather details about each buildings layout.24 In at least twocases, attackers utilized public taxis to approach their

target.25 Bombs were left in both taxis by the terrorists, and

both later exploded, killing the two drivers and at least onebystander.26The ten militants were briefed in Pakistan using digitalphotos and maps obtained from the Internet to familiarizethem with the citys layout and the locations of their targets.Meanwhile, LeT had set up a remote command post in asafe house or hotel that Indian officials believed was inLahore or Karachi, Pakistan. The safe house was filled withcomputers, televisions, voice-over-Internet phones (VOIP),and satellite phones manned by six LeT terrorists whomaintained contact with the terrorist teams as they movedthrough the city.27There is no definitive account of which attack occurredfirst, but one of the earliest reports of violence came fromthe Leopold Caf, a historic restaurant and watering holepopular with foreigners and locals.28 Shortly after 2100,Hafiz Ashad and Naser entered the Leopold and beganspraying the patrons inside with machine-gun fire.29 Oneof the two men also lobbed a grenade into the tightlypacked caf. According to one eyewitness account, theassault began with what sounded like a light bulb shattering, and then screams erupted as the crowded restaurantwas raked with gunfire.30 Photos from the attack showbullet holes in the caf window and the walls and otherdamage from the explosion.31 Indian investigators say theterrorists remained inside the Leopold for about five minutes, during which time they killed ten peopleamongthem two Americansprior to heading toward the TajMahal Palace Hotel.32At about the same time diners were under attack at theLeopold, Abu Ismail Khan and Mohammad Ajmal Kasabentered the crowded Chhatrapati Shivaji Terminusor Victoria Stationand began firing indiscriminately at people onthe platforms.33 I was firing and Abu was hurling hand grenades, Kasab later recalled in court.34 I was in front of Abuwho had taken such a position that no one could see him. Ifired at a policeman after which there was no firing from thepolices side. A total of 58 people died and 104 were injuredbefore a small band of police drove the attackers from thestations terminal.35 Outside, the two militants fled across apedestrian bridge and headed toward the Cama & AlblessHospital. Together, the pair ambushed a van carrying policeofficers and counterterrorism officials, killing six out of theseven law enforcement officials riding inside. Wronglybelieving all of the vehicles occupants were dead, the militants dumped several of the bodies on the road and thencommandeered the van for themselves. Constable Arun

= Combo of Teams 1 and 2

12PM 7:30PM 11PM 9AM

Operationsend atTridentHotelwithbothmilitantsdead.

Events on28 November

Defending Mumbai from Terrorist Attack 177

Jadhav, the only officer who survived the attack, switched on

his radio and transmitted live audio from the back of thevehicle as the militants careened through the streets, shooting at targets of opportunity.36 Jadhav said the two men inthe van also fired at police officers as they drove: One ofthem laughed and said, Look, theyre wearing [bulletproof]jackets, after killing one such officer.37When the van approached the Metrobig Cinemas, thegunmen slowed the vehicles speed and opened fire on thelarge crowd gathered on the sidewalk, killing ten people.38The duo then attempted to reach the Oberoi-Trident Hotelbut was turned back by police barricades.39 When the vandeveloped a flat tire, they abandoned it and stole a Skodaautomobile. 40 The pair headed toward the sea withunknown intent. Their journey was halted when theyencountered a roadblock at Girgaum Chowpatti andbecame involved in a firefight with police that left Khandead and Kasadthe attacks only survivorwounded andin police custody.The third LeT teamShoaib and Javadsprinted intothe lobby of the Taj Mahal Palace Hotel, an iconic buildinglocated near the citys waterfront that attracts an elite clientele of businesspeople and holiday travelers, and began firing into the crowded room.41 A gunman just stood therespraying bullets around, right next to me, said SajjadKarim, a British diplomat who was inside the hotel duringthe attack.42 I managed to turn away and I ran into thehotel kitchen. . . . All of a sudden another gunmanappeared in front of us, carrying machine gun-type weapons. And he just started firing at us. . . . I just turned andran in the opposite direction. Firing wildly and tossinggrenades, the gunmen managed to kill about twenty peoplein the first few minutes of their assault.43 Shortly after theattack began, the LeT team that attacked the LeopoldCafAshad and Naserarrived in the lobby of the TajHotel and added their firepower to the carnage alreadyunfolding. Together, the four militants ascended to theupper floors of the hotel to round up hostages and fortifytheir position.The fourth LeT teamAbdul Rehman Chotta andFahadullahentered the Oberoi-Trident Hotel through themain doors about fifteen minutes after the attack began atthe Taj Hotel.44 After the militants peppered the hotels restaurant with machine-gun fire, they ignited their IEDs andshot at whoever had not escaped from the lobby. We tookthe lift to the lobby and heard bangs as the door opened,a British business traveler remembered. 45 A Japaneseman, one of four men in the lift, was shot and wounded. I

frantically pressed the close door button but had to move

the shot mans foot for the doors to close. As was the caseat the Taj Hotel, after the initial burst of violence and killing,the attackers headed for the hotels upper floors, collectedhostages, and prepared themselves for a response from theIndian security forces gathering outside.The fifth and final LeT attack teamBabar Imran andNazirassaulted a community center owned and operatedby Chabad Lubavich, a Hasidic outreach movement.46 Thefive-story building housed a rabbi and catered almost exclusively to Jews visiting India. Unlike the other targets, theChabad House was not a well-known landmark and wasfrequented neither by businesspeople nor Westerners.47 Theattackers targeted the building because they were told bytheir handlers in Pakistan that the lives of Jews were worth50 times those of non-Jews. A spokesperson for the Chabadgroup said Rabbi Gavriel Noach Holtzberg, age twentynine, telephoned the Israeli consulate to report gunmen hadentered the facility.48 In the middle of the conversation, theline went dead, the spokesperson said. Both Holtzberg andhis wife were killed sometime during the attack. Accordingto an account from an unidentified medic who entered thecenter shortly after the Indian government killed the attackers, many of the Jews in the house survived Imran andNazirs initial raid and subsequently were tortured verybadly.49At the end of the initial assaults on 26 November, four ofthe five LeT attack teams were still operational. One terrorist was dead and another had been captured, but theremaining eight militants had all taken hostages andstrengthened their positions inside the Chabad House andthe Taj and Oberoi Hotels. Sporadic gunfire between thegrowing number of Indian security forces gathering outsideand the terrorists occurred throughout the night and intothe early morning of 27 November. During this sameperiod, Mumbais first responders, a mixture of police officers and local counterterrorism officials, were secondedor replaced entirelyby military forces.50 The NationalSecurity Guards (NSG), Indias elite commando force, alsoarrived from New Delhi.Throughout the standoff at the Taj Hotel and the othertwo locations, the militants used cellular phones to keep incontact with LeT commanders in Pakistan, who were monitoring events in Mumbai by watching Indian televisioncoverage.51 The LeT commanders told the terrorists occupying the Taj Hotel to set fires so that people could see thehotel burn on television, suggesting that the attack waschoreographed with media coverage in mind.52

178 Chapter 14

Box 14.3 THE MUMBAI ASSAILANTS

Team 1: Hafiz Ashad and Naser attack the Leopold Caf near the Taj Mahal Palace Hotel. They spend five to ten minutes in caf, toss a grenadeinto the crowd of diners, then head for the Taj to join up with their comrades. At the Taj Hotel, they head to the upper floors with members ofTeam 3 and help take hostages. Both die when Indian security forces assault the Taj Hotel.Team 2: Mohammad Ajmal Kasab and Abu Ismail Khan assault the Chhatrapati Shivaji Terminus and are forced to flee outside after theyencounter the police. They move to the Cama and Albless Hospital where they ambush a police van, steal it, and attempt to drive to the OberoiTrident Hotel. The team is prevented from reaching the hotel by a police roadblock. The pair abandon the police van and then steal another car. AtGirgaum Chowpatti, a shootout ensues with police that ends with Khan dead and Kasab in police custody.Team 3: Shoaib and Javad head directly to the Taj Hotel and begin killing guests in the lobby area. The pair head upstairs, take hostages, anddo as much damage to the hotel as possible with their grenades and IEDs. When these run out, they take to igniting mattresses. Both men die aftera protracted game of cat and mouse with Indian commandos in the burning hotel.Team 4: Abdul Rehman Chotta and Fahadullah enter the main entrance of the Oberoi-Trident Hotel, proceed to the hotels restaurant, andattack diners there. They ignite two IEDs in the lobby and then head to the buildings upper floors, firing as they go. They take hostages and arekilled when NSG commandos raid the hotel.Team 5: Babar Imran and Nazir throw grenades at a gasoline station, then force their way into a community center called the Chabad House thatcaters to Jews. The pair take hostages, some of which appear to have been tortured before they were killed. NSG commandos use helicopters to land onthe centers roof. Imran and Nazir perish in the ensuing gun battle.

Unknown to the terrorists, the Indian government claimed

that it had intercepted virtually all of the conversationsbetween the attackers and their handlers back in Pakistan.Transcripts of the conversations that have been released detailhow LeT commanders kept the teams in Mumbai informedabout the movement of Indian security forces, offering advicesuch as throw one or two grenades at the Navy and policeteams, which are outside.53 The commanders also remindedthe teams that everything is being recorded by the mediaand that they needed to inflict the maximum damage.When team members grew tired or frustrated, their leadersencouraged them to keep fighting. Dont be taken alive, oneof the voices from Pakistan instructed.The EndgameOn the morning of 27 November, Indian commandosmounted an assault on the Oberoi Hotel and began room-toroom searches through the hotels 877 units.54 It was laterrevealed that at least 380 people were trapped in the hotel atthe time of the attack.55 Indian forces spent the rest of the dayand part of the next morning freeing hostages and chasingdown the two terrorists fortified inside the massive building.56When the operations concluded, both terrorists were dead.

The NSG employed a helicopter to land commandos on

the roof of the Chabad House on the morning of 28 November.57 Brother you have to fight, a LeT commander told amilitant inside during their final conversation. This is amatter of the prestige of Islam.58 The two gunmen managed to keep their Indian opponents at bay for almosttwelve hours, despite the buildings small size (in relation tothe seized hotels). 59 Six people were killed inside theChabad before the standoff was broken.60The assault on the Taj Hotel began at about the sametime as the operation at the Oberoi Hotel, but not until themorning of 29 Novembernearly two and a half dayslaterwas the landmark hotel secured.61 The difficulty atthe Taj Hotel was the number of guestsabout 450 people,many of them hiding in their roomswho needed to belocated. The task was made all the more difficult by thenumerous fires that raged inside the building (LeT attackershad been throwing grenades and igniting mattresses forseveral hours).62, 63 We were working in two teams, combing the hotel top to bottom said Sunil Kumar, an NSGcommando.64 We cleared the sixth floor and roof withoutincident. Then the fifth. Then the fourth. By the time wegot to the third floor, it was too late. There were simply toomany rooms. Many wouldnt open, even with the master

Defending Mumbai from Terrorist Attack 179

key. We had to enter by force to get people out who were

too scared to evacuate.65 As the commando teams creptthrough the smoke-filled hotel, hostages trapped upstairsunfurled banners that said Save Us from the windows oftheir rooms.66 From Pakistan, the message from the LeTcommanders was indisputable: The hostages are of useonly as long as you do not come under fire. If you are stillthreatened, then dont saddle yourself with the burden ofthe hostages. Immediately kill them.67 A total of thirty-twopeople were killed in the hotel during the three-day ordealbefore it was retaken by Indian forces.68

The AftermathMore than 160 people died, and over 300 people sustained injuries during the 60-hour rampage.69 In the wake ofthe attacks, Indian investigators quickly identified the attackers as Pakistani. It was not difficult to link the attackers toLeT once their nationality was established. By the time theinvestigation concluded, Indian officials alleged that elements within the Pakistani intelligence services had helpedLeT with the assaultor, at the very least, had known aboutthe attack and done nothing to prevent it. The governmentof Pakistan initially denied there was any connectionbetween that country and the attack.70 However, faced withhours of intercepted phone calls and a mountain of forensicevidence, Pakistani officials were ultimately forced to concede the assault was planned in their country and that thegunmen had trained in LeT camps located there. In 2009,Pakistan charged LeTs military chief and six less influentialsuspects in the Mumbai attacks and brought them to trial.US officials say, however, that the trial seems hopelesslystalled over legal complications and conflict with India.71, 72Kasab, the only gunman who survived the attack, initiallyconfessed to taking part in the attack, and he went on toprovide a great deal of information about his recruitment inPakistan, his training, and his fellow attackers.73 He laterchanged his story in court and argued that he was a touristwho had been framed by the Mumbai police. Kasab wasconvicted of murder, damage to public property, and a hostof other minor charges in May 2010. It was not a simple actof murder, the presiding judge said of the attacks at the conclusion of Kasabs trial. It was war.74 Kasab was sentencedto death. More than thirty-eight other people, most of whomlive in Pakistan, have been charged in connection to theattacks. LeT commander Rehman and at least nineteen others have been found guilty in absentia by Indian courts.

KEY TAKEAWAYS Predicting how a terrorist group might launch anattack is a daunting task. The best analyses considerthe broadest range of credible alternatives and thennarrow the list down to those that are most attentiondeserving. Structured Brainstorming provides a good methodfor ensuring that all possible options have beenconsidered; its power is that it stimulates creativethinking. Classic Quadrant CrunchingTM is amore rigorous and systematic process that usuallygenerates a robust set of alternatives because it forcesthe analyst to think about the problem from a widevariety of very different optics. When generating a list of indicators to guidecollection, analysts should focus their energies ondeveloping truly diagnostic indicators that can drivethe analysis and focus the attention of investigatorson what really matters, especially when time is ofthe essence. Collectors usually prefer working with ashort list of tailored indicators as opposed to a longlist of all possible indicators that might be relevant. In a crisis environment, imprecise and often incorrectreporting is the norm, especially when relyingon eyewitness reports. Always include with suchinformation caveats as, for example, initial reports.

Scenarios and Indicators

15 Iranian Meddling in Bahrain

his case provides a framework for tackling problems

when information is scarce. It highlights a commonproblem for intelligence analysts who have deep substantiveexpertise but are confronted with questions for which thatexpertise is necessary but insufficient to answer policy makers questions. For analysts, there is a great temptation tostart with what is known and then build a plausible analysisaround that information. A much more robust approach,however, starts with the analytic questions that need to beanswered, a full explication of the potential explanations,and a robust list of collectible indicators that can help differentiate among possible answers.While much is known in this case about the history ofthe region, internecine fighting, claims, and counterclaims,there is no direct information in the case that would helpanalysts deliver judgments about the truth of the Bahrainiclaims, Iranian denials, or opposition counterclaims.Nevertheless, US interests in the regionnot the least ofwhich include force protection issues surrounding the stationing of the US Fifth Fleet in Manama Baymake this anissue with high-level policy maker interest. In situationssuch as this, it is incumbent upon the analyst to identify notonly what is known and unknown, but also to list all possible explanations and to construct a focused collection strategy to help rule out explanations as new information iscollected in the future.The following techniques guide analysts through aprocess that helps them identify key questions in the caseusing Starbursting; explore possible alternatives for theclaims and counterclaims using Morphological Analysis;explicate the key dimensions of the problem using StructuredBrainstorming; and create specific indicators that will help

guide future collection and analysis using Indicators. Taken

together, these techniques force divergent thinking to ensurethat all angles of the problem have been actively considered.TECHNIQUE 1: STARBURSTINGStarbursting is a form of structured brainstorming thathelps to generate as many questions as possible. It is particularly useful in developing a research project, but it canalso be helpful to elicit many questions and ideas about conventional wisdom. This process allows the analyst to consider the issue at hand from many different perspectives,thereby increasing the chances that the analyst may uncovera heretofore unconsidered question or new idea that willyield new analytic insights.Using this technique, analysts can quickly determinewhat is known, what is knowable, and what will probablynot be knowable in the foreseeable future. Even moreimportant, it quickly helps identify the key questions towhich additional resources should be devoted.Task 1.Starburst the Bahraini government claim that Bahraini elements are being trained in Iranian-backed Hezbollah campsspecifically established to train assets from the Gulf in a plotto overthrow the monarchy.Step 1: Use the template in Figure 15.1 in the book or drawa six-pointed star and write one of the following words ateach point of the star: Who, What, How, When, Where, Why.Step 2: Start the brainstorming session, using one of thewords at a time to generate questions about the topic. Do

183

184 Chapter 15not try to answer the questions as they are identified; justfocus on generating as many questions as possible. (SeeFigure 15.2.)

Analytic Value Added: As a result of your analysis,

which questions or categories do you believe deserve further investigation? Are there any issues or questions inwhich your knowledge, based on the case, is particularlystrong or deficient? Many of the questions are knowable inthe Who, What, and When categories, such as who theBahraini opposition figures are, what their chief complaints

Step 3: After generating questions that start with each of

the six words, the group should either prioritize the questionsto be answered or sort the questions into logical categories.

Figure 15.2 Starbursting Bahrain Example

Who are the main Bahraini opposition figures? Who are fringe opposition figures? Who has shown a proclivity toward Iran in the past? Who supports them financially, ideologically, administratively? Who are their role models/what provides their ideological inspiration? What biographical information do we have for the main leaders? Who are their mentors? (Professors, Religious Leaders, Academic Advisors, Spiritual) Who has specific titles? (Official/Unofficial) Who are their friends, enemies, aliases, family members? What are the main oppositiongroups?

WHO?

Why are they prominent?

Why are they trying to change the social order? Why are they feared?

WH

Why are they Shia?

Where do they live?

Where have they been arrested?

WHEN?

Where are the alleged training camps?

What is their agenda?

HOW?

EER

Where have they lived?

What inspired them?

?ATHW

Y?

Where do they communicate ideas?

What communities are they

influencing?

What languages do they speak?

What have they said publicly aboutIran and the opposition efforts inBahrain? What has been said about them? How involved are they in thecommunity? How are they perceived in thecommunity?

Where does their family live?

How often do they travel?

Where do they vacation?

How do they communicate their

message? (examples: elections,sermons, Facebook)

Where do they travel?

Where do they own property? Where do they work? Where do they bank?

When did they get the attention

of the government? When have they been arrestedor detained?

How do they conduct training?

How do they raise money?

When have they traveled to

Lebanon or Iran?

How do they communicate with

colleagues?

When did they start becoming

involved in their cause?

How do they propose to realize their

agenda? How do they view the following: Iran,US, West, Hezbollah?

Iranian Meddling in Bahrain 185

are, and when they came to the attention of the Bahraini

government. Some, however, are much more difficult toanswer, such as where the alleged camps are, who has traveled there, and for what purpose. Equally important arequestions about who funds them, how they are funded, andwhy in particular they are feared. The Starburst helps toidentify the full range of questions, which can then be prioritized by analysts according to relevance, accessibility, oranother criterion. The process of identifying questions forprioritization easily translates into a strategy that can beused by a single analyst or a group to tackle an issue moreefficiently.TECHNIQUE 2: MORPHOLOGICAL ANALYSISMorphological Analysis is a method for systematicallydealing with complex, nonquantifiable problems for whichlittle information is available. It is especially useful in identifying possible variations of a threat or the way a set ofdriving forces might interact in ambiguous or informationpoor situations. Morphological Analysis works throughtwo common principles of creativity techniques: decomposition and forced association. By breaking down the problem and reassembling the various alternative dimensions, ithelps generate a comprehensive list of possible outcomes,including low-probability/high-impact and nightmarescenarios that could have adverse implications for policymakers. This process helps to identify credible alternatives.Analysts can develop collection strategies to tackle themand indicators to help them determine whether or not ascenario is unfolding.Task 2.Conduct a Morphological Analysis of the claims, counterclaims, and other possible explanations for events in thecase.Step 1: Define the set of dimensions in the case. Forexample, the main dimensionsGroup, Activity, Method,

and Impacthave already been identified in the confidential report by the Bahraini government and could be used toframe the analysis. (See Table 15.5 in the book.) Thecounterclaims by the Bahraini opposition and Iran couldalso serve as additional alternative expressions of thedimensions.Step 2: Create additional dimensions as needed.Step 3: Consider all the combinations of dimensionsto create a list of possible alternative scenarios. (SeeTable 15.6.)Identifying the main claims, counterclaims, and nullhypothesis are easily accomplished by looking down thecolumns: Bahraini opposition members receiving clandestinetraining in Iranian-backed Hezbollah camps with thepurpose of overthrowing the Khalifa monarchy. Bahraini opposition members receiving clandestinefinancial support with the purpose of overthrowingthe Khalifa monarchy. Bahraini opposition members who are overtlycampaigning for minority Shia rights but arereceiving no support. No activity.The table also helps identify several alternatives,including: Bahraini opposition members who are unwitting offinancial support that is aimed at overthrowing theKhalifa monarchy. Equally interesting is the possibility that unaffiliatedor rogue opposition members are receiving trainingin camps but the activity has no impact because theBahraini elements lack the organizational structure

Table 15.6 Bahrain Morphological Analysis Example

DimensionsGroup

Bahraini Opposition Members

Unaffiliated Opposition

No Activity

Activity

Receiving Training in Iranian-backed Hezbollah Camps

Financial Support

No Support

Method

Clandestine

Overt

Unwitting

Impact

Overthrow the Khalifa Monarchy

Obtain Greater Shia Minority Rights

No Impact

186 Chapter 15that would enable them to put the training intoaction once they return to Bahrain.Step 4: Eliminate any combinations that are impossible,impractical, or undeserving of attention.Nonsensical combinations should be discardedforexample, a scenario in which individuals receiving thetraining are unwitting of it.Step 5: Refine the scenarios so that they are clear andconcise. Bahraini opposition members are receivingclandestine training in Iranian-backed Hezbollahcamps with the purpose of overthrowing the Khalifamonarchy. Bahraini opposition members are receivingclandestine financial support with the purpose ofoverthrowing the Khalifa monarchy. Bahraini opposition members who are overtlycampaigning for minority Shia rights are receivingno Iranian support. Bahraini opposition members are receiving financialsupport with the purpose of overthrowing theKhalifa monarchy but are unwitting of the source ofthat funding. Unaffiliated or rogue opposition members arereceiving clandestine training in camps that has notyet had an impact in Bahrain.Analytic Value Added: Which scenarios are mostdeserving of attention? Do any assumptions underliethe scenarios? Certainly, the main claims and counterclaims deserve attention, but equally important in thiscase is the possibility that the opposition is unwitting thatit is receiving support from Iran. In this scenario, there isa possibility that cooptation and influence by Iran areoccurring, but the opposition is not yet aware of thatactivity. It also raises the possibility that only select individuals associated with otherwise legitimate Bahrainiopposition groups may be aware of the activity while thelarger organization is not.Are there any information gaps that affect your abilityto assess the likelihood of a scenario? Information is lacking about the locations of the alleged training camps, theindividuals who have traveled there, or the specifics relatingto alleged financial support such as bank accounts oramounts of transfers. These gaps limit our ability to assessthe likelihood of several of the scenarios.

TECHNIQUE 3: STRUCTURED BRAINSTORMING

Brainstorming is a group process that follows specific rulesand procedures designed for generating new ideas and concepts. The stimulus for creativity comes from two or moreanalysts bouncing ideas off each other. A brainstorming session usually exposes an analyst to a greater range of ideasand perspectives than the analyst could generate alone, andthis broadening of views typically results in a better analyticproduct. (See Box 15.1 in the book.)Structured Brainstorming is a more systematic twelve-stepprocess for conducting group brainstorming. It requires afacilitator, in part because participants are not allowed to talkduring the brainstorming session. Structured Brainstormingis most often used to identify key drivers or all the forces andfactors that may come into play in a given situation.Task 3.Conduct a Structured Brainstorming exercise to identify allthe factors that could help determine whether or notBahraini opposition figures are being aided by the Iraniangovernment.Step 1: Gather a group of analysts with knowledge of thetarget and its operating culture and environment.Step 2: Pass out sticky notes and marker-type pens to allparticipants. Inform the team that there is no talking duringthe sticky-notes portion of the brainstorming exercise.Step 3: Present the team with the following question: AreBahraini opposit ion groups being aided by the Iraniangovernment?Step 4: Ask them to conduct a Structured Brainstormingexercise to identify all the factors that could help determinewhether or not Bahraini opposition figures are being aidedby the Iranian government.Step 5: Ask the group to write down responses to thequestion with a few key words that will fit on a sticky note.After a response is written down, the participant gives it tothe facilitator, who then reads it out loud. Marker-type pensare used so that people can easily see what is written on thesticky notes when they are posted on the wall.Step 6: Post all the sticky notes on a wall in the order inwhich they are called out. Treat all ideas the same.

Iranian Meddling in Bahrain 187

Encourage participants to build on one anothers ideas.

Usually an initial spurt of ideas is followed by pauses as participants contemplate the question. After five or ten minutesthere is often a long pause of a minute or so. This slowingdown suggests that the group has emptied the barrel of theobvious and is now on the verge of coming up with somefresh insights and ideas. Do not talk during this pause, evenif the silence is uncomfortable.Step 7: After two or three long pauses, conclude thisdivergent-thinking phase of the brainstorming session.A list of brainstorming results appears in Figure 15.3Step 8: Ask all participants (or a small group) to go up tothe wall and rearrange the sticky notes by affinity groups(groups that have some common characteristics). Somesticky notes may be moved several times; some may also becopied if an idea applies to more than one affinity group.Step 9: When all sticky notes have been arranged, ask thegroup to select a word or phrase that best describes eachgrouping.

See Figure 15.4 for an example of affinity-clustered results.

Step 10: Look for sticky notes that do not fit neatlyinto any of the groups. Consider whether such an outlier isuseless noise or the germ of an idea that deserves furtherattention.Step 11: Assess what the group has accomplished. Whatare the main dimensions that the group has identified?Use this opportunity to refine the clusters. Take a stepback and ask what the main emphasis of the cluster is. Forexample, family, financial, or professional problems mightreflect vulnerabilities to recruitment.Step 12: Present the results, describing the key themes ordimensions of the problem that were identified.Analytic Value Added: What affinity clustersemerged? What are the key dimensions of the problem?The main affinity clusters were Family, Outside Influences,Malleable Personal Ideas, Vulnerability, Opportunity to BeInfluenced, and Foreign Actors. Upon subsequent

Figure 15.3 Bahrain List of Brainstormed Ideas

Love/Marriage

Foreign Media

Iranian Regime

Marriage/Relationships

Vulnerability

Iran

Personal Attributes

Money Needs

History of Employment

Language Spoken

Green Revolution

Unemployment

Age

Neda

Criminal Record

Ethnicity

Malleable Personal Ideas

Connection to Organized Crime

Religion

Beliefs

Narcotic Use/Distribution

Intelligence

Personal Goals

Public Statements against the West

Mentor(s)

Values

Degree of Organization

Associates

Need for Adventure

Administrative Savvy

Wealth

Need for Attention

TV Shows/Foreign Media

Ownership in Bahrain

Anger

Chance

Ownership in Iran

Injustice

How Often They Travel to Iran?

Location

Education

Travel

Social Affiliations

Religious Education

Accounting

Ties to the West

Social Background

Children

Support in West

Discontent

Family History

TV Shows

Skill of Iranian Officers

Family Ties

Contacts in Foreign Countries

Iranian Aggressiveness

188 Chapter 15

Figure 15.4 Bahrain Affinity Clusters

Family

TV Shows

Ownership in Iran

Love/Marriage

Foreign Media

Criminal Record

Children

Green Revolution

Connection to Organized Crime

Family History

Narcotic Use/Distribution

Family Ties

Malleable Personal Ideas

Marriage/Relationships

Beliefs

Personal Attributes

Personal Goals

Opportunity to Be Influenced

Language Spoken

Values

Degree of Organization

Age

Need for Adventure

Administrative Savvy

Ethnicity

Vulnerabilities

TV Shows/Foreign Media

Religion

Need for Attention

Chance

Intelligence

Anger

How Often They Travel to Iran?

Injustice

Travel

Discontent

Accounting

Outside InfluencesMentor(s)

Public Statements against the West

Contacts in Foreign Countries

Associates

Vulnerability

Education

Money Needs

Foreign Actors

Religious Education

History of Employment

Skill of Iranian Officers

Social Affiliations

Social Background

Iranian Aggressiveness

Ties to the West

Wealth

Iranian Regime

Support in West

Ownership in Bahrain

Iran

refinement, it becomes apparent that the clusters center on

the presence or absence of: Vulnerabilities Pro-Iranian influences Pro-Iranian beliefs Opportunities for cooptationThese dimensions of the problem clearly focus on factorsthat could help determine whether or not Bahraini opposition figures are being aided by the Iranian government.TECHNIQUE 4: INDICATORSIndicators are observable or deduced phenomena that can beperiodically reviewed to track events, anticipate an adversarys plan of attack, spot emerging trends, distinguishamong competing hypotheses, and warn of unanticipatedchange. An indicators list is a preestablished set of actions,conditions, facts, or events whose simultaneous occurrence

would argue strongly that a phenomenon is present or about

to be present or that a hypothesis is correct. The identification and monitoring of indicators are fundamental tasksof intelligence analysis, as they are the principal means ofavoiding surprise. In the law enforcement community, indicators are used to assess whether a targets activities or behavior are consistent with an established pattern or leadhypothesis. These are often described as descriptive indicators that look backward. In intelligence analysis, indicatorsare often described as predictive indicators that look forward.Preparation of a detailed indicator list by a group ofknowledgeable analysts is usually a good learning experience for all participants. It can be a useful medium for anexchange of knowledge between analysts from differentorganizations or those with different types of expertiseforexample, counterterrorism or counter drug analysis, infrastructure protection, and country expertise. The indicatorlist can become the basis for conducting an investigation ordirecting collection efforts and routing relevant informationto all interested parties. Identification and monitoring

Iranian Meddling in Bahrain 189

of indicators or signposts that a scenario is emerging can

provide early warning of the direction in which the future isheading, but these early signs are not obvious. The humanmind tends to see what it expects to see and to overlook theunexpected. Indicators take on meaning only in the contextof a specific scenario with which they have been identified.The prior identification of a scenario and associated indicators can create an awareness that prepares the mind to recognize and prevent a bad scenario from unfolding or help agood scenario to come about.In this exercise, instructors should encourage students tothink creatively about how to get information. In a highlydigital society, how might Bahraini opposition members usesocial media to gather information? What social mediaindicators might help analysts? What kind of informationmight be found there on associations, travel, interests,familial ties, or education, for example?Task 4.Using the Structured Brainstorming results to prompt yourthinking, create tailored indicators for each of the main scenarios developed in Task 2: Morphological Analysis.In the example below, we have focused on social mediaindicators due to space constraints and the fact that theBahraini government and opposition members haveactively used social media to organize and monitor recentprotest activities in Bahrain.Step 1: Create a list of the most attention-deserving scenarios to track for this case.For this example, we will use three scenarios generatedfrom the Morphological Analysis in Task 2: Bahraini opposition members are campaigningovertly for minority Shia rights and are receiving noIranian support. Bahraini opposition members are receiving financialsupport with the purpose of overthrowing theKhalifa monarchy but are unwitting of the source ofthat funding. Bahraini opposition members are receiving clandestinetraining in Iranian-backed Hezbollah camps with thepurpose of overthrowing the Khalifa monarchy.Step 2: Work alone, or preferably with a small group, tobrainstorm a list of indicators for each scenario.Use the dimensions developed in Task 3 to promptthinking.

Step 3: Review and refine each set of indicators, discarding

any that are duplicative and combining those that are similar.Step 4: Examine each indicator to determine whether itmeets the following five criteria. Discard those that arefound wanting.1. Observable and collectible. There must be somereasonable expectation that, if present, the indicatorwill be observed and reported by a reliable source. Ifan indicator is to monitor change over time, it mustbe collectible over time.2. Valid. An indicator must be clearly relevant to theendstate the analyst is trying to predict or assess,and it must be inconsistent with all or at least someof the alternative explanations or outcomes. It mustaccurately measure the concept or phenomenon atissue.3. Reliable. Data collection must be consistent whencomparable methods are used. Those observingand collecting data must observe the same things.Reliability requires precise definition of the indicators.4. Stable. An indicator must be useful over time toallow comparisons and to track events. Ideally, theindicator should be observable early in the evolutionof a development so that analysts and decisionmakers have time to react accordingly.5. Unique. An indicator should measure only one thingand, in combination with other indicators, shouldpoint only to the phenomenon being studied. Valuableindicators are those that not only are consistentwith a specified scenario or hypothesis but also areinconsistent with all other alternative scenarios.Scenario 1: Bahraini opposition members are campaigning overtly for minority Shia rights and are receiving noIranian support.In this scenario, the indicators center on the lack of vulnerabilities, influences, beliefs, or opportunities that wouldfacilitate cooptation by Iran. For example, there would befew or no apparent marital, family, money, professional, orcriminal problems, and no Iranian-related influences orbeliefs that would create an opportunity for Iran to influence, or coopt, the target. One potential pitfall in situationssuch as these is the failure to consider deceptive practices.For example, the absence of activities may be the result ofoperational security or a specific effort to conceal the activity. As a result, it is necessary to note the absence of activityacross the dimensions of the problem and over time.

190 Chapter 15 No demonstrated marital or familial problems

No favorable expressed opinions on Khomenei

No resumed progression indicating professional

problems

No favorable expressed opinions on Hezbollah

No inconsistency between education/training and

job No inconsistency between social media picturesshowing standard of living and reported income No inconsistency between geographic location ofhome and reported income No business problems highlighted by public recordsdata No articles or social media data on arrests,criminality, or drug or alcohol abuse No articles or social media data on perceivedinjustices toward person of interest or family Social media information reflecting marital harmony Articles or social media data illustrating soundfinances Articles or social media data indicating close-knitfamily Resumed progression indicating professional success Articles/social media data indicating drug/alcoholabstinence Articles/social media data indicating history oflawfulness No pro-Iranian content in social media postings orpublished articles by mentors, professional associates,or friends Articles or social media data indicating thatnumerous friends or immediate family members livein the United States or Europe

No membership in Iranian-backed opposition group

No favorable expressed opinions on IranianRevolution Presence of favorable expressed opinions on UnitedStates/West No favorable expressed opinions on Syrian regime No suspected ethnic Persian names in social network No indications in articles/social media of travel toIran No indications in articles/social media of travel toEurope, Asia, or Africa No indications from organizations website dataof large number of employees, branches, orinternational presence Resumed data indicating training in accounting Resumed data indicating successful experiencesmanaging large organizations Presence of social media picture postings withgeocoordinates from foreign locationsScenario 2: Bahraini opposition members are receivingfinancial support with the purpose of overthrowing the Khalifamonarchy but are unwitting of the source of that funding.In this scenario, the indicators focus on financial connections between individual opposition members and theiraffiliated groups or parties and any Iranian-linked organizations or individuals. These may be hidden. The presence ofpro-Iranian beliefs or significant personal vulnerabilitiesmay or may not be present in this scenario.

No articles/social media postings that include

favorable citations of pro-Iranian TV/movies/books

Publicly available financial information that links to

shell or front companies in third countries

Visits to United States or from Americans/Europeans

Unexplained influx of donations from dubious

sources

Descriptions in articles or social media of antiIranian influences

No articles or social media postings indicatingsupport for transnational Shiism Presence of articles or social media postingsindicating transparency of lifestyle or personalconduct No public expressions of desire to travel to/live inIran

Iranian Meddling in Bahrain 191

No resumed data indicating training in accounting

Little or no resumed data indicating successfulexperiences managing large organizations Inconsistency between social media pictures showingstandard of living and reported income Inconsistency between geographic location of homeand reported income Presence of public records data indicating businessproblems Presence of articles or social media data on arrests,criminality, or drug or alcohol abuse Some pro-Iranian content in social media postings orpublished articles by mentors, professional associates,or friendsScenario 3: Bahraini opposition members are receivingclandestine training in Iranian-backed Hezbollah campswith the purpose of overthrowing the Khalifa monarchy.In this scenario, multiple vulnerabilities are present andare compounded by more significant pro-Iranian influencesand beliefs developed over time through contact withIranian sympathizers or associates. Direct contacts withIran may also be observed. Social media references to marital or familialproblems Resumed progression indicating professionalproblems

Evidence of personal trauma (loss of family member,

for example) Some pro-Iranian content in social media postings orpublished articles by mentors, professional associates,or friends Little or no presence of articles or social mediaindicating that numerous friends or immediate familymembers are living in the United States or Europe Some articles/social media postings that includefavorable citations of pro-Iranian TV/movies/books No or little evidence of frequent visits to UnitedStates or from United States/Europe Few or no descriptions in articles or social media ofpro-Western influences Some descriptions in articles or social media of proIranian influences Articles or social media postings indicating supportfor transnational Shiism No articles or social media postings indicatingtransparency of lifestyle or personal conduct Public expressions of desire to travel to/live in Iran Favorable expressed opinions on Khomenei Favorable expressed opinions on Hezbollah Unfavorable expressed opinions on Green Revolution Membership in Iranian-backed opposition group Favorable expressed opinions on Iranian Revolution

Inconsistency between education/training and job

No favorable expressed opinions on United States/West

Inconsistency between social media pictures showing

standard of living and reported income

Favorable expressed opinions on Syrian regime

Inconsistency between geographic location of home

and reported income Presence of public records data indicating businessproblems Presence of articles or social media data on arrests,criminality, or drug or alcohol abuse Presence of articles or social media data on perceivedinjustices toward POI or family No private chats demonstrating marital harmony

Descriptions in articles or social media of antiWestern views

Descriptions in articles or social media of proIranian views Presence of suspected ethnic Persian names in socialnetwork Indications in articles/social media of travel to Iranor Hezbollah Indications in articles/social media of travel toEurope, Asia, or Africa

No articles or social media data illustrating sound

finances

Possible indications from organizations website

data of large number of employees, branches, orinternational presence

No articles or social media data indicating close-knit

family

Social media picture postings with geocoordinates

from foreign locations

192 Chapter 15Analytic Value Added: Are the indicators mutuallyexclusive and comprehensive? Have a sufficient numberof high-quality indicators been generated for each scenario to enable an effective analysis? Are the indicatorscollectible, and if so, what should be the collection priorities? The indicators in this case were generated on thebasis of the dimensions developed in Task 3, and thereforereflect the range of issues identified in the divergent phaseof Structured Brainstorming. This has resulted in a highnumber of indicators per dimension that analysts can reasonably expect to collect. The collection priorities for thiscase should focus on using the indicator sets to rule out thepossibility that opposition members are engaged in activities to overthrow the Khalifa regime, rather than ruling inactivity. Once the list has been narrowed, additional analysis and collection can be conducted to review thoroughlythe basis for judgments about activities consistent with oneor more of the scenarios. Some of the most interesting indicators surround the financial dealings of the oppositiongroups and members, their social networks, and the contentand quality of their social media activities.CONCLUSIONThe standoff between the government and opposition didnot abate in the months following the arrest of the eightopposition leaders. In June 2011, King Hamad sought todeescalate tensions by creating the Bahrain IndependentCommission of Inquiry (BICI). The five-person commissions mandate was to determine whether the events ofFebruary and March 2011 involved violations of international human rights laws and norms and to make recommendations to the government. 1 In a 500-page reportreleased in November 2011, the commission detailed government abuses and offered recommendations, some ofwhich the government took steps to implement.2 The commission found that force and firearms were used in anexcessive manner that was, on many occasions, unnecessary, disproportionate, and indiscriminate.3 The report alsodocumented 35 deaths, 559 allegations of torture, and 1,624complaints of employment termination as a result of theuprising in Bahrain.4 By early 2012, several of the boards recommendations had been implemented, including compensating families of deceased protestors and victims of torture,reviewing convictions, and promising to investigate allegations of torture.5 On 8 January 2012, Bahrains cabinet proposed granting more power to the elected legislature inorder to achieve greater balance between the executive and

the legislative, but no effort was made to increase Shia representation in the political sphere.6In addition to general recommendations to establishmore independent institutions to investigate and overseecurrent and future claims of abuses, the commission offeredspecific recommendations to address the following: The use of force, arrest, treatment of persons incustody, detention, and prosecution in connectionwith the freedom of expression, assembly, andassociation. Demolition of religious structures, termination ofemployees of public and private sectors, dismissal ofstudents, and termination of their scholarships. Media incitement issues. Better understanding and appreciation of humanrights, including respect for religious and ethnicdiversities.7In many respects, however, the commissions recommendations and the governments response were too littleand too late. For example, the government instituted a newcode of conduct calling on police to be respectful of humanrights principles; however, the governments detention ofhundreds of opposition members in the months precedingand following the commissions report only fueled opposition calls for reforms and sparked additional protests thatwere met with government force.8 In addition, the arrestand sentencing of forty-eight Bahraini doctors and nursesto five to fifteen years in prison for treating injured protestors fanned the flames of dissent and elicited stern rebukesfrom international institutions.9 UN Secretary General BanKi-Moon, through his spokesperson, expressed his deepconcern over the harsh sentences handed down in Bahrainto civiliansmedical professionals, teachers and othersbythe Bahraini military Court of National Safety, pointing outthat proceedings were conducted under conditions thatraised serious questions of due process irregularities.10 Inthe months following the report, clashes between police andprotesters continued, prompting the Office of the U.N. HighCommissioner for Human Rights to issue a statement onworrying reports about the use of tear gas, rubber bullets,and birdshot pellets. The OHCHR said reliable sourcesindicated that a number of deaths were linked to the use oftear gas fired by security forces into crowds and called onthe government of Bahrain to investigate the alleged use ofsuch excessive force.11

Iranian Meddling in Bahrain 193

BahrainiIranian relations cooled further in the wake of

the protests. The Bahraini government, in its official capacity and through unofficial forums and social networkingsites, accused almost every opposition leader of being influenced by or connected to Iran. It also accused internationalhuman rights organizations that had voiced support for theopposition movement of collusion with Iran. Both sideswithdrew their ambassadors in 2011.Whether or not any of the 14 February protesters hadlinks to Iran or received training and support via Hezbollah,however, remains an unanswered question. The Bahrainigovernment publicly offered no evidence of direct Iranianmeddling or support to the arrested opposition activists,and the opposition leaders remained in detention through2011. In November 2011, Bahrain issued new accusations,stating that it had arrested five members of an underground

terrorist cell with direct links to the Iranian Revolutionary

Guard Corps who were plotting to attack Bahraini government buildings and the causeway linking Bahrain to SaudiArabia.12 Bahrain released neither the names nor any evidence proving the alleged links, and protests continued wellinto 2012 unabated.KEY TAKEAWAYS In the absence of direct reporting, use divergenttechniques such as Starbursting and StructuredBrainstorming to develop a robust set of questionsand issues for research. Indicators help focus research on relevant, collectibleinformation that can be used to focus collection andmitigate the human tendency to see what one expectsto see and to overlook the unexpected.

Assessment of Cause and Effect

Scenarios and Indicators

16 Shades of Orange in Ukraine

ne of the most important ways that analysts can help

policy makers prepare for uncertain future outcomes is to identify the key factors at play and explaintheir dynamics. It is sometimes tempting to offer predictions about how a situation will turn out, but single-pointforecasts of distant outcomes are nearly always incorrectand seldom are relevant to the considerations required forsound policy decisions. Effective foreign and security policy must be applicable to a range of possible outcomes,and policy makers need a good sense of which factors theycan influence as they attempt to maximize the chancesthat events will conform to the nations interests. Moreover, they must consider the potential opportunity costsof policy optionsthe impact that a given approach to onesituation might have on an important goal in anotherpolicy area.In this case, students face the temptation to focus theiranalysis on which candidate is most likely to win the presidential election. The case narrative concentrates largely ondomestic developments in Ukraine, as it is designed to simulate the focus of analysts responsible for understandingthe countrys internal politics. Such a focus can come at theexpense of identifying critical external factors, however.Box 16.2 on Russia and Box 16.3 on Georgia in the caseprovide clues about the kinds of external factors that couldaffect the outcome of the election. The StructuredBrainstorming, Outside-In Thinking, and Simple Scenariostechniques help analysts overcome the temptation to offersingle-point electoral predictions or focus on too narrowa set of driving factors. Taken together, they frame an

analytic process that can identify all relevant factors

direct and indirect, external and internaland aid in understanding the interrelationships among them. Instructorsshould encourage analysts to consider carefully the processby which they complete the tasks in these exercises, becauseit is applicable to many analytic support situations.TECHNIQUES 1 & 2: STRUCTUREDBRAINSTORMING AND OUTSIDE-IN THINKINGBrainstorming is a group process that follows specific rulesand procedures designed for generating new ideas and concepts (see Box 16.4). The stimulus for creativity comes fromtwo or more analysts bouncing ideas off each other. Abrainstorming session usually exposes an analyst to agreater range of ideas and perspectives than the analystcould generate alone, and this broadening of views typicallyresults in a better analytic product.Outside-In Thinking helps analysts who are familiarwith issues related to their own fields of specialization consider how factors external to their areas of expertise couldaffect their analyses. This technique is most helpful whenconsidering all the factors at play at the beginning of ananalytic process. Outside-In Thinking can reduce the risk ofanalytic failure by helping analysts identify external factorsand uncover new interrelationships and insights that otherwise would be overlooked.Using these two techniques together prompts analysts toconsider the full range of factors that could shape the outcome of the election.

195

196 Chapter 16

Box 16.4 EIGHT RULES FOR SUCCESSFUL

BRAINSTORMING1. Be specific about the purpose and the topic of thebrainstorming session.2. Never criticize an idea, no matter how weird, unconventional,or improbable it might sound. Instead, try to figure out howthe idea might be applied to the task at hand.3. Allow only one conversation at a time and ensure thateveryone has an opportunity to speak.4. Allocate enough time to complete the brainstormingsession.5. Engage all participants in the discussion; sometimes thismight require silent brainstorming techniques such asasking everyone to be quiet for five minutes and write downtheir key ideas on 3 5 cards and then discussing whateveryone wrote down on their cards.6. Try to include one or more outsiders in the group to avoidgroupthink and stimulate divergent thinking. Recruit astutethinkers who do not share the same body of knowledge orperspective as other group members but have somefamiliarity with the topic.7. Write it down! Track the discussion by using a whiteboard, aneasel, or sticky notes.8. Summarize key findings at the end of the session. Ask theparticipants to write down their key takeaways or the mostimportant things they learned on 3 5 cards as they departthe session. Then, prepare a short summary and distribute thelist to the participants (who may add items to the list) and toothers interested in the topic (including those who could notattend).

Task 1.Conduct a Structured Brainstorming of the factors that willdetermine the outcome of the Ukrainian election.Step 1: Pass out sticky notes and marker-type pens to allparticipants. Inform the team that there will be no talkingduring the sticky-notes portion of the brainstormingexercise.Students will be limited to the case study for this exercise, but it is important to point out that in real-life situations, it is helpful to include in the brainstorming groupboth experts on the topic and generalists who can providemore diverse perspectives. When only those working theissue are included, often the groups perspective is limited to

the stream of reporting it reads every day; as a result, key

assumptions remain unchallenged, and historical analogiescan be ignored.Step 2: Display the following focal question for the team:What are all the factors that will determine who will be thenext Ukrainian president?Step 3: Ask the group to respond to the question by writing a few key words on their sticky notes. After a response iswritten down, the participant gives it to the facilitator, whothen reads it out loud. Marker-type pens are used so thatpeople can easily see what is written on the sticky noteswhen they are posted on the wall. Urge participants to useshort phrases rather than long sentences.Step 4: Post all the sticky notes on a wall in the order inwhich they are called out. Treat all ideas the same. Encourage participants to build on one anothers ideas. Usuallythere is an initial spurt of ideas followed by pauses asparticipants contemplate the question.It is important to emphasize the importance of avoidingmirror imaging. In a classroom situation, many studentsmay not know much about the Ukrainian political landscape; this is why it is important to ensure that all participants read the case study with the relevant backgroundmaterial carefully. They should have the case study at handfor quick reference.By using the case narrative, students should quicklyidentify the internal political factors that will most likelyshape the election landscape. These include the most likelycandidates and their bases of support and the election environment, including media freedom and role of nongovernmental organizations (NGOs) working in the country.Step 5: After five or ten minutes there is often a longpause of a minute or so. This slowing down suggests thatthe group has emptied the barrel of the obvious and isnow on the verge of coming up with some fresh insightsand ideas. Do not talk during this pause, even if the silenceis uncomfortable.Step 6: After two or three long pauses, encourage Outside-In Thinking by asking the group specifically to focuson identifying external factors that could affect the outcomeof the Ukrainian election. Use the mnemonic STEEP +2(Social, Technological, Economic, Environmental, Political,plus Military and Psychological) to catalyze the process.

Shades of Orange in Ukraine 197

During this phase, students should begin to note the

potential role of the United States, European Union (EU),Russia, international institutions such as the Organizationfor Security and Cooperation in Europe (OSCE), and foreign NGOs. In addition, the use of STEEP +2 should elicitfactors such as the roles nontraditional media, cell phones,and social media sites may play in sharing information andrallying support. During this phase students might note theRose Revolution in Georgia, the psychological impact thatthis event might have on Ukrainians, and the possibility oflinks between the opposition in both countries.Give the students a few minutes of brainstorming andpauses to think about the issue and jot down a few ideas.Then go around the room and collect the sticky notes. Readthe responses slowly and post them on the wall or thewhiteboard in random order as you read them. A list ofbrainstorming results appears in Figure 16.3.

Step 7: Ask all participants (or a small group) to go up to

the wall and rearrange the sticky notes by affinity groups(groups that have some common characteristics). Somesticky notes may be moved several times; some may also becopied if an idea applies to more than one affinity group.If only a subset of the group goes to the wall to rearrange the sticky notes, then ask those who are remaining intheir seats to form small groups and come up with a list ofkey drivers or dimensions of the problem based on thethemes they heard emerge when the instructor read out thesticky notes. This keeps everyone busy and provides a useful check on what is generated by those working at thewhiteboard.Step 8: When all sticky notes have been arranged, ask thegroup to select a word or phrase that best describes eachgrouping.

Psychological impact of Rose Revolution

Role of technology Likelihood of a coup L ikelihood of debilitating violence against one orboth of the leading candidates Role of organized crime P rospects for NATO and EU enlargement and membership for Ukraine

198 Chapter 16See Figure 16.4 for an example of affinity-clusteredresults.Only two clusters are shown in Figure 16.4, but four orfive themes usually emerge from this part of the exercise.In this case, a notional set of groups might include thefollowing:

wooing one or more of its significant members away from

Yushchenkos camp?Expected candidates and their bases of support: Howthe candidates conduct their campaigns, including theirability to garner support from voters and business leaders,will affect voter turnout and financial support. The degreeof corruption and fraud are key unknowns. Leonid Kuchmas maneuvering.Role of the media: The media are largely controlled bythe government in Ukraine and present few, if any, oppos Expected candidates and their bases of support(Viktor Yushchenko, Viktor Yanukovych).ing political viewpoints. The opposition at their Februaryconvention showed a creative use of technology and non Role of the media.traditional media to broadcast their message. Also, there Russian influence.is an underlying assumption that control of the media will US/EU/Western influence.only help the incumbent, when it is possible that the lackof alternative perspectives could encourage an engaged Business interests.electorate to seek out nontraditional sources of informa Nongovernmental organizations.tion. A gap that additional research could fill is the Popular sentiment.extent to which the opposition is tapping other formsof communication and, if it is, what these forms ofStep 9: Assess specifically how each of these forces andcommunications are.factors could have an effect on the problem and, using thisRussian influence: The case narrative highlightslist of forces and factors, generate a list of areas for addistrong motivations to discourage a Yushchenko presitional collection and research.dency, but the case does not identify specifically RussiasKuchmas maneuvering: Kuchma is taking steps to alterpotential means for influencing a transition. Russiasthe constitution to deprive the new president of significantmeans of influencing the outcome and indications thatpowers. Kuchma has been accused in the past of unscrupuMoscow is exercising those means are an avenue for furlous dealings, raising questions about just how far he will gother research. If Russia sees Ukraine as its most importantto ensure Yanukovychs victory and how effective he mightforeign policy issue, how far will it go to protect its interbe in doing so. Would he try to prolong his own rule by proests in Ukraine?voking a crisis? Would he take ruthless steps to silence theUS/EU/Western influence: The United States and otheropposition? Or would he attempt to divide the opposition byWestern countries, including international organizations,have provided aidvia foreign NGOs and internationalFigure 16.4 Ukraine Brainstorming Affinity Cluster Examplesinstitutions such as theCouncil of Europe, the OSCE,etc.to fledging civil societyorganizations in other countries. To what extent are theyRUSSIANfunding these organizations inROLE OFNewRussia?

INFLUENCEMEDIAmediaUkraine and to what effect?RussianState ofenergyBusiness interests:MediaRussianmediainterestscoverabusinessgefreedomUkrainianbusinesspeople areinterestsianRusss andseinin a position to influence thebusalpolitic nsctioconneelection by providing financial support to the candidatesand enabling access to themedia. Some businesspeoplehave withdrawn their support

Shades of Orange in Ukraine 199

for Yanukovych and are backing Yushchenko. Which businesspeople are supporting the main candidates, how strongis their support, and how might their support tip the balance in one direction or the other?Nongovernmental organizations: NGOs are operatingin Ukraine. To what extent can NGOs organize the kinds ofactivities that took place in Georgias Rose Revolution? Towhat extent is Kuchma taking preemptive action to preventsuch activities?Popular sentiment: How does the Ukrainian electorate perceive the candidates and the contest in general?What are their perceptions of Western or Russian involvement? And what will be their level of voter turnout andactivism?Analytic Value Added: What key factors will influence the outcome of the election? What gaps deserveadditional attention? The value added by this combinationof Structured Brainstorming and Outside-In Thinking isnot only the list of driving factors but also a clear exposition of why the factors could influence the outcome andhow additional collection can narrow the range of uncertainty by filling important information gaps. This processcan focus information collection tasks on the most meaningful and potentially fruitful avenues of inquiry becauseanalysts have focused on factors that they have reason tosuspect will influence the outcome and the specific information needs surrounding them. Some gaps are knowable,and information can be collected. Some of them are notknowable, but the mere act of considering them helps analysts identify the variables at play and place bounds aroundtheir uncertainty.

Task 2.Conduct a Simple Scenarios analysis to consider the rangeof possible outcomes and driving factors that will shape theoutcome of the Ukrainian election.Step 1: Clearly define the focal issue and the specific goalsof the Simple Scenarios exercise.In this case, the task above defines the focal issue, butstudents may want to consider whether any other focalissues warrant further consideration.Step 2: Make a list of forces, factors, and events that arelikely to influence the future.Students can draw from the list of factors developedusing Techniques 1 and 2 or brainstorm a list of factors thatwould have some effect on the issue being studied.Step 3: Organize the forces, factors, and events that arerelated to each other into five to ten affinity groups that areexpected to be the driving forces in how the focal issue willevolve.Again, students can use their previous list and/or tailoror augment it to include the most relevant grouping of factors. For this case, those notional groups of factors includedthe following: Kuchmas maneuvering. Expected candidates and their bases of support. Role of the media. Russian influence. US/EU/Western influence.

TECHNIQUE 3: SIMPLE SCENARIOS

Business interests.

The Simple Scenarios technique helps analysts develop an

understanding of the multiple ways in which a situationmight evolve. The technique can be used by an individualanalyst or a group of analysts. In either situation, the analytic value added of Simple Scenarios lies not in the specifics of the scenarios themselves but in the analytic discussionof which drivers will affect a particular scenario, the implications of each scenario for policy makers, and the indicators that will alert policy makers to the fact that such afuture is unfolding.In this case, the simple act of creating multiple scenariosfor how the situation will unfold forces the analyst to moveaway from calling the winner of the election and insteadconsider how the drivers can vary to produce radically different results.

Nongovernmental organizations. Popular sentiment.Step 4: Write a brief description of each or use thedescriptions previously developed.Kuchmas maneuvering: Kuchma is taking steps to alterthe constitution to deprive the new president of significantpowers. Kuchma has been accused in the past of unscrupulous dealings, raising questions about just how far he will goto ensure Yanukovychs victory and how effective he mightbe in doing so. Would he try to prolong his own rule byprovoking a crisis? Would he take ruthless steps to silencethe opposition? Or would he attempt to divide the opposition by wooing one or more of its significant members awayfrom Yushchenkos camp?

200 Chapter 16Expected candidates and their bases of support: Howthe candidates conduct their campaigns, including theirability to garner support from voters and business leaders,will affect voter turnout and financial support. The degreeof corruption and fraud are key unknowns.Role of the media: The media are largely controlled bythe government in Ukraine and present few, if any, opposing political viewpoints. The opposition at their Februaryconvention showed a creative use of technology and nontraditional media to broadcast their message. Also, there isan underlying assumption that control of the media willonly help the incumbent, when it is possible that the lackof alternative perspectives could encourage an engagedelectorate to seek out nontraditional sources of information. A gap that additional research could fill is the extentto which the opposition is tapping other forms of communication and, if it is, what these forms of communicationsare.Russian influence: The case narrative highlights strongmotivations to discourage a Yushchenko presidency, but thecase does not identify specifically Russias potential meansfor influencing a transition. Russias means of influencingthe outcome and indications that Moscow is exercisingthose means are an avenue for further research. If Russiasees Ukraine as its most important foreign policy issue, howfar will it go to protect its interests in Ukraine?US/EU/Western influence: The United States and otherWestern countries, including international organizations,

have provided aidvia foreign NGOs and international

institutions such as the Council of Europe, the OSCE, etc.to fledging civil society organizations in other countries. Towhat extent are they funding these organizations in Ukraineand to what effect?Business interests: Ukrainian businesspeople are in aposition to influence the election by providing financialsupport to the candidates and enabling access to the media.Some businesspeople have withdrawn their support forYanukovych and are backing Yushchenko. Which businesspeople are supporting the main candidates, how strong istheir support, and how might their support tip the balancein one direction or the other?Nongovernmental organizations: NGOs are operatingin Ukraine. To what extent can NGOs organize the kinds ofactivities that took place in Georgias Rose Revolution? Towhat extent is Kuchma taking preemptive action to preventsuch activities?Popular sentiment: How does the Ukrainian electorateperceive the candidates and the contest in general? Whatare their perceptions of Western or Russian involvement?And what will be their level of voter turnout and activism?Step 5: Generate a matrix with the list of drivers down theleft side, as shown in Table 16.3.Step 6: Generate at least four different scenarios: a bestcase, a worst case, mainline, and at least one other.

Ukrainian Business Interests

Shades of Orange in Ukraine 201

Best Case: Democratic Transition.

Worst Case: Constitutional Coup. Mainline: Triumph of the Oligarchs. Additional: Ukraines Rose Revolution.Step 7: The columns of the matrix are used to describethe scenarios. Each scenario is assigned a positive or negative value for each driver. The values are strong or positive(+), weak or negative (), and blank if neutral or no change.An easy way to code the matrix is to assume that the scenario occurred and ask, Did driver A exert a strong, weak,or neutral influence on the outcome?Step 8: This is a good time to reconsider both the driversand the scenarios. Is there a better way to conceptualize anddescribe the drivers? Have any important forces been omitted? Look across the matrix to see the extent to which eachdriver discriminates among the scenarios. If a driver has thesame value across all scenarios, it is not discriminating andshould be deleted or further defined. To stimulate thinkingabout other possible scenarios, consider the key assumptions that were made when deciding on the most likely scenario. What if some of these assumptions turn out to beinvalid? If they are invalid, how might that affect the outcome, and are such alternative outcomes included withinthe available set of scenarios?For the purposes of the matrix, it is best to disaggregatethe candidates so that Yushchenkos opposition andYanukovychs government-supported maneuvering areindependent drivers. The media have the same value acrossall scenarios, which might have marked the driver for deletion, but in this case, the medias role can vary widely. As aresult, the driver should be retained, and the variationshould be described in the story for each scenario. Forexample, in the story for the best-case scenario, state mediacoverage is heavily tilted toward Yanukovych, butYushchenko receives some coverage and significant fundingfrom some oligarchs. In the alternative scenario, on theother hand, Yushchenko is shut out from the mainstreammedia, but his following grows through public appearancesand his Internet presence.One interesting outcome of this coding exercise is thesimilar coding for the worst-case and mainline scenarios.Upon further examination, this is because a fundamentalassumption for both is that the presidency is stolen,whether through maneuvering in the legislature or throughunfair and fraudulent conduct of the election.

Step 9: For each scenario, write a one-page story to

describe what the future looks like and/or how it mightcome about. The story should illustrate the interplay of thedrivers.Key elements in the one-page stories for the four scenarios we have generated might include these:Best case (democratic transition): Electionsare held as scheduled. The campaigns proceed with littlediscord. State media coverage is heavily tilted towardYanukovych, but Yushchenko receives some coverageand significant funding from some oligarchs, includingDnipropetrovsk clan leader Viktor Pinchuk. Russiasends funding to Yanukovych but refrains from blatantinterference or endorsement, hoping to leave the dooropen to pragmatic relations with whoever wins the election. Kuchma fails to win two-thirds majority approval ofthe Rada for the constitutional reform bill. Pressure fromthe OSCE, the Council of Europe, the United States, andthe European Union deters Kuchma from the most egregious options to cook the election books. Meanwhile, theUS bilateral relationship with Russia improves andincludes a pledge by both sides to respect the will of theUkrainian people on both the presidential election andNATO membership.Worst case (constitutional coup): The Rada approvesthe constitutional reform bill by a vote of 3000, with OurUkraine and other opposition groups boycotting the vote.True to his word, Yushchenko, along with Tymoshenko,leads a massive campaign of protests and civil disobedience.Aside from several thousand demonstrators in Kyiv, however, the Ukrainian people are unmoved, and Kuchmaseizes the opportunity to declare a state of emergency.Kuchma strikes a deal with Russia to join the CommonEconomic Space and gets a long-term gas deal on favorableprice terms for Ukraine. In response to Western criticism,Kuchma pulls Ukrainian troops from Iraq, and Putin offersdirect support of Kuchmas actions by crediting Kuchmasstrong leadership in averting a full-blown crisis.Mainline (triumph of the oligarchs): Kuchmas constitutional reform bill fails by a narrow margin. Donetsk clanhead Renat Akhmetov strikes a deal with Dnipropetrovskclan head Viktor Pinchuk, aligning all of Ukraines businessclans behind Yanukovych. Kuchma chief of staffMedvedchuk travels to Moscow in April to get a briefingfrom Russias intelligence chiefs on the lessons learned fromthe Rose Revolution in Georgia, and the regime cracks downon foreign NGOs and arrests leaders of a nascent youth

202 Chapter 16organization in May. In August, key Yushchenko ally YuliaTymoshenko dies in a car bombing, and Kuchmas pastinvolvement in the killing of opposition journalist Gongadzeprompts speculation that his government arranged theassassination. With US and EU support, the OSCE withdraws its election-monitoring team, declaring that the newcircumstances preclude a free and fair election. Yushchenkomanages to qualify for a runoff election in the first round ofvoting on 31 October, but he loses the runoff vote toYanukovych. Ukrainian NGOs claim the vote involved massive fraud, but the regime precludes alternative vote countefforts, and opposition calls for protest spark little actionfrom the public.Additional scenario (Ukraines Rose Revolution):Kuchmas constitutional reform bill falls short of winning atwo-thirds majority in the Rada. Ukraines oligarchs align insupport of the Yanukovych campaign, and Russia intervenesheavily in support of Yanukovych, fueling a nationalist backlash that benefits the Yushchenko candidacy. It also reinforces the determination of international organizations andWestern-financed NGO groups to organize alternative votecounts and strict election monitoring. Activists fromGeorgias Rose Revolution train their Ukrainian counterparts in civic organization and popular mobilization.Yushchenko is shut out from the mainstream media, but hisfollowing grows through public appearances and his Internetpresence. Much as in Georgias Rose Revolution, the regimeclaims its candidate won the election, but the public protestsagainst the perception of massive fraud and the governmentcannot rely on security forces to stop the demonstrators,who peacefully take over state television and key ministriesand declare Yushchenko president. Sensing the inevitable,Yanukovych concedes the election to Yushchenko, andKuchma and his key associates flee to Russia.Step 10: For each scenario, describe the implications forthe decision maker. The implications should be focused onvariables that the United States could influence to shape theoutcome.Following are some examples: Best case (democratic transition): US diplomaticoutreach to Russia and a bilateral agreement torespect the Ukrainian democratic process are keymeans of holding Russian influence in abeyance. Worst case (constitutional coup): The key variablein this scenario is the vote in the Rada, over whichthe United States exerts little influence.

Mainline (triumph of the oligarchs): The

withdrawal of the election-monitoring team removesthe key means through which the United States canencourage free and fair elections. Additional (Ukraines Rose Revolution):Engagement via election monitoring and support tocivil society organizations helps ensure a democraticprocess can be followed, if the sides allow it tobe. These organizations can be encouraged to usenontraditional media to get their message out.Step 11: Generate a list of indicators for each scenariothat would help you discover that events are starting to playout in the way envisioned by the scenario.Some general indicators might include the following, butinstructors should encourage analysts to define the indicators with as much specificity as possible. For a more robustindicators process, employ a full Indicators and IndicatorsValidatorTM process.1 Best case (democratic transition): Stateinstitutions uphold the letter and intent of law.Instances of harassment attributed to the governmentare rare. Few complaints are filed with the CentralElection Commission. Opposition media flourishesand gains a stronger representation among sources ofinformation. Russia takes a hands-off approach. Worst case (constitutional coup): Theconstitutional reform bill passes. Instances ofviolence during the campaign occur against bothcandidates. Government institutions take measuresto strengthen presidential powers. Mainline (triumph of the oligarchs): Theoligarchs resist the urge to split their forces andresources and instead remain united in support ofYanukovych. State and partisan lines are blurred.Instances of violence during the campaign intimidatethe opposition and reduce turnout for or frequencyof rallies. Additional (Ukraines Rose Revolution):Opposition media do not cower in response tointimidation. New media sources pop up as othersare shut down or their operations are constrained bygovernment activities. New media sources are usedas an organizing force by opposition groups. Theoligarchs split their support for the main candidates.The Russians play a vocal, partisan role in favor ofYanukovych; there are signs of a popular backlash insupport of Yushchenko. The opposition redoubles its

Shades of Orange in Ukraine 203

efforts in the face of intimidation tactics resulting in

more rallies, more media coverage, and higher voterturnout.Step 12: Monitor the list of indicators on a regular basis.Analytic Value Added: What judgments should analysts highlight in response to US policy makers questionsabout what will influence the outcome of the Ukrainianelection? It is often helpful to advise students before theyembark on this portion of the exercise that forecasting isone of the hardest tasks an analyst faces. The Simple Scenarios technique is not a means that will produce a resultthat can then be parroted to policy makers. Rather, thetechnique is designed as a means to identify and activelyconsider how each outcome could come about. This processcan help the analyst knowand warn policy makersifone future or another is emerging. The goal is to help policymakers understand the dynamics at play and the most plausible outcomes that can be produced by various permutations of the dynamics.Analysts should therefore identify not only the implications identified in the exercise but also the key indicatorsthat would suggest that an outcome is occurring. Forexample, the level and nature of Russian involvementanexternal factorfigure as a key driver in several scenarios.Students should be able to define the hallmarks of Russianbehavior that would contribute to the relevant scenarios.In the best-case scenario, Russia would take a relativelyhands-off approach, while in the worst-case scenario, theRussians would most likely aid and abet Kuchmas grip onpower.Another way to test the students understanding of theanalytic value added is to have them develop a graphicalrepresentation of the key findings of the previous threeexercises. This exercise encourages analysts to distill the keyjudgments, drivers, and assumptions about the range ofpossible outcomes rather than create a tome that simplysummarizes the results.Yet another means of testing students understanding isto ask them how confident they are that a particular outcome will occur. Then ask what would need to occur toincrease or decrease their confidence. This questioningmethod often helps students identify indicators, gaps, andassumptions that they have not yet considered. Next, askthem how they could track the indicators, close the gaps,and check the assumptions that they have identified. Thisprocess can become the basis for an information collectionstrategy that will guide further research.

CONCLUSIONUkraines presidential transition wound up producing whatbecame known popularly as the Orange Revolution, butin retrospect it is apparent that this outcome was far frompreordained; several other alternative scenarios came closeto being realized (see Figure 16.5 for a chronology of thisperiod). Constitutional reform, for example, proved to be anear miss. On 8 April 2004, Ukraines Rada fell just sixvotes short of the two-thirds majority needed to passKuchmas constitutional reform bill.2 Opposition blocs boycotted the vote, and the government failed to garnerenough support from independent deputies to carry theday. The Rada chair declared the bill dead until sometimeafter the presidential elections, and the leaders of pro-government parties in the legislature voted to unite behindYanukovychs candidacy.3The campaign turned out to be a bare-knuckled contest.The governments intended tactics became clear in themayoral election in Mukachevo held in April, when theregime employed gross falsifications and pure thuggeryat the polling stations to defeat a popular Yushchenko ally,alarming opposition groups.4 As the presidential campaignprogressed over the summer into the fall, Kuchmas operators pulled out all the stops to bolster Yanukovych, butmany of their tactics proved counterproductive. The government regularly issued so-called temnykyinformalguidance on coverageto media organizations. Statecontrolled television coverage amounted to little more thancrude propaganda, and the refusal to broadcastYushchenko only encouraged larger attendance at his campaign events by voters curious to learn about him. 5Yushchenkos campaign also faced near-constant harassment. At one point, a truck attempted to force his motorcade from the road, and in September he was taken ill witha mysterious malady that nearly took his life. Austrian doctors diagnosed the illness as dioxin poisoning; Yushchenkoaccused the Kuchma regime of involvement, but the perpetrators were never identified. The poisoning leftYushchenkos once handsome face badly scarred, but it alsocemented his image as a courageous opponent of theregimes brutality and redoubled his determination to winthe presidency.6Like the Kuchma regime, Russia intervened massively insupport of the Yanukovych campaign, but if anything itsefforts backfired. To all appearances, Russian PresidentPutin made the Ukrainian election a personal mission,meeting with Kuchma on an almost monthly basis during

204 Chapter 16the campaign, coming out publicly in favor of Yanukovychin July, and even campaigning for Yanukovych in Ukraineon the eve of the election.7 Dozens of Russian political consultants descended upon Ukraine, appearing frequently onUkrainian- and Russian-language television shows praisingYanukovych and criticizing Yushchenko.8 Hundreds of millions of dollars in Russian money poured into Yanukovychcampaign coffers.9 The Kremlins campaign came across asa transparent attempt to impose its will on Ukraine and mayactually have hurt Yanukovych.10Arrayed against the Kuchma regime, Russia, andYanukovych were Ukraines opposition groups and a rangeof NGOs. For several years, the United States, Europe, andprivate donors had been funding Ukrainian NGOsinvolved in voter education, judicial reform, and electionmonitoring, and these groups in turn had developed anextensive network of local activists and officials trained inelection laws and community organization.11,12 In parallel,several independent Internet media sites were established,including the cyber-newspaper Ukrainska Pravda, whichbecame a key source of news on the Yushchenko campaign,and the website Maidan, which served as a virtual civicorganization in cyberspace for regime opponents.13 In lateMarch 2004, a Ukrainian student organization named Pora(Its Time) emerged, modeled on groups that had helpedto topple presidents in Serbia and Georgia; it providedboth formal and informal support for the Yushchenkocampaign, despite harassment by the regime that Poraactivists sometimes captured on cell-phone cameras.14 TheUnited States adopted a neutral stance toward the candidates but pressed the Kuchma government to ensure a freeand fair electoral process.15 In May 2004, then DeputyAssistant Secretary of State Steven Pifer told the HouseInternational Relations Committees Subcommittee onEurope thatthe US Government does not back any particular candidate in the election; our interest is in a free and fairelectoral process that lets the Ukrainian people democratically choose their next president. We would beprepared to work closely and eagerly with whomeveremerges as president as the result of such a process.16

He added that the single most important issue now

on our bilateral agenda is the conduct of the Ukrainianpresidential campaign and election and the upcomingpresidential election . . . will affect Ukraines strategic coursefor the next decade.17 Monitors from the Organizationfor Security and Cooperation in Europe (OSCE) worked

toward this end on the ground, keeping a watchful eye

on the conduct of the campaign and the preparations forvoting.18The voting on 31 October divided the country. It produced a virtual tie between the two leading candidates,with Yushchenko officially garnering 39.90 percent of thevote compared to 39.26 percent for Yanukovych. Yanukovych won 71 percent of votes in the east and south, andYushchenko took 78 percent of the western and centralregions. OSCE monitors reported numerous irregularities,and fed-up journalists at state-run television stationsbalked at obeying the regimes temnyky, signaling important fractures in the Kuchma governments power base.19,20The precipitous drop in votes for Communist candidateSymonenko compared to both his own performance in1999 and his partys support in the 2002 Rada election suggested that some of his votes had been fraudulently reallocated to Yanukovych, and an enraged Symonenko urgedhis supporters to vote against both candidates in the runoff election that was to be held on 21 November, asrequired by Ukraines election laws.21The run-off was marred by massive falsification.22 TheCentral Electoral Commission declared Yanukovych thewinner with 49.5 percent of the vote versus 46.6 percent forYushchenko. Opposition groups immediately rejected theresults, citing independent exit polls that indicatedYushchenko had won 53 percen