How useful are our intelligence agencies? To what degree are they blinded by prejudice and institutional needs?

Intelligence about foreigners — state and non-state actors — is vital for the US government, of course. But how useful has US intelligence been in practice? This post is the first in a series examining several perspectives from the large literature on this important subject. The other chapters are

At one extreme we have totalitarian states (in many forms), in which even technically capable intelligence services can become warped so that they maintain the ruling elites’ view of the world — no matter how inaccurate (for examples of this see Christopher Andrew’s articles listed at the end). At the other end are the idealized intelligence services of fiction (e.g., Ian Flemming’s Secret Service and Tom Clancy’s CIA). Real-life western agencies fall somewhere in between. But where? Histories of the CIA (such as Tim Weiner’s Legacy of Ashes) often highlight the many failures; a balanced accounting is difficult to find.

Analysis has its own ethical considerations, and these largely involve applying the desire to bring truth to power. As often noted, on the wall of the Central Intelligence Agency (CIA) headquarters building is the inscription, “Ye shall know the truth, and the truth shall make you free.” Unfortunately, no one has yet developed the formula by which an intelligence analyst, let alone a whole agency or set of agencies, can know the truth, the whole truth, and nothing but the truth. No analyst in my acquaintance ever received a crystal ball from the supply room. Every analyst has been enjoined to emphasize objectivity and to avoid bias, but the reality is that every analyst — in intelligence, in law, in medicine — brings biases to the case at hand. An emergency room physician, knowing that a life-threatening disease may have many of the same symptoms as a simple case of influenza, but experiencing influenza 500 to 1,000 times more frequently than the life-threatening disease, is highly subject to being fooled by the exception to the rule.

In this case, bias, understood as the formation of conclusions based on the accumulation of data, is both unavoidable and efficient. What would happen to waiting times in the emergency room, not to mention to costs, if a doctor overrode the evidence of hundreds of “routine cases” and ordered extensive tests on every patient to rule out the “1 in 100” or “1 in 1,000” occurrence?

Intelligence analysts use bias in the same way and are therefore subject to the same vulnerability to nonlinear or aberrational events. An analyst who in 1990 had approached colleagues preparing a National Intelligence Estimate on the future of the Soviet Union by suggesting that it would simply go out of business, devolving the Baltics and the Central Asian Republics, and renouncing the political monopoly of the Communist Party, would have been hard pressed to provide evidence to support such an outcome. Even as “an alternative outcome,” that panacea of intelligence reformers, would this outcome have had credibility? Or would it have been dismissed, in large part because the analyst would have found it hard to produce “evidence” supporting that alternative?

Much has been made since 2003 of the politicization of analysis, and the collateral mistake of policymakers in “cherry picking” analysis. First and foremost, politicization, that is, the distortion of analysis to fit a desired policy or political outcome, must be considered the cardinal sin of analysis. To a great degree, politicization can only be done within the intelligence agencies. Once the intelligence reaches policymakers, they will read, react to, and interpret the intelligence within the framework of policy preferences, prior experience, and personal intellectual (or ideological) preconceptions, reform efforts notwithstanding. It is at least possible that intelligence analysis can change the mind of a policymaker.

But this cannot be predicted, and it certainly cannot be legislated. Finally, an analyst or even an analytic agency may fall into the same conventional (though incorrect or obsolete) wisdom shared with policymakers. Analysts and policymakers, as the phrase goes, may be “drinking the same Kool-Aid.”

Long-term study of the strengths and weaknesses of Western intelligence, based on the methodologies that already exist, has already begun to provide answers which go beyond anything that analysis of only the recent past can provide. To follow the conventional distinction between ‘secrets’ and ‘mysteries’, during the twentieth century we were frequently very good at discovering our opponents’ secrets when it mattered most but more confused than we should have been by the mysteries of what they intended to do.

The American and British success in discovering the capability and deployment of enemy armed forces shortened the Second World War, stabilised a Cold War which might otherwise have turned into hot war, and has since helped to make possible a series of breathtakingly rapid military victories. The importance of intelligence reports on military strengths and capabilities went far beyond the data they provided.

Studies of the Cold War frequently forget the truth of Eisenhower’s dictum that intelligence on ‘what the Soviets did not have’ was often as important as information on what they did. Shortage of reliable intelligence in the early 1950s generated the destabilising American myths of the ‘bomber gap’ and the ‘missile gap’ — the delusion that the Soviet Union was increasingly out-producing the United States in both long-range bombers and ICBMs.

In 1955 US Air Force intelligence estimates calculated that by the end of the decade the Soviet Long-Range Air Force would be more powerful than U.S Strategic Air Command, whose head, General Curtis Le May, became dangerously attracted by the idea of a pre-emptive strike to prevent the Soviet Union achieving nuclear superiority. The introduction of the U-2 spy-plane in 1956, followed 4 years later by the first imagery intelligence (IMINT) from spy satellites, provided proof that the Soviet nuclear strike force was not overtaking that of the United States. The U-2 missions, wrote Eisenhower, ‘provided proof that the horrors of the alleged “bomber gap” and “missile gap” were nothing more than the imaginative creations of irresponsibility’.

Without the IMINT revolution, US policy to the Soviet Union would doubtless have continued to be confused by other destabilising myths about the extent of the Soviet nuclear strike force. It is difficult to exaggerate the importance of the National Technical Means (NTMs) and the analytical tools devised to interpret them, chiefly by US intelligence, to Western policy during the Cold War.

… Of the ‘mysteries’ which confused us during the Second World War and the Cold War, the one we were worst at unravelling was the mindset of our opponents-in particular, the understanding of fanaticism. That remains a serious problem because of one fundamental, largely unnoticed continuity between the threats that faced us in the 20th century and those that face us in the 21st One of the very few to draw attention to that continuity has been Elie Wiesel, Nobel laureate, holocaust survivor and human rights activist. Several years before 9/11 Wiesel said this: ‘The principal challenge of the 21st century is going to be exactly the same as the principal challenge of the 20th century: How do we deal with fanaticism armed with power?’

Tomorrow’s post provides another perspective three: the struggle to find an accurate AND institutionally useful narrative

When I first saw the memo from the FBI’s counterterrorism center in Newark, declaring that I’m “a threat to National Security,” not to mention an “agent of a foreign power,” I was incredulous. These can’t be real FBI documents, I thought to myself. Someone is pulling my leg. Sadly, no.

… My alleged “crime” is to have written in too much detail about the possibility that agents of a foreign power (Israel) had some degree of foreknowledge of what happened on September 11, 2001. This is clearly what upset the FBI—because, if that is true, then where were our intrepid G-men while Israeli agents were crawling all over the place as the Twin Towers burned? They did indeed arrest five of them and interrogate them for months before quietly deporting them; and I, apparently, became a “threat to National Security” by noticing this inconvenient fact.