Enlist IT tools - not more people - to improve intel

Jun. 27, 2010 - 06:00AM
|

The near-success of the Christmas Day bomber on a flight from Amsterdam to Detroit has been characterized as a failure to "connect the dots." Now, additional teams of specialists are being formed to run down clues.

Throwing resources at a problem is an understandable knee-jerk reaction, but the human capacity to process enormous quantities of information quickly is limited. Connecting the dots requires cutting-edge technologies that augment human capabilities. Without revamping the information systems used in the intelligence community, more eyeballs will, at best, yield diminishing returns and, at worst, exacerbate problems.

The intelligence community has invested heavily in building unparalleled tools for collecting information — generating dots — while slighting tools to connect them. The most commonly reported weakness is that analysts have to search multiple databases to access information and that they cannot integrate the data they find from their searches.

Although technologies to amalgamate databases exist, they have not been widely deployed within the intelligence community. Giving analysts the same capabilities that are commonplace for most Web surfers is necessary, but on its own will not help connect the dots.

Since 9/11, intelligence-sharing has improved, but the data deluge caused by increased capabilities has negated these gains. Data is now measured in the petabyte — 1 million gigabytes — roughly equivalent to 20 million four-drawer filing cabinets filled with text.

A public search engine might process many petabytes of data daily, and presumably the intelligence community collects data on a comparable scale. Without knowing what to look for, searching these enormous quantities of data will only increase information inundation. An effective system not only helps analysts find information, it helps them make connections.

A search on variants of the Christmas bomber's name, Umar Farouk Adbulmutallab of Nigeria, would have resulted in hundreds or thousands of responses. One person could not examine this volume of material in a timely manner. Even if a team of analysts examined the material in a reasonable time frame, without knowing what to search for, team members might focus on different issues and not connect the dots.

Information systems that can draw basic conclusions would help analysts identify critical information. Businesses are investing in the Semantic Web, in which information is given some context by facilitating smart or semantic searches. This approach is starting to be embraced by industry, with some search engines, such as Microsoft's bing.com, starting to link searches.

A system with these capabilities might see Umar Farouk Abdulmutallab as a person who has certain characteristics, such as parents and a last known location. Nigeria and Yemen can be understood as places where one might be at a given time. With this context, the system could identify disparate warning signs as potentially relating to the same person.

Despite analysts' efforts, the ways of doing business have proved inadequate. Increasing personnel represents more of the same. Adopting revolutionary IT could be a much-needed game-changer.

Ironically, many of the technologies underpinning these revolutions in information systems were funded by the Defense Advanced Research Projects Agency, and were motivated by the need to enhance data-sharing in the intelligence community. Now, the Semantic Web is used by governments to increase the public's access to data and by businesses to connect the dots. But efforts to deploy this technology within the national security system have moved more slowly.

There are understandable reasons for this. The infrastructure for launching an intelligence-collection satellite is proven and the data gathered is easily measured. But it can be difficult to quantify the impact of a new information system, and its implementation is an enormous technical and bureaucratic challenge.

But without better information systems, the intelligence community will be hamstrung in its efforts to transform information into intelligence.

———

Aaron Mannes is a researcher at the University of Maryland's Laboratory for Computational Cultural Dynamics. James Hendler is a professor at Rensselaer Polytechnic University in New York and a former chief scientist for information systems at the Defense Advanced Research Projects Agency.