TRENDING

Intelligence units mine the benefits of public sources

Open Source Center draws, analyzes info from a variety of public databases

By Patience Wait

Mar 17, 2006

Sometimes the most useful information is in plain sight'as long as you know where to look for it.

In the ongoing struggle against Latin American drug smuggling, for instance, an article in a local Mexican newspaper might report that a semi belonging to Company X was stopped at the U.S. border in Texas and a quantity of cocaine was seized. The article includes the name of the truck driver.

That information might be useful at the border crossing in San Diego, where trucks belonging to that same company also cross. Or the driver might be identified at the wheel of another rig a couple of weeks later. In either case, some extra scrutiny may be warranted.

That's the sort of information the Open Source Center is compiling and pushing the intelligence community to use, said Eliot Jardines, assistant deputy director in the Office of the Director of National Intelligence (ODNI).

'Getting the [intelligence community] to accept open source as the source of first resort is my number one goal,' Jardines said. 'In the past we tended to value information in proportion to how hard it was to get.'

(While the phrase 'open source' is used most commonly in the IT community to refer to software whose source code is publicly available, in the intelligence community it refers to any data source that is available to the general public.)

The data can be free'such as local property records, voter registrations and political-campaign contributions'or it can be for sale, such as credit reports, commercial satellite imagery and New York Times articles. The data might already be in database format, or it may have to be converted, as are transcripts of TV and radio broadcasts. All of it, however, is grist for the mills of analysts throughout the intelligence community. The center is the clearinghouse for much of it.

'We like to make a distinction between collection and acquisition,' Jardines said. Traditionally, the IC has owned the means of collection, with an emphasis on covert intelligence gathering. But open-source intelligence revolves around acquiring information that someone else has collected, organized and published, he said.

Foreign targets

The Open Source Center was created in November as part of the reorganization of the IC to improve intelligence and information gathering activities in support of counterterrorism and international law enforcement activities. Its data gathering ac- tivities are not directed at U.S. citizens, Jardines stressed. Domestic agencies such as the FBI and the Homeland Security Department are chartered to investigate within the country's boundaries, while the intelligence community concentrates on the rest of the world, he said.

But OSC is not exactly a new organization, nor is the gathering of publicly available information for intelligence purposes new to the government. OSC is a revamped and expanded version of an existing CIA program, the Foreign Broadcast Information Service.

FBIS was founded in 1941 to gather and analyze radio, television and print stories for clues on events happening in closed or hostile societies, such as Nazi Germany during World War II and the Soviet Union during the Cold War.

'The Open Source Center has global responsibilities and em- ploys a combination of technology and people around the world to carry out its mission,' Naquin said. Federal statutes prohibit disclosing the number of employees in OSC or the size of its budget.

Similarly, neither Naquin nor Jardines could discuss specifics of the technologies being used in OSC to conduct pattern recognition and data mining of open sources of information. But Naquin said the center's technology requirements cover such areas as data filtering, digitization, foreign-language processing, multimedia production and delivery, and interagency collaboration.

'Because [the center] operates primarily in an unclassified domain, it has been able to employ technologies for multimedia production, collaboration and delivery that are more difficult to employ in classified do- mains due to security concerns,' Naquin said. 'In fact, the unclassified domain is often regarded as a 'test bed' for which successful technologies can be exported to classified domains.'

A vast sea of data

The OSC monitors somewhere between 150 and 300 jihadist Web sites it considers the most significant, Naquin told the Washington Post last year. The center also tracks about 500 television stations, with access to about 20,000 around the globe. Throw in blogs, radio broadcasts, even bumper stickers and graffiti (they reflect local public sentiment 'on the ground'), and one can see just how vast a sea of data is covered by open-source data mining.

Volume alone is not the only difficulty in managing the data. Just as the government has found technical barriers to information sharing across agencies, OSC has encountered obstacles to effective data searches, but not of the same magnitude, Jardines said.

'We have the same sort of issues but to a far lesser extent,' he said. 'Because this is open-source information, it can be readily shared. The problem is that generally things are organized by publisher ... rather than thematically.'

Also, while OSC is supposed to acquire information and make it available to the numerous open-source programs scattered throughout the intelligence community and Defense Department, there is not a single network in place available to all those organizations.

'We have a number of networks that provide open-source reporting or sensitive-but-un- classified reporting that don't talk to each other. I see that as a problem we have to address immediately,' Jardines said.

There is a working group in place to identify the requirements for sharing open-source information throughout the broader community. Addressing the problem of multiple networks without interoperability is on its list.

'One of the non-negotiables I will impose [is single sign-on],' Jardines said. 'We may not take the five or six separate networks and tell them, 'You have to do away with yours,' [but] we're looking for connectivity.'

Another technology challenge is data storage. While open-source information is changing all the time, Jardines said none of it should be discarded.

'Ultimately, hard-disk space is cheap as opposed to the costs to identify [and] capture data,' he said. 'We might want to be selective in what we capture, but once it is captured, why get rid of it?'

Jardines said he and OSC are working closely with Gen. Dale Meyerrose, the CIO for ODNI'and by extension for the entire intelligence community'on a national open-source strategy, which may be completed fairly soon.

Technology may be a challenge for OSC, and there may be policy questions that need to be resolved, but one of the biggest hurdles is culture change'convincing agencies that have always emphasized covert intelligence that information available in the public domain has real value.

According to the joint congressional inquiry into the terrorist attacks of Sept. 11, 2001, the intelligence community was 'more inclined to use open-source material as a last resort, not as a primary source, no matter how compelling the information.'

Classified isn't better

'It is very tempting to equate classification with value. The higher the classification, the higher the value is a common, albeit erroneous, assumption,' Jardines said. 'Classification has nothing to do with the value of information; it only indicates the degree of damage done to national security should the sources and methods used to collect it be compromised.'

As new employees enter the intelligence field, that will be- come less of an issue, he said. Young people who have grown up with the Internet already understand the value of open sources.

And there's another satisfied customer.

'The number of open-source items provided in the President's Daily Brief have increased,' Jardines said. 'I would say we're scoring some wins with our most important customer.'