Open Source Intelligence

A ThreatsWatch Symposium

By ThreatsWatch | September 16, 2008

Open source intelligence (OSINT) is, for lack of a universally-accepted formal definition, information of value that you don't have to steal with spies or technical means. It can be free or you might have to pay for it but the thrust is that it is not classified or otherwise restricted by a government entity. To an extent we all use OSINT every day for any decision of substance; you identify a question that needs to be answered, you gather information that will help you make an informed decision, you process or analyze that information, and you make a decision based on the outcome of your analytic process. You don't need a spy or satellite to tell you which car to buy; in many cases a government doesn't need either of those tools to help it decide whether or not to take some kind of political action against an adversary. A simplistic comparison to be sure, but at the other end of the spectrum consider that all global business runs on OSINT, not secrets.

As a matter of fact, most of the information (later processed into intelligence) national and military decision-makers need to operate can be obtained via open sources (estimates range from 80%-90%), and the value of OSINT has been demonstrated numerous times over the last several years. Acknowledging these facts, this year's Office of the Director of National Intelligence OSINT Conference focused on the issue of the "decision advantage" OSINT can provide. While it was not an expressed target of the conference, there were numerous discussions about the role social network(ing) and related tech and practices could benefit the IC, something we can do for every -INT but given the related revolution taking place outside of SCIFs worldwide it made the conference a particularly apropos environment.

ThreatsWatch had the pleasure of discussing relevant issues with a number of current and former intelligence practitioners at this year's Open Source Intelligence Conference: Robert Stede, intelligence community analyst and technologist; Bob Gourley, former Chief Technology Officer at the Defense Intelligence Agency and currently CTO of Crucial Point LLC; Matt Burton, former analyst at the Defense Intelligence Agency and currently a consultant to the intelligence community; and Jack Holt, OSD Public Affairs.

TW: The promotion of OSINT as a peer –INT has been going on for some time now, though I don't think I'm going out on a limb when I say that earlier efforts have fallen short. Do you think what was said at this conference – combined with that you're seeing in the trenches - backs up the assertion that OSINT is finally coming into its own?

Robert: Is it coming into its own? It is and it isn't. OSINT's ubiquity is both a pro and con. It permeates everything in intelligence. If you take away the rules and customs of secrets, all you have left is OSINT, which is knowledge. Some diagram OSINT's role or place in intelligence as one of many circles in a Venn diagram; I see it as a cornerstone upon which the foundation of all intelligence is built. You can use the finest materials to build your house but if the construction is shoddy it's all for naught. Today, IC culture and expenditures drive us to buy fancy fixtures and molding, not invest in good architecture up front. The priorities of the Community are also backwards when it comes to sharing and collaboration. The tools and tech we use are serviceable; it's a rehab of our culture that needs to take place if we're going to deal with future mysteries and puzzles. We need to divorce ourselves from hierarchy and promote self-forming networks of trusted peers. Fears about reaching out to those who know better than you also has to be curtailed. Rewards for collaboration with peers inside and outside the wire need to be addressed if we ever want to know the truth. I don't expect this change to happen for a while; we are notoriously slow to understand and respond to emerging threats and concerns for our nation.

Jack: I believe OSINT is coming into its own primarily because the institutions that performed the function previously have been disbanded. USIA was a primary source of intel on cultures, norms, and attitudes in prior years and could give us an idea of who the thought leaders were on certain topics or areas. That information has not been readily available to an expeditionary force. OSINT has given us back that capability.

Matt: I'm not down in the trenches anymore, so I can't say. In my opinion, it would be best for open source intelligence if the term "OSINT" went away completely. The big reason it isn't a mainstay is because it's unclassified, and I think the OSINT label only highlights that when customers come across it. OSINT is not a separate INT. Rather, it comprises all the other INTs we normally use: photos, electronic, people...public satellite imagery is simply unclassified IMINT. Why call it something else that makes it look untrustworthy?

Bob: The nature of the world is changing. More and more is available in open sources. And with our IT systems we can bring the sum total of that to the desktop of our analysts. With social media we can apply more brains to racking and stacking and sorting. So you could argue that OSINT is of growing importance. But what OSINT could have told us that Putin was about to invade Georgia, or that he would mount a sophisticated deception effort to manipulate Western diplomats? Or that he would continue to say he will withdraw but had no intentions of doing so for weeks after the country was crushed? Could you argue that our newfound love affair with OSINT led us to believe we could know Putin's intent? Did OSINT contribute to yet another failure?

TW: We saw the senior representatives of a number of IC entities get up and talk about their OSINT efforts at a fairly meta-level. Not a day goes by and one of those entities (DHS) gets hammered by Congress for falling down on the job with regards to OSINT. With the benefits so obvious, why do you think it is so hard to turn vision into reality?

Robert: Attitude. The IC's attitude has not changed. If it's the first resort, shouldn't it be funded first?

Jack: I believe part of the problem comes from legacy thinking; the rest is people protecting their rice bowls.

Matt: I'm not familiar with the report, but it's probably a combination of things: execs being unwilling to be the first to take the leap, preferring that one of their peers do it first; middle managers who clog the flow of new ideas between desk workers and executives; and desk workers being stuck in their ways, unwilling and unable to learn a new business practice on their own.

TW: We heard a lot about "cognitive diversity" and the value that it can bring to analytic problems. Since OSINT is more or less tailor made for such an approach, how come there has not been a more significant outreach to SMEs outside the IC? We've done it in the past with classified works.

Robert: There's a fear of showing "our hand." We hear this stupid notion of the "Washington Post Test: If it can be ran in the Washington Post, it's bad." I think we need to be working with whomever, wherever them may be. And it was Dan Butler who said we need a little more humility to accomplish this. With a little more humility we can put our questions to a trusted network of groups, individuals, or machines. I would also argue that we could leverage our own citizens by just placing our puzzle pieces out there. We would be taking a large risk, but would feel that the return would be just as big. We have to face the reality that we cannot do this work alone and we have to grow into it.

Jack: Outreach to other SMEs will come more easily as we work through re-framing the environment to bring better understanding of the global information environment to the legacy thinkers. Example is how we've made a totem pole of "strategic communication" with no real understanding of what it is. SC is a process of identifying the strategically important publics for mission success and then finding a way to tell them what we're doing. That has a big IC component, an operational component, and a public affairs component. And, for example, if public affairs doesn't avail itself of the intel then the command is speaking without intelligence. We've seen and heard enough of that.

Matt: Haven't thought about this. A few possibilities: maybe it's a lot easier to get outsiders to help us when we have classified info to show them. Or maybe, because those outsiders already work with this OSINT day in and day out, there's no need to ask them for their input; it's already out there. Global Futures is the only such outreach effort I know of. Any others?

TW: Play king for a day; how would you re-align the current OSINT budget for your agency (or the community); what would you do with a 10% increase in funding?

Robert: More people with the right mindset; people for whom sharing is the default setting. If we all don't share or realize that we can't do it all, it's going to hurt our team. There are hundreds of people who wanted to be here in Analyst X's chair; if Analyst X is being introverted and unwilling to share what they know, that person is hurting far more than helping.

If I couldn't break the manpower ceiling I'd move to enable all of our data to be interoperable. There are two things that run concurrent through any information: space and time. Linking information by place would get us off and running in the right direction to integrate a multi-discipline community for multi-disciplinary problems.

Jack: More investment should be made in people. Collection and analysis is primary with training in the available products, processes, and the technologies available. Then investment in making the technologies shareable across the community.

Matt: Systems and training. Give the systems guys some of that budget and ask them to built a JWICS-to-Web port. We have to get the Web at the desk of every single analyst, and do it in a way that keeps them from having to switch machines. As long as the two networks are segregated, there will be a psychological barrier between "JWICS, where I do my work," and "The computer where I check my Gmail."

TW: If you had to draw a line in the National Intelligence Priorities Framework - stuff that would exclusively be handed over to OSINT practitioners outside the IC – where would it be drawn? If you could focus more OSINT on a given mission(s) which one(s) would they be?

Robert: I couldn't really say. You could throw them all over the fence if you wanted. They're just questions.

Jack: I see this as a complete package where someone gathers, another analyzes, another speaks, and the communication is whole. IC monitors and analyzes while PA discusses and puts back and every interaction changes the environment. To build to more understanding.

Matt: My first-hand experience in the various intelligence domains is limited. Bottom line though is that I'm sure they could all probably do with a healthy injection of more unclassified information.

Bob: Without a doubt, the best thing I heard any IC senior say was Glenn Gaffney stated he believes it to be our responsibility to field an infrastructure with the ability to enable IC users to mashup data. We owe this to today's analysts and tomorrow's; and we must build an infrastructure that will let tomorrow's users do things like mash up data in ways we might not even imagine. Seeing a senior IC leader articulate a vision on mashups really left me feeling like the community is in great hands.

As a runner up, I have to say it was Director Hayden's quoting of the great master of operational Intelligence Vince Fragromene. Vince was quoted as having taught Director Hayden that "if you live by SIGINT you die by SIGINT", meaning that you cannot simply trust any single source for an assessment. Vince and others drummed that lesson into my head as well, and I saw time and time again how important it is to seek every possible source that has information or context on a particular situation. It is also critically important to know the strengths and weaknesses of every source. For example, HUMINT might reveal intentions, but if a Human said it then it could also be a lie. ELINT might reveal a precise location but if a parameter is captured wrong it may totally mis-identify the radar that is there. SOSUS might actually be a bottom bounce hit and the Sub may actually be way far away vice close. Imagery might have been taken at the time the activity was over or may be taking pictures of fake targets. The lesson is, you must use every possible source and you must know its weakness. And the reason Director Hayden was telling us about Vince's lesson was it is even more important to OSINT. Just because it is said does not mean it is true.

Robert: "Here's some money. Go do this." Actually, I like Secretary Gates' speech last year discussing the need to have individuals like John Boyd. Boyd's mantra about choosing what to be is great. Either you go off to "be someone" or you "do something." We need more "do," because right now, we have a large "say-do" gap.

Jack: Everything Glenn Gaffney said was the best thing along with Gen. Hayden. I don't recall anything disturbing but that is probably because if disturbing statements are made then impact is being had.

Matt: The best thing was an astonishing policy opinion stated in private by a very senior person in the community; I'd rather not run the risk of upsetting him, but needless to say it was very positive. The worst thing was also said during a private conversation with another senior: In the context of FISA, wiretapping, telco immunity, etc: "These people share their lives online, then complain about invasion of privacy." This is disturbing in the context of the conference because it demonstrates a lack of awareness of how online communities work.

As one speaker at the conference noted, a lot of long-time intelligence practitioners doubt the value of OSINT and they chaff at the thought that freely available information is more valuable than secrets. More to the point, they consider any efforts to more effectively integrate freely available information into the intelligence business to be anathema and entertaining such change is a threat to national security. The threat however, is in not pursuing such a course of action, because it is abundantly clear that our adversaries - particularly non-state adversaries - are doing it and in many cases eating our lunch in the process. It was easy to dismiss al-Qaeda on 9/10/01; not so much the day after, and they managed their far-flug and clandestine operation without the trappings and "benefits" of our intelligence community. Making full use of OSINT is not simply a trendy thing to do, but a vital aspect of any effort designed to reboot the intelligence community. We appreciate your time and thank you for participating in this discussion.