tag:www.schneier.com,2016:/blog//2/tag:www.schneier.com,2013:/blog//2.4810-2016-09-03T04:58:35ZComments for Intelligence Analysis and the Connect-the-Dots MetaphorA blog covering security and security technology.Movable Typetag:www.schneier.com,2013:/blog//2.4810-comment:1368012Comment from SOF on 2013-05-11SOF
It's like missing and exploited kids. We developed sensor nets, gps shoes, bracelets and tracking interfaces. Wearable computer research made advances. Trouble with security is they go around it or are distracted and can't pay attention and attention must be paid. Make Trax. More government projects? Low income housing is growing. The reverse Darwinism crowd is funded.]]>
2013-05-11T16:37:19Z2013-05-11T16:37:19Ztag:www.schneier.com,2013:/blog//2.4810-comment:1365299Comment from Otter on 2013-05-10Otter
@ AtomBoy

"Like I've always told my conspiracy theorist friends: If you geometrically increase the number of dots and remove the sequential numbers, you can draw any picture you want."

Authorities are often the worst buyers into conspiracy theories.

Endless, ludicrous stories there.

Some actually true. Some not.

Usually there is huge over reactions from the basis of statistically small events with investigative agencies with Gestapo powers. Usually these statistically small events are genuinely scary.

]]>
2013-05-09T13:31:58Z2013-05-09T13:31:58Ztag:www.schneier.com,2013:/blog//2.4810-comment:1359462Comment from Coyne Tibbets on 2013-05-09Coyne Tibbets
I don't think numbering the dots is that much easier after an "event".

Even then, the dots can be numbered in many different ways. This is the source of many conspiracy theories. For a current example consider this "numbering" in Infowars: Tsarnaev Brothers had a CIA Connection.

With dot-relationships like this, how does one decide which are valid and which are irrelevant?

Some might dismiss this as, "Well, that's just conspiracy theorists." But the same problem also results in convictions of the innocent; both legally and in the public mind.

As a result, it seems to me that it is just as specious to say, "Well, we were able to connect the dots afterward..." as it is to complain about "not connecting" them beforehand.

]]>
2013-05-09T06:03:34Z2013-05-09T06:03:34Ztag:www.schneier.com,2013:/blog//2.4810-comment:1358159Comment from Aristotle Pagaltzis on 2013-05-08Aristotle Pagaltzishttp://plasmasturm.org/
One of those articles by Malcolm Gladwell that I do recommend concerns exactly this issue:

In it, he draws a distinction between puzzles and mysteries. He defines a puzzle as a situation where you lack information, and finding more of it is the key to solving the puzzle. A mystery, on the other hand, he defines as a situation where you already have plenty of information and the challenge is to make sense of it. In a mystery, it’s not lack of information you need to address, but the abundance of it; simply gathering more will achieve nothing but muddy the waters further.

]]>
2013-05-08T19:50:26Z2013-05-08T19:50:26Ztag:www.schneier.com,2013:/blog//2.4810-comment:1357756Comment from Joseph R. Jones on 2013-05-08Joseph R. Joneshttp://jrj.org
Like I've always told my conspiracy theorist friends: If you geometrically increase the number of dots and remove the sequential numbers, you can draw any picture you want.]]>
2013-05-08T16:38:12Z2013-05-08T16:38:12Ztag:www.schneier.com,2013:/blog//2.4810-comment:1357436Comment from AtomBoy on 2013-05-08AtomBoyhttp://www.bbc.co.uk/news/science-environment-22358861
@ Paul Rennault

"If the FBI had spent less time, money, and effort manufacturing cases (see Trevor Aaronson's The Terror Factory), harassing peace activists (see www.stopfbi.net), 'enforcing' websites' Terms of Service (see the prosecution of Aaron Swartz), and being hand-maidens to the MPAA (see the extra-territorial, and from all appearances extra-judicial) prosecution of Kim Dotcom, is it possible that they might have had some time, money, and effort to 'connect the dots'?
Didn't the Russians point a finger at these two?"

That is what I am thinking. 700,000 people? Really?

I liked Bruce's statement about increasing the size of the haystack to look for needles, in this context.

They probably are doing what all such agencies have a tendency to do, and as they have a history of doing -- looking in New York for a coin they dropped in LA. Because they like NY better. :/

]]>
2013-05-08T13:54:30Z2013-05-08T13:54:30Ztag:www.schneier.com,2013:/blog//2.4810-comment:1356939Comment from Autolykos on 2013-05-08Autolykos
@paul: Your case is far from unusual. I'd be hard pressed to find a chemistry nerd who didn't occasionally blow stuff up with homemade explosives in his youth. About half of my final chemistry class in college was that kind of guy - and the other half had an unhealthy interest in poisons (that probably decides whether they'll study chemistry or pharmacy). Police would be quite busy if they tried to monitor them all. And they might never find an actual terrorist; I can tell from most plots that they were pretty clueless about chemistry.]]>
2013-05-08T10:12:37Z2013-05-08T10:12:37Ztag:www.schneier.com,2013:/blog//2.4810-comment:1356836Comment from John Magic on 2013-05-08John Magic
Haha just stop invading other countries and terrorism will be less of a problem. People get upset when their country/region is invaded by the US and they want revenge. It's kind of natural if you think about it.

John Magic
Sweden

]]>
2013-05-08T09:52:50Z2013-05-08T09:52:50Ztag:www.schneier.com,2013:/blog//2.4810-comment:1355951Comment from Privacy Law Site on 2013-05-07Privacy Law Site
We're seeing hindsight bias playing out again now with the Cleveland kidnapping case. TV commenters are breathlessly asking how local police could possibly have not known what was going, why they did not search the house (because he may have left a kid on school bus) or because he (may or may not) have covered up the house windows, or just because. It's all obvious in retrospect.]]>
2013-05-08T03:27:51Z2013-05-08T03:27:51Ztag:www.schneier.com,2013:/blog//2.4810-comment:1355523Comment from Dirk Praet on 2013-05-07Dirk Praet
It's not just about connecting the dots, it's also about sharing the dots. Giving one kid the entire page of a coloring book will allow him to connect the dots quite easily and find the animal that's hiding. If however you divide that one page among several kids, they will have to work together to literally figure out the bigger picture. As long as they don't, they're unlikely to find anything.

I believe many intelligence failures can or could have been avoided by TLA's working together and sharing information in a much more efficient way than is happening today. Unfortunately, most - if not all - are secretive little kingdoms with their own little princes, constantly fighting amongst each other over budgets, jurisdictions and esteem. The DHS only came about because everyone in Washington DC and beyond knew this was never gonna happen, so they decided to create yet another one to avoid the Kafkaian nightmare that would have ensued trying to align and integrate existing agencies.

As every engineer knows, technical solutions often are far easier to solve than the political issues that come with them, and throwing more resources at a problem not necessarily a better strategy than trying a different approach.

]]>
2013-05-08T01:02:26Z2013-05-08T01:02:26Ztag:www.schneier.com,2013:/blog//2.4810-comment:1355522Comment from RSaunders on 2013-05-07RSaunders
It's not finding a needle in a haystack, it's finding a particular blade of hay in a haystack. There are lots and lots of of "leads" and facts. The "security enhancing" actions proposed by politicians, like adding security cameras, only add more hay.

Like refusing to be terrorized, we just need to accept that preventing evil is science fiction (and usually dystopian SF at that). Nobody wants to prevent auto crashes with 5 MPH cars, and nobody really wants to spend what would be required to prevent terrorism.

]]>
2013-05-08T01:02:08Z2013-05-08T01:02:08Ztag:www.schneier.com,2013:/blog//2.4810-comment:1355292Comment from MingoV on 2013-05-07MingoV
After the 9/11 disaster, we learned that a flight instructor had reported to the FAA and the FBI that some Arab men wanted passenger jet flying lessons but not landing lessons. We supposedly learned from this failure to act.

But, after the Boston bombing, we learn that the brothers had been questioned before, that the Russians informed us that the older brother was dangerous, and that the Saudis told us the same thing. When two nations, one of which we are not on good terms with, tell the US government that an immigrant is dangerous, then we should act. (There were grounds to deport the older brother.) This wasn't a situation of picking out a potential terrorist among millions of non-terrorists. It was a situation of completely ignoring evidence of criminal intent. It was inexcusable.

]]>
2013-05-07T23:02:05Z2013-05-07T23:02:05Ztag:www.schneier.com,2013:/blog//2.4810-comment:1355236Comment from Jake on 2013-05-07Jake
I just want to know who comes up with silly names like "Terrorist Identities Datamart Environment".

datamart?

]]>
2013-05-07T22:43:14Z2013-05-07T22:43:14Ztag:www.schneier.com,2013:/blog//2.4810-comment:1355200Comment from hoodathunkit on 2013-05-07hoodathunkit
It's not a haystack. The Ruskies (NKVD) sent a warning to their counterparts at the FBI, and the Saudis sent a warning to State. Those should have intersected; that was the sole purpose for the creation of DHS. Once alerted, a cursory background check would have found other indicators: parents split with faith differences, non-mainstream demands put on suspect's wife, career/job at loose ends, financially unsuccessful, etc. In combination, those should have enabled a letter or search warrant for monitoring his computer.

In turn, sharing his (known now) Chechen connection back to NKVD could have obtained more details from them in trade. Same with Saud, the reason for his visa denial was never explored.

]]>
2013-05-07T22:23:38Z2013-05-07T22:23:38Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354944Comment from option12 on 2013-05-07option12
It's not a sailboat... it's a schooner!]]>
2013-05-07T20:06:09Z2013-05-07T20:06:09Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354908Comment from Paul Renault on 2013-05-07Paul Renault
I want to add to the needle-haystack analogies:

The FBI is adding straw to the haystack - not the data. In fact, IMHO, they're building lots and lots of haystacks that they know are completely devoid of needles - just so they can get paid to search them.
If the FBI stopped doing this, they would have more eyes/hands/ears/brains working on that haystack - and they'd find that needle much quicker (and without the easy solution of burning the haystack down...).

]]>
2013-05-07T19:51:58Z2013-05-07T19:51:58Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354731Comment from LongHairedWeirdo on 2013-05-07LongHairedWeirdo
Re: Finding a needle in a haystack, @John Henry.

The phrase dates back to the 1600s where they didn't have industrial magnets or metal detectors, and may not refer to a sewing needle - I've seen one statement of "finding a pine needle in a haystack". If it does refer to a sewing needle, it might be a handmade one, made from a splinter of wood or bone.

The expression hasn't aged well; the key is finding one special something in a large stack of similar-enough somethings. I think finding a needle in a needlestack might well be the best translation to modern ideas.

I agree with your central point but you do not stress one fact enough, the fact that stories work. President Obama said that his number one regret of his first term was he had wished he had spent more time with telling stories rather than developing policy.

The battle between complex policy and simple stories is a losing battle for complex policy in a democracy where 50% of the population has an IQ below 100 . But that is OK. Because the truth is that the stories that we tell each other and which we tell ourselves are not inevitable. The fundamental problem is that we have let a "security narrative" dominate our society. It is the story we tell ourselves and each other. But there is a civil liberation narrative that can be told. The Americans who passed the Bill of Rights told that story. We can too.

You once said that poor hackers hack machines and great hackers hack people. Hacking people is more than just psychology. It is poets and novelists and screenwriters. It was Freud who said, "no matter where I have been analytically some poet has been there first."

The challenge that faces those who care about liberty is developing a new narrative of liberty. So long as civil liberties are always on the defense they will always fight from a weakened position and will more often than not lose. Let's stop bemoaning the calamity of hindsight bias and start telling our own stories, to ourselves and to each other. We should not be afraid of fighting on that battleground; we can win it.

]]>
2013-05-07T17:48:16Z2013-05-07T17:48:16Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354563Comment from David Dyer-Bennet on 2013-05-07David Dyer-Bennethttp://dd-b.net/
We need to revise the old chestnut; hindsight is not 20/20, it is 20/10. Or maybe 20/5.
]]>
2013-05-07T17:07:32Z2013-05-07T17:07:32Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354538Comment from john henry on 2013-05-07john henry
Perhaps a bit off topic but the "needle in a haystack" analogy has long bothered me.

What is so hard about finding a needle in a haystack? Simply pass the hay over a magnet or a metal detector. Might be a bit time consuming but you could put a 3rd grader to work doing this.

And the needle would be found fairly quickly.

I do like the "needle in a needlestack" analogy that Harvey used. I assume that you meant a specific needle in a pile of similar needles. I'd not heard it before but am going to appropriate it.

It makes much more sense in this and any other context where the haystack metaphor might be used.

John Henry

]]>
2013-05-07T16:53:27Z2013-05-07T16:53:27Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354525Comment from John Henry on 2013-05-07John Henry
Why do we hear nothing about the boys' uncle Ruslan? Other than his interview calling them losers.

He was married to the daughter of CIA biggie Graham Fuller. He registered a Chechan activist group using Fuller's home as the address.

He apparently worked for USAID in Kazakhstan.

His is currently a very well connected oil/gas lawyer.

Did he and/or Fuller pull strings to get the boys admitted to the US? Did they pull strings to get them US citizenship?

How did Uncle Ruslan become a citizen? Did he get any special treatment?

]]>
2013-05-07T16:46:36Z2013-05-07T16:46:36Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354455Comment from Harvey MacDonald on 2013-05-07Harvey MacDonaldThis is like finding a needle in a haystack?

No.

In this case we are looking for which bad guy in a huge list of bad guys is about to do something that the media will care about. A more correct analogy would be:

This is like finding a needle in a needlestack.

]]>
2013-05-07T16:17:20Z2013-05-07T16:17:20Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354446Comment from bcs on 2013-05-07bcs
Most human information processing consist almost entirely of ignoring the right information (this is why image and audio compression works so well). The problem with needle-in-a-haystack problems, is that when the haystack is bigger than any one person can examine, our heuristics quit working. OTOH, once you know where to look, it's easy to hand someone (after the fact) a small haystack.]]>
2013-05-07T16:13:02Z2013-05-07T16:13:02Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354411Comment from SystemAnalysis on 2013-05-07SystemAnalysis
@Paul Renault,

You are correct to criticize the FBI and other agencies for manufacturing cases. It's theater. However, we don't even know for a fact that these two bombers were actually working on their own. It's possible, but there's a lot of evidence that suggests that they were patsies for the CIA, and other CIA contractors on scene. Were they? We may never know.

We just don't know what the hell happened, who did it, or why.

No matter what actually happened, the US media is certainly developing a narrative far from the truth with an agenda to continue the senior government's objectives (regardless of party).

What is reported and what is are almost always completely different things, and the most important things aren't reported at all.

The surveillance state is just the next stop in the course of empires. If you study the historical precedents, it is all completely predictable.

]]>
2013-05-07T15:51:57Z2013-05-07T15:51:57Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354378Comment from Brandioch Conner on 2013-05-07Brandioch Conner
Bruce, I think your comment "... think of it as a million unnumbered pictures superimposed on top of each other" does not go far enough.

It is more like pictures of the lives of every person living in this country at any particular moment (over 300 million). Reduced to dots. And then those dots overlaying the dots of the pictures of every other person that they've been in contact with or occupied the same space as or etc, etc, etc.

So they will never be able to "connect the dots" PRIOR to an attack. All they can do is reconstruct the person's life AFTER they've been identified.

]]>
2013-05-07T15:38:26Z2013-05-07T15:38:26Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354273Comment from kodewords on 2013-05-07kodewords
It's become self-evident that the government is unable to connect the dots with these things. But surveillance is big business. Lobbyists love a surveillance state with unlimited funding, and by spending lots of money and building massive datacenters and collecting massive amounts of data, politicans can pretend to be "doing something."

In other words, data mining and monitoring and collection is going to keep happening whether it's effective or not.

]]>
2013-05-07T14:48:02Z2013-05-07T14:48:02Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354190Comment from Mike B on 2013-05-07Mike B
This article is a bit ironic coming from someone who is known to be able to decode a box of Alphabits :-)]]>
2013-05-07T14:20:14Z2013-05-07T14:20:14Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354147Comment from David Jones on 2013-05-07David Jones
I agree with the premise of the article, but there comes a point where enough is enough. Since 2001, we've been subjected to grievous violations of privacy. America isn't America any more. I fully believe it's fair to criticize LE for not stopping this. If this was 1999, fine. But in 2013, after going through every citizens hind-quarters with fine toothed combs, every day...nice job LE. They deserve the criticism IMO. And if you think that by *not* criticizing them, that they won't look to grab even more power...good luck with that. Maybe criticizing them will actually wake people up and they won't be granted more power that won't help. Want to stop the criticism? Stop going through my email.

Dave

]]>
2013-05-07T14:06:23Z2013-05-07T14:06:23Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354138Comment from paul on 2013-05-07paul
This whole "connect the dots" and "proactive" thing makes me think of an old buddy. We blew up all the things we legally could, and a few that we shouldn't have. He had a pal who had played with synthesizing nitroglycerin. We roughed out all manner of designs for IEDs, talked about where one might plant them for maximum effect. Disaffected? We both left college before graduating.

Obviously should have been on a watch list, maybe taken into custody just in case. And yet 30 years later, we're relatively productive unconvicted members of our communities. If the FBI had spent any time on us it would have been wasted and then some.

]]>
2013-05-07T14:03:28Z2013-05-07T14:03:28Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354066Comment from paranoia destroys ya on 2013-05-07paranoia destroys ya
Steve Jobs, one of the most successful business people in recent memory, further noted "You have to trust that the dots will somehow connect in your future."

This principle would serve us better than people seeking to blame others for political gain over perceived intelligence failures.

]]>
2013-05-07T13:24:47Z2013-05-07T13:24:47Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354057Comment from MT on 2013-05-07MT
In hindsight, we know who the bad guys are. Before the fact, there are an enormous number of potential bad guys.

It's even worse than that, they aren't bad guys until they actually do something so there is no possible reliable indicator of potential badness until the act is in progress. People ask why the Tsarnaev brothers weren't picked up some time ago but the fact is that at that time they hadn't done anything wrong that would justify such measures. Unless the public is willing to inter and prison millions of people based on no evidence on the off chance that they might commit a crime, or alternately continue turning the country into the worlds largest open air prison (take that Austrailia!).

That doesn't even take into account the factor that enhanced security and military measures induce far more crime and violence in retaliation than they prevent.

]]>
2013-05-07T13:20:17Z2013-05-07T13:20:17Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354053Comment from Rich on 2013-05-07Rich
But, but, but... the FBI just thwarted a "Terror Attack" in Montevideo, MN. It seems we have way too many 'terrorists' and not enough 'clowns with a couple pipe bombs' (or more accurately, 'with a couple suspected pipe bombs'). ]]>
2013-05-07T13:17:24Z2013-05-07T13:17:24Ztag:www.schneier.com,2013:/blog//2.4810-comment:1354003Comment from Frank on 2013-05-07Frank
Yes, yes, yes. This hits three or four nails right on the head. Somebody high up in the executive branch needs to hire Bruce.]]>
2013-05-07T12:49:54Z2013-05-07T12:49:54Ztag:www.schneier.com,2013:/blog//2.4810-comment:1353975Comment from Brian on 2013-05-07Brian
I also wonder if the Monday morning quarterbacking isn't just about hindsight bias but also about people wanting to feel safe. Implicit in the argument that the government should have connected the dots is the belief that it must be possible to do so.

That seems like a comforting belief to have, as opposed to the alternative world where sometimes it might not be possible to find a terrorist before they attack. "Intelligence failures" like this poke holes in that comforting belief, which could be part of the source of the reaction.

Even if the FBI spent all its time and effort on terrorism, that doesn't mean the job becomes easier. As Bruce pointed out, more hay on the haystack doesn't make finding the needle easier. It's entirely possible that without the extra data point of "these guys committed a terrorist attack" pegging them as terrorists wasn't really possible.

You're proving his point with the comment about the Russians. That seems like smoking gun style information after we already KNOW they're terrorists. But without that extra piece of information, connecting the dots might not be so easy. The Russians aren't magic terrorist detectors either, they have their own set of dots they're trying to connect, and passing those dots along to other countries doesn't necessarily make the problem any easier to solve.

]]>
2013-05-07T12:31:25Z2013-05-07T12:31:25Ztag:www.schneier.com,2013:/blog//2.4810-comment:1353920Comment from Mike on 2013-05-07Mike
Yeah, the Russian government warned us about Tamerlan Tsarnaev. But I'm sure that's just one dot in a million. I'm sure the Russians name ten of thousands of potential terrorists a year, no reason to give that any more weight than say a person buying fertilizer. ]]>
2013-05-07T12:12:17Z2013-05-07T12:12:17Ztag:www.schneier.com,2013:/blog//2.4810-comment:1353891Comment from Paul Renault on 2013-05-07Paul Renault
I'll ask a question I've asked elsewhere here - hopefully I can get a cogent response:

If the FBI had spent less time, money, and effort manufacturing cases (see Trevor Aaronson's The Terror Factory), harassing peace activists (see www.stopfbi.net), 'enforcing' websites' Terms of Service (see the prosecution of Aaron Swartz), and being hand-maidens to the MPAA (see the extra-territorial, and from all appearances extra-judicial) prosecution of Kim Dotcom, is it possible that they might have had some time, money, and effort to 'connect the dots'?