The Hopelessness of Finding Terrorists through Eavesdropping

People in the Administration have been working feverishly of late to make the case that grossly intrusive and unconstitutional surveillance of American citizens is the only way to Unearth Terrorists and Save Civilization. Supposedly the data mining techniques developed by banks and credit card companies could be harnessed to process all the data in emails and phone calls and the guilty parties would be relentlessly driven into the daylight. Therefore, we should feel that trading our constitutional rights to privacy is a small price to pay for this increment of safety.

As if to bolster this, they have just lately (and will continue to do so increasingly leading up to the election) trucked out a couple of really pathetic “terrorist cells” of deranged and disconnected individuals in a couple of cities who are charged with “plotting terrorism.”

Unfortunately, the actual effectiveness of all this surveillance is completely an illusion: the processing of eavesdrop-acquired data through data mining techniques will NOT lead us to terrorists and will NOT enhance our safety. Bruce Schneier, a true expert in security issues and the CEO of Counterpane Internet Security, Inc., has prepared an interesting analysis of why it can’t work, and I strongly recommend that you read it.

When I was at US Bank, I worked implementing a real-time credit card transaction analysis system that identified transactions that were likely to be fraudulent, so they could be stopped before they took place. This software worked because we could “train” it (for about 9 months) to look at millions of transactions a day, and over time as we told it in retrospect which of the transactions had been fraudulent, it began to extract patterns of transactions that it could project would be fraudulent. And even after a long period of training, we got false positives — things that looked bad but were in fact just out of character for some particular cardholder, but were OK.

So, millions of transactions a day and very typical patterns of fraudulent transactions make credit card data mining systems work. However, to quote Schneier,

Terrorist plots are different. There is no well-defined profile, and attacks are very rare. Taken together, these facts mean that data mining systems won’t uncover any terrorist plots until they are very accurate, and that even very accurate systems will be so flooded with false alarms that they will be useless.

Do I have a better solution to finding terrorists here in the US? No, I don’t. I’m not a counter-terrorism expert. But I can see, through thoughtful analysis, what won’t work, and as a citizen I can only encourage the government not to waste its time on these things. No doubt they feel they must do something, but doing the wrong things won’t increase safety and will just waste time and money.