The culmination of post-9/11 policies and the steady erosion of civil liberties in the service of "fighting terrorism" has opened up nearly 300,000 people to additional scrutiny because ¯\_(ツ)_/¯ .

The list has increased 10-fold during Obama's stay in office, growing from 47,000 at the end of Bush's term to 680,000, 40% of whom the government is sure represent some sort of a threat, even if it can't quantify that in any specific way.

When U.S. officials refer to "the watchlist," they typically mean the TSDB, an unclassified pool of information shared across the intelligence community and the military, as well as local law enforcement, foreign governments, and private contractors. According to the government's watchlisting guidelines, published by The Intercept last month, officials don't need"concrete facts" or "irrefutable evidence" to secretly place someone on the list—only a vague and elastic standard of "reasonable suspicion."

It's long been noted that articulable facts are unwanted guests in the War on Terrorism (and War on Drugs) discussion. Instead, hunches and gut feelings are elevated to places formerly occupied by Fourth Amendment protections.

This group of people, shrugged into "nomination" by a variety of government agencies, is then shared with law enforcement, private contractors and foreign governments. That's at least 280,000 people being vetted with impunity by a variety of TSDB end users -- people who are deemed too dangerous to go unsurveilled but not dangerous enough to arrest or investigate further.

If there's any good news here, it's that at least some form of filtering is used to keep the database from swelling exponentially.

Most people placed on the government's watchlist begin in a larger, classified system known as the Terrorist Identities Datamart Environment (TIDE). The TIDE database actually allows for targeting people based on far less evidence than the already lax standards used for placing people on the watchlist. A more expansive—and invasive—database, TIDE's information is shared across the U.S. intelligence community, as well as with commando units from the Special Operations Command and with domestic agencies such as the New York City Police Department.

Those running TIDE have actually celebrated the fact that they recently added the millionth name to the database, failing to see that the constantly-increasing database is actually an admission of failure. If the system was working, the number of names should remain nearly constant, as those who aren't threats are removed from the list (something which apparently never happens) and those that are threats are rounded up (or otherwise disposed of).

And there seems to be a hint of racial profiling contained within the TSDB numbers.

The top five U.S. cities represented on the main watchlist for "known or suspected terrorists" are New York; Dearborn, Mich.; Houston; San Diego; and Chicago. At 96,000 residents, Dearborn is much smaller than the other cities in the top five, suggesting that its significant Muslim population—40 percent of its population is of Arab descent, according to the U.S. Census Bureau—has been disproportionately targeted for watchlisting.

Beyond this nearly-suspicionless watchlisting, there's further privacy concerns, prompted by the addition of millions of pieces of biometric data from American citizens, something that ramped up immediately following the Boston Bombing.

In the aftermath of last year's Boston Marathon bombing, the Directorate of Terrorist Identities began an aggressive program to collect biometric data and other information on all Americans on the TIDE list. "This project includes record by record research of each person in relevant Department of State and [intelligence community] databases, as well as bulk data requests for information," the documents note.

The DTI also worked on the subsequent Chicago Marathon, performing "deep dives" for biometric and other data on people in the Midwest whose names were on the TIDE list. In the process, the directorate pulled the TIDE records of every person with an Illinois, Indiana, or Wisconsin driver license.

That the many agencies tasked with counterterrorism are operating on instinct rather than articulable suspicion is nothing surprising. The large number of people with "no known terrorist affiliation" being added to a terrorist watchlist is the natural progression of bad policies with 12 years of momentum behind them.

As an additional note, it appears the US government attempted to "scoop" The Intercept by leaking a much more friendly recap of the leaked watchlist details to the AP, at least according to the this tweet by Jeremy Scahill, one of the post's authors.

US government, pissed we were publishing our story, tried to undermine us by leaking it to other news organization right before we published

US government, pissed we were publishing our story, tried to undermine us by leaking it to other news organization right before we published

A look at the AP piece seems to confirm this, as it presents something more akin to press release than a serious dive into the numbers. (More confirmation at the Huffington Post. The government claims its scoop "theft" was a "mistake.") It also makes no mention of the information appearing at The Intercept first. The AP's "story" presents this as mostly a triumph by the government, with only the briefest aside about privacy or civil liberties concerns.

This excerpt is indicative of the (very short) article's credulousness:

The database's growth is a result of the government's response to a failed attempt to blow up a commercial airliner over Detroit on Christmas Day in 2009. The terror operative's name was included in the database before the attack, but it was not on a list that would have prevented him from getting on a U.S.-bound airplane. Since then, the government has lowered the standards for placing someone on the no-fly list and intelligence agencies have become more diligent about submitting names to the TIDE database.

The database was created after the 9/11 terror attacks when it became clear that the government's terror watch list was ineffective. The watch list was once maintained in a rolodex and in paper notebooks, according to redacted photographs provided by the National Counterterrorism Center.

And here's the entirety of the "discussion" about the possible negative of an inflated, nearly-suspicionless watchlist of terrorist suspects.

The government does not need evidence that links someone to terrorism in order for the person to be included in the database. This is among the reasons the database and subsequent terror watch lists have been criticized by privacy advocates.

The AP says it has "learned," but it looks more like it was handed some talking points and an ultra-tight deadline. With thousands of news outlets pulling in the AP feed, this will allow the government to get out ahead of the leak, or at the very least, present a cohesive media presence that presents a "fair" portrayal of its out-of-control counterterrorism databases.

By "no recognized terrorist group affiliation", I take it they really mean "not affiliated with recognized muslim terrorist groups". They recognize many non-muslim terrorist and anarchist groups, the majority of those in the US are in this 40%, but the media and the government almost never call those groups out by name. They usually just name individuals, not their group affiliations.

The public panic over "terrorism" and the support for anti-terrorism measures is easier to manage when you can point to outside groups, a demonized "other", but it's much harder when the threat is actually within your own.

Houston and San Diego

And of course this is likely due to the large populations of Hispanic immigrants in these cities. And those kids being detained at the border? You guessed it. Terrorists in training. At least this is how DHS likely looks at it.

So after Boston, they started collating more information about Americans? Forgive me, but the Tsarneav's were not American - so this doesn't help for those sort of threats. It is interesting that these agencies reaction to almost everything is to "collect more". As Boston showed once again, it's quality and timeliness that counts, not quantity. These two were already known, and but for a careless clerical error may have been picked up. 9/11 was in that category too. The more you collect, the more you miss, it simply guarantees a lot of agents will be chasing shadows.

it's not just the database that's out of control! no matter what action is taken, terrorism is going to be with us, unfortunately, for a long time to come. my fear is that the actions and ways of governments atm, rather than combating it, is actually encouraging more of it and making it more widespread!

There's been several instances to tell us about who is on the list and how accurate it is. The student that was allowed to leave and then not return because an agent messed up his paperwork. It had to go to court because there was no way to remove someone from that list or challenge it.

There was another case some years ago, under Bush. A school teacher who was rather verbal in his disapproval of the administration. He got to Canada and then couldn't return.

Another American citizen was allowed to go overseas and then was blocked from returning by plane. I believe he took a cruise line to get back.

On the whole, with the list hidden from public view it is a sure place to put the problem children you'd rather not hear from as far as the government is concerned. This isn't about terrorism. This is about preventing embarrassment. When the list is hidden, who can complain and get results. At best you go to court and find out either you don't have standing or you can't get the data to prove it in court.

Either way comes down to the same thing. A government gone rabid. 280,000 people are terrorists? Come on here, who is trying to fool whom?

Re: Re: I never realised...

There is an interesting part missing in The Intercept story, if you read the doc itself. The DNA collection, which is follow up to Assange disclosure of State Dept using diplomatic personel to do it. The issue might araise of contamination, if done by untrained person in unsanitary conditions, and clumsy handling therafter. Then, the database itself becomes worthless.

According to doc, secret DNA collection seems to strech into US citizens too.

This lines up with several state legislations pushed in past decade to mandatory collect DNA during arrests. This suggests, there is another secret program going on.This also lines up with IBM getting contracts with many foreign governments for citizens databases.

If even 1% of the people on the list are terrorists, that's 6,800 terrorists in the US. If that's the case, we're doomed. You could arrest one per day over the next 18 years and still have enough slip through to have an attack every month.

If less than 1% of the people on the list are terrorists... what is even the point of keeping a list of people who have less than a 1% chance of being a terrorist?

Re: Re:

Like I already stated, those in the US intelligence community recognize and track many non-muslim terrorist and anarchist groups. Violent acts by muslim groups are quickly called "terrorism" by the government and the media, but violent acts by non-muslim groups rarely are. Even worse, the majority of attacks in the US were by non-muslim groups.

The point is that the government and media narrative paints a very misleading picture of the threats here in the US. The fact that they didn't call out non-muslim groups by name in the graph isn't surprising.

Re:

Re: Re: Re:

Even worse, the majority of attacks in the US were by non-muslim groups.

What attacks are you referring to? I can think of only one attack inside the US that was conducted by an organized terrorist group. Either you're very wrong or you're very right (and there are attacks going on that aren't getting reported) :-).

Re:

the Tsarneav's were not American

They were America residents. The term "Americans" is about as elastic as everything else in this discussion. For legal purposes, being in America with a valid resident style visa would be enough to make you "American".

The Boston bombings (and for that matter everything back to the Murrow building event) shows that there are many risks inside the US as well - from native born Americans and those in the country via immigration, visa, or other less honest means.

The list has increased 10-fold during Obama's stay in office, growing from 47,000 at the end of Bush's term to 680,000, 40% of whom the government is sure represent some sort of a threat, even if it can't quantify that in any specific way.

I always get a laugh when it comes to claims like these. I have a problem with any political discussion that starts with claiming X did more than Y. It shows a political leaning and an attempt to shade one party or the other, a clear indication that what's to follow was written with that goal in mind.

Tim, news flash for you: During the Bush administration, the list went from a single person (the first one they put on) to 47,000 - that is a massive 470,000% increase! Perhaps you should be thankful that Obama's government has worked to slow down the insane increases that happened during the Bush "administration".

Oh, and I noticed that they managed to work "bulk data collection" into the story too... make sure you touch all the hot button points along the way!

Re: Re:

Re: Re:

"the Tsarneav's were not American

They were America residents. "

Ah, you can't agree with or dispute any of the criticisms in the article, so straight to splitting hairs in a comment...

You're both right. They were American residents, and I believe both had actually become naturalised citizens before the time of the bombing. But, that doesn't necessarily make them "Americans" depending on who you ask and which law you're applying.

"I have a problem with any political discussion that starts with claiming X did more than Y."

Even if it's true? No wonder you're always wrong if you have to try and find ways to avoid facts that make you uncomfortable. Note that the statement didn't say "Obama is worse/better than Bush because X", it simply compared numbers under both administrations and noted that one was significantly higher.

"During the Bush administration, the list went from a single person (the first one they put on) to 47,000 - that is a massive 470,000% increase!"

Ah, we're back to half-truths again, are we? While, of course, smugly trying to attack other for the same.

It has to be noted that while furiously typing your own "facts" (uncited, of course), you absolutely failed to refute the fact you were addressing. Whether or not Bush increased the list massively in his time, the fact is that it has also increased disproportionately under Obama's watch. You don't have to adhere to any of the moronic "sides" to see and admit this.

Of course, you won't admit it, but you do take the time to ensure that you use a different metric to make your facts sound higher (percentages), whereas it would look smaller if you used the same metric as the article (raw numbers). Because of this, you expect us to believe that an increase by 47,000 is worse than an increase by 633,000 just because they started from a lower number. You won't state why, of course, except for your own misplaced belief in your own correctness and the belief that everybody should take your faulty opinions at face value.

Re: Re: Re: Re:

There's a wide range of groups across the social-political spectrum, from "pro-life" groups that bomb abortion clinics to "eco-terrorist". They're all tracked as "terrorist" groups by the government, they're just rarely called "terrorist" in public.

Re: Re: Re: Re:

And yes, these groups do sometimes make the news. However, they're rarely called "terrorist". Even the term "eco-terrorist" has seem to fallen out of favor in the media post-9/11. They'll usually just name individuals, with the pretense that they'd acted alone, and downplay any involvement of a larger organized group.

Given how the FBI like to manufacture crime these days

Re: Re: Re: Re: Re:

Ah I see. While the number of attacks is relatively large, the impact is much smaller, and also much more focused. I think it might be appropriate to devote less resources to abortion and eco terrorism than to other kinds. It isn't necessarily an anti-Muslim conspiracy.

Statistical Math

I see both sides of the comment discussion re: comparing the size of the list increases under G.W. Bush and Obama. But the true math error is division by zero, as there was no list when G.W. Bush came into office, and the classification of citizens as terrorists was based on the "us vs. them" mentality, which assumes that Americans will never take up arms against their own country. We only have to look back to the days of the civil war to see that this thesis is false. So the real problem is the assumption behind the lists... which give a false sense of security based on quantity of data collected, neglecting the data quality issues.

Re: Re:

Dear Whatever,

You know, there is a definite unemployment problem in the US today and what with inflation and high costs and all, I was just wondering if you'd be kind enough to post the email address of those who are paying you to post here, and likely elsewhere.

If they are willing to pay you actual money for posting the drivel you do so often, its quite likely that many here would be able to earn a little extra income for posting similarly pathetic drivel in their spare time, for your employers.

The Institution of Paranoia

I think what everyone is overlooking here, is that this is just the No-Fly list.

In other words, this is just the list of folks that the USG doesn't care enough about to prevent them from knowing they're on a list, which they will learn as soon as the USG stops them from boarding a plane.

I'm certain there must be dozens if not hundreds of other lists the government agencies keep tabs on regularly to let them know where all of the suspected "bad people" are at any given time.

For example, there must be at least one list that lets people fly because their jobs necessitate them flying to other countries, even though they are considered a "risk".

These people would become aware of their being spied on if they were to be detained due to taking a plane, so they need to be allowed to fly while still being watched closely.

And what about the No-Hire lists. Lists of people suspected to be hackers... or worse... who are caused to be fired from any hi-tech jobs as soon as the employer registers them for income tax purposes.... or suspected bomb makers, or suspected anti-government artists and writers, or.... well the list of lists just goes on and on....