On Wednesday, the Department of Justice published three declassified documents on a new government website hosted on Tumblr called "IC (Intelligence Community) On The Record."

In a press call that preceded the release of the documents, three senior intelligence officials defended their record. The general counsel of the Office of the Director of National Intelligence, Robert Litt, portrayed the documents as evidence of an intelligence agency working within the rules and making few mistakes.

"This is not an egregious overreaching by a greedy agency seeking to spy on Americans," he said. "It's an inadvertent collection of a relatively small number of US person communications."

At least one of the documents, however, spent a fair amount of time criticizing the NSA for overreaching and changing its story.

The longest item is a previously secret October 2011 document from the Foreign Intelligence Surveillance Act Court (FISC) showing that the NSA "frequently and systematically violated" its own oversight requirements. The agency collected as many as 56,000 e-mails and communications by Americans with no connection to terrorism.

The federal judge authoring the opinion, FISC Judge John Bates, concluded that there is no way to know with certainty how far the government’s intelligence and surveillance capabilities have actually gone. In his 85-page opinion, Bates noted that his court originally approved the NSA's ability to capture a more limited and targeted amount of data.

“In conducting its review and granting those approvals, the Court did not take into account NSA’s acquisition of Internet transactions, which now materially and fundamentally alters the statutory and constitutional analysis,” the judge wrote.

Bates also wrote that the NSA had been misleading about the scope of the NSA's surveillance program.

In the footnote to that section, he added:

The Court is troubled that the government’s revelations regarding NSA’s acquisition of Internet transactions mark the third instance in less than three years in which the government has disclosed a substantial misrepresentation regarding the scope of a major collection program.

. . .

Contrary to the government’s repeated assurances, NSA had been routinely running queries of the metadata using querying terms that did not meet the required standard for querying. The Court concluded that this requirement had been “so frequently and systematically violated that it can fairly be said that this critical element of the overall…regime has never functioned effectively.”

“Meaningful review... is not feasible”

Later on in the October 2011 opinion, the judge notes:

Ninety percent of the approx 13.25 million total Internet transactions acquired by NSA through its upstream collection during the six month period works out to be approx. 11.9 million transactions. Those 11.9 million transactions would constitute the universe of SCTs [single communication transactions] acquired during the six month period, and .197 percent of that universe would be approx. 23k wholly domestic SCTs. Thus, NSA may be acquiring as many as 46,000 wholly domestic “about” SCTs each year, in addition to the 2,000-10,000 [multiple communication transactions] reference above.

. . .

In absolute terms, tens of thousands of non-target, protected communications annually is a very large number.

The government distinguishes between what it calls “single communication transactions” as well as “multiple communication transactions” (MCTs). The former represents a single e-mail or text message between two parties that is intercepted en route. By contrast, MCT represents a bulk of messages that transit collectively. At times, over-collection occurred because the NSA collects communications in bulk, and separating the e-mails of a particular target is a difficult task.

For example, when you pull up your smartphone and download your IMAP e-mail, synching what’s on your phone with what’s in your inbox, you’re probably collecting several messages at once. And even though the government might be interested in one of those messages, it collects and stores the other non-suspect messages as well.

How did the government realize that it was over-collecting—despite the fact that this data collection program had been going on for three years up until that point?

“[The NSA and the Department of Justice] were having a discussion and a light bulb went off in somebody's head and they said: ‘Oops, this may be a problem,’” Litt said on the press call. He was speaking in response to a question from a CNBC reporter.

Earlier, Litt said that the NSA “was not capable and is still not capable of breaking those down into separate e-mails.”

Judge Bates had a one last scathing conclusion: neither the NSA nor the Court has the ability to know with certainty how many Americans’ rights have been violated.

The sheer volume of transactions acquired by NSA through its upstream collection is such that any meaningful review of the entire body of the transactions is not feasible. As a result, the Court cannot know for certain the number of wholly domestic communications acquired through this collection, nor can it know the number of non-target communications acquired or the extent to which those communications are to or from United States persons or persons in the United States... Internet service providers are constantly changing their protocols and the services they provide and often give users the ability to customize how they use a particular service. As a result, it is impossible to define with any specificity the universe of transactions that will be acquired by NSA’s upstream collection at any point in the future.

Ultimately, the court ruled the "upstream collection" of millions of MCTs was constitutionally deficient and needed to be altered. Those "upstream" programs constitute about nine percent of the total Internet communications collected by the NSA. The newly released court document provides the most conclusive evidence of the NSA's overreach in its surveillance and demonstrates, in the government's own words, the vast scope of those programs.

It's at least somewhat reassuring that the FISC court judge (Bates) was not a rubber-stamp after all. The problem here with the 'judicial review' process is that the FISC court can't issue any orders that can be enforced by a separate enforcement mechanism.

Eventually it's going to come out that they grab everything they can from every data source they can get their hands on - authorized or not - and store it, and there are no safeguards and no oversight what-so-ever against high level individuals having access to it all.

"... the Department of Justice published three declassified documents on a new government website hosted on Tumblr ..."

I had to re-read that three times, and after the third time, my head *still* exploded. The DOJ has a new website that is hosted on Tumblr where it publishes newly released documents - What. The. Fuck?!??

"... the Department of Justice published three declassified documents on a new government website hosted on Tumblr ..."

I had to re-read that three times, and after the third time, my head *still* exploded. The DOJ has a new website that is hosted on Tumblr where it publishes newly released documents - What. The. Fuck?!??

Well of course, Yahoo buys Tumblr and pays DOJ to post stuff on Tumblr. DOJ probably figures it makes them seem like they stay current and maybe they think it targets certain people they think its important to target, and Yahoo's Tumblr gets more attention.

“[The NSA and the Department of Justice] were having a discussion and a light bulb went off in somebody's head and they said: ‘Oops, this may be a problem,’” Litt said on the press call.

This sums up the disinformation campaign right here. They're trying to back pedal and say that they too, like all of the pissed off Americans, suddenly realized they were collecting "too much" information (Overcollecting? Look for this to be the mass media buzz-word for the next month) and immediately put a stop to it.

Yeah fucking right.

Do you mean to tell me they set up the world's most advanced data center to engulf the full outpouring of the internet and didn't REALIZE they were doing exactly that?

The CIA has always been the masters of disinformation, and this stinks of their strategy.

Americans don't care if you're illegally collecting 1 or 1 billion people's data. They care that it violates the Constitution.

We need to get these Nazi-like assholes out of our government and take this country back before it's too late.

"The court is concerned" in reference to the lies the NSA keep telling to the public...Sorry but I'm just baffled that they are CONCERNED! This are friggin lies under oath damnit! at least have the decency to fake it better, concerned my @SS! Even the "declassified" documents downplay the issue. I smell PR campaign from a mile. no one will be punished for this, so stay away Mr. Snowden or you will be the next victim of this mess right after our privacy and freedom of speech.

The Judicial branch is our last hope. If they truly find the NSA in violation of the constitution, my faith in our system of government and its checks and balances may be restored.

The founding fathers were really brilliant men.

This really is the big question. Do we realistically have a hope that we really will see anything done about this or is it all just going to be swept under the rug? I'm sincerely hoping that we really do get some meaningful action on this before much longer.

At this point I'm wondering just when is the NSA and Obama administration going to realize that their deny and deflect strategy on this is making our country at the least look incompetent and at worse the biggest bully who has shown that we're not afraid to use our power to get what they want. Even our allies are looking more complicit in this activity, just look at the article posted here earlier about the UK shaking down the Guardian, and I'm pretty surprised that at the very least China's not taken a potshot at us saying "You say WE'RE bad? Look in a mirror lately?"

And for those of you outside of the U.S. it looks like you were right we ARE a country run by bullies. Makes me wonder if this was the actual aim of 9/11 all along not so much to destroy us but to prove just how we are the biggest bunch of hypocrites who still possess a military strong enough to cow anyone who disagrees with that assessment. If that was the aim I think it's pretty clear they succeeded. -_-

Well, not sure what to say. I mean, the administration is still trying to catchup to the leaks with belated declassified information. Snowden told us about it already and we are just hearing now from the Administration. This was done to satisfy certain legal requirement from a third party. The administration would not deliberately release declassified information unless someone is asking the question.

And this is the evidence they're happy to give to prove they aren't doing anything wrong. I imagine we'll never see the court opinions stating massive abuses, or get any kind of actual investigation into the NSA

Nice to see all the lies and contradictions starting to catch up with the Intelligence Community. It seems they lie and obfuscate everything so readily that they don't know how to stop, even when they'd be better off just shutting up.

It's at least somewhat reassuring that the FISC court judge (Bates) was not a rubber-stamp after all. The problem here with the 'judicial review' process is that the FISC court can't issue any orders that can be enforced by a separate enforcement mechanism.

I read this differently than you. The FISC is certainly a rubber stamp. They even tell the NSA how to tweak their applications to get them stamped. In this one rare case, the violation was so egregious that even the FISC couldn't bring themselves to stamp it.

This really is the big question. Do we realistically have a hope that we really will see anything done about this or is it all just going to be swept under the rug? I'm sincerely hoping that we really do get some meaningful action on this before much longer.

Even if the NSA and the Obama administration roll over on this, promise to reform everything, and act very contrite ... how do we know they're being honest? Will we just have to believe them until another leaker (with sufficient access to secret documents) is willing to risk their life and freedoms in order to bring us the truth?

The culture of lies, secrecy, gag orders, and "trust us, it's for the greater good" that surrounds the NSA is the root of the problem, and I don't see it ever going away. Not to mention the many other TLA agencies -- do we really think that there's no comparable oversight problems within the CIA, the FBI, or the DHS?

While I'm ranting, and just because it's not mentioned near enough, I would like to add that it's complete bullshit that "we" Americans are treating just the domestic surveillance as problematic. The dragnet of all communications needs to be stopped, period.

This makes me wonder if even the President is really in control here. Has the national security state taken on such a life of its own that even those who are supposed to be in charge can't control it? Even the FISC says that the NSA is going too far, only to get ignored.

I love how the court order berates the NSA for "overreach" and "overcollecting" but stops short of saying: shut it down.

If its unconstitutional, and FISC agrees that it is...why not send an injunction? Why not say, the next person using this system to collect information will be criminally prosecuted.

On another note, this is why we need hackers. This is one of the reasons I love reversing so much...we all need to unite, hack the government. Then we can seed a torrent using the machines of high ranking officials...let the fuckin martyrs be the corrupt...

It's at least somewhat reassuring that the FISC court judge (Bates) was not a rubber-stamp after all. The problem here with the 'judicial review' process is that the FISC court can't issue any orders that can be enforced by a separate enforcement mechanism.

I read this differently than you. The FISC is certainly a rubber stamp. They even tell the NSA how to tweak their applications to get them stamped. In this one rare case, the violation was so egregious that even the FISC couldn't bring themselves to stamp it.

Valid point, different perspectives. If Judge Bates were just a rubber stamp, he could have let this case and others slide by without an opinion, but he did not. And yes, you are right to say that his role is advisory to enable the NSA to comply with the law (which is really the root problem).

As for 'egregious,' I think what is lacking here is a balanced understanding of perspectives. From an individual's point of view, it is unacceptable and an outrage that personal communications (content or metadata) are collected and possibly analyzed and retained indefinitely. I get that, and I don't like it at all, either. From a General Alexander's perspective, 10,000 people is a tiny fraction of the scale of the NSA's mission and activity. So, for all practical purposes, it might seem 'statistically' insignificant to the NSA.

It's at least somewhat reassuring that the FISC court judge (Bates) was not a rubber-stamp after all. The problem here with the 'judicial review' process is that the FISC court can't issue any orders that can be enforced by a separate enforcement mechanism.

I take a different view of that, as they grant 99% of the requests given. The FISC is most certainly a rubber stamp organization, and despite that, while getting limited information on what is actually going on, they managed to find it objectionable. If the FISC tells you you are going to far, it's like Charlie Sheen telling you you ought to lay off the drugs.

The fact is that:1. Congress has no oversight over the NSA because they lie to them and ignore their authority.2. Judiciary has no oversight over the NSA because they lie to them and ignore their authority.3. Executive is their boss and defends whatever they do.

It is up to the people and whatever remains of the free press to assert their demands to curb these abuses.

The Judicial branch is our last hope. If they truly find the NSA in violation of the constitution, my faith in our system of government and its checks and balances may be restored.

The founding fathers were really brilliant men.

Congress is also a reasonable place for hope. They can pretty swiftly take action along the lines of defunding activities (defunding the Utah facility would hopefully rein in the plans to record everything forever, although the NSA is already extensively doing Google-style distributed data warehousing and processing) and removing statutes such as FISA that have been twisted into claims of legal support for these programs. Congressional action also would quite clearly dispel the argument that all 3 branches have approved of the NSA's activities (to whatever degree you can really even say Congress was informed - what is up with conferring just with Republican members of Congress?).

Smith v. Maryland should be repealed by Congress, or at the very least make it clear that it does not legally support vacuuming up everything. There needs to be an idea in privacy law that even if discrete items might be considered "public" information, there are problems with the mass aggregation and processing of every scrap of such information - both by government and corporations (which the government has been using to collect and launder such info). Documenting what I ate for breakfast will not catch terrorists to begin with!

To more thoroughly attack the problem, Congress needs to not only remove the enabling statutory provisions (which really is what FISA ever was, even 35 years ago), but also put in place new statutes that expressly bar wholesale collections of haystacks in search of needles - with consequences. Currently, not only is there no consequence for violations, but also responsibility has been diffused to where it is very hard to punish anyone after the fact (it has been a long time since there has been a Truman-style "the buck stops here" president).

However, Congress will only act when there is more public opinion against these programs. Its starting to penetrate the minds of more average folk.

Some problems with looking to the judiciary include:- Time to resolve anything. This is a Supreme Court level issue, and will take years to get through there.- Admitted incompetence in handling computer tech issues. We have Supreme Court Justices who have acknowledged ignorance of simple email. The government also lays out a lot of misplaced metaphors and hides the ball in presenting these issues - look at how the government miscast relatively trivial computer matters in the Auernheimer case.- History of deference on national security issues. Standing has not been the only problem in the courts. There is a long track record - specifically in going after these programs - of the courts granting a national security exemption to the executive's activities.

The FISC undersold their capability for oversight. It should be within their power to appoint competent inspectors working on behalf of the court - that is what courts often do for difficult subject matter. However, the big investigatory commissions historically have been the province of Congress.

It would be nice to believe that the judiciary could take an essentially lawless executive branch, and get them to observe the rule of law. However, time and again it has been made clear that the executive has been doing whatever they please, and then twisting the law in unbelievable ways to justify themselves after the fact. Look at the FISA amendments: they were tailored to make these kinds of programs more legal, but they were already doing this before getting this so-called enabling/supporting legislation. There is also a very troubling thread throughout the justifications for the NSA's activities: an expressed belief that most or all of this somehow falls outside of any constitutional protections, and that any restraint on their part is essentially being done as a favor rather than in observance of any rule of law. Sadly/frighteningly, restraint seems mainly to be due to there being finite computing resources in the world.

In view of their track record, what should not be allowed to happen is to believe the executive branch should they make a show of capitulating. There will be words about stopping "these" (but not those) programs, but nothing really will change. It will just be hidden more thoroughly, and the full scope will never be disclosed.

Finally, if any of the threads that are popping in some of this are true, domestic federal law enforcement is rotten to the core, and is only pretending to follow the rule and spirit of the law when they think people might be watching. Had no one else heard of the notion of "parallel construction" of evidence (even if not under that label)? The FBI, and apparently many other agencies, has been doing it for a long time. Too many people have bought into the ends justifying the means, that criminals of all kinds should be rooted out and punished severely, and that the "real world" is filled with uncivilized terrorists and criminals, so the US is justified in likewise being uncivilized and abandoning our principles.

It's at least somewhat reassuring that the FISC court judge (Bates) was not a rubber-stamp after all. The problem here with the 'judicial review' process is that the FISC court can't issue any orders that can be enforced by a separate enforcement mechanism.

I take a different view of that, as they grant 99% of the requests given. The FISC is most certainly a rubber stamp organization, and despite that, while getting limited information on what is actually going on, they managed to find it objectionable. If the FISC tells you you are going to far, it's like Charlie Sheen telling you you ought to lay off the drugs.

I take a different view than that. Anyone applying for a FISA order knows what standard they have to meet (or what facts they need to purport to have/need/omit) and will tend to only seek a FISA order that they have high confidence the FISC will grant. The effect is making the FISC appear to be a rubber stamp, particularly when so much of the substantive matters are redacted, and we the public can only speculate as to how permissive or reserved the FISC really is.

I think that the FISC judges struggle to figure out where the line is between ideological principal and the practical realities in each and every application. Any Federal judge has almost certainly agonized over the application of constitutional rights in the face of challenging factual situations, and none that I have met (an admittedly small sample) have seemed like the kind of person to take matters like this lightly or with blind trust in an agency.

It's at least somewhat reassuring that the FISC court judge (Bates) was not a rubber-stamp after all. The problem here with the 'judicial review' process is that the FISC court can't issue any orders that can be enforced by a separate enforcement mechanism.

I take a different view of that, as they grant 99% of the requests given. The FISC is most certainly a rubber stamp organization, and despite that, while getting limited information on what is actually going on, they managed to find it objectionable. If the FISC tells you you are going to far, it's like Charlie Sheen telling you you ought to lay off the drugs.

Judge Bates is not Charlie Sheen. You are making an emotional analogy, which is understandable, but ultimately not helpful.I think the 99% figure is a result of the guidelines the FISC judge must follow, which are based on the inherent flaws in the Patriot Act and related laws and court decisions. If the FISC judge were to invent his/her own rules for judging what is fair or reasonable or lawful, I'm sure he would be replaced. The fundamental flaws here are: (1) the laws creating this entire system; and (2) the inherent one-sidedness of this process. There is no appeals court nor review process to review Judge Bates as far as I know.

The Judicial branch is our last hope. If they truly find the NSA in violation of the constitution, my faith in our system of government and its checks and balances may be restored.

The founding fathers were really brilliant men.

Congress is also a reasonable place for hope. They can pretty swiftly take action along the lines of defunding activities (defunding the Utah facility would hopefully rein in the plans to record everything forever, although the NSA is already extensively doing Google-style distributed data warehousing and processing) and removing statutes such as FISA that have been twisted into claims of legal support for these programs. Congressional action also would quite clearly dispel the argument that all 3 branches have approved of the NSA's activities (to whatever degree you can really even say Congress was informed - what is up with conferring just with Republican members of Congress?).

Your post was very wide ranging so I didn't quote it all, but I wanted to say I agree with you. In a nutshell, both the Legislative and judiciary have to step up and assert their role to achieve efective checks and balances.

This judge's comments are about the NSA's procedures of several years ago. The technology of years ago. The programming of years ago. The more limited personal communication practices people had years ago. A single data collection "mistake" today could involve millions of Americans.

This makes me wonder if even the President is really in control here. Has the national security state taken on such a life of its own that even those who are supposed to be in charge can't control it? Even the FISC says that the NSA is going too far, only to get ignored.

Precisely. I realized some years ago that the growth & scope of Federal government was not simply out of control, but was in fact actually uncontrollable.

This is not a republican vs. democrat vs. (fill in the blank) issue. Two years ago (give or take) the consistently excellent PBS series Frontline broadcast an episode on the stupefying growth of the intelligence community. A significant take-away from that episode was that, truly, no one knows how much money is actually being spent on feeding this demon.

There are so many individual tentacles sprouting from some uncertain number of squid-like intelligence "organizations", each with different grades of security classification & different rosters of who is cleared to be knowledgeable; no one can give a full or accurate comprehensive accounting of anything. This opacity is made all the more dense by the fact that so much of the machinery of it is in the hands of the "private sector".

And in this context I find that the term "Private Sector" is a simply sublime oxymoron.