NSA Director Keith Alexander, who admitted in court documents kept classified since 2009 that the agency had illegally surveilled thousands of phone numbers.

The next time the National Security Agency claims that it works only within the strict oversight of the judicial branch and other watchdogs, its critics will have a new story to tell in response: That in 2009, the agency was found to be routinely misleading those overseers, and that it took another four years for those violations to become public.

From 2006 to 2009, the NSA was found by its judicial regulators in the Foreign Intelligence Surveillance Court (FISC) to be illegally surveilling thousands of phone numbers both inside and outside the United States without reasonable suspicion, according to documents released Tuesday by the Office of the Director of National Intelligence in response to a Freedom of Information Act request from the Electronic Frontier Foundation. On a daily basis, NSA analysts searched massive records of phone call metadata for matches with a so-called "Alert List" of 17,835 numbers, of which only 1,935 met the legal requirement of "reasonable articulable suspicion" necessary to track those numbers.

"In its report to the Court, NSA stated the alert list only contained telephone identifiers that satisfied the RAS [reasonable articulable suspicion] standard," NSA director Keith Alexander wrote in a statement to the FISC summarizing the NSA's violations. "In reality, the majority of identifiers on the alert list were [counter-terrorism] identifiers that had not been assessed for RAS." Alexander went on to emphasize in his statement that the searches merely raised an alert if they were connected to a suspicious number, in which case a counterterrorism analyst could "make a [reasonable articulable suspicion] determination."

"They were basically doing an end-run around the Foreign Intelligence Surveillance Court, conducting suspicionless searches so that they could collect suspicions in order to do more searches," says Trevor Timm, an activist with the EFF who has participated in the FOIA request and analyzed the documents. "They seemed to believe that they could automatically search the data that came in from the phone companies every day and they wouldn’t have to follow court rules."

The portion of the Patriot Act that allows the NSA to collect the metadata of millions of Americans and foreigners, known as Section 215 or more specifically the Business Records provision, has come under renewed scrutiny following the revelations of NSA contractor Edward Snowden. Snowden's first leak to be published by the Guardian newspaper in June revealed that the NSA, via the FBI, had issued a Section 215 order to Verizon demanding all phone record metadata from American customers of its Business Network Services for a three month period. A follow-up story from the Wall Street Journal confirmed that AT&T and Sprint have also been the subject of such orders and Congressmen Saxby Chambliss and Diane Feinstein revealed soon after that the data demands were a routine request that had been made on a regular basis for years.

The NSA responded in June by protesting that the information collection was carefully overseen by judiciary, congressional and executive branch regulators. In his 2009 statement to FISC, however, Alexander was more contrite:

In retrospect, the Business Records Order did not provide NSA the specific authority to employ the alert list in the manner in which it did. The agency's failure to describe the alert process accurately to the court unintentionally precluded the Court from determining for itself whether the NSA was correctly implementing the Court's orders. Although I do not believe any NSA employee intended to provide inaccurate or misleading information, I fully appreciate the severity of this error.

The documents also cast doubt on statements Alexander made in congressional hearings earlier this year, in which he stated that the metadata collected under Section 215 had only been accessed by analysts less than 300 times in 2012. One footnote in the court filing reveals that two analysts alone conducted 280 queries of the database in just late December 2008 and early January 2009. Along with the daily "alert list" checks of thousands of numbers, Sanchez points out that either the NSA's activities have changed drastically since 2009, or Alexander was using a creative definition of "access" in his 2012 count. "It's kind of hard to square," says Julian Sanchez, a researcher with the Cato Institute who spotted the apparent discrepancy. "Every time the NSA makes a statement, it seems they were using words in a very special way that normal people don’t use them. At some point it stops being convincing."

In a statement about the newly-released court documents Tuesday, Director of National Intelligence James Clapper said that the 2009 ruling had resulted in reforms such as the NSA creating the position of Director of Compliance, instituting new "technological safeguards" and "procedures to ensure the accuracy and precision in FISC filings." He didn't note any disciplinary action taken as a result of the misconduct, which remained classified for another four and a half years following the three-year-long period of illegal surveillance.

In fact, the court documents seem to paint a picture not of any deception on the part of the NSA, but rather of a surveillance apparatus that's too large for anyone to fully monitor or even understand, says the EFF's Trevor Timm. He pointed to quotes like one from an intelligence official to Politico, who said that "there was nobody at NSA who really had a full idea of how the program was operating at the time."

"This is a breathtaking admission—the NSA’s surveillance apparatus, for years, was so complex and compartmentalized that no single person could comprehend it," Timm wrote in a blog post on the EFF's website. "The intelligence officials also acknowledged that the court has to base its decisions on the information the NSA gives it, which has never been a good basis for the checks and balances and oversight that is a hallmark of American democracy."

Tuesday's Freedom of Information release resulted in hundreds of pages of newly public documents, which the EFF--and we at Forbes--have only begun to dig through. Check them out here, and stay tuned for more.