With the latest release of documents about the NSA and the FISA Court (this one in response to an ACLU/EFF Freedom of Information Act request) we now have yet more evidence that the NSA’s compliance with the court’s orders has been poor. We learn, for example, that, according to the court, “the NSA exceeded the scope of authorized [metadata] acquisition continuously during the more than [redacted] years of acquisition under these orders.” And, “NSA’s record of compliance with these rules has been poor.”

Extraordinary powers require extraordinary oversight. But we’re gradually beginning to see the full scope of the FISA Court’s inadequacy as an oversight institution. The latest disclosures follow other evidence that this court has had less than a stellar record in enforcing its rulings. Previous documents revealed, for example, that the NSA repeatedly violated court-imposed limits on its surveillance powers, and that the agency experienced numerous so-called “compliance incidents” such as staff using the agency’s tremendous powers to spy on love interests. And as my colleague Jameel Jaffer points out, the record suggests that the government has felt free to make bolder, less-supportable arguments before the secret FISA Court than it’s willing to make before real courts that are open to the public.

It has often been pointed out that the FISA Court is not a normal court, a big reason being that all of its proceedings are ex parte (that is, there is no adversarial proceeding, the court only hears from one side) and that it operates within an ocean of secrecy and compartmentalization. My colleagues Patrick Toomey and Brett Max Kaufman yesterday detailed the sorry story of how these characteristics allowed the court to stretch the law to permit bulk metadata collection.

There are a lot of proposals out there to fix the structural problems with the FISA Court, and I’m not going to get into them. I just want to make the point that the problems with the court as oversight institution run very deep—reaching to the very heart of what “a court” is—and that sweeping changes are needed if the NSA is to receive anything close to the degree of oversight it needs.

It’s significant that most or all of the NSA compliance problems now coming to light are self-reported. To begin with, that means we don’t know what kinds of problems were not reported. And unfortunately in any human institution the discipline required to self-report legal violations is, to put it mildly, not something that can be relied upon. It’s not exactly a “check and balance” to assume people will tell on themselves. Even if we could count on all problems eventually being self-confessed to a court, that is not good enough, because in the time that passes before officials ‘fess up—a matter of years in many of the cases we’ve learned about—real privacy invasions and damage can be done.

More fundamentally, the reason all these violations have been self-reported is that the court does not have the administrative capacity to ferret out any violations on its own. Like all courts, the FISA Court issues orders: “do this,” “don’t do that.” But unlike most courts, it has no way of actually knowing whether its orders are being followed. When a court issues an order in a normal adversarial court proceeding, one side in that case always has an incentive to watch the other like a hawk, and if the court’s orders are not carried out the judge is sure to hear about it from the aggrieved party. That is even true of criminal warrants that are issued ex parte; if the police exceed the scope of a warrant, a defendant can have any resulting evidence thrown out in court. But in the netherworld of the NSA, the FISA Court appears to be sending its orders into a black hole, with no way of finding out whether they are being complied with. Except through self-reporting.

Courts are not administrative agencies, like the EPA or USDA, that have inspectors on the ground able to watch over an operation and verify first-hand exactly what is taking place. Courts are generally not institutions that are set up for that kind of boots-on-the-ground oversight of what’s happening out in the world. It’s true that courts do have the power to appoint special masters and receivers—but it’s not clear what kind of a special master could oversee such a gigantic, compartmentalized beast as the NSA, and I don’t believe there is any precedent for one to operate on a permanent basis and at the scale and with the secrecy that would be required.

The fact is, human beings have a hard time reliably ascertaining truth. That is true even in the absence of extreme secrecy and compartmentalization, not to mention the existence of a gigantic and complex bureaucracy with thousands of moving parts. On a deep epistemological level, the adversarial judicial process that has evolved within the Anglo-American legal tradition is a response to that problem—an attempt to increase the reliability of truth determinations through the public, competitive clash of highly interested parties.

And implicit in that system is the premise that those interested parties actually know something of what’s going on. Creating a FISA Court “Devil’s Advocate” position may help steer the court away from the kind of one-sided interpretive rulings that inevitably result when a judge only hears one point of view, but a court-bound advocate won’t solve this institution’s core lack of administrative capacity to engage in boots-on-the-ground verification of its rulings. The FISA Court is like a robot that lacks eyes or arms—no matter how sophisticated its brain, it can’t go out and explore the world.

When Congress created the FISA Court, it made a departure from all the accumulated experience of our centuries-long legal tradition. It created an institution that was novel and untested, with secrecy and the lack of adversarial proceedings short-circuiting the normal operation of normal courts in very profound ways. It did this out of a recognition that our giant yet secretive national security agencies are something new in human history, that require new, more robust forms of checks and balances. But it now seems very clear that this deformed mutant version of a “court” is not up to the job.