The Fisa courts, set up under the Foreign Intelligence Surveillance Act, are intended to provide legal oversight. Photograph: Jim Lo Scalzo/EPA

A former federal judge who granted government surveillance requests has broken ranks to criticise the system of secret courts as unfit for purpose in the wake of recent revelations by NSA whistleblower Edward Snowden.

James Robertson, who retired from the District of Columbia circuit in 2010, was one of a select group of judges who presided over the so-called Fisa courts, set up under the Foreign Intelligence Surveillance Act, which are intended to provide legal oversight and protect against unnecessary privacy intrusions.

But he says he was shocked to hear of recent changes to allow more sweeping authorisations of programmes such as the gathering of US phone records, and called for a reform of the system to allow counter-arguments to be heard.

Speaking as a witness during the first public hearings into the Snowden revelations, Judge Robertson said that without an adversarial debate the courts should not be expected to create a secret body of law that authorised such broad surveillance programmes.

"What Fisa does is not adjudication, but approval. This works just fine when it deals with individual applications for warrants, but the 2008 amendment has turned the Fisa court into administrative agency making rules for others to follow."

"It is not the bailiwick of judges to make policy," he added.

The comments, during the morning session of a PCLOB public workshop held in a Washington hotel, are the most serious criticism yet from a recently serving Fisa judge.

Until now, Fisa judges have mainly spoken anonymously to defend the court process.

Robertson says he was generally impressed with how "careful, fastidious and scrupulous" the court process had been, but felt the so-called ex parte system (where only the government is able to make its case to the judge) needed urgent reform.

"This process needs an adversary. If it's not the ACLU or Amnesty, perhaps the PCLOB can be that adversary."

Later on Tuesday afternoon, the workshop also heard from a number of other experts who called for the decisions of the Fisa courts to be made public.

James Baker, a Department of Justice lawyer who has represented the government in surveillance requests before the Fisa court, said that an unclassified summary of its findings could be produced fairly easily in future cases, although it would be harder do this retrospectively.

He said this was preferable to trying to redact existing orders. "Not everything that the Fisa court does is reflected in a [written] opinion," he said. "If the court writes the summary, it can write what it wants to say."

A panel of technical experts also gave evidence that legal attempts to separate US citizens from foreign surveillance targets online were increasingly flawed, because of the difficulty of identifying geographic locations in an era of cloud computing and virtual private networks.

Steven Bellovin, a computer expert at Columbia University, revealed that the NSA had even patented a system of locating addresses by triangulating round-trip times for data packets to travel between known internet nodes, but said such technology still often failed to separate foreign and domestic internet traffic.

The quartet who gave evidence argued that technological solutions to protecting privacy were necessarily limited and less preferable than introducing better policy checks and balances.

Nevertheless, the day-long PCLOB "workshop" produced little sign that the oversight board was preparing to propose radical new policy in its report to President Obama.

James Dempsey, a PCLOB member, criticised civil liberties campaigners for not doing more to suggest alternative ways for the government to gather intelligence. He also suggested the scale of intelligence that needed to be collected made it difficult to see how authorities could go back to granting individual warrants rather than blanket approvals.

Rachel Brand, another seemingly unsympathetic board member, concluded: "There is nothing that is more harmful to civil liberties than terrorism. This discussion here has been quite sterile because we have not been talking about terrorism."