Lawyers for a terror suspect set free this week by the home secretary called for the immediate repeal of the entire control order system today, saying his three-and-a-half year ordeal had done him and his family lasting psychological harm.

The government's controversial regime of using virtual house arrest against suspects suffered a fresh blow when the decision by Alan Johnson to revoke a second control order rather than disclose "secret" evidence in the case was made public.

Johnson has written to lawyers for the man, a former imam to the Iraqi community in Britain, who is known for legal reasons only as AE, telling them that he made the decision after considering the impact of a law lords' ruling in June that he was obliged to disclose the detail of the allegations against him.

AE's solicitor, Mohammed Ayub, said a control order was first served on him in May 2006. "For three-and-a-half years he and his family have had to live a life which has been controlled in every respect by the Home Office: where he lives, when he may go out, where he may go, who he may meet, what employment he may undertake," he said.

"During the whole of this period, he has never been told on what basis he is suspected of being a terrorist and he has never had a fair hearing before a court at which he could challenge the allegation made against him."

Ayub said that AE, his wife and three children might not get over the psychological harm of the past three years. "AE will consider carefully his legal options in respect of seeking damages from the home secretary for the wrong that has been done him and his family." The lawyer said that control orders were "fundamentally oppressive and unfair", flawed in their conception, execution and review.

In previous high court rulings involving AE, the judges have said that he was an imam in an unnamed town in the north of England, who had received terrorist training and taken part in terrorist activities.

Earlier this month the home secretary insisted the control order regime remained viable despite the law lords' ruling in June.

The decision on AE is to be implemented with immediate effect, before a high court hearing was due to start on 1 October, when AE's lawyers were due to challenge the order.

A Home Office spokeswoman said revoking the order did not change the assessment that control orders remained the best available disruptive tool for managing the risk posed by suspected terrorists they could not prosecute or deport.

There are now 14 remaining control orders in force and lawyers are expected to challenge Johnson to disclose the "secret" evidence involved in each. Johnson has asked Lord Carlile, the official reviewer of terrorism laws, to report back on whether the regime should continue.