Defence, security and intelligence grandees will next week launch a critique of the government's strategy for dealing with terror, including a demand that ministers should show that Britain has "unambiguously" renounced the use of torture.

They will argue that the government must demonstrate that it is "on the right side" when it comes to the abuse of detainees and the practice of extraordinary rendition.

The 180-page final report of the Institute of Public Policy Research's security commission, to be published on Tuesday, will recommend that the government ensures that its own agents employ only legal methods and robustly challenge alleged or suspected torture .

The commission also warns that control orders and other measures that "subvert the rule of normal law" can provide a propaganda coup for radical jihadi groups.

The criticism of official policy across a range of counter-terrorism and human rights issues is expected to prove highly influential as many of its figures come from the security and defence establishment.

The commission was jointly chaired by the former Nato secretary-general, Lord Robertson, and the former Liberal Democrat leader, Lord Ashdown. Its final report has been signed off by commissioners who include the former chief of the defence staff, Lord Guthrie; the former British ambassador to the UN, Sir Jeremy Greenstock; and the former Cabinet Office security and intelligence coordinator, Sir David Omand.

The two-year study concludes that a commitment to the rule of law at home and abroad and a willingness to uphold and protect human rights are fundamental to the legitimacy of a national security strategy.

Their recommendations include:

• Suspected terrorists should be dealt with using the standard criminal justice system;

• Ways need to be found to use intercept evidence in criminal trials without prejudicing national security;

• Deporting terror suspects to countries that practice torture on the basis of diplomatic assurances is unacceptable unless accompanied by robust independent monitoring to ensure their safety;

• Britain should sign the International Convention for the Protection of All Persons from Enforced Disappearance.

The report also examines the government's Prevent strategy to counter radicalisation and terrorism within Britain and warns that there have been occasions where enforcement activity has backfired. "Prevent must not become Provoke. Some of the pursuit and disruption tactics employed have had just such unintended consequences.

"This damages a counter-terrorism strategy that must be enacted 'amongst the people' and that is largely predicated on community trust and confidence in the state and its agents," says the final report seen by the Guardian.

It says the police and security services must recruit more Muslim staff and also further review the use of language to build on the decision not to use phrases such as the "war on terror". On torture, the authors say they pass no judgment on the allegations in the case of Binyam Mohamed, the British resident released after more than four years' detention at Guantánamo Bay.

"However, in our view, in ordering the closure of Guantánamo Bay, the ending of CIA practices of enforced disappearances and secret detentions forbidding torture, President Obama is re-establishing Amercian legitimacy in the eyes of the rest of the world. In the UK, we too must consider what more we can do to be unambiguously on the right side of these issues."

On the rule of law in Britain, the security commission stops short of calling for control orders and lengthy period of pre-charge detention to be scrapped, but says that both involve suspending the legal right of habeas corpus - the right of the accused to hear the evidence against them.

"We recognise the difficulties in this area but we must also recognise that developments that subvert the rule of normal law, however well-intentioned, can be a propaganda coup for the radical and neo-jihadi groups we are trying to combat," they warn.

The security and defence experts add that legitimate grievances of isolated minority groups, including over British foreign policy, have to be addressed.

They say that a divided and grievance-ridden society is unlikely to prove a resilient one when subjected to the extreme disruptions that future security scenarios might bring.