British forces are avoiding detaining suspected insurgents in Afghanistan and Iraq because of fears they will be liable under the European human rights convention, a key Bush administration lawyer told the Guardian yesterday.

John Bellinger, legal adviser to secretary of state Condoleezza Rice, said detention operations had been "enormously complicated" by the application of human rights protection to war zones outside Europe.

"Human rights law was arguably intended to apply to civilian populations," he said. "There has been an effort to extend it to military operations that are becoming more like counter-insurgency operations."

He also said the US wanted to try Binyam Mohamed, a British resident held at Guantánamo Bay, for war crimes at a special military commission. The case is the subject of a legal struggle in the high court, which heard yesterday that the US authorities were refusing to hand over documents which could support Mohamed's claims that he has been tortured.

Bellinger, in London for an international meeting of lawyers, said: "Human rights law is clearly regionally intended to deal with the state's treatment of its own people. Importing concepts from human rights law is not necessarily the solution."

The appeal court ruled in 2005 that British soldiers in Iraq are bound by the Human Rights Act where they have "sufficient control" over detainees. In a second landmark case last year, the government admitted "substantive breaches" of the European human rights convention over the death and torture of Iraqi civilians in the custody of British soldiers.

The effect, Bellinger said yesterday, was palpable. "European countries will go to great lengths to avoid detaining anyone in these places," he said.

British defence officials expressed puzzlement at Bellinger's remarks. "British forces are undergoing detention operations in Afghanistan; there is no suggestion that they have been scared off," said one. However, British commanders have asked for assurances any detainees handed over to the Afghan authorities will be treated properly, and not passed to the US.

Richard Stein, a defence lawyer for Binyam Mohamed, said yesterday: "In Europe, the courts take over and act while the Americans use might and power to operate outside the law." Bellinger told the Guardian that the US would strive to prevent Mohamed's return to Britain. "Our defence department proposes to try Binyam Mohamed for war crimes."

Mohamed, 30, denies any connection with terrorism, including claims he was involved in a "dirty bomb" plot, and says any confessions he may have made were extracted during torture. An Ethiopian national and British resident, he was held in Pakistan in 2002, when he was questioned by an MI5 officer.

He was later secretly rendered to Morocco, where he says was tortured. The US subsequently flew him to Afghanistan and he was transferred to Guantánamo in September 2004.