Human rights campaigners have called for a full criminal investigation into the rendition of a Pakistani man by UK and US forces to Afghanistan, following a supreme court judgment describing his subsequent detention at the notorious US prison at Bagram as unlawful. Yunus Rahmatullah has been imprisoned ever since he was handed over by the SAS to American forces in Iraq in 2004, and has never been charged.

Lawyers for the man argued before the UK's highest court that the government should apply pressure on the US to release him. The court of appeal had previously issued a writ of habeas corpus – an ancient law that demands a prisoner is released from unlawful detention – requiring the UK to seek Rahmatullah's return or at least demonstrate why it could not. However, the US authorities refused to cooperate, arguing that they would discuss Rahmatullah's situation with the Pakistani government.

Lawyers for William Hague and Philip Hammond, the foreign and defence secretaries, had argued that they had no power "to direct the US" to release him and that it would be inappropriate for the courts to instruct them to ask the US authorities to return Rahmatullah.

Rejecting this argument, a panel of seven supreme court judges ruled that the UK did not need to have actual custody to exercise control over his release. The UK's most senior judges also declared that there was clear evidence that Rahmatullah's rendition and detention was a breach of international human rights law, despite "memorandums of understanding" Britain had agreed with the US over treatment of detainees.

Lord Kerr said: "The, presumably forcible, transfer of Mr Rahmatullah from Iraq to Afghanistan is, at least prima facie, a breach of article 49 [of the fourth Geneva Convention]. On that account alone, his continued detention post-transfer is unlawful."

Kerr also said that he would have "little hesitation in dismissing" arguments from former US assistant attorney general Jack Goldsmith asserting that al-Qaida operatives found in occupied Iraq were excluded from protection under the Geneva Conventions during armed conflict.

However, the court was split 5-2 in a decision to reject arguments by Rahmatullah's lawyers that there was more that the UK government could do following the American's refusal to respond to the habeas corpus writ. Rahmatullah was represented by legal charity Reprieve and solicitors Leigh Day, who argued that the UK should have made more effort to demand his release.

In a dissenting judgment, Lady Hale and Lord Carnwath said: "Where liberty is at stake, it is not the court's job to speculate as to the political sensitivities which may be in play."

Reprieve's director, Clive Stafford Smith, said: "This powerful supreme court decision has huge ramifications. Clearly there will now have to be a full criminal investigation. But if the US has 'dishonoured' its commitment to the UK in this case for the first time in 150 years, and continues to violate law as basic as the Geneva conventions, this also throws other extradition agreements with the UK into doubt."

Reprieve's legal director, Kat Craig, added: "The UK government has nowhere left to turn. The highest court in the country has expressed serious concerns that grave war crimes may have been committed as a result of which a police investigation must be initiated without delay."

Yunus Rahmatullah and Amanatullah Ali, both Pakistani men, are suspected of having travelled to Iraq to fight for al-Qaida. MI6 is understood to have tracked them as they travelled across Iran and into Iraq early in 2004. After they settled into a house in southern Baghdad a decision was taken to raid the building.

In an operation codenamed Op Aston, SAS troopers attacked the house, shooting dead two men and capturing several others. After the detainees were handed over to the US military, Rahmatullah and Ali were held at a prison at Baghdad international airport, according to a British military source. They were then flown to Afghanistan and taken to the US-run prison at Bagram, north of Kabul, a transfer that was in breach of the Geneva conventions.

SAS troopers detaining suspected militants in central Iraq were usually accompanied by one US special forces soldier, and the prisoners were then deemed to be the legal responsibility of the US government. For reasons that are currently unclear, no US forces accompanied the SAS on the Op Aston raid.

The aircraft that took the two men to Afghanistan is thought to have been the Boeing 737 that had a few days earlier carried Abdel Hakim Belhaj and his pregnant wife Fatima Bouchar to Tripoli, in a so-called rendition operation mounted by MI6 and Muammar Gaddafi's intelligence agencies. This operation is now the subject of a Scotland Yard inquiry.