Images released by the authorities in Dubai showing seven of the 11 suspects wanted in connection with the killing of Mahmoud al-Mabhouh. Photograph: AP

A hit squad that murdered a senior Hamas official in Dubai used the stolen identities of six British citizens and faked at least five other European passports, it emerged today.

The international investigation into the assassination of Mahmoud al Mabhouh last month is certain to examine the role played by Israel, where four of the victims of identity theft, three Britons and a German, currently live.

Some reports last night suggested that a team of 17 people were involved in the meticulously planned killing, of whom six have yet to be identified.

UK authorities issued the British passports used by the assassins in Dubai last month, official sources said. They confirmed the killers had not altered the names and numbers in the passports, but the photographs had been changed.

At least five of the Britons whose identities were used by the assassins in Dubai live in Israel. All deny any involvement.

"I am obviously angry, upset and scared – any number of things," Melvyn Mildiner told Reuters. "And I'm looking into what I can do to try to sort things out and clear my name."

Israeli television named two other British-Israelis whose identities were used, Paul Keeley and Stephen Hodes, and a German passport holder, Michael Bodenheimer.

The revelation fuelled already widespread speculation in the region over Mossad involvement in the killing of Mabhouh and an elaborately orchestrated plot involving 11 assassins mostly posing as tourists in Dubai, with some wearing wigs and false beards.

Mabhouh was one of the founders of Hamas's military wing and had been wanted by Israel for his role in the 1989 kidnapping and killing of two Israeli soldiers on leave. His participation was acknowledged by Hamas last month.

The Israeli government would not comment tonight on allegations of its involvement in Mabhouh's killing, which, if confirmed, would trigger a diplomatic row with Britain, and the other three European nations, whose passports were used: Ireland, Germany and France.

The Irish Republic said the three Irish citizens named by Dubai police as suspects did not exist. The German government said the passport number of the sole German suspect was either incomplete or wrong. The authorities in Paris refused to comment on the authenticity of a French passport used by one of the killers.

Dubai public prosecutors have issued warrants for "premeditated murder" against the 10 men and one woman in the suspected assassin team, the first step in obtaining a "red notice" from Interpol to track down wanted fugitives abroad.

The police in the emirate described the killing as meticulously planned, with the hit squad arriving on different flights and checking into different hotels. They were seen on surveillance video tailing Mabhouh from the moment he arrived in Dubai. His murder in his hotel room near Dubai's international airport took 10 minutes and early forensic tests suggest he was suffocated. The assassins left on flights to Europe and Asia before the body was discovered on 20 January.

Authorities in the United Arab Emirates identified Austria as a possible "command centre" for the elaborate operation. In a sign that the country would be seeking formal assistance from European states, Dubai's attorney general, Essam al-Humaidan, told the National, an Abu Dhabi-based newspaper, that the UAE had signed judicial treaties with certain European countries, enabling the "extradition of the suspects wherever they hide".

Michael Levi, a Cardiff University professor of criminology and an expert on identity theft, said that if the British travel documents used by the assassins predated the introduction of biometric e-passports, they would not be hard to tamper with.

"The sort of organisation that can pull off a hit like that will be able to make those sort of changes," he said. "The point is that in any system, you only have to be good enough to pass the scrutiny of the people you're going to come across."

The Foreign Office said it was too early to speculate on who could have carried out the identity theft against the British citizens involved.

In recent decades, the Mossad has gained a reputation for using passports of other countries. Britain had a diplomatic row with Israel in 1987 over its use of forged British passports. In 1997, a Mossad hit squad used doctored Canadian passports in a botched attempt to kill the Hamas leader, Khaled Meshal. In 2005, two suspected Israeli agents were jailed in New Zealand for obtaining the country's passports illegally.

A former Mossad official, Rami Yigal, told Israel Army radio the assassination "doesn't look like an Israeli operation"

• This article was amended on 16 February 2010. The original read, " ...the other three European nations whose passports were used: Ireland, Germany and France". This has been corrected.