Theresa May, the home secretary, described the revelations as 'shocking and disturbing'. Photograph: Olivia Harris/Reuters

The home secretary has ordered a public inquiry into the undercover infiltration of political groups after an independent inquiry confirmed that Scotland Yard had spied on the family of Stephen Lawrence.

Theresa May's decision came after an inquiry conducted by Mark Ellison QC found that the Metropolitan police planted "a spy in the Lawrence family camp".

In a dark day for the Met's reputation, May branded the revelations "profoundly shocking and disturbing", adding that "policing stands damaged today". She warned that the "full truth" had yet to emerge.

It also emerged that three other police officers could be prosecuted over sexual relationships they formed while pretending to be political activists while undercover.

May's decision to order a public inquiry comes after three years of revelations surrounding the undercover officers who have been sent to infiltrate political groups over the past 40 years. Investigations by the Guardian have revealed how the spies routinely formed sexual relationships with the activists they had been sent to spy on and stole the identities of dead children to fortify their cover while they lived among campaigners in deployments usually lasting four or five years.

The officer who spied on the Lawrences monitored the family during the inquiry led by Sir William Macpherson into the Met's bungled handling of the investigation into their son's murder.

The police spy – known only as N81 – gathered personal details about Doreen and Neville, the parents of the murdered teenager, as well as intelligence that Ellison said could be seen as giving the Met a "secret advantage" over the family during the inquiry.

N81 secretly passed "fascinating and valuable" intelligence on to a senior Met officer who was involved in formulating the then Met commissioner Sir Paul Condon's defence of his force's conduct during the murder investigation.The officer who received the information was Richard Walton, then a detective inspector involved in writing the Met's submissions to the inquiry. Walton is now the commander of the Metropolitan police counter-terrorism command. Ellison found that the opening of such a channel of communication at that time was "wrong-headed" and "inappropriate".

N81 worked in the Met's controversial undercover Special Demonstration Squad (SDS), along with Peter Francis. Last summer, the Guardian revealed that Francis had also spied on the Lawrence family, at a time when they were seeking to bring the racist killers of their son to justice.

Ellison concluded: "The mere presence of an undercover MPS officer in the wider Lawrence family camp in such circumstances is highly questionable in terms of the appearance it creates of the MPS having a spy in the family's camp."

In a speech on Thursday in the House of Lords during which she fought back tears, Lady Lawrence, said: "It's been 21 years of struggle and no family should have to do that. It's the job of the justice system and the police service to give service to the whole community not just to one section. That is what I have been campaigning about for the last 21 years. We weren't asking for anything special, just for something that we should have had just like any other citizen of this country."

Ellison said there was not enough surviving evidence to assess Francis's claim that he had been asked to find information to undermine the Lawrences and their campaign.

Meanwhile, Francis welcomed the announcement of the public inquiry, adding: "When the full truth comes out about the police's work and activities, across the UK, against political campaigns and protests since 1968, I think they will be very shocked."

However, Ellison's verdict stood in contrast to a report by the police that largely exonerated themselves over any wrong-doing in relation to the undercover infiltration of the Stephen Lawrence campaign. At a press conference, Craig Mackey, the deputy commissioner of the Metropolitan Police, did not apologise for the espionage.

He said: "We've worked with both Doreen and Neville over the years, and the Met has worked hard to build a trusting relationship with them to help us achieve our aim of convicting all those responsible for Stephen's death – an aim we still hold.

"I understand that today they must feel that all the trust we have worked to build is shattered by what they've heard and read. As a police officer, and a human being, that's a terrible situation to be in."

David Cameron tweeted that the review was "profoundly shocking. It's important we have a full inquiry".

Labour leader Ed Milliband said the report by the QC, Mark Ellison, contained "deeply shocking conclusions", adding that it was "important to get truth and justice for Doreen and family".

Neville Lawrence, Stephen's father, said: "What the home secretary has announced today is 21 years overdue. Mark Ellison's report has simply corroborated what I have known for the past 21 years and our long fight for truth and justice continues."Prosecutors will now have to decide whether to charge three undercover spies over sexual relationships during their covert missions. The Crown Prosecution Service is examining a file of evidence gathered by Operation Herne, the internal police investigation into the alleged misconduct of the undercover spies since 1968.

May also said that a further inquiry by Ellison would be necessary to establish whether the undercover deployments had caused the wrongful convictions of campaigners.

Ellison found that the extraordinary secrecy shrouding the work of the undercover spies meant that they took part in criminal trials without revealing their true identities. Crucial information was therefore not disclosed to the court or the defendants, according to Ellison, who also found that "undercover officers sometimes failed to correct evidence given in court which they knew was wrong". In the last three years, as revelations about the covert deployment emerged, 56 activists have had their convictions quashed after evidence gathered by undercover officers was withheld from their trials.