The independent inquiry by Mark Ellison found the spy and other undercover officers gathered personal details about the murdered teenager's parents. Photograph: PA

Theresa May has announced a public inquiry into the work of undercover police officers shortly after the publication of an independent inquiry that found Scotland Yard had spied on Stephen Lawrence's family.

Her move follows an independent inquiry into potential corruption and the role of undercover policing in the Lawrence murder inquiry, which found that Scotland Yard planted "a spy in the Lawrence family camp" – whose existence was previously concealed.

May told the Commons that revelations contained in the review about the original murder investigation were "deeply troubling".

The independent inquiry by Mark Ellison also found the spy and other undercover officers gathered personal details about Doreen and Neville Lawrence, the parents of the murdered teenager.

The information obtained included "discussion of the progress, reasons and details of the decisions made by the Lawrence family" connected to their campaign to force the Metropolitan police to investigate their son's racist murder properly.

May also went beyond the Lawrence case to announce a review of successful prosecutions where investigations involved undercover officers. It came after Ellison voiced considerable concern at the wider problems caused by the blanket of secrecy shrouding undercover police operations and demanded the review, which he will now lead.

He said he wanted to revisit prosecutions "to assess if material non-disclosure [of the role of SDS officers] may have occurred in any case in which there has been a conviction".

Hours later, it emerged in the afternoon that prosecutors are deliberating whether to charge three police officers over sexual relationships they formed while undercover.

The Crown Prosecution Service is examining a file of evidence gathered by Operation Herne, the internal police investigation into the alleged misconduct of the undercover spies over the past 40 years.

Mick Creedon, the Derbyshire chief constable, who is in charge of the investigation, said in his report that "there are not and have never been any circumstances where it would be appropriate for such covertly deployed officers to engage in intimate sexual relationships with those they are employed to infiltrate and target".

Ellison, working for the home secretary, also criticises the police for arranging a meeting between the spy in the Lawrence family camp – known only as N81 – and an officer who was part of the internal team drafting the response by Sir Paul Condon, the then Met commissioner, to the inquiry led by William Macpherson into the Met's handling of the murder investigation.

The officer who received the information was Richard Walton – then working on the Lawrence review team. Walton is now the commander of the Metropolitan police counter-terrorism command.

The lawyer found that the opening of such a channel of communication at that time was "wrong-headed" and "inappropriate".

He added: "The reality was that N81 was, at the time, an MPS [Metropolitan police service] spy in the Lawrence family camp during the course of judicial proceedings in which the family was the primary party in opposition to the MPS."

He also said: "The mere presence of an undercover MPS officer in the wider Lawrence family camp in such circumstances is highly questionable in terms of the appearance it creates."

N81, whose name has not been revealed, was deployed in a group "positioned close to the Lawrence family campaign". The spy gathered "some personal details relating to" the murdered teenager's parents.

Elsewhere in the report Ellison also concluded that a public inquiry would have "limited" potential to uncover further evidence regarding corruption in the original murder investigations, a conclusion likely to frustrate the Lawrence family. But his comment about an inquiry was focused on a review of those murder investigations – not the activities of undercover police officers.

Ellison said there were also serious concerns that material on the corruption issue believed to have been held by the Met could not be found.

His report also concludes:

• There were "clear defects" in the level of information the Met police gave to the 1998 public inquiry into the murder.

• There are reasonable grounds for suspecting a detective, John Davidson, in the original Lawrence investigation acted suspiciously, and may have had a relationship with the father of one of the suspects.

• There are not reasonable grounds to suspect the conduct of other officers involved in the initial investigations, though he adds that there are still some outstanding lines of inquiry.

Neville Lawrence, Stephen's father, said: "What the home secretary has announced today is 21 years overdue. Mark Ellison's report has simply corroborated what I have known for the past 21 years and our long fight for truth and justice continues."

The prime minister, David Cameron, said on Twitter: "Like the home secretary, I find the conclusions of the Stephen Lawrence review profoundly shocking. It's important we have a full inquiry."

The home secretary said the "outstanding" lines of inquiry about allegations of corruption in the Lawrence case would be examined by Keith Bristow, the director general of the National Crime Agency.

MPs were told Bristow had been asked how best these inquiries could be taken forward into Davidson and other officers. Bristow had been asked to report back to the home secretary, May told the Commons.