Communist prisoners are held during the Malaya emergency of the late 1940s. Photograph: Jack Birns/Time & Life Pictures

The Foreign Office intervened to stop a criminal investigation into the alleged massacre of 24 unarmed villagers by British troops, in a cover-up that puts Britain's colonial past under renewed scrutiny. Newly disclosed documents reveal that in the 1990s UK officials pressured Malaysian authorities into aborting a police inquiry into the alleged killings by Scots Guards in Malaya in 1948.

They reveal that Malaysian police officers contacted Interpol and were due to visit the UK in 1993 to interview soldiers involved in the shootings, only for the Foreign Office to pressure the country's high commissioner into halting the visit. One memorandum states that senior Foreign Office officials later met Malaysian police chiefs to discuss closing the inquiry shortly before it was aborted.

The documents, released by Malaysian sources ahead of a judicial review related to the massacre, also reveal how a Metropolitan police investigation in 1970 into the allegations was "terminated" because an incoming Conservative government did not want the darker aspects of Britain's colonial past exposed.

The plantation workers were shot in cold blood by a 16-man patrol of Scots Guards in December 1948. Many of the victims' bodies were found to have been mutilated and their village of Batang Kali was burned to the ground. No weapons were found when the village was searched during a military operation against Chinese communists in the post-second world war Malayan emergency.

The British government has refused to apologise for the incident or offer reparations, and last November it said it would not hold a public inquiry into an incident that campaigners dub "Britain's My Lai massacre". A recent letter from Treasury solicitors indicates that the government is not prepared to discuss whether the killings were lawful or not.

News of the suppressed investigations follows last week's disclosure of government reports in the high court revealing the extent of British brutality during the Mau Mau rebellion in Kenya.

Four Kenyans who allege that rebels were sexually abused, castrated and beaten while detained under colonial rule in the 1950s will continue their claim for compensation this week.

A separate legal challenge over allegations of colonial brutality will shortly commence, with the Bindmans law firm launching a judicial review over the government's refusal to hold an independent inquiry into the Malaysian killings.

The case is being brought on behalf of four survivors, including one whose father was allegedly beheaded. The government's refusal to reopen the matter has provoked claims that it is carrying on a cover-up that has lasted more than half a century.

Documents released by Tan Kai Hee, founder of the Action Committee Condemning the Batang Kali Massacre, also show that the lead investigating officer in the 1970 Scotland Yard inquiry, Detective Chief Superintendent Frank Williams, had major concerns "over a political aspect to the case".

The defence secretary, Denis Healey, had instructed the Met to investigate the incident and arranged for Williams to travel to Malaysia to interview other witnesses and disinter the bodies for forensic examination.

However, the inquiry was abruptly stopped. Previously unreleased documents citing Williams's investigation state: "It was patently clear that the decision to terminate inquiries in the middle of the investigation was due to a political change of view when the new Conservative government came into office."

Author Ian Ward, who wrote an account of the massacre, Slaughter and Deception at Batang Kali, said: "Personally I was appalled at the cover-up that was instituted and has been systematically renewed year after year since it occurred."

Speaking from Malaysia, Ward said that, since his book was published in 2009, he had obtained notes of a 1994 meeting between two senior Foreign Office officials and a police chief in Malaysia during which they discuss winding up the investigation.

The four Malaysians represented by Bindmans are Loh Ah Choi, who was seven at the time and claims to have witnessed his uncle being shot dead; Lim Kok, whose father's corpse was allegedly beheaded by the soldiers; Chong Nyok Keyu, whose uncle was allegedly shot and killed; and Wooi Kum Thai, whose father was also killed.

John Halford, a partner at Bindmans, said: "This case is all about questions which, when they are finally answered, will bring about accountability for a massacre that was as immoral as it was unlawful. For six decades successive governments have gone to extraordinary lengths to evade answering these questions."

A letter seen by the Observer reveals that the government is threatening the four claimants with legal costs of up to £100,000 if the quest for an inquiry fails. "Many will question whether that is something a government with nothing to hide and confidence these killings were legal would resort to," said Halford.

Tan Kai Hee said the British government's decision to refuse a public inquiry into the allegations had left him shocked and deeply disappointed. He added: "Many of us had believed for much of our lives that Britain stood for openness, fairness and honour. The current impasse is neither honourable nor moral."

The official British position maintains that the villagers were killed as they attempted a mass escape into the jungle, having been warned they would be shot if they tried. Some of the troops involved have, however, claimed that there was no escape attempt and that they were ordered to take the men out in groups and shoot them.

The original investigation, in December 1948, amounted to a "superficial cover-up", according to Halford. Conducted by the Malaya attorney-general, Sir Stafford Foster Sutton, it did not question any survivors or carry out forensic examinations. An undated memo records the attorney-general's view that there "was something to be said for public executions".