William Hague makes a statement in the House of Commons, London, setting out plans to compensate elderly Kenyans tortured during the Mau Mau uprising. Photograph: PA

Britain is to pay out £19.9m in costs and compensation to more than 5,000 elderly Kenyans who suffered torture and abuse during the Mau Mau uprising in the 1950s, the foreign secretary, William Hague, has said.

Hague told the House of Commons that the payment was being made in "full and final settlement" of a high court action brought by five of the victims who suffered under the British colonial administration.

"We understand the pain and the grief felt by those who were involved in the events of emergency in Kenya. The British government recognises that Kenyans were subjected to torture and other forms of ill-treatment at the hands of the colonial administration," he said.

"The British government sincerely regrets that these abuses took place and that they marred Kenya's progress to independence. Torture and ill-treatment are abhorrent violations of human dignity which we unreservedly condemn."

Hague said Britain would also support the construction of a memorial in the Kenyan capital, Nairobi, to the victims of torture and abuse during the colonial era.

Hague told MPs that 5,228 Kenyans would receive compensation under the terms of the settlement agreed with their solicitors, Leigh Day.

He stressed that the government continued to deny liability for the actions of the colonial administration and indicated it would defend claims brought from other former British colonies.

"We do not believe that this settlement establishes a precedent in relation to any other former British colonial administration," he said.

The announcement was warmly welcomed by Martyn Day, senior partner at Leigh Day.

"I take my hat off to Mr Hague for having the courage to make today's statement and to announce this settlement with our clients," he said.

"These crimes were committed by British colonial officials and have gone unrecognised and unpunished for decades. They included castration, rape and repeated violence of the worst kind. Although they occurred many years ago, the physical and mental scars remain.

"The elderly victims of torture now at last have the recognition and justice they have sought for many years. For them, the significance of this moment cannot be over-emphasised."

The settlement comes after a lengthy legal battle between a number of elderly victims and the British government.

The Mau Mau movement emerged in central Kenya during the 1950s to take back seized land and push for an end to colonial rule.

Supporters were detained in camps and thousands were tortured, maimed or executed.

Last year the high court ruled that three Kenyans tortured during the unrest could pursue their compensation claims against the government.

The Foreign Office had attempted to thwart the bid, claiming the actions were brought outside the legal time limit and there were "irredeemable difficulties" in relation to the availability of witnesses and documents.

It did not dispute they suffered "torture and other ill-treatment at the hands of the colonial administration".

Lawyers for Wambugu Wa Nyingi, Paulo Muoka Nzili and Jane Muthoni Mara argued that it was an exceptional case in which the judge should exercise his discretion in their favour.