The past is constantly being re-considered, revised and rewritten. As new evidence appears – new sources or files are unearthed – historians are given the chance to measure what they know about an event and to test interpretations against the new information.

So for decades the generally accepted picture of the British Empire, on which the sun never set, was that it brought free trade, Christianity and education to India and Africa. When the indigenous populations demanded independence and Macmillan’s “winds of change” swept over Britain’s colonial possessions, they were handed over – in a largely peaceful process – to local leaders.

So while the French were engulfed in the eight horrific years of the Algerian War, the British handover of power was characterised by smiling new leaders being photographed with the queen at meetings of the Commonwealth.

Another way of looking at this story, however, was highlighted last year by the compensation paid to Kikuyu plaintiffs in respect of torture they suffered in the 1950s, when British rule was threatened by the Mau Mau. A different narrative began to emerge of British colonialism, captured in titles such as David Anderson’s Histories of the Hanged, Caroline Elkins’ Britain’s Gulag and Ian Cobain’s Cruel Britannia.

The expulsion of the inhabitants of the British territory of Diego Garcia to make way for a US naval base during the Vietnam War was dramatised in Adrian Jackson’s A Few Man Fridays.

It appeared that the British Empire had kept itself on the road by means of ethnic cleansing, concentration camps, torture and massacres.

Hidden cache of history

During the Mau Mau legal case it came to light that thousands of government files relating to the colonial past had not been revealed, let alone made public under the 1958 Public Records Act. Numbers of 8,000 or 9,000 were bandied about. More recently, Ian Cobain has suggested in The Guardian that there may be over a million Foreign and Commonwealth files, some going back to the Crimean War, hidden away at the ironically-named HMG Communications Centre at Hanslope Park, near Milton Keynes.

On 22 January, 27 fellows of the British Academy, all practising historians, academic lawyers or political scientists, wrote to the Foreign Secretary expressing their concern about the concealment of these files. Their letter was published in The Guardian the following day. While aware of existing advisory mechanisms, they have asked the government to explain what material is being held and offered to assist in expediting the process of releasing these documents to the public. A number of parties are also exploring legal procedures to challenge exemptions claimed by the government under the Public Records Act and more recent legislation.

Why does all this matter? Our relationship with history never stands still. The study of the colonial past of Britain and indeed of other European countries has changed radically in the past 20 or 30 years, for three reasons. First, public recognition of the evils of Holocaust has forced a reconsideration of other examples of genocide, often committed in post-colonial countries but not exempting colonial ones.

Second, in the study of the history, politics and literature of empire since Edward Saïd’s Orientalism and the “subaltern studies” of the 1980s, a post-colonial critique of colonial rule and the colonial legacy in the contemporary world has been developed around Indian-born intellectuals now with chairs in the US, such as Gayatri Chakravorty and Homi Bhabha.

Third, since the end of the Cold War, the preservation of human rights has been a touchstone in national and international affairs. Regimes are assessed not only in terms of their power and wealth but in terms of their human rights record. This applies as much to the UK as to Russia and China.

Public interest

So what is at stake here? Why can we not simply agree with the line that national security requires official secrecy and that some questions in the past should not be pried into too closely? There are two reasons, which relate to both the nature of our democracy and to the world we live in.

To begin with, an obsession with official secrets betrays an authoritarianism that undermines free society and a paternalism that reduces citizens to subjects. There is a sharp contradiction when GCHQ operations tell us that the government will stop at nothing to unearth information about its own and other people, while people’s right to call their government to account for past actions is stone-walled.

That secrecy, moreover, removes the right to justice of former colonial peoples who have been abused. Today, they can no longer be trampled upon or ignored, and in our multi-cultural and multi-faith societies they and their descendants live, not somewhere else, but among us. Is it surprising that a sense of injustice sometimes leads to acts of violence against former colonial powers or that racial troubles erupt in our suburbs? The release of these papers to enable scholarly research and a mature debate about Britain’s role in the world will be a tribute both to democracy and to justice.