Manage your subscription

How to stop Excel errors driving austerity economics

A global row has erupted over errors in research used to bolster the case for austerity. Open data might have spared us the pain, says Velichka Dimitrova of the Open Knowledge Foundation

26 April 2013

By Velichka Dimitrova

You never know where an Excel slip-up will lead

(Image: Dan Kitwood/Getty)

Another economics storm has hit the headlines but this time it is not the grand global gambles of the banks that matter – it is the minutiae of Excel spreadsheet entries. At the centre of it are two professors from prestigious Harvard University, Carmen Reinhart and Kenneth Rogoff.

Debt redeemed

Most widely reported among their three grounds of criticism is a single spreadsheet mistake that meant five countries were excluded from the sample used by Reinhart and Rogoff. The University of Massachusetts team say that, taken together, the problems they have identified in the original paper resulted in significant error in the conclusion that a large public debt cripples growth.

After correcting the mistakes, the researchers found that countries with a lot of public debt suffer only “modestly diminished” average GDP growth rates, rather than the stronger effect shown in the 2010 paper. Their analysis shows that when countries owe about 90 per cent of their GDP, they do indeed tend to have slower economic growth – but the trend does not follow the data at all well. This means that if a country’s public debt exceeds the threshold, its economic performance will not necessarily deteriorate. It may even get better.

The debate continues. Reinhart and Rogoff have admitted they made mistakes but say their core point remains valid. Herndon has responded that his results do not show that public-debt-to-GDP ratio above a certain threshold has a significant impact on growth.

Even if peer review did not apply, does this really exempt researchers publishing in the May issue from sharing their data and code? It is the policy of The American Economic Review “to publish papers only if the data used in the analysis are clearly and precisely documented and are readily available to any researcher for purposes of replication”. Note that there is no differentiation between users of the data as peer-reviewers or as researchers who want to replicate the results&colon; data and code should be available to any researcher.

Coding errors happen, but the greater problem was this failure to allow others to review and replicate the results by making the data openly available. If this had been done upon publication in 2010 – or if the idea of open data was universally applied – it might not have taken three years to prove these results wrong, and the odd case of Reinhart and Rogoff’s slip of the keyboard wouldn’t be making such huge waves around the world.

Profile

Velichka Dimitrova is project coordinator of Open Economics at the Open Knowledge Foundation, which advocates the release of datasets and code along with published academic articles. It defines open data as “a piece of data or content… free for anyone to use, reuse, and redistribute – subject only, at most, to the requirement to attribute and/or share alike”.