Share this story

An economics paper claiming that high levels of national debt led to low or negative economic growth could turn out to be deeply flawed as a result of, among other things, an incorrect formula in an Excel spreadsheet. Microsoft's PowerPoint has been considered evil thanks to the proliferation of poorly presented data and dull slides that are created with it. Might Excel also deserve such hyperbolic censure?

The paper, Growth in a Time of Debt, was written by economists Carmen Reinhart and Kenneth Rogoff and published in 2010. Since publication, it has been cited abundantly by the world's press politicians, including one-time vice president nominee Paul Ryan (R-WI). The link it draws between high levels of debt and negative average economic growth has been used by right-leaning politicians to justify austerity budgets: slashing government expenditure and reducing budget deficits in a bid to curtail the growth of debt.

This link was always controversial, with many economists proposing that the correlation between high debt and low growth was just as likely to have a causal link in the other direction to that proposed by Reinhart and Rogoff: it's not that high debt causes low growth, but rather that low growth leads to high debt.

However, the underlying numbers and the existence of the correlation was broadly accepted, due in part to Reinhart and Rogoff's paper not including the source data they used to draw their inferences.

A new paper, however, suggests that the data itself is in error. Thomas Herndon, Michael Ash, and Robert Pollin of the University of Massachusetts, Amherst, tried to reproduce the Reinhart and Rogoff result with their own data, but they couldn't. So they asked for the original spreadsheets that Reinhart and Rogoff used to better understand what they were doing. Their results, published as "Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff," suggest that the pro-austerity paper was flawed. A comprehensive assessment of the new paper can be found at the Rortybomb economics blog.

It turns out that the Reinhart and Rogoff spreadsheet contained a simple coding error. The spreadsheet was supposed to calculate average values across twenty countries in rows 30 to 49, but in fact it only calculated values in 15 countries in rows 30 to 44. Instead of the correct formula AVERAGE(L30:L49), the incorrect AVERAGE(L30:L44) was used.

There was also a pair of important, but arguably more subjective, errors in the way the data was processed. Reinhart and Rogoff excluded data for some countries in the years immediately after World War II. There might be a reason for this; there might not. The original paper doesn't justify the exclusion.

The original paper also used an unusual scheme for weighting data. The UK's 19-year stretch of high debt and moderate growth (during the period between 1946 and 1964, the debt-to-GDP ratio was above 90 percent, and growth averaged 2.4 percent) is conflated into a single data point and treated as equivalent to New Zealand's single year of debt above 90 percent, during which it experienced growth of -7.6. Some kind of weighting system might be justified, with Herndon, Ash, and Pollin speculating that there is a serial correlation between years.

Recalculating the data to remove these three issues turns out to provide much weaker evidence for austerity. Although growth is higher in countries with a debt ratio of less than 30 percent (averaging 4.2 percent), there's no point at which it falls off a cliff and inevitably turns negative. For countries with a debt of between 30 and 60 percent, average growth was 3.1 percent, between 60 and 90 it was 3.2 percent, and above 90 percent it was 2.2 percent. Lower than the low debt growth, but far from the -0.1 percent growth the original paper claimed.

As such, the argument that high levels of debt should be avoided and the justification for austerity budgets substantially evaporates. Whether politicians actually used this paper to shape their beliefs or merely used its findings to give cover for their own pre-existing beliefs is hard to judge.

Excel, of course, isn't the only thing to blame here. But it played a role. Excel is used extensively in fields such as economics and finance, because it's an extremely useful tool that can be deceptively simple to use, making it apparently perfect for ad hoc calculations. However, spreadsheet formulae are notoriously fiddly to work with and debug, and Excel has long-standing deficiencies when it comes to certain kinds of statistical analysis.

It's unlikely that this is the only occasion on which improper use of Excel has produced a bad result with far-reaching consequences. Bruno Iksil, better known as the "London Whale," racked up billions of dollars of losses for bank JPMorgan. The post mortem of his trades revealed extensive use of Excel, including manual copying and pasting between workbooks and a number of formula errors that resulted in underestimation of risk.

The paper's problems also highlight the value of peer review. Although the paper was included in the American Economic Review, it was included as a "Papers and Proceedings" article, taken from a conference and published without being reviewed. A peer reviewer might have sought the Excel spreadsheet at review time and could have exposed these flaws sooner. In turn, this may have prevented an apparently flawed paper from receiving the widespread attention that it got. The response is similarly unreviewed, so it too may be flawed.

290 Reader Comments

Um... so not an Excel error, but an error in a formula someone happened to put in a spreadsheet, which happens how many thousands of times a day across the world? You wouldn't get that from the title of this article (which is a pretty interesting article, BTW).

And whether or not the politicians used this paper to justify their preexisting beliefs will be exposed now that the paper has been discredited. If they continue to push austerity as hard as before then it's reasonable to guess their belief for austerity came before the paper. On the other hand, if they reconsider their stance at least a little then perhaps not.

Quote:

What about things like centralization of power and inflation?

Keep in mind that inflation is dangerous when there is a lot of it, and deflation (as we see in Japan) is dangerous as well. In fact, they're desperately trying to achieve the same inflation benchmark the US is striving for (2-3%).

The problem with this analysis is that it assumes that growth is the only important metric. What about things like centralization of power and inflation? Higher public spending does both and they are both strong negatives.

Where is this purported inflation that you claim we are suffering in the US (we have high public spending, after all)? How are those gold investments working out for you?

The occasions where I have to use excel I always find extremely frustrating. The formula handling is counterintuitive, and more than once it has led to embarrassing errors (not by me... yet). This is why I think it is much better to define tables in Matlab, Maple or Mathematica and use their much more transparent mathematical operations.

The occasions where I have to use excel I always find extremely frustrating. The formula handling is counterintuitive, and more than once it has led to embarrassing errors (not by me... yet). This is why I think it is much better to define tables in Matlab, Maple or Mathematica and use their much more transparent mathematical operations.

Unfortunately Excel is ubiquitous and sort of nice on the eyes. I really would like to see an alternative because for serious applications Excel is useless. The half-ass work (in Excel, always in Excel) I have come across in my short time in industry is scary.

I personally like Octave but without a nice GUI my co-workers roll their eyes as they undoubtedly think of DOS.

The problem with this analysis is that it assumes that growth is the only important metric. What about things like centralization of power and inflation? Higher public spending does both and they are both strong negatives.

Where is this purported inflation that you claim we are suffering in the US (we have high public spending, after all)? How are those gold investments working out for you?

Not that I'm saying go buy gold, but this is akin to a gold bug having said, "How is that stock market working for you?" on march 9, 2009 when the dow and S&P hit 12 year lows. If someone is investing long term, short term collapses are usually not a cause to panic but an opportunity to invest more.

The problem with this analysis is that it assumes that growth is the only important metric. What about things like centralization of power and inflation? Higher public spending does both and they are both strong negatives.

Both of these assertions are not necessarily true. They are also not necessarily negative.

First of all, higher public spending does not necessarily lead to centralization of power. What if city governments increased public spending -- does that qualify as "centralization of power"? I'm not so sure given that there are tens of thousands of cities in the US.

Secondly, higher public spending does NOT always lead to increased inflation. In fact, in the economic situation we are currently in at the zero lower bound (where interest rates want to go lower but can't because then they would be negative), increased public spending does not increase inflation. Just look at the fact that inflation has remained very low since the financial crisis even during periods of increased government spending. It's also hard to argue in the current economic situation that increased government spending would be a negative. The majority of economists state that increased government spending would be a big positive to the economy as it would put to use resources that currently lie fallow (such as unemployed people).

The problem with this analysis is that it assumes that growth is the only important metric. What about things like centralization of power and inflation? Higher public spending does both and they are both strong negatives.

Where is this purported inflation that you claim we are suffering in the US (we have high public spending, after all)? How are those gold investments working out for you?

Not that I'm saying go buy gold, but this is akin to a gold bug having said, "How is that stock market working for you?" on march 9, 2009 when the dow and S&P hit 12 year lows. If someone is investing long term, short term collapses are usually not a cause to panic but an opportunity to invest more.

Generally people hold gold long term as a hedge against risk, not as a means of profiting. In the long term, there should be no return on gold, since its price should be relatively stable. If someone is expecting to make a return on gold, they're probably not in it long term so its not really analogous.

The occasions where I have to use excel I always find extremely frustrating. The formula handling is counterintuitive, and more than once it has led to embarrassing errors (not by me... yet). This is why I think it is much better to define tables in Matlab, Maple or Mathematica and use their much more transparent mathematical operations.

Unfortunately Excel is ubiquitous and sort of nice on the eyes. I really would like to see an alternative because for serious applications Excel is useless. The half-ass work (in Excel, always in Excel) I have come across in my short time in industry is scary.

I personally like Octave but without a nice GUI my co-workers roll their eyes as they undoubtedly think of DOS.

Try RStudio. It is both pretty to look at and produces correct statistical values. As an added bonus it produces charts that actually convey information instead of obfuscating it. Plus it's free.

Some economists made a mistake in an Excel formula, and that invalidates their specific analysis. Fair enough.But it does not follow that Excel is at fault (as the title suggests) or that all "pro-austerity" studies would be similarly flawed (the topic of the study seems a digression to the point of the article).This article looks like storm in a bucket.

The problem with this analysis is that it assumes that growth is the only important metric. What about things like centralization of power and inflation? Higher public spending does both and they are both strong negatives.

Where is this purported inflation that you claim we are suffering in the US (we have high public spending, after all)? How are those gold investments working out for you?

Inflation is much higher than what is reported. In fact, deflation is natural not inflation. So take the reported rate add a couple percent because of bad metrics and add on what would have been natural deflation and the rate is much higher then you think.

The problem with this analysis is that it assumes that growth is the only important metric. What about things like centralization of power and inflation? Higher public spending does both and they are both strong negatives.

Both of these assertions are not necessarily true. They are also not necessarily negative.

First of all, higher public spending does not necessarily lead to centralization of power. What if city governments increased public spending -- does that qualify as "centralization of power"? I'm not so sure given that there are tens of thousands of cities in the US.

Secondly, higher public spending does NOT always lead to increased inflation. In fact, in the economic situation we are currently in at the zero lower bound (where interest rates want to go lower but can't because then they would be negative), increased public spending does not increase inflation. Just look at the fact that inflation has remained very low since the financial crisis even during periods of increased government spending. It's also hard to argue in the current economic situation that increased government spending would be a negative. The majority of economists state that increased government spending would be a big positive to the economy as it would put to use resources that currently lie fallow (such as unemployed people).

Cites can't print money. Therefore, assuming no money comes from the national government all of it was gained from taxation which centralizes power by taking from the many to be spent by the few.

I'd argue that inflation is not low at all. It is under-reported and doesn't even factor in what would have been natural deflation.

I would expect better from Harvard professors. Instead of taking ownership for their failings they instead point at a new paper that they published. I hope this hubris catches up with them at some point.

These "errors" strike me as awful convenient. Couple with the non-release of the data until pressed for it.

Yep, when every error compounds to bolster support for their conclusion I think it's safe to call them "errors".

Also note that Rogoff appears to have practiced politically motivated economics in past roles:

Quote:

from http://www.cepr.net/documents/publicati ... 007_04.pdfDuring [Rogoff's tenure as chief economist with the IMF], Argentina went from being an IMF poster child to being an outlaw when it defaulted on its debt in December of 2001. In the years prior to its default, the IMF consistently erred on the high side in its projections of growth for Argentina. In the years after it defaulted the IMF's projections consistently underestimated Argentina's growth by large amounts. It would be almost impossible to produce this pattern of errors if the IMF was not using political criteria in its growth projections for Argentina.