This copy is for your personal, non-commercial use only. To order presentation-ready copies for distribution to your colleagues, clients or customers, click the "Reprints" link at the bottom of any article.

April 16, 2013

Oops! Reinhart-Rogoff Research Tripped Up by Typo

A simple Excel spreadsheet error punches a hole in a highly cited economic paper by Carmen Reinhart and Kenneth Rogoff that has been used to bolster policies favoring a restraint in government spending.

Reinhart and Rogoff (left), famous for their book This Time is Different, which examined 224 historical banking crises and looked at the factors involved in postcrisis recoveries, have become something of a brand name in the field of economics, and have continued co-authoring influential research subsequent to their 2009 bestseller.

One such collaborative effort was a highly cited 2010 paper called “Growth in a Time of Debt,” which argued that countries with debt-to-GDP ratios above 90% were marked by far slower growth than countries with lower ratios.

That finding tended to bolster fiscal conservatives like former Republican vice presidential candidate Rep. Paul Ryan of Wisconsin and pro-austerity European politicians.

The error was uncovered in a paper released Monday by three University of Massachusetts researchers who persuaded Reinhart and Rogoff to share the data from their 2010 study; apparently, many researchers had been frustrated by an inability to replicate their results.

The three scholars, Thomas Herndon, Michael Ash and Robert Pollin, found three problems in the Reinhart-Rogoff paper: a simple coding error; exclusion of available data that would have been unfavorable to their conclusions; and a survey design error that breaks with scholarly conventions and had the effect of giving greater than due weight to their conclusion.

The spreadsheet glitch was the most significant. A simple typo in Reinhart and Rogoff’s data, typing “L44” and instead of “L49” yielded a data set excluding the experience of five countries—Denmark, Canada, Belgiuim, Austria and Australia. Had the broader set been included, Reinhart and Rogoff would have seen that countries with a debt-to-GDP ratio of greater than 90% actually achieved growth of 2.2% rather than negative 0.1%.

Reinhart and Rogoff responded Tuesday afternoon with a brief defense of their work, promising a fuller explanation “in due course.” The economist duo say their critics (and other cited works as well) agree with their principal finding: “On a cursory look, it seems that that Herndon, Ash and Pollen also find lower growth when debt is over 90% ... However, these strong similarities are not what these authors choose to emphasize.”

Nevertheless, reaction from liberals rolled in swiftly. Paul Krugman, writing Tuesday in his New York Times blog, The Conscience of a Liberal, said he never agreed with the Reinhart-Rogoff study. “But even I never dreamed that a large part of the alleged result might reflect nothing more profound than bad arithmetic,” he wrote.

Dean Baker of the liberal Center for Economic and Policy Research is asking: “How much unemployment was caused by Reinhart and Rogoff’s Arithmetic Mistake?” Both he and Mike Konczal, a Roosevelt Institute fellow and progressive blogger, are calling for, as Baker states, “a major reassessment of the deficit reduction policies being pursued in the United States and elsewhere;” or as Konczal puts it, “a larger deficit right now would help us greatly.”

Despite the errors found in the study, other studies have found similar results to those by Reinhart-Rogoff, and the two economists have published additional studies supporting their results, according to the The New York Times blog Economix.

The University of Massachusetts critique may have especially pronounced implications in Europe, where debate of austerity measures rages in countries such as Britain, France and Portugal. President Francois Hollande of France faces a revolt from his own ministers as he tries to push through measures aimed at meeting EU deficit targets.