Joshua Angrist writes to TNR on Levitt

He defends Levitt's work, his own work, and the use of instrumental variables in economics...

In "Freaks and Geeks" (April 2, 2007) Noam Scheiber praises my work with Alan Krueger on the economic effects of compulsory schooling but argues that economics Ph.D. students today are obsessed with headline-grabbing trivia of little substantive importance. The root cause is said to be excessive attention to a good or clever research design at the expense of the relevance of the underlying question. Scheiber’s story is engaging and he lands a few punches, but his account is misleading in two important ways. For one thing, he exaggerates the problem of small-bore studies. America’s half-dozen top Ph.D. programs produce scores of Ph.D. students every year. Most of this work is still on traditional topics. At MIT (where Levitt studied), we continue to supervise empirical theses on, among other things, health insurance, immigration, unions, and human capital. High-quality research on these traditional topics gets students high-quality jobs. I’ll plead guilty, however, to being especially pleased when students manage to come up with clean identification–that is, they have a convincing strategy for uncovering causal effects. Clean identification is not a fetish; without it, little of value is learned. On this score, our students typically do better than the empiricists of H.G. Lewis’s generation. In two of the most dynamic empirical microeconomics subfields, the economics of education and economic development, there has been a virtual credibility revolution, with the increase of randomized field trials as well as compelling natural-experiments research designs (see, e.g., the work done at MIT’s Poverty Action Lab).

Second, in his rush to tar some up-and-comers with the "cute-o-nomics" brush, Scheiber misses a central feature of the clean-identification research agenda, best explained by example. One of the enduring scientific and policy questions in Labor economics is the sensitivity of hours worked to changes in pay (this matters for tax policy, for example). The best evidence labor economists have on the relation between wages and hours worked comes from a small experiment (by Ernst Fehr and Lorenz Goette) involving the wages of bicycle messengers in Switzerland. The second best comes from a study of stadium vendors by Gerald Oettinger. Who cares about the riders of Veloblitz or snack sellers at Camden Yards? We care because economics is predicated on the notion that a few simple principles explain behavior in many settings. These studies produce results that are convincing and may well be general, though, as always in science, it will take replication to know for sure. Some of the studies Scheiber dismisses can be understood in this spirit. Finally, as to the merits of Freakonomics the book: My 17-year old daughter picked it up on her own last year. She never knew economics could be so cool. Even better, she now asks me what I’m up to. So I tip my cap to Dubner and Levitt–I hope to see many of their readers in an economics classroom some day.

The irony is that Steven D. Levitt’s best known theory — that legalizing abortion in 1970-1973 cut crime later — contradicts Noam Scheiber’s hypothesis that Levitt’s work is brilliant but trivial. It’s an extremely important topic, but one that Levitt notoriously botched up in a variety of ways, from miscoding his programming to failing to do simple reality checks on his results.

As a journalist, I have to admit that, overall, economists have acquitted themselves better in terms of critically evaluating Levitt’s most famous Freakonomics theory than have journalists, such as, oh, Scheiber, who never mentions in his New Republic article critical of Levitt the flaws economists such as Joyce, Foote, Goetz, Dills, and Miron have identified in Levitt’s theory. Though the mills of academic economics have ground slowly, they are grinding up Levitt’s theory exceeding small.

In contrast, the book reviews of Freakonomics in prominent publications such as the New York Times were disgracefully credulous. Ten minutes of Googling would have revealed that there had been considerable empirical controversy over Levitt’s abortion-crime theory for a number of years before Freakonomics’ publication, but few reviewers did even this minimal due diligence.

I think you can argue that showing the relationship between abortion and crime is trivial because there really isn’t much you can do with it after establishing the fact.
I think one of the main strengths of economics is the policy implications, where you analyze what the government or other institutions should do in light of the results of one’s research. But what would such policy implications be of showing the causual relationship between abortion and crime rates? That abortion is good and should be legalized? Of course not, there are moral and political considerations to take in as well, so in the end the entire thing becomes a triviality. Obviously this is not true of all of his research, but one can do a similar thought experiment to the one above and realize that a few of the points he makes in Freakonomics are indeed quite trivial.

Off the top of my head, the main critique of Levitt’s book from the very beginning was that it was too trivial, to which the ultimate response was, and is, so what?

Even if it is, I know a number of people who have read Levitt’s book who would otherwise not read a book on economics. Etc.; that argument has been made.

An interesting aside is the treatment that most economists receive from the media in general. I’m thinking of Richard Posner’s “Public Intellectuals” wherein he lists many (though not all) of the most highly regarded experts who have been nothing but wrong in their declamations and predictions, and yet no one ever seems to notice, or bothers downgrading their credibility rating.

I’ve seen some arguments on Levitt’s work (see above) but nothing that would come close to how frighteningly wrong the media’s selected, anointed and never criticized experts have been.

I will say, however, that the examples on labor supply elasticity that Angrist cites do worry me. I would have to go back and read the papers, but unless the experiments took place over longer periods of time, they likely capture only short-term labor supply elasticities, which are inevitable lower than long-term labor supply elasticities. So you might not want to take those elasticities to the bank for policymaking purposes.