statistics vs. econometrics – heckman’s approach

Over at Econ Talk, Russ Roberts interviews James Heckman about censored data and other statistical issues. At one point, Roberts asks Heckman what he thinks of the current identification fad in economics (my phrasing). Heckman has a few insightful responses. One is that a lot of the “new methods” – experiments, instrumental variables, etc. are not new at all. Also, experiments need to be done with care and the results need to be properly contextualized. A lot of economists and identification obsessed folks think that “the facts speak for themselves.” Not true. Supposedly clean experiments can be understand in the wrong way.

For me, the most interesting section of the interview is when Heckman makes a distinction between statistics and econometrics. Here’s his example:

Identification – statistics, not economics. The point of identification is to ensure that your correlation is not attributable to an unobserved variable. This is either a mathematical point (IV) or a feature of research design (RCT). There is nothing economic about identification in the sense that you need to understand human decision making in order to carry out identification.

In contrast, he thought that “real” econometrics was about using economics to guide statistical modelling or using statistical modelling to plausibly tell us how economic principles play out in real world situations. This, I think, is the spirit of structural econometrics, which demands the researcher define the economic relation between variables and use that as a constraint in statistical estimation. Heckman and Roberts discuss minimum wage studies, where the statistical point is clear (raising wages do not always decrease unemployment) but the economic point still needs to be teased out (moderate wage increases can be offset by firms in others ways) using theory and knowledge of labor markets.

The deeper point I took away from the exchange is that long term progress in knowledge is not generated by a single method, but rather through careful data collection and knowledge of social context. The academic profession may reward clever identification strategies and they are useful, but that can lead to bizarre papers when the authors shift from economic thinking to an obsession with unobserved variables.

One Response

Econometrics, going back to Frisch, is literally the conjunction of statistical reasoning with pure economic theory: the definition is in the Econometric society constitution! What Heckman means is that pure questions of efficiency, consistency, causal inference, etc. are not econometrics. Rather, the use of theory to permit inference is. The classic example is supply and demand. You see two pairs of prices and quantities sold. On the basis of those two pairs, the supply and demand curves cannot be drawn. We need to know whether supply conditions have shifted, or demand conditions, or potentially both. That fact, in conjunction with statistical work utilizing the fact, is an example of econometrics.

Structural econometrics in the modern sense goes further by fully modeling the data generating process including the nature of error terms and how they result from aggregated decisions. Almost always there is an assumption that some sort of equilibrium play is occurring. What was once called “reduced form” econometrics does not fully model that process, but still has a model in the background. Today, “reduced form” generally refers to the type of “identification before all else/let the data speak” type of analysis that Heckman finds utterly pointless for most questions of economic interest, since to him economists must be modeling decision making at some point if the analysis they are doing can be called “economics”.