Tuesday, April 3, 2012

Not feeling the 'physics envy'

Kevin Clarke and David Primo have an op-ed in the New York Times that critiques the dominance of what they call "hypothetico-deductivism" in political science - the idea that in order to study politics "scientifically," one must follow a specific method:

This might seem like a worthy aspiration. Many social scientists contend that science has a method, and if you want to be scientific, you should adopt it. The method requires you to devise a theoretical model, deduce a testable hypothesis from the model and then test the hypothesis against the world. If the hypothesis is confirmed, the theoretical model holds; if the hypothesis is not confirmed, the theoretical model does not hold. If your discipline does not operate by this method — known as hypothetico-deductivism — then in the minds of many, it’s not scientific.

Such reasoning dominates the social sciences today. Over the last decade, the National Science Foundation has spent many millions of dollars supporting an initiative called Empirical Implications of Theoretical Models, which espouses the importance of hypothetico-deductivism in political science research. For a time, The American Journal of Political Science explicitly refused to review theoretical models that weren’t tested. In some of our own published work, we have invoked the language of model testing, yielding to the pressure of this way of thinking.

The NYT piece is a summary of the argument that they make more extensively in their book A Model Discipline. I have yet to read the book, so I can't speak to any differences/nuances developed there that don't necessarily come out in the op-ed. As far as I can tell, Primo and Clarke's main point is that political science has become too much of a methodological monoculture and that we should not be opposed to engaging in theoretical work that is not necessarily empirically testable or empirical work that doesn't aim to "test" a pre-determined theory.

I entirely agree with the call for diversity in methods - the question should guide the choice of tool and not the other way around. There's plenty of great theoretical work that is impossible to test systematically, but nevertheless useful. Moreover, empirical research that isn't guided by any particular theory can still generate interesting questions and find surprising relationships between variables. The rise of "big data" makes the search for these sorts of correlations even more relevant since there is so much information that has yet to even be examined by political scientists.

But I don't get the sense that the next generation of political scientists is necessarily being taught that "hypothetico-deductivism" is the way to do research. Sure, maybe journals have underlying biases, but I don't think that bias is unique to methods - top-tier journals have a reputation to protect and will by necessity be risk-averse in publishing anything "new." As far as training goes, from talking to professors during graduate student visits, I certainly did not get the sense that all theoretical models must have empirically testable implications and that all empirical research should be backed up by theoretical models (however haphazard). I actually brought up the topic of empirical testing in a few of my conversations with some of the formal theorists and got more or less the same response as what Primo and Clarke seem to be arguing. Maybe I'm underinformed since I'm only an incoming graduate student, but if there's significant pressure towards "hypothetico-deductivism," I'm definitely not picking up on it.

The last thing that slightly irked me about the article, and this is likely more the fault of the New York Times op-ed board trying to make an editorial on the philosophy of science as applied to poli-sci appealing to the general public, is the subtle invocation of the old trope that "social science" isn't a "hard" science like physics or chemistry and social scientists shouldn't bother trying to emulate the "real" sciences. Often this tends to be accompanied by lazy bromides about how human behavior is "inherently unpredictable" and that it's impossible to predict the really important events in political history. However, this doesn't seem to be what Primo and Clarke are arguing at all (which is why I'm puzzled by their use of the phrase 'physics envy' - as Erik Voeten pointed out, even physicists don't subscribe to hypothetico-deductivism as the only way to do physics). Rather than dismiss rigor in political science, they're calling for more of it - for more creative and insightful methodological approaches to poli-sci questions. In this sense, I think they're presenting an argument similar to the one Dan Nexon made last week on the "overprofessionalization" of academia. Certainly Primo and Clarke are not calling for total method-free "thinking" about politics - that's what political analysis and the NYT/WaPo op-ed pages are for.

If there's a square peg and round hole problem, it's definitely not between social science and "science."