economics and sociology, part cdlxvii: comments on a blog post by noah smith

A few weeks ago, economics columnist Noah Smith wrote a blog post about how economics should raid sociology. This raises interesting questions about how academic disciplines influence each other. In this case, why has sociology not been a good a receptor for economics?

I start with an observation, which Smith also alludes to: Sociology has already been “raided” by economics with only moderate success. In contrast, economists have done very well raiding another discipline, political science. They have done fairly well in establishing pockets of influence in public policy programs and the law schools. By “success,” I do not mean publishing on sociological topics in economics journals. Rather, “success” means institutional success: economists should be routinely hired sociology programs, economic theory should become a major feature of research in graduate programs, and sociological journals should mimic economics journals. All of these have happened in political science but not sociology.

Here’s my explanation – Sociology does not conform to the stereotype that economists and other outsiders have of the field. According to the stereotype, sociology is a primarily qualitative field that has no sense of how causal inference works. In some accounts, sociologists are a bunch of drooling Foucault worshipers who babble endlessly in post-modern jargon. Therefore, a more mathematical and statistical discipline should easily establish its imprint, much as economics is now strongly imprinted on political science.

The truth is that sociology is a mixed quantitative/qualitative field that prefers verbal theory so that it can easily discuss an absurdly wide range of phenomena. Just open up a few issues of the American Sociological Review, the American Journal of Sociology or Social Forces. The modal article is an analysis of some big N data set. You also see historical case studies and ethnographic field work.

It is also a field that has import traditions of causal identification, but does not obsess over them. For example, in my department alone, there are three faculty who do experiments in their research and one who published a paper on propensity scores. Some departments specialize in social psychology which is heavily experimental, like Cornell. There are sociologists who work with data from natural experiments (like Oxford’s Dave Kirk), propensity scores (like IU’s Weihua an), and IV’s (I actually published one a while ago). The difference between economics and sociology is that we don’t reward people for clever identification strategies or dismiss observational data out of hand. If possible, we encourage identification if it makes sense. But if an argument can be made without it, that’s ok too.

So when economists think about sociology as a non-quantitative field, they simply haven’t taken the time to immerse themselves in the field and understand how it’s put together. Thus, a lot of the arguments for “economic imperialism” fall flat. You have regression analysis? So does sociology. You have big N surveys? We run the General Social Survey. You have identification? We’ve been running experiments for decades. One time an economist friend said that sociology does not have journals about statistical methods. And I said, have you heard of Sociological Methodology or Sociological Research and Methods? He’s making claims about a field that could easily be falsified with a brief Google search.

In my view, economics actually has one massive advantage over sociology but they have completely failed to sell it. Economists are very good at translating verbal models into mathematical models which then guide research. The reason they fail to sell it to sociology is for a few reasons.

First, economists seem to believe that the only model worth formalizing is the rational actor model. For better or worse, sociologists don’t like it. Many think “formal models = rational actor model.” They fail to understand that math can be used to formalize and study any model, not just rational choice models.

Second, rather than focus on basic insights derived from simple models, economists fetishize the most sophisticated models.* So economists love to get into some very hard stuff with limited applied value. That turns people off.

Third, a lot of sociologists have math anxiety because they aren’t good at math or had bad teachers. So when economists look down at them and dismiss sociology as whole or qualitative methods in particular, you loose a lot of people. Instead of dismissing people, economists should think more about how field work, interviews, and historical case studies can be integrated with economic methods.

I am a big believer in the idea that we are all searching for the truth. I am also a big believer in the idea that the social sciences should be a conversation not a contest of ego. That means that sociologists should take basic economic insights seriously, but that also means that economists should turn down the rhetoric and be willing to explore other fields with a charitable and open mind.

>Second, rather than focus on basic insights derived from simple models, economists fetishize the most sophisticated models.*

I think this is incorrect overall. Theory as a practice has gotten more sophisticated, but economists as a whole have focused towards increasingly simple models. I’m surprised someone made you read a paper like that, it sounds unusually arcane. 90+% of economists just don’t care about high theory anymore, and basically don’t interact with modern theorists at all. (Not to get too snarky, but the fact that you don’t know that is evidence that economists really have done a poor job of exporting our methodology to you sociologists!)

Another reason this stuff hasn’t taken off in sociology, I think, is that a lot of the stuff sociologists might care about isn’t amenable to formal modeling. To take something with crossover appeal, I think a sociologist might care about the fact that what economists call “preferences” are socially constructed. The economics literature on things like advertising and peer effects has (to my limited knowledge) been remarkably atheoretic, which I interpret as a sign that writing down formal models of those effects is extremely difficult.

Reblogged this on Crowd Thought en el Mercado de Capitales Chileno and commented:
I second this idea. This is a particularly interesting idea in the case of financial markets, where a dialogue between social sciences that study this object need to sit down together and create a new ‘space’ of possibilities for a better understanding of financial markets and banking.

ZC: I would politely disagree. If you look at how economists are selected and trained, the emphasis is on more and more sophisticated math. When I was an undergrad in the early 90s at Berkeley, you could get a BA degree with about a semester of calculus and the two year calculus requirement was “honors.” Now, two years is standard. At the graduate level, econ programs now routinely teach real analysis and measure theory. The situation is a bit different at the level of research practice.

As you correctly note, the place of “theory” has declined in economics with a healthy return of empirical work. But still, the level of math required for modern economics is far and beyond anything required in the rest of the social sciences. In the post, I don’t say this is a bad thing, but what it does mean is that economics can (and does) occasionally veer into math for math’s sake, which can make it hard to really connect with other social scientists.

I disagree that economic theory is ‘math for math’s sake’ – rather, higher math often simplifies arguments, as long as you understand the math. The most common “infinite dimensional Banach space” is just the set of continuous functions on the real line with a norm that gives completeness (say, the supremum of the function). This is a concept anyone who has studied analysis will know, which essentially means anyone who has done graduate work in economics. Doing things in continuous time, or with a continuum (an “infinite dimension”) of strategies, or similar often makes arguments much more transparent, not less.

As an aside, though, the phrase “infinite dimensional Banach space” has, according to Google Scholar, literally been used only one time in the history of Econometrica, the most theoretical general interest journal in the field. This is in Rust’s famous “GMC Bus Engine” paper, a very very very highly cited article on how to empirically estimate so-called “DCDPs”, or problems where people need to dynamically choose how to make a discrete choice (like when to invest, or when to sell an asset, or when to return to school, etc.). In general, these choices depend on a variety of factors that need to be estimated if counterfactuals like a new government policy are to be considered: people’s discount rates, or their beliefs about future income, and so on are not visible to the outside social scientist.

As someone who reads the soc lit pretty heavily, the big soc v. non-soc distinction is not rational choice vs. others (both since economists regularly estimate “behvaioral” models and since there is now enormous evidence about which contexts a rational choice approximation is reliable for guiding counterfactual work). Rather, the big difference is about whether counterfactuals are worth considering (sociologists are much more willing to do purely descriptive work, whereas economists are much more interested in counterfactual or hypothetical questions – it’s tough to imagine Arrow’s Impossibility Theorem coming from a sociologist).

Afinetheorem said : “…just the set of continuous functions on the real line with a norm that gives completeness (say, the supremum of the function). This is a concept anyone who has studied analysis will know…”

This comment summarizes the whole math issue in economics. The amount of knowledge required to even process this sentence is beyond almost all non-economist social scientists and many life and hard scientists. A lot of physics majors don’t take real analysis. Most computer scientists or chemists don’t. I say this as person who has a degree in math. I love math, but this sort of comment is exactly the type of thing that decreases communication across fields.

Also, I disagree on some other issues. For example, rational choice is still a big divide in sociology and behavioral econ doesn’t quite capture the difference. I am not an expert in that field, but my understanding is that most behavioral models are modifications of neo-classical models, not rejections or substitutions. Correct me if I am wrong.

PS. I was required to read that paper circa 1994 – 22 (!) years ago, so it is very possible that I got the terminology wrong. But it was in the context of studying equilibrium theorems in more and more general contexts. Since you can put norms on spaces of continuous functions, they would certainly be Banach spaces. But this is a minor quibble.

“The difference between economics and sociology is that we don’t reward people for clever identification strategies or dismiss observational data out of hand. If possible, we encourage identification if it makes sense. But if an argument can be made without it, that’s ok too.”

there’s a push-back in economics against overly “clever” identification (see: ‘cute-onomics’ as pejorative), especially against clever instruments. but cleverness is not an end in itself, only to the extent that it helps make a convincing case for identification. convincing identifications should always be rewarded.

a _causal_ argument cannot be made without identification, by definition. if sociologists are lax about identification (i am generally ignorant of sociology and have no idea if that’s so) people will rightly ignore their results.

i think Noah’s point was that sociologist grad students should join the ‘Identification Taliban.’ That is, they should study Mostly Harmless Econometrics and apply its methods and philosophy of inference, and stringently discount results that suffer from failure of the exogeneity assumptions. and they’d better do this quick, because economics grad students are already beating them to it. There is nothing unique to economics about these methods. Angrist and Pischke didn’t invent them. So saying ‘we sociologists have our own methods’ doesn’t really cut it, either.

Sam: There are different arguments that can be made. Certainly, “A causes B” is one of them and if you make that argument, soc journals will demand an identification strategy, or some pretty strong incriminating descriptives. In this respect, econ and soc are no different. That is why soc programs have experimentalists or folks who do propensity/IV/Diff etc.

However, you can also make the argument “theory X predicts that A correlates with B.” That is not an identification issue. It is about conceptually understanding what X implies. My experience is that economists no longer tolerate this while it is still normal in other social sciences. Or if they do, they certainly don’t admit it in public.

And yes, “clever id” is now receding, but it reveals something important about the culture of economics. It’s a field that rewards cleverness, which has its ups and downs. The up part is that you screen for high intelligence, which is good. The down is that clever arguments are often delicate arguments – they make lots of assumptions, they can’t be replicated, they strain credulity, they use weird data. Soc has its own system of good and bad rewards, but joining the ID Taliban isn’t one of them and that’s a plus in my view. ID is great, but it’s not the only thing by far.

I agree that this type of math decreases communication across fields. I disagree that this means it is “math for math’s sake”. Take any problem where what you do today affects what is possible tomorrow. What is the best thing to do today? What will people do today under various institutional arrangements, and how do they compare? I mean, there are very well stated questions, are they not? But by far the simplest way we have to solve problems of this type is to write down a dynamic program and look at what happens to the so-called value function – this is specialized terminology, I agree, but it is the simplest and not the most complex way to look at the problem. I don’t even know how *in principle* we could answer questions like “How badly does a new tax on capital gains distort people’s retirement savings behavior?”, or “Do managers upgrade equipment in the same way a firm’s owner would do, or is there a wedge, and if there is a wedge, how large is it?”, or “Which counterfactual policy can best prevent white flight from inner cities?” in the absence of at least a bit of math.

Physics is a great example, because rather than economists aping physicists, the average economist know literally zero about the mathematics most common in modern physics (differential systems, gauge theory, etc.). Analysis, real and functional, is everywhere in economics because analysis turns out how to be useful for turning qualitative verbal arguments into formal statements where properties like “If a increases, what will happen to b” can be investigated.

I remember one time, talking to a sociologist friend, that we decided everyone cares about how choice under constraints aggregates into outcomes, but economists care most about the choice and the outcomes, and sociologists care most about describing the constraints. That division of labor seems much more appropriate than sociologists radically changing their mathematical training, or sociologists dropping their useful descriptive work to ape the identification taliban.

Afinetheorem: Let me appeal to a sound economic insight to clarify my argument – diminishing marginal returns. If you have a policy question, or any other social science question, you will probably make a lot of progress the first time you write it down and solve it. But then, we have can ask about if the model holds in X condition or generalized in Y way. Sooner or later, you’ll hit diminishing returns. You may learn a lot of neat math if you want to solve it in the most general situation, but you likely won’t really get that much extra empirical insight of it.

Here’s the way I learned the lesson in graduate school – game theory. The basic ideas of game theory are amazingly important and they clarify a lot of issues. I think that social scientists who don’t understand basic game theory and study interaction are really missing the boat. But when you read the research literature, you quickly get into “math for math’s sake.” Can we prove that an equilibrium exists for a game with infinitely many players in infinitely many time periods who are engaged in a Bayesian signalling game with priors that have some crazy distribution? After reading such papers, I soon realized that it was math for math’s sake. I just couldn’t take it, it clearly didn’t allow me to appreciate the empirical issues I cared about.

Of course, this is subjective. Perhaps such theorem proving is allowing you to really make some big insights. But when I read economics that seems to be successful, the math tends to be modest but the economist has spent a lot of time thinking about the right model and really learning the topic at hand. There’s an important lesson in there.

But this is such a strange complaint since in the case you mention, you are talking about papers which are pure theory – that is, tool building. The mathematics necessary to prove that, say, the Law of Large Numbers holds in certain circumstances is way beyond the ability of most social scientists. But I would worry if we were applying statistical techniques to situations where no one has even checked whether those techniques are appropriate, yes? Applied papers, partially due to the preferences of referees, do not tend to introduce complexity for its own sake.

Take a look at the newest issue of Econometrica. The main papers are:
– A paper that shows how to estimate preferences for location when they depend on amenities which may be endogenous to your neighbors. They explicitly discuss why a simpler non dynamic model misses quantitatively important effects.
– A model of growth where firms produce ideas which can be kept or sold; the paper has no novel mathematical contributions.
– A statistical model for estimating the risk premium from stock market data, explicitly noting why simpler methods are not consistent estimators of this parameter
– An explicit “model-free” estimate of a certain model of rational bubble, which is very straightforward mathematically
– A model of Nash equilibrium that permits players to be uncertain about how actions map into consequences, and a proof that certain types of learning will converge to this
– A model of why people negotiate up until the deadline when wasting time is costly
– A paper showing how to aggregate beliefs when people disagree not only about what is desirable but also about the probability of various states of the world coming to be.

These all seem to be pretty well-founded questions, and in the cases where the math gets hairy, there is explicit justification for why such a digression is necessary.

Afinetheorem: I am not against tool building in general. What I am against tool building that does not serve the core interest of its discipline. And this criticism is not limited to economics. Any field can sit around and build model after model without an interest. For example, there are branches of physics that have been accused of endless model building (see Lee Smolin’s book The Trouble with Physics). In sociology, we have a version of this problem, but it’s not math specific.

I have not read the articles you cited so I can’t make a judgment on them or your summaries. But here is what I suggest: write down a list of classic problems of economics.This might include why we have business cycles, or ways to design optimal tax policies, or calculating the return on education. Then for each article, ask yourself if (a) this model resolves an issue associated with such a core problem and (b) the model is in some way tied to data/observation or can lead to a concrete policy design. The more you score yes on (a) and (b), the more it’s a tool that matters. Otherwise, it’s math for math’s sake as we can always cherry pick a few ideas and then have fun modelling them.

Finally, in terms of getting a grip on the barrier between economics and other fields, ask yourself, for each article, what is the *minimal* level of knowledge needed to properly understand the article. If it’s a year of calculus, then the model will be accessible to many other social scientists. If it’s measure theory, then almost no one will care except other specialists.

if a new tool is developed it is best for theorists to rigorously prove its properties. for example a new statistical estimator needs proofs to questions like: is it consistent? unbiased? what are its small-sample properties? what assumptions does it rely on? how can we test them? a new equilibrium concept will need proofs of uniqueness, existence, etc. analysis is the math that allows you to answer questions like that.

So either we give Lars Hansen a nobel prize and remain inscrutable to some sociologists, or sociologists understand everything we do but each time we do GMM we say three hail-maries and pray what comes out makes sense, and then when someone asks us why it works we respond ¯\_(ツ)_/¯

Luckily, 99% of the math in economics is calculus, linear algebra, basic probability theory and statistics. economics departments force their graduate students to take courses in this rigorous style even though most will never produce papers like that. the idea is that students should be able to understand these rigorous demonstrations of the properties of the estimators/equilibrium concepts they use. I don’t think this should be a great barrier to intra-disciplinary communication

Sam: The point isn’t that economists should stop doing math. The blog post never said that nor have any of the commenters. In fact, the original post praised economists for using math! One of the issues, though, is how much math? If a theorem has wide application (the Law of Large numbers) and is central, then the math is justified. But that isn’t always the case. Sometimes, people are just into theorem proving and lose sight of economic insights. You need to be able to see the difference. Otherwise, you can easily spend a career doing models that don’t promote the empirical field that you care about.

I was responding to your point in this exchange between you and Kevin:

AFT: “The most common “infinite dimensional Banach space is just the set of continuous functions on the real line with a norm that gives completeness (say, the supremum of the function). This is a concept anyone who has studied analysis will know…”

FR: “This comment summarizes the whole math issue in economics. The amount of knowledge required to even process this sentence is beyond almost all non-economist social scientists and many life and hard scientists.”

Now you say: “If a theorem has wide application (the Law of Large numbers) and is central, then the math is justified. But that isn’t always the case”

I disagree that math is only justified ex-post by application. How could you know before writing a theorem that it will be widely applied? And many great theorems are never “applied” in any straightforward sense, yet are useful in other ways: no one ever dismantled their democracy after reading Arrow’s theorem. Furthermore, this implies that there is an alternative option of somehow proving the theorem without the math. I don’t think this is either possible or desirable. I’ve never seen a non-formal proof of any non-trivial result. And even if one existed, how can we discern a proof from persuasive rhetoric?