history, the stock market, and predicting the future

So the stock market has been freaking out a bit the last couple of weeks. Secular stagnation, Ebola, a five-year bull market—who knows why. Anyway, over the weekend I was listening to someone on NPR explain what the average person should do under such circumstances (answer: hang tight, don’t try to time the market). This reminded me of one of my pet quibbles with financial advice, which I think applies to a lot of social science more generally.

For years, the conventional wisdom around what ordinary folks should do with their money has gone something like this. Save a lot. Put it in tax-favored retirement accounts. Invest it mostly in index funds—the S&P 500 is good. Don’t mess with it. In the long run this is likely to net you a reliable 7% return after inflation, about the best you’re likely to do.

Now, it’s not that I think this is bad advice. In fact, this is pretty much exactly what I do, with some small tweaks.

But it has always struck me how, in news stories and advice columns and talk shows, people talk about how this is a good strategy because it’s worked for SO LONG. For 30 years! Or since 1929! Or since 1900! (Adjust returns accordingly.)

And yes, 30 years, or 85, or 114, are all a long time relative to human life. And we have to make decisions based on the knowledge we’ve got.

But it’s always seemed to me that if what you’re interested in is what will happen over the 30+ years of someone’s earning life (more if you’re not in academia!), you’ve basically got an N of 1 to 4 here. I mean, sure, this may be a reasonable guess, but I don’t think there’s any strong reason to believe that the next 100 years are likely to look very similar to the last 100. Odds are better if you’re just interested in the next 30, but even then, I’m always surprised by just how confident the conventional wisdom is around the idea that the market always coming out ahead over a 25- or 30-year period—going ALL THE WAY BACK TO 1929—is rock solid evidence that it will do so in the future.

Of course, there are lots of people who don’t believe this, too, as evidenced by what happened to gold prices after the financial crisis. Or by, you know, survivalists.

Anyway, I think this overconfidence in the lessons of the recent past is something we as social scientists tend to be susceptible to. The study that comes most immediately to mind here is the Raj Chetty study on value-added estimates of teachers (paper 1, paper 2, NYT article).

The gist of the argument is that teachers’ effects on student test scores, net of student characteristics (their value added), predicts students’ eventual income at age 28. Now, there’s a lot that could be discussed about this study (latest round of critique, media coverage thereof).

But I just want to point to it—or rather, broader interpretations of it—as illustrating a similar overconfidence in the ability of the past to predict the future.

Here we have a study based on a massive (2.5 million students) dataset over a twenty-year period (1989-2009). Just thinking about the scale of the study and taking its results at face value, it’s hard to imagine how much more certain one could be in social science than at the end of such an endeavor.

And much of the media coverage takes that certainty and projects it into the future (see the NYT article again). If you replace a low value-added teacher with an average one, the classroom’s lifetime earnings will increase by more than $250,000.

And yet to make such a leap, you have to be willing to assume so many things about the future will be like the past: not only that incentivizing teachers differently and making tests more important won’t change their predictive effects (which the papers acknowledge), but, just as importantly, that the effects of education on earnings—or, more specifically, of teacher value-added on earnings—will be similar in future 20-year periods as it was from 1989-2009. And that nothing else meaningful about teachers, students, schools, or earnings will evolve over the next 20 years in ways that mess with that relationship in a significant way.

I think we do this a lot—project into the future based on our understanding of a past that is, really, quite recent. Of course knowledge about the (relatively) recent past still should inform the decisions we make about the future. But rather a lot of modesty is called for when making blanket claims that assume the future is going to look just like the past. Maybe it’s human nature. But I think that modesty is often missing.

6 Responses

I agree with a lot of what you said. But I disagree with your view on why we value evidence from the past. When we say “invest in index funds, they did well in the past”, we are not simply extrapolating from past trends to the future. We have a theory of why individuals cannot beat the market and past evidence supports that theory. That theory (in addition to past trends) is what really makes us confident about that particular strategy.

I’m not so much questioning the theory of why individuals can’t beat the market as the expectation that the future is going to be like the past in terms of returns and so on. Like I said, my money is in index funds too. If you’re going to put your money somewhere, that seems like the best bet. It’s more the idea that 7% is a safe, predictable return in the long-long run — that the next century will look like the last century — that seems a bit crazy to me.

I agree with Kerokans point about there being theoretical reasons, more than just induction, behind the investment advice.

However, overall I agree with your (Epopp) post. I view this as the classic issue of considering threats to external validity. That is, the eternal question of are conclusions that are valid for these data also valid for other data (in this case: future data)? And that usually boils down to having a theory of the two data generating processes (for studied data and generalized to data). Something which is certainly challenging in the complex and ever evolving world that includes education and economic returns.

I suppose I made it sound very basic now. But I believe that carefully considering external validity is not at all basic, rather it is a constant and serious challenge that is easy to fail at. And that it is often worthwhile to remind ourselves of these (basic) issues. As you did in this very reasonable post/argument (Epopp).

I completely agree with Elizabeth that data says essentially nothing about external validity, and with Kerokan that theory is how we get out of this. If those of us that are more theoretically inclined get upset when people start hyping “big data” or talking about an “empirical turn in the social sciences” or similar, it is exactly because the joint distribution of empirical variables is fundamentally unable to offer compelling reasons to believe in external validity. Our beliefs about whether value-added in teaching or the return to the stock market will continue in a similar way to the past are totally dependent on deductive understanding of causal factors rather than inductive understanding of what has happened before.

I second what Kerokan said. Also, much financial advice isn’t of the form “do this” but rather “do this instead of that.” So you’re simply encouraged to invest index funds, but rather to invest in index funds as opposed to managed funds because of theoretical principles.