A longstanding academic theory for describing how markets work has finally died. Its impact on investment theory and practice was enormous, but has also led to some risks.

The so-called efficient market hypothesis (EMH), which basically assumes that investors are rational and that it's impossible to beat the market because prices reflect all available information, has been under fire for years.

We recently spoke with him on FS Insider about his book and the theory that will replace the old one. It's called the adaptive market hypothesis (AMH), which looks at the market less as a high-performance supercomputer and more as a biological organism or ecosystem—an ecosystem that, unfortunately, has become less diverse and more fragile in recent years.

The Adaptive Markets Hypothesis

The basic premise is that market participants are human, Lo noted. This has implications, including that while these people can come together and form a great decision-making apparatus — the market itself — they still make mistakes. They then learn from these mistakes and adapt to changing market conditions.

Ultimately, this led Lo to formulate an understanding of the market not in mechanistic terms, as we see under the EMH, but in terms of evolutionary biology and neuroscience.

“The very idea that markets work perfectly and frictionlessly really defies the fact of human biology,” Lo said. “It takes time and effort and a number of participants in order to make the market efficient. That suggests that the notion of efficiency isn’t an all-or-nothing phenomenon, but it’s really a matter of degree.”

In competitive markets with lots of investors engaged in research, efficiency is higher, Lo noted. In markets where mob mentality persists or emotional decision making takes over, we see markets become inefficient.

Ultimately, the move from efficiency to inefficiency is a byproduct of human biology and adaptation.

“It’s evolution at the speed of thought,” he said. “That’s really the theme of adaptive markets.”

Behavior and the Environment

While many behavioral economists have come up with all sorts of interesting counter examples to the idea that markets are efficient, in order to dismantle EMH, one needs to actually provide a newer and better framework, not just point out where the old one is wrong.

To fill in the gap, Lo emphasizes the dynamic between investor and the market environment.

“The environment is really what shapes our behavior,” he said. “While there are many different behavioral biases and anomalies, in order to figure out which one … is going to be relevant at any one point in time, in order to develop a theory of behavior, you need to understand the context and environment.”

One application of this idea is to evaluate markets based on how humans react to threats, incorporating neuroscientific research into the economic decision-making process. The hardwired fight-or-flight syndrome, for example, should be taken into account when designing mutual fund products, Lo noted.

Systemic Shocks

While the efficient market hypothesis has created tremendous value for investors, allowing people to get access to investment returns at relatively low cost, it has downsides as well.

Here's what Lo had to say on this point:

“If we take a look at the impact of efficient markets, it's created tremendous value for investors, portfolio managers, and even regulators in allowing people to get access to very, very valuable investment returns at relatively low cost. Case in point: Vanguard just crossed the $4 trillion mark in terms of its assets and I would argue that Jack Bogle has done more for the investor than all academics combined. He's been a tremendous source of value in terms of allowing people to invest in a very efficient, cost effective manner. However, there is an issue with regards to the popularity of these passive index funds and the issue is that more and more now, we're going to be experiencing the same kinds of ups and downs at the same time. So, the analogy of a crowded theater comes to mind. If somebody shouts fire in a crowded theater and we all want to get out at the same, the exists are going to be plugged up and there are going to be some serious problems. So, now that all of us are indexing ... the one unintended consequence is we're all now going to be herding in a way, we're all going to be experiencing the same ups and downs at the same time so if the stock market drops by 10% next week, my guess is that we're going to see a much, much bigger reaction than before index funds became popular. So, diversity, which is something that biologists talk about all the time as an important feature of a robust ecosystem, we're losing and now have less diversity and, as a result, the potential for a systemic shock has actually increased and we need to worry about that.”