Imagine you sent 100 bombers on a mission. Fifty crashed, and 50 dropped their payloads and returned. What if you looked at the 50 that returned, called the mission successful and then wrote a report that never mentioned the 50 lost planes?

That’s essentially what many mutual funds and investment data providers do when they report the performance of actively managed mutual funds and ETFs, according to Charles Ellis, the founder of consulting firm Greenwich Associates, who has also long been known as the dean of American investment advisors.

Money has been shifting dramatically from active to passive management for the past two decades, but more money is still managed actively. Actively funds of all kinds, including money market funds, manage about $15.4 trillion and passively managed funds and ETFs manage $6.7 trillion, according to Thomson Reuters’ Lipper.

Ellis was a proponent of active management—which means hiring a manger to pick stocks in an effort to beat the market — early in his career. But he has since flipped the script, steadily becoming a critic of active management over the years. His recent book, Index Revolution, is a case in point.

“You’ve got millions of people using (active management) for their retirement savings,” Ellis said. “But people aren’t making money. It’s all about sales for the fund companies.”

Active management is falling short, he says, because of the way the market has changed. Decades ago, most trading was done by amateurs, making it easy for skilled professionals who had access to proprietary data to beat the market. Now, 99 percent of trading is done by experts and computers who are so good that they can’t beat each other. “The willing losers have been outnumbered,” Ellis said.

Burying key data

In an interview with CNBC, he talked about one of his big problems: The way mutual funds report their numbers, which tends to make active management look better than it is. “Individuals and institutions are getting humongously biased data,” Ellis said. “You know the old joke: doctors bury their mistakes. Mutual funds close or merge theirs.”

Usually, that’s because they weren’t performing well, though there can be other reasons, such as not gathering enough assets to achieve an economy of scale, according to Tom Roseen, head of research services at Lipper. In the past three years, 2,229 mutual funds and ETFs closed, according to the Investment Company Institute, the industry group representing mutual funds.

The data on closed funds is to a large extent lost to the eyes of individual investors as they evaluate the returns of asset classes and as they make decisions about which fund companies, fund families and funds to invest in. Investors who want to try and get a more accurate picture can ask their investment advisers whether the analysis they’re looking at includes the performance of closed funds, and how the data is incorporated.

“The way the numbers are reported in advertisements and promotional materials gives investors a false “enhanced” impression of the capabilities and performance of active managers,” Ellis wrote in his book. “In a race to the bottom — “All the other kids do it, Mom” — the poor and mediocre performers get deleted from the database as though they had never existed.”

For individual investors who are investing with a long time horizon, such as for retirement, it’s important to understand that active management generally does worse than passive management, or indexing, over time.

"The way the numbers are reported in advertisements and promotional materials gives investors a false “enhanced” impression of the capabilities and performance of active managers."-Charles Ellis, Founder of Greenwich Associates

Over a 15-year investment horizon, 92.33 percent of large-cap managers, 94.81 percent of mid-cap managers, and 95.73 percent of small-cap managers failed to outperform on a relative basis, when performance data is calculated with the results of closed funds included, according to the 2017 SPIVA U.S. Scorecard. That means an investor would likely have had better results with a low-cost index fund.

Take, for instance, multi-cap growth funds. Over a 15-year period, only 13.79 percent of fund managers beat their benchmarks over time, according to SPIVA. But if a data provider excluded the performance of the closed funds, the asset class (and active management in general) looks much better: With the losers excluded, 43 percent of funds beat their benchmarks.

Because the chance that an active manager can beat the market, already relatively small, declines over time, the difference between the two data sets gets larger over time. In short, active management’s underperformance gets worse over time, and it looks even worse if the results of closed funds are not included.

Survivorship bias

There is no industry standard on how to report closed and merged funds, according to Ellis and interviews with about a half-dozen other industry experts. Practices vary across fund companies. Morningstar and Lipper, the two main data providers, retain the performance data on the closed funds and consider it best practice to present data that way, but it’s up to clients whether to request it, representatives of both companies said. Perhaps not surprisingly, clients, including investment advisers, analysts and fund companies, sometimes ask for data without the closed funds, said Roseen.

Mergers often happen because “(fund companies) will want to hide their poor performance,” he said. “Many times you’ll see a fund wanted to get rid of their black eye. A lot of times the fund manager drove a fund off a cliff.”

Data scientists readily recognize the industry’s problem: It’s called survivorship bias. Index providers like S&P Dow Jones Indices, which produces the SPIVA data, advocate for producing the data without survivorship bias, which means including the performance of the closed funds. But with little regulation on this point, there’s no reason for anybody to change.

“Even in my conversations with active managers, they say they know this is the correct. It’s one of the facts that cannot be refuted,” she said.

The question of how to deal with the poor performance of closed funds gets trickier when it comes to judging a particular fund, versus looking at the performance of whole asset classes or management styles over time. Closed funds are often merged into existing funds, but in those cases, the mutual fund company typically retains only the performance of the surviving fund. If the closed fund lost three percent a year over 15 years, and the surviving fund earned five percent, investors would see the five percent.

"The fund marketplace is competitive and fast-moving. When funds decide to merge or liquidate, they do so with shareholders' best interests in mind, under strict regulations set by the SEC and with close oversight by independent directors,” said ICI Chief Economist Sean Collins.

“Data from merged or liquidated funds remains readily available and analysts choose whether to incorporate that data or not,” Collins points out. “The information that matters for a new investor is the strategy, objectives, and performance of the fund they are buying, not of funds that no longer exist."

To Ellis, there’s a great deal of obfuscation that disguises a fundamental point: Long-term individual investors stand the best chance with passive investments. “If you as an investor would like to have a top quartile investment return over the next decade: Index,” Ellis said.