Luck and Skill in Baseball and Investing

The following is our quarterly investment letter that was recently sent to clients.

75 years ago this month, a thrilling World Series concluded what many baseball historians decidedly describe as the best baseball season ever. That season is notably remembered for two historic accomplishments: Joe DiMaggio’s 56-game hitting streak and Ted Williams finishing the season with a .406 batting average. In the 75 seasons that have followed, both of these marks remain untouched.

However, Ted Williams was the 28th baseball player to hit over .400 (40%) in a season when he did so in 1941 and no one has done it since. What makes the disappearance of .400 hitters so puzzling is that the average batting average has not changed – hovering around .265 for the past century. It is not as if the game shifted from hitter-friendly to pitcher-friendly rules or that pitchers began to dominate the game. Baseball players still get base hits with the same frequency today as in 1941. This begs the question: why is it that 28 baseball players eclipsed the .400 batting average in the first 65 years of the sport while no one has done so in the past 75 seasons?

After all, athletes today are stronger, faster, and healthier. Advancements in technology now allow athletes to train and perform more efficiently. In the most recent Olympic games, 60 Olympic records were broken. This year’s marathon winner shaved 80 minutes off the winning time of the 1904 Olympics – an improvement of nearly 40%. With records so frequently broken (or shattered) in swimming, cycling, and track and field, the lack of any .400 hitters in baseball over the past 75 years presents a dilemma.

Paradox of Skill

The late Harvard professor and baseball superfan, Stephen Gould, examined the disappearance of .400 hitters in baseball to try to understand the causes. His research debunked popular theories that changes to the game like pitching specialists, a prevalence of night games, or the development of the slider and split finger fastball were driving factors. He also argued that the extinction of .400 hitters was not simply because superstar talents like Ted Williams don’t exist anymore. Gould instead claimed that the disappearance of .400 hitters was about increased skill, not reduced skill.

What transpired in baseball after World War II was an increase in the skill level of all players. The game began to draw from a larger population of talent so that a sport comprised largely of white players from the northeast for its first century expanded to include black Americans and eventually drew players from places such as the Dominican Republic and Japan. Better training techniques coupled with a wider source of talent meant that that skill among pitchers and hitters alike increased. As a result, the absolute level of skill increased but the relative skill level decreased. In statistical terms, the standard deviation of skill declined.

Where this phenomenon gets more interesting, and relevant to the investing world, is in what statisticians term the ‘paradox of skill’. The implication of this paradox is that greater skill does not reduce the dependence on luck, but instead increases it. As the average skill level of all baseball players increases, results (batting averages) cluster and any deviation between players is likely to be the result of luck rather than skill. Improved skill compresses the extremes so that the likelihood of what Ted Williams did in 1941 is, statistically, about 20 times less likely today as it was then.

Long-time investment strategist and Columbia professor, Michael Mauboussin says, “Perhaps nowhere is the paradox of skill more evident than in the world of investing.” Consider some of the factors that make it harder for skilled fund managers to outperform their peers.

Democratic Dissemination of Information. The ubiquity of the Internet and the passage of Regulation Fair Disclosure (“Reg FD”) in 2000 mean that the distribution of information is far more democratic today than it was 20 or more years ago. Any short-term information advantage that a select group of professionals had in the past are all but gone.

Increased Sophistication. Fund manager’s today are smarter, more skilled, and have access to better tools. The esteemed Charles Ellis wrote in a 2014 Financial Analysts Journal article, “Over the past 50 years, increasing number of highly talented young investment professionals have entered the competition for a faster and more accurate discovery of pricing errors…They have more advanced trading than their predecessors, better analytical tools, and faster access to more information.”

Weaker Participants Exit the Competition. The spoils of success draw the best and the brightest to the investment world and they also push the weak participants (in this case, mom and pop investors) out. Despite the persuasion of Jim Kramer and the deep pocketed stock brokerage firms, mom and pop investors continue to slowly exit the active investing game, leaving fewer unsophisticated investors to compete against the pros.

Similar to baseball, the growing size, sophistication, and skill level of the investment management industry means that luck plays a more pronounced role than talent in determining relative success. Whereas evidence demonstrates that the average fund manager’s skill has significantly improved over the past 30 years, we may never see the superstar fund managers of yesteryear for the same reasons we may never again have a .400 hitter in baseball. In a 2014 paper titled “Scale and Skill in Active Management”, the researchers write, “We find that the average fund’s skill has increased substantially over time” but then go on to say that “This improvement in skill has failed to boost fund performance…We argue that the growing industry size makes it harder for fund managers to outperform despite their improving skill.”

Luck and Skill

As luck becomes more pronounced in the investment universe because of this paradox of skill, it inevitably becomes more challenging to differentiate skill from luck. Investors are more likely to be tricked into treating the past success of a mutual fund manager as the result of talent when it is more likely the result of good luck. If luck, not skill, is the key determinant of relative performance among mutual funds, then choosing mutual funds based on past performance success is a recipe for failure.

Evidence supports this case. Past performance makes for a great marketing gimmick but an atrocious determinant of future performance. Morningstar – a company that made its fame and riches on the star rankings of mutual funds – has, on several occasions, debunked its original star-rating methodology that relied on past performance. In fact, recent research suggests that investors would be better off buying funds with poor recent performance and selling funds with good recent performance. Such evidence provides a telling indictment about the utility of past performance as an indicator of future performance.

When we introduce our investment process to a prospective client, we are occasionally asked what criteria we use to select the best stock pickers, bond managers, or mutual fund stars. The answer is that almost none of the investments we utilize are managed by a person or team of managers seeking to locate the best underlying investments. While Wall Street promotes this vision of star managers consistently delivering great results, evidence suggests that this glamorous approach to investing generally just delivers higher fees, more taxes, and an exciting story, not better returns.

Instead, nearly all the investments we employ in your portfolio are managed not by people but by a quantitative discipline. We rely on passive investments to take advantage of reliable sources of excess return. The industry refers to this approach as evidence-based investing. The author of more than a dozen investing books, Larry Swedroe, well summarizes this approach:

Thus, the logical conclusion should be that the strategy most likely to allow you to achieve the best results is to focus instead on the selection of passively managed funds that provide you with the desired amount of exposure to the well-documented factors that explain the differences in returns of diversified portfolios—factors such as beta, size, value, momentum and profitability/quality for equities and term and default for bonds—and do so in a low-cost and, for taxable investors, tax-efficient manner.

The Moneyball Approach

In the bestselling book, Moneyball, author Michael Lewis describes a low-budget Major League Baseball franchise that defies conventional wisdom to build consistent winners with low cost young talents and inexpensive castoff veterans. The team’s front office relies on statistical data and historical evidence to construct a winning roster rather than the long-standing traditional method of relying on the judgment of human scouts. This Moneyball approach suggested understanding what quantitative metrics were reliable predictors of success and then using these metrics to determine the value of a player relative to the marketplace. This approach subsequently changed the game.

Evidence-based investing is the financial world’s version of Moneyball. In two worlds (baseball and investing) where results are often more the result of luck than skill and the biases of human judgment can impair decisions, it is advantageous to find metrics (or factors, as the investment world calls them) that predict success over long time periods and then rely on them without the impact of human judgment.

Sadly, the faith in hiring a team of human analysts, fund managers, or scouts to deliver a winning strategy is too deeply embedded to go away. The investment universe will likely never depart from this quest because it is highly profitable for those businesses selling the holy grail and because giving up on the quest would be an admission of human failure.

We are humble enough to admit that we do not know what the future holds and that we’re not the unique investor who can reliably differentiate skill from luck. We do know that there will be periods during which an evidence based approach underperforms. In fact, two of the most robust predictors of future returns (value and momentum) tend to have underperformance cycles that last between 3-4 years. However, in an industry or a sport where short-term outcomes are so often clouded by luck, one is better served to focus on the process rather than the short-term outcome. We remain humbly and diligently committed to a process that helps you achieve long-term successful outcomes.

Share This Story, Choose Your Platform!

Jason Lina is a Partner, Lead Advisor, and Chief Investment Officer at Resource Planning Group where he advises clients in financial planning matters covering investments, taxes, estate planning, retirement planning, personal finance, education planning, and insurance. He is a Chartered Financial Analyst (CFA) charterholder and a CERTIFIED FINANCIAL PLANNER™ (CFP) with over 18 years of professional experience in the financial services industry.
Jason earned his MBA from the Tuck School of Business at Dartmouth College, and received his undergraduate degree (BA, History) from Davidson College where he was captain of the golf team and a first-team Academic All-American.
You can find Jason's personal biography and contact information here.