How do index funds outperform hedge funds?

Passively managed index funds are hailed as a great and inexpensive way for investors to get positive returns over a long time horizon. Multiple studies show that for regular folks, holding index funds is a far better strategy than trying to beat the market. But these funds break the first rule of investing: When new stocks are added to an index, or when stocks are removed, the funds essentially buy high and sell low. They have no choice.

Funds that track an index such as the S&P 500 (INDEXSP:.INX) outsource their investment decisions. When those who control the composition and weighting of indices -- such as Standard & Poor's Dow Jones Indices, which maintain the S&P 500 -- decide to switch out one company in the index for another, the fund managers have to follow. The index is changed to better reflect the larger economy or sector, not to maximize returns. This is not an infrequent occurrence. This year, the S&P 500 brought in and expelled 17 companies.

Recently, Facebook (NASDAQ:FB) got such a vote of confidence from S&P Dow Jones Indices; it joined the S&P 500 and the S&P 100 (INDEXSP:SP100). Abercrombie & Fitch (NYSE:ANF), the struggling teen clothes retailer, was kicked out. Therefore index fund managers had no choice but to buy Facebook and sell Abercrombie. The same thing happened recently when JC Penney (NYSE:JCP) exited the S&P 500.

"Does inclusion in the S&P 500 suddenly make Facebook more profitable? Did inclusion in the benchmark add magic to Facebook’s business model? No," wrote James Osborne, a financial planner in Colorado. "And those index funds will now have the privilege of paying about 4% more for the stock than before the announcement."

The index fund has no choice but to track the index, even if it is a foolish move. Not only is the stock more expensive than before it was added, but it is also a less attractive buy from a price-to-earnings or price-to-book basis.

“The managers are forced to do something that makes us think twice about using a whole lot of index funds,” Pete Benson, co-owner and partner of Beacon Capital Management and a Franklin, Tennessee-based investment advisor, told Minyanville. “[It's] the managers who are supposed to have control, but this obviously hurts their overall performance.”

Compounding the problem is the tendency for deleted stocks to perform better after leaving the index. Index funds tend to buy good past performers, and shun companies with growth or comeback potential until their growth has already come and gone.

This matters now more than ever since passively managed funds have grown in importance in recent years. According to Pensions and Investments, global index funds' assets under management grew by $1 trillion in the year that ended June 30, 2013. The total is now a whopping $7.3 trillion.

This is a major change from when low-cost index funds started out in the 1970s. There are multiple reasons for this. One is that people understandably have little faith in richly paid stock pickers. According to S&P 500 Indices Versus Active Funds Scorecard, far more than half of active managers underperform their respective benchmarks. In the wake of Occupy Wall Street, many are wary of lining the pockets of hotshot fund managers with their savings.

So how do index funds perform relatively well despite the requirement to buy stocks that have already had a significant upswing?

One explanation is that funds offer built-in diversification across many sectors. This is considered safer than investing in single companies because even if one company in an index has a terrible quarter, it only has a small effect on the investor's nest egg. Lower fees compared to actively managed funds also make index funds attractive.

"If you had $1,000,000 invested in the S&P 500, only about $200 would have been in Abercrombie & Fitch," says Ben Sullivan, a financial planner with Palisades Hudson Financial Group. "By paying an active manager an extra 1% to try to avoid such issues, it would have cost the same investor $10,000 in additional fund management fees."

Fund managers also anticipate the changes in indices' constitutions by buying companies long before they are added, even if it means trailing the benchmark for a while, or paring holdings in a laggard company that is likely to fall out of the index. By the time a failing company is removed, an index fund's holdings in it are already just a small portion of the whole.

"One issue is that [index funds] can be gamed; we know which [companies] are likely to break into the index. Managers try to front-run these trades," says Phil DeMuth, a financial advisor and Forbes contributor. He says that indices with lesser known, thinly traded companies, such as the Russell 2000 (INDEXRUSSELL:RUT), are particularly easy for managers to game.

The information on this website solely reflects the analysis of or opinion about the performance of securities and financial markets by the writers whose articles appear on the site. The views expressed by the writers are not necessarily the views of Minyanville Media, Inc. or members of its management. Nothing contained on the website is intended to constitute a recommendation or advice addressed to an individual investor or category of investors to purchase, sell or hold any security, or to take any action with respect to the prospective movement of the securities markets or to solicit the purchase or sale of any security. Any investment decisions must be made by the reader either individually or in consultation with his or her investment professional. Minyanville writers and staff may trade or hold positions in securities that are discussed in articles appearing on the website. Writers of articles are required to disclose whether they have a position in any stock or fund discussed in an article, but are not permitted to disclose the size or direction of the position. Nothing on this website is intended to solicit business of any kind for a writer's business or fund. Minyanville management and staff as well as contributing writers will not respond to emails or other communications requesting investment advice.