Hypothetical performance data skews future ETF performance

A new report by Vanguard found that exchange-traded funds that are successful in attracting assets may only do so because of the hypothetical performance data that is available on most ETFs.

As of March 31, 2012, more than $1.2 trillion was invested in 1,400 U.S.-listed ETFs, which sought to track more than 1,000 different indexes, according to “Joined at the hip: ETF and index development.” Many of these launched using new indexes based on narrow market segments and alternative weighting methods that often lack live performance history, the paper said. Among the indexes created for use in ETFs, more than half included back-filled or back-tested data from before the date the indexes were first publicly available, and it is difficult for investors to discern which data are hypothetical and which are live.

The paper found that more than half of ETFs are launched with an index that has been in existence for less than six months. “Using an event-study analysis to look at the performance of indexes before and after ETF creation, we find that ETFs are most likely to be created with indexes that have performed well relative to the broad U.S. stock market before the inception date, but that such performance, on average, does not persist.”

Investors need to be cautious in considering historical index performance, especially for indexes constructed for use in new ETFs.

There were only six ETF index providers in 2000, compared to 75 at the end of November 2011.

Of the 1,070 indexes that serve as ETF benchmarks, Vanguard was able to obtain index-live date information for 775 indexes from index-provider or ETF sponsor websites.

The report focused on the 637 indexes that went live after 2000. Of these, Vanguard found that 370 indexes had either back-filled performance data or live performance data of at least six months, while 219 did not have any back-filled data, and 48 had back-filled data or live performance of less than six months.

Back-testing is done by retroactively applying the index methodology to the historical data as if the index had existed previously, the report stated.

The median time between index and ETF creation decreased from almost three years in 2000 to just 77 days in 2011. As a consequence, most indexes have little live performance history for investors to assess in the context of a new ETF investment.

Through its research, Vanguard found that back-filled data was not a reliable indicator of how the index would perform after it was launched. It analyzed the performance of 775 indexes, 370 of which had at least six months of back-filled data and six months of live data from 2000 through 2011. What it found is that although 87 percent of the indexes outperformed the broad U.S. stock market for the time in which the back-filled data were used, only 51 percent outperformed the broader market after the index was launched.