What's the story behind Morningstar's Star Ratings?

When FiduciaryNews.com published a story that, despite the provocative title, merely relayed the fairly uniform feelings of many financial advisers (see “Have Mutual Fund Rating Agencies Lost Their Mojo?” FiduciaryNews.com, January 15, 2013), no one expected the Spanish Inquisition.

In the piece, we quoted a source who said, “Morningstar was honest enough to state exactly that in a recently published article on their site where they pointed out that their proprietary star rating system is not a valid indicator of future investment performance or even relative performance.” To be clear, the article didn’t single out Morningstar, although this particular quote did.

Within hours, FiduciaryNews.com received an e-mail from Morningstar stating we “take exception” to that particular quote. Furthermore, they insisted “it’s simply not true.” In researching the original article, so many sources said the same thing we did not expect the quote to be challenged.

In fact, we thought the sentiment viewed Morningstar in a most favorable light. Still, since Morningstar brought up the issue, like any good journalist, I immediately began to track down the truth.

When contacted for the citation of the “recently published article,” the original source toldFiduciaryNews.com it had been removed from the Morningstar site. We then moved to other sources who provided similar comments to the original source (but weren’t quoted in the published article). They continued to believe Morningstar had always maintained their Star Rating systems were not predictive.

Our sources were able to provide documentation going as far back as 1996 and 2005 where Morningstar admitted the star rating system was not predictive. These dates were important because the 1996 admission addressed a popular academic study which showed the star ratings were not predictive.

In 2002 Morningstar changed its algorithm for calculating its star ratings system, so for the 2005 comment to be consistent with the 1996 comment (albeit just for “the short-term”) would imply the Morningstar policy had not changed.

In 2010 a new study was done – this time by Morningstar – that compared the predictability of fund expense ratios versus the predictability of star ratings. While that study “concluded” both were predictive, a mathematical analysis by Edwin Choi proved the Star Ratings were less reliable than simply looking at the fund’s expense ratio.

I reviewed still more (independent) studies that concluded Morningstar Star Ratings had a better predictive rating after the 2002 change in methodology. Indeed, I had hopedFiducairyNews.com would have been in a position to break the news that Morningstar was going on the offensive by stating their ratings were predictive.

Alas, despite an aggressive beginning, all they admitted was, “it all depends on what you mean by predictive” and that higher Morningstar ratings generally did result in “better outcomes.” In fact, after presented with the DALBAR No-Action Letter (which requires advisers using independent ratings to repeat the ubiquitous mantra “Past Performance Does Not Guarantee Future Results"), Morningstar reminded me their license agreement requires precisely that.

When it was all said and done, the quote in the original was left intact. It was challenged, sure, (and if you want to see the full story and see exactly what Morningstar said, read “Morningstar Star Ratings: Do They or Don’t They Predict?” FiduciaryNews.com, January 29, 2013). But we’ll leave readers to make up their own minds as to how successful that challenge was.