Ignore Morningstar's Stars

This week, Morningstar published an article warning investors not to rely too heavily on the firm's well-known and well-regarded star ratings. Morningstar's one- to five-star scale is akin to the old Siskel and Ebert"thumbs up/thumbs down." It's that iconic.

Meanwhile, investors are buying mutual funds again. The industry saw $4 billion in new money during the week of Jan. 21st, reversing a trend of redemptions, according to the Investment Company Institute.

No doubt many of these fund buyers are choosing their investments with advice from
Morningstar
, a firm that analyzes stocks and funds in two ways. First, it uses its quantitative star rating system, which weighs the fund's returns compared with its peers. Stars are assigned based on below-average, average or above-average return over a five-year period.

Morningstar's analysts take a more subjective approach to fund analysis. They examine portfolios, meet with fund managers divine industry and market trends and do the old "kick the tires" routine that all "bottoms-up" analysts are supposed to excel at.

Our panel of mutual fund experts isn't all that impressed. They say the star ratings are not only not a good indicator of how funds will perform in the future, they're not even a good starting point for research.

While its analysts are good, our panel noted that Morningstar does have some conflicts of interest because the company provides services to the mutual fund industry. Morningstar charges funds to use its star ratings, and other proprietary data and research in investor letters. The fee is charged based on the number of funds in the family, but our panelists were able to produce one Morningstar licensing agreement that pegged the price for one fund at about $8,000 a year.

In an e-mail, a Morningstar spokesperson replied. "We do not charge a fee for firms to use the star rating in advertisements. We do charge firms for any other type of use of the star rating, such as direct shareholder communication like a direct mail or e-mail campaign. The fee we charge is based on the number of funds managed by a fund company, regardless of how the funds are rated."

The panel also wondered why Morningstar only produces fund family reports for Fidelity, American Funds and Vanguard. Morningstar says it's a matter of user demand--since those are the largest fund families, it's not surprising that they generate the most interest.

Morningstar's take on all of this?

"The star rating isn't a complete solution, but rather an aid that helps you to narrow the field and improve your chances for success," the company said. "Our Fund Analyst Picks do an even better job of that than the star rating, and because there are fewer picks than five-star funds, it also narrows the list to fewer choices."

Wiener: Once a year or so Morningstar does its mea culpa (see link) and basically says its star ratings are not really the be all, end all way to rank, or even research and choose funds. Yet, the Morningstar stars are probably the most heavily advertised ranking the fund industry uses (and I believe Morningstar takes in some kind of fee for allowing fund companies to use them in their advertising).

Since we already know, and Morningstar admits, that the fund rating is virtually useless, should the fund industry be allowed to use them in their advertising?

Bold: I think Morningstar does a terrific job of compiling hard statistical data on mutual funds. However, we part ways when they start to make subjective calls as to the merits of one fund vs. another. Their methodology for assigning star ratings is entirely based on average returns; it can't account for when a really good manager has one bad year that throws off the trailing averages. In fact, it doesn't even consider whether the same manager has been there for the entire period being evaluated.

It would be really great if picking funds were as simple as looking at how many stars an independent research firm assigns to it. Unfortunately, it's not that easy, and I fear many individual investors rely to heavily on those stars.

Wiener: The question is where to draw the line on the "independence" of the mutual funds raters. If they are selling their services to the same companies they are rating, are they any better than S&P or Moody's? Look at the mess those companies got us into.

Editor's note: Here, Morningstar explained that it only charges for use of the stars in client communications. One panelist obtained a Morningstar proposal from 2005 that asked for about $8,000 a year for this. Morningstar says that fund companies do not pay Morningstar to issue star ratings or write research.

Wiener: So, you use the stars gratis in your ads, which helps promote the brand. Then, once the brand is created you, have to pay to use them in your direct communications--and I assume this would mean adviser-only communications that advisers then use to sell the funds to their clients. Smart, but full of conflicts of interest.

Gates: Many academics have studied the predictive power of Morningstar's star ratings. Many of the studies imply to me that the ratings have negligible value. For instance, Patrick McGuigan from Pace University wrote an analysis that was published in 2006 that summarized the rating system by saying, "A five-star fund may be a beacon, a bright oasis amidst the uncertainty, but like the stars in the sky, their predictive ability may be millions of miles away."

However, I am even more leery of Morningstar's analyst reviews. These reviews are the subjective reports that the firm's analysts periodically publish. My sense is that these reviews may actually be worse than the quantitative star ratings. In other words, while I think the star ratings may not necessarily help, I believe the analyst reports may actually hurt retail investors.

One of TFS' analysts has been in touch with Morningstar in the recent weeks, trying to obtain data to confirm this suspicion. If they provide us with a data set of their analysts' selections, we will perform a rigorous analysis and make all of the results and our data publicly available.

Bold: I think it's also interesting that Morningstar puts out monthly research that helps you pick the best funds, but only if you buy American Funds, Vanguard or Fidelity. Call me skeptical, but why do they even bother to follow funds from other companies when evidently those three have a lock on all the great funds?

Or, could it be that those three fund companies pay [Morningstar] to write the research?

Gates: Taking a step back, I think it is funny that Morningstar has two different rating systems--star ratings and then analyst picks. They are very distinct from each other. If I were to design a firm that really wanted to help retail investors pick funds, I would put all of my research into it and pick just one set of "good funds."

Having two sets of funds probably helps their marketing, though. :)

Adam and Dan are right--there are a lot of other potential conflicts of interests with Morningstar's business model. One example is that they manage money themselves. In addition to collecting money from fund companies for star ratings, Morningstar also accepts payments from fund families to attend their conferences.

These conflicts are another reason why I suspect their star ratings are better indicators than their analyst picks. Of course, the star ratings are based on quantitative analysis of historical returns. For the most part, they cannot be biased. Either way, I sure hope Morningstar releases the historical information that TFS requested on their analyst picks. We sent the request over to some of their top analysts, including Russel Kinnel, so are cautiously optimistic we will get the data that will help settle this once and for all.

Wiener: In February 1998, Vanguard fired Husic Capital and hired Primecap Management to run its Capital Opportunity fund. Morningstar had a one-star rating on the fund at the time. The fund eventually hit five-star status in April 2000.

Over the period that it was growing into its stars, the fund gained 244%, vs. 58% for Vanguard 500 Index. Too bad for those who only look for five-star funds, as it was one of the best examples of manager outperformance ever displayed.