Beware Consumer Ratings

If you use consumer reviews to make buying decisions, here's why you may be making poor choices.

When people want to decide what type of big screen TV or mobile device to buy, they often turn to product reviews posted online. This information might be to their disadvantage, however, depending on the way prospective consumers put their findings to use.

Social learning — our tendency to learn by observing other people — leads us to tend to make purchasing decisions based on the number of online reviews a product has, more than the rating other consumers gave the product, a new study suggests. We will overlook a product with a few excellent reviews in favor of a product with many good reviews.

A Popularity Bias

Web sites and apps tend to display the average rating for a product, along with the number of reviews, Derek Powell, lead author of the study, told TheDoctor. People seeking guidance for new purchases may use that information to make poor decisions.

ADVERTISEMENT

“People seem to have a popularity bias. They treat popularity as a useful cue to the quality of something. But it can actually lead people to believe a product is better than it really is, or cause them to shy away from a quality product,” Powell, a postdoctoral researcher in psychology at Stanford University, explained.

Looking at millions of consumer reviews of actual products available on Amazon, Powell and his colleagues determined that the number of reviews for a product was unrelated to its rating. In other words, the number of reviews was an unreliable indicator of a product’s quality.

The popularity bias was so strong, buyers even chose the case with the most reviews, even when both cases had the same low rating.

The researchers asked 132 adult participants to look at smartphone cases, presented in pairs. The participants were shown the average user rating and the number of reviews for each phone case. They were asked which case in each pair they would buy, based on that information.

Participants consistently chose the phone case with the most reviews, across different combinations of ratings and reviews. The popularity bias was so strong, they even picked the case with the most reviews, even when both cases had the same low rating. These results would indicate that, statistically speaking, participants were more likely to choose the lower quality phone case. A second experiment, done online with a nearly identical experimental design, produced similar results.

ADVERTISEMENT

The study findings have important implications for both consumers and retailers. Consumers might want to do more to educate themselves about a product’s quality. They may want to read some reviews to get a sense of how good a product is. “If a product has many reviews, stop and ask yourself if it really is better,” advises Powell.

The researchers developed a statistical model of how people should choose products. According to this model, when faced with two products with poor ratings, consumers should choose the product with fewer reviews, since it is less likely to be that bad.

Retailers hoping to offer their customers the best product feedback need to think about what information is most useful for consumers, Powell said. They should avoid providing their customers with the kind of misleading information “average ratings” tend to offer.

“Our findings highlight the power of social cues to guide behavior, but also the relatively simplistic mechanisms by which people sometimes process those cues,” the authors write. Future research will examine whether decision-making improves when consumers are given more specific information, such as how many five-star reviews a product has, rather than the average rating, Powell said, as well as decision-making in other consumer-driven areas, such as choosing a restaurant.