Customer Reviews – Trust, but Verify

During the last two decades customer reviews and ratings became an ubiquitous part of e-commerce. Over 88% of consumers gratefully use them to select goods and services, whether they purchase them online or off. There are numerous studies that tie in availability of customer reviews to increases in sales, occupancy rates and other business benefits. Nevertheless, there are some people who question the authenticity or usefulness of customer reviews and ratings at large. There are certainly examples of attempts to manipulate consumers with fake reviews or inflated scores, but overall the usefulness of publicly available product/service reputation information is undeniable.

While there is a law to protect consumers from predatory marketing practices, it is important for consumers to learn how to assess the meaningfulness of information they use for purchasing selections. I would like to illustrate this point with personal experiences I had during my recent travel to Italy.

After researching the reputation of multiple airlines, that fly between San Francisco and Italian destinations, I booked the flight with Turkish Airlines. The company seem to combine high levels of satisfaction (according to online customer reviews) and very attractive rates.

While in air service and schedule reliability where quite good, the experience of a layover in Istanbul was a complete buzz killer. The airline did not live up to their online reputation by failing to assist us in any way. Upon return, I took a closer look at the online reviews and noticed that nearly 92% of them was published by passengers who traveled within very close distance and did not experience layovers. In retrospect their experiences were not all that relevant in supporting my selection. Iberia, that has similar reputation scores and not prohibitively more expensive, would have been a better choice as much higher percentage of their reviews were published by the inter-continental travelers.

Since I could not get any promised assistance from Turkish Airlines, I used Bookings.com to find the hotel with the airport shuttle service. I booked the site’s highest ranked hotel since my phone’s screen was too small for reading multiple reviews. Upon arrival to the Istanbul’s airport I discovered that the Coresh Suites Hotel shuttle service did not exist, the taxi costs equaled our room rate, the hotel could not possibly be reviewed as high as Bookings.com ratings implied. The bed was tiny and uncomfortable, and the bathroom was not clean. Overall, it was a miserable experience.

A day later I received an email from Bookings.com with request to review my experience. Looking at their request form I understood what caused discrepancy between the hotel high score rating and my actual experience – Bookings.com asks guests to rate a limited number of customer experience attributes, and then algorithmically generates an overall score that they call a customer review. In my opinion such an approach makes their “customer reviews” too easily manipulable and not trustworthy. Therefore from now on I will book my hotels from TripAdvisor.com.

The customer reviews work well when consumers can read actual descriptions of customer experiences and apply them to their own expectations of experience. The details of the described experience, the language and phraseology help consumers to recognize authenticity and ascertain meaningfulness of the content to their own situation. The statistical averages offer very little utility and the algorithmically produced ones are almost as good as “faked”, regardless of the intent.

Thank you Jeff. I am familiar with the articles about BU study, but not with the details like number of reviews they analyzed and methodology they used. Our own research of Amazon and Yelp content (60M reviews) comes 4% below of there number.

BTW, my overall travel experience was very, very good I just focused on outliers to illustrate the point that not all reputation sites are created equally.