Have you ever bought a product based on a review? One of the advantages of online shopping is the community of fellow shoppers who are willing to share their honest, real-life experiences with items that are being sold. But what happens when that trust is broken? It isn’t the first time, but an Amazon (NASDAQ:AMZN) vendor has been publicly outted for paying customers to post positive reviews.

Reviews have long been an important tool for consumers who want to ensure they don’t buy something that turns out to be a lemon. After all, it’s not like you can make an informed decision based on a company’s advertising. An entire industry of publications, from automotive magazines to monthly video game reviews, has grown around the need for independent advice on whether a product is worth buying, with perhaps the best known product testing and review publication of all being Consumer Reports.

The popularity of the Internet increased the field of product testers and reviewers. Many publications launched websites and, freed from publication limitations, gained the ability to post more reviews than ever.

Review collectives

Online retailers such as Amazon, Dell (NASDAQ:DELL), and Apple (NASDAQ:AAPL) took the concept of reviewing a product to the next level, crowd sourcing it. The reasoning behind that seems solid. First of all, not everyone wants to take the time to look up reviews on a product or to read a detailed review; seeing a star rating system right beside the product is simply easier.

And while professional product reviewers can be depended on to provide candid and detailed opinions, in some eyes they can be overly critical. Or maybe the test unit they received was faulty. Having real people who actually bought and used the product chime in with their thoughts on it, assign it a rating and then averaging that number should offer potential buyers a pretty good idea of what to expect.

The flaw in the system is that it’s difficult to ensure those crowd-sourced ratings are real. Professional reviewers are bound by ethical and legal guidelines. The Federal Trade Commission, for example, requires a reviewer who has a connection to a manufacturer or receives payment for a positive review to disclose this information. Customers leaving a comment and review aren’t bound by the same rules, a loophole that companies have been known to exploit.

A conflict of interest and a reason to be wary

Such was the case with VIP Deals. As reported in the New York Times (NYSE:NYT), Amazon caught on to the fact that the company, which was selling $59.99 leather cases for the Kindle Fire, was offering the cases for free to customers who wrote a review, preferably a five-star review. Buyers were shipped the case and once they completed the review, VIP Deals refunded their purchase price. By the time Amazon shut the seller down, 310 of the 355 reviews for the case were perfect five stars. Sales figures aren’t available, but you can bet with that consistently high rating and an in-demand product, VIP Deals moved a lot of cases.

Competition for online buyers is fierce, and vendors are certainly aware that consistently high review scores from buyers can drive sales, so it’s not surprising that some might be tempted to pay for favorable reviews as a highly effective (and relatively inexpensive) way to overcome the competition. It’s highly unlikely that the VIP Deals incident is an isolated one–the vendor just happened to get caught. The key for consumers is to treat star rating systems and buyer comments found on online retailers with a healthy dose of skepticism. Watch for reviews that seem strangely bubbly, include no downsides or negatives, and that are part of a cluster presenting overwhelmingly high ratings. Check other websites to see if shoppers there are of a similar mind, and check in to see what those professional reviewers say.

Amazon isn’t going to suffer from this one incident, but given the publicity it has received, Amazon sellers can expect to be scrutinized more closely to make sure they aren’t gaming the rating and review system. Pervasive mistrust isn’t good for customers or vendors, and it’s certainly not good for Amazon. Eventually, the FTC may get involved as more retail transactions move online and customer ratings become even more influential. An FTC official interviewed in the New York Times article put the issue of fake reviews bluntly: “We’re very concerned.”

As of this writing, Brad Moon did not own a position in any of the stocks named here. He is, however, a professional reviewer for a website focused on consumer electronics.