Share this story

Amazon is a gigantic international marketplace filled with all sorts of goods from countless manufacturers and vendors—a selection so broad, it can easily overwhelm shoppers. Though the company doesn't really curate what's sold on its platform, it does do the equivalent of showing off certain products in the window with its "Amazon's Choice" label. The problem is that nobody outside Amazon knows how those choices get chosen... and some of those "choice" products are basically crap.

Several media outlets have tried and failed to learn how it all works, but this week members of the Senate have come knocking on Amazon's metaphorical door with some pointed questions. Democrats Bob Menendez of New Jersey and Richard Blumenthal of Connecticut are calling on Amazon to explain why certain products get that coveted Amazon's Choice badge to determine if the moniker "deceives consumers into purchasing products of inferior quality."

A search for a product like dish detergent returns more than 20,000 results, Blumenthal and Menendez write in a letter (PDF) addressed to Amazon CEO Jeff Bezos. Given that volume, consumers "look for distinctive product features to help narrow the extensive search results," and those shoppers "reasonably rely" on the Amazon's Choice label "to guide their final purchasing decisions."

The label does indeed seem to guide purchasing decisions, the senators note in their press release. A research study showed that products granted Amazon's Choice status can see a threefold sales increase—but products that have the badge and then lose it see sales slump by 30%.

Further Reading

Amazon introduced the "Choice" feature in 2015 as a way to create a default for shoppers talking to their Alexa-enabled devices to buy goods. "Alexa, buy dish soap" is one thing if you buy the same kind of dish soap every month, but what if you have no purchase history? You can't easily browse 20,000 items—or even 20 items—when you're talking to a speaker. So Amazon's Choice product became what Alexa would suggest for you up front.

These "choice" products can be of dubious quality, as highlighted by a BuzzFeed News report from June. BuzzFeed found dozens of instances of consumers complaining, both in Amazon reviews and elsewhere on social media, about Amazon's Choice products that broke down or just plain didn't work—including a thermometer where the product description itself said the item was "widely inaccurate."

"I took the Amazon's Choice label as an endorsement by Amazon, although I later found out that's more of an algorithmic term," one Amazon shopper told BuzzFeed at the time. "I find that to be confusing at best, misleading at worst."

Pay to place?

The letter gives Amazon until September 16 to respond to a whole host of questions about how the Amazon's Choice label gets assigned, starting with a request for a detailed description of the process and whether it's purely algorithmic or if human hands are ever involved. It also asks if all Amazon shoppers see the same "choice" products or if the label varies between shoppers based on individual purchase and browsing histories.

The senators also want to know what metrics Amazon uses to make the call. As part of that, the letter also asks Amazon to define what "highly rated" means, in context, especially given the challenges the company has faced in the past with fraudulent, paid-for reviews and review recycling, in which a lesser product is swapped into the listing of a different product that already has a positive rating.

"Amazon bears the responsibility of providing its customers with accurate information" so shoppers can make informed decisions, the senators write. "Unfortunately, Amazon has failed to fulfill this responsibility."

Given Amazon's dominance of online retail, this is actually a reasonable issue to be concerned about. I consider myself pretty decent at online shopping and yet I failed to consider that Amazon's choice was not human-curated. More reasons to stick with human-curated reviews like Wirecutter and Consumer Reports whenever possible, I guess.

How is this any different than manufacturers and distributors paying to get shelf placement in physical stores? You think products are just randomly placed? No, the best locations (eye level) go to the highest bidder.

In the same way that google's or facebook's use of advertising is very easy to abuse. So is amazon's

They have the whole platform very efficiently making them money. Sellers get a better spot in the searches depending on many things including standing with the company. There is probably at least 1-2 corners cut on ethics, or objectively good for the customer changes.

At the same time hordes of people making a living on amazon are trying to game the algorithm, and trying to outsmart one of the richest most tech reliant companies in the world.

It should surprise no one that this is the reality of Amazon with everything else going on.

Amazon ratings are worse than worthless. I'm not even too keen on review review sites like fakespot, etc. I buy a lot of I.T. gear on Amazon which I'm pretty familiar with so I don't rely on Amazon reviews for that.

But occasionally when I'm looking for some no-name thing, it's a crap shoot.

I would also like some insight into how Amazon recommends things. I find it irritating when I order a bunch of stuff and it will all arrive 1-2 days except one thing which will be shipped from China and should arrive in October. If you are going to recommend a product it should already be in a warehouse in the destination country.

How is this any different than manufacturers and distributors paying to get shelf placement in physical stores? You think products are just randomly placed? No, the best locations (eye level) go to the highest bidder.

I had never thought of this. What is the equivalent of an end-cap on Amazon?

Amazon's Choice needs to go away, they already have enough influence by putting stuff at the top of the results.

There is a bigger problem that needs to be addressed though, search sorting. If I sort by anything other than "featured", 99% of the results disappear into the void which means that even without "Amazon's Choice" I'm still going to be stuck with seeing the "featured" listings first.

Given Amazon's dominance of online retail, this is actually a reasonable issue to be concerned about. I consider myself pretty decent at online shopping and yet I failed to consider that Amazon's choice was not human-curated. More reasons to stick with human-curated reviews like Wirecutter and Consumer Reports whenever possible, I guess.

Consumer Reports is kind of in a class by itself, but most other online review sites are as subjective as an Ars phone review. And no review site (again, with the possible exception of CR) does a good job with long-term durability.

In some ways I'd trust a couple thousand reviews from random buyers more than I would a single review from a self-appointed expert, but that assumes that Amazon is curating their reviews and preventing abuse.

Why don't we just audit the algorithm(s) responsible for the ranking? Who needs Amazon to explain it when you could look at the source?

It seems like every time senators want to grill a big tech company it just goes nowhere because most of the time in the hearings is spent on explaining the tech, conducted as if the participants were hearing about these issues for the very first time. If we could actually look at some code it'd be far easier to sick some engineers on it to analyze and explain how junk keeps floating to the top, without needing some lengthy hearing.

Anything short of showing the public how Amazon internally decides its rankings would be a waste of time at this point. You're never going to get straight answers from the horse's mouth.

The last time I bought something on Amazon was last fall or maybe two years ago. Between useless ratings and reviews and useless Amazon's choice algorithm and not having a good selection of the kind of products I would buy online, I don't bother anymore.

Several years ago people complained that customers would go see the items at brick and mortar stores and buy off Amazon.

Now it's the other way around : I check Amazon for price range and buy from a website that has a brick and mortar store behind it. Which is kind of odd, but there you have it: Amazon, the most complete price comparison app!

How is this any different than manufacturers and distributors paying to get shelf placement in physical stores? You think products are just randomly placed? No, the best locations (eye level) go to the highest bidder.

I had never thought of this. What is the equivalent of an end-cap on Amazon?

How is this any different than manufacturers and distributors paying to get shelf placement in physical stores? You think products are just randomly placed? No, the best locations (eye level) go to the highest bidder.

It's the same class of behavior. However, it's vastly magnified since this is Amazon, and the badge is a more powerful effect (I would think).

Why don't we just audit the algorithm(s) responsible for the ranking? Who needs Amazon to explain it when you could look at the source?

It seems like every time senators want to grill a big tech company it just goes nowhere because most of the time in the hearings is spent on explaining the tech, conducted as if the participants were hearing about these issues for the very first time. If we could actually look at some code it'd be far easier to sick some engineers on it to analyze and explain how junk keeps floating to the top, without needing some lengthy hearing.

Anything short of showing the public how Amazon internally decides its rankings would be a waste of time at this point. You're never going to get straight answers from the horse's mouth.

I work for a company with Amazon sales in the 8 figures, and can tell you that we have no idea why certain products receive this designation, but it is definitely not human curated.

And don't get me started on reviews. They are so tremendously important that is is no surprise that they are so heavily games, but they seriously are.

One of my favorites is "ASIN Squatting" where an unscrupulous seller finds an old retired product with many/good reviews and takes the listing over to sell their own entirely unrelated product. Amazon has even made this harder to discover by removing the ability to quickly navigate to, say, page 6 of reviews to see if people were once reviewing a different product.

How is this any different than manufacturers and distributors paying to get shelf placement in physical stores? You think products are just randomly placed? No, the best locations (eye level) go to the highest bidder.

I had never thought of this. What is the equivalent of an end-cap on Amazon?

The home page and "people who bought this also bought" areas.

You would expect Choice to be something that people are actually buying a lot of as opposed to something promoted.

Amazon ratings are worse than worthless. I'm not even too keen on review review sites like fakespot, etc. I buy a lot of I.T. gear on Amazon which I'm pretty familiar with so I don't rely on Amazon reviews for that.

But occasionally when I'm looking for some no-name thing, it's a crap shoot.

Ratings fraud is prevalent on any site in which there is an incentive (usually monetary) to commit such fraud. See IMDB or Yelp for more examples; both of those sites are also useless to me, at least as far as trusting the ratings.

The person who comes up with a solution for a fraud-proof ratings system is going to be very wealthy.

Just wanted to add a callout to FakeSpot.com, which tries to use algorithms to predict how reliable the reviews for a product are. It has stopped me from buying a few stinkers. It also has some price history, so generally I look up all listings on there unless I know the product and don't care about the reviews.

At least there's a partial solution to this. Do price comparisons. I've been finding Amazon to have non-competitive prices to even brick and mortar more and more frequently. If they start losing sales maybe they won't push garbage so hard.

How is this any different than manufacturers and distributors paying to get shelf placement in physical stores? You think products are just randomly placed? No, the best locations (eye level) go to the highest bidder.

I had never thought of this. What is the equivalent of an end-cap on Amazon?

The home page and "people who bought this also bought" areas.

Actually, it's the top "Sponsored" results for any given search. The large bulk of Amazon shoppers' behaviour is 'spear fishing', which means they enter some search they think gets at what they want. Then, for most verticals, something over 65% of all clicks go to the top 3-5 results.

Despite how the algorithm is now, and how it finds every microscopic niche possible, it's weighted in a fundamentally anti-consumer way.

The recommendations need to point to something that's sold by Amazon or a very trusted partner that will ship from a local warehouse and have stock to a Prime customer within the Prime shipping deadline.

It needs to recommend a product from a real company with a real history, and a product that has a combination of positive reviews (understanding the limitations of that pooch screw) and high sales volume (likewise something that is getting gamed.) Yes this is probably going to disadvantage Chinese sellers who make flashlights, bras, Lightning cables and music stands (this week: next week they'll be selling shoes and hard drive enclosures.) If a Chinese brand is a legitimate brand, then that's fine, but it probably should set a higher bar and have those companies prove they're operating as a real company with some realistic expectation that they'll be around next month and not re-constituted as a new entity with a new name when Amazon find something to penalize them on or their reviews drop too low.

Likewise, they need to be rooting out products that exist from multiple vendors. If you have identical widget X, cranked out in China and branded by 5 different companies, none of them should be the "choice" since they're all the exact same %@#$ing thing.

I would also like some insight into how Amazon recommends things. I find it irritating when I order a bunch of stuff and it will all arrive 1-2 days except one thing which will be shipped from China and should arrive in October. If you are going to recommend a product it should already be in a warehouse in the destination country.

Wouldn't be a problem if Amazon's search tool was anywhere as useful as Newegg's. Though I have to fault Amazon and Newegg for allowing anyone s dog to set up a storefront.

I had to wait 2 months for pads and rotors to be delivered (EBC, so they're good quality at least), what's worse is the "Buissness address" was in AZ. Meanwhile I finally got them to admit they didn't have the parts innstock., And they were being shipped from. EBC themselves to their shop in a bulk(heh) order. So I quite literally was unaware I was in a group order for car parts.

Just wanted to add a callout to FakeSpot.com, which tries to use algorithms to predict how reliable the reviews for a product are. It has stopped me from buying a few stinkers. It also has some price history, so generally I look up all listings on there unless I know the product and don't care about the reviews.

But even Fakespot goes wonky.

Anyways Amazon used to have a "Downvote" on reviews but they went to the EBay method of it's POSITIVE OR NOTHING.

I work for a company with Amazon sales in the 8 figures, and can tell you that we have no idea why certain products receive this designation, but it is definitely not human curated.

And don't get me started on reviews. They are so tremendously important that is is no surprise that they are so heavily games, but they seriously are.

One of my favorites is "ASIN Squatting" where an unscrupulous seller finds an old retired product with many/good reviews and takes the listing over to sell their own entirely unrelated product. Amazon has even made this harder to discover by removing the ability to quickly navigate to, say, page 6 of reviews to see if people were once reviewing a different product.

How is this any different than manufacturers and distributors paying to get shelf placement in physical stores? You think products are just randomly placed? No, the best locations (eye level) go to the highest bidder.

Well they don't label them as "Grocery Store's Choice" for one (they do sometimes for their own in house brands though but they don't have good shelf space). They also don't include fake reviews.

Why don't we just audit the algorithm(s) responsible for the ranking? Who needs Amazon to explain it when you could look at the source?

It seems like every time senators want to grill a big tech company it just goes nowhere because most of the time in the hearings is spent on explaining the tech, conducted as if the participants were hearing about these issues for the very first time. If we could actually look at some code it'd be far easier to sick some engineers on it to analyze and explain how junk keeps floating to the top, without needing some lengthy hearing.

Anything short of showing the public how Amazon internally decides its rankings would be a waste of time at this point. You're never going to get straight answers from the horse's mouth.

Amazon likely claims their algorithm is a trade secret.

That's neat but if the trade secret contains steps that are harmful to the consumer then the offending bits need to be exposed.

"This item appeared in the top of your search because of paid endorsement from the manufacturer."

That's not trade secret, that's just keeping the consumer informed that the manufacturer paid for a ranking. It helps to separate popular items that are popular because of paid endorsement, versus popular due to quality.