One fella named Chip S. came along to say that the whole argument was a ridiculous strawman; nobody really believes that result.

Chip S. was annoyed because I had posited, without doing research on it, that this study would be the type of thing Big Media and Freakonomics types would write about:

You can easily see journalists writing an article that wows the public. Call the Freakonomics guys. Can’t you envision a section of a chapter talking about this surprising result, and discussing the likely reasons for it? Perhaps the shoppers were paralyzed by indecision when presented with so many choices. Valuable information, certainly, for any marketer to know.

Valuable — and almost certainly wrong.

Was I wrong that this would be the sort of study Big Media and Freakonomics types would jump on?

Please. Would I ask the question so publicly if the answer were unflattering to my predictive powers?

I looked this evening. There is a New York Times article pushing the idea. There is a book that features the study as an example of how choice can paralyze people. There are blog posts making the same point. And plenty of my commenters came out in support of the jam study’s conclusion — seemingly overlooking Manzi’s assertion (as reported by my post) that, if anything, studies show the opposite of what the jam experiment purports to show.

Why, as it turns out, even the Freakonomics peoplewrote about it. What is especially funny about their post is that they clearly love the counter-intuitive finding, admit that it fits their preconceptions — and then quote someone showing the study’s results cannot be replicated consistently . . . and then . . . and then conclude that, hey, maybe people should streamline choices anyway:

So even if jam studies of the future prove inconclusive, it still seems wise to streamline choices whose complexity might otherwise hamper a good outcome.

In other words, the result is just so much fun, it’s a shame to toss it overboard just because it cannot be shown to be accurate.

People love this so much, I thought I would look for Manzi talking about this in a written format, and let him make the argument himself, rather than have me report to you about something I heard in a podcast. So I found Manzi discussing this in more detail at the Corner and thought it was worth quoting at length:

What are the odds that we would see one randomly chosen group of about 100 of the people who were given a coupon have a redemption rate that is ten times as large as another similarly sized random group of people given the exact same coupon? It’s larger than you might think. Consider an example. A recent in-store coupon executed by a large-format grocery-store chain was distributed to more than 1.3 million shoppers. I randomly divided them into about 13,000 groups of 100 shoppers each. I then randomly paired each of these groups with one other, creating about 6,500 randomly matched pairs of randomly selected groups of 100 shoppers. In a little over 9 percent of these pairings, the redemption rate was at least ten times as high in one group as in its matched pair. The jam experiment, by this simplified and indicative metric, would fail to achieve standard measures of statistical confidence required to reject the hypothesis that this was just random variation.

And while the specifics will vary for any given coupon – based on characteristics like product category, average redemption rate, time of year, and so forth – this indicative analysis almost certainly understates the actual probability of seeing this much difference between the two groups in the experiment. The two groups of jam buyers were not assigned randomly. Because the experiment was done for a total of ten hours in only one store, and because shoppers were grouped in hourly chunks, there are all kinds of reasons why the people who happened to show up during the five hours of limited assortment might have different propensity to respond to one-dollar-off coupons for a specific line of jams than those who arrived in the other five hour period. Maybe a soccer game finished at some specific time, and several of the parents who share similar propensities versus the average shopper came in nearly together, or maybe a bad traffic jam in a part of town with non-average propensity to respond to the coupon dissuaded several people from going to the store at one time versus another. Remember, all of the inference is built on the purchase of a grand total of 35 jars of jam. This is one reason why rigorous retail experiments, when a lot of money is at stake, are typically executed for dozens of randomly assigned stores for a period of weeks — and even sample sizes like that are pushing the envelope of causal inference.

But the result is at least interesting, and the right way to figure out whether or not the result is valid and generalizable is replication. Over the past ten years, a number of such experiments have been done by academics to evaluate the asserted paradox of choice for product categories ranging from mp3 players to mutual funds, and a paper was published in February (Scheibehenne, et al.) that conducted a meta-analysis of 50 of them (h/t Tim Harford). Across all of these experiments, the average effect of increasing choice on consumption or satisfaction was “virtually zero.” Further, this meta-analysis showed a positive average effect of increasing choices for those experiments that, like the jam experiment, tested the effect of choice on consumption quantity, rather than some measure of satisfaction, as the outcome. That is, when it comes to sales, more choice is better.

This is consistent with all of the unpublished assortment experiments that I’ve seen, and should not be surprising. As a store adds more and more products to a given product line assortment – say, canned soup – sales will rise sub-linearly with product count.

The key, again, is whether repeated experiments produce a predictable result — not how much fun the answer is, or whether it is in line with your preconceptions.

This reminds me of when Walmart made the ‘strategic decision’ to limit the brands and number of options they carried on their shelves. For example they would only carry Great Value and Glad brand garbage bags but not Hefty-thus limiting the choice and hoping to steer customers to their store brands. If I recall correctly they lost business and have since reversed this decision. I am pretty sure we can all agree Walmart knows a thing of two about making $$$.

The National Recovery Administration (NRA) was the primary New Deal agency established by U.S. president Franklin D. Roosevelt (FDR) in 1933. The goal was to eliminate “cut-throat competition” by bringing industry, labor and government together to create codes of “fair practices” and set prices. The NRA was created by the National Industrial Recovery Act (NIRA) and allowed industries to get together and write “codes of fair competition.” The codes were intended to reduce “destructive competition” and to help workers by setting minimum wages and maximum weekly hours, as well as minimum prices at which products could be sold…

In 1935, the U.S. Supreme Court unanimously declared that the NRA law was unconstitutional, ruling that it infringed the separation of powers under the United States Constitution. The NRA quickly stopped operations, but many of its labor provisions reappeared in the National Labor Relations Act (Wagner Act), passed later the same year. The long-term result was a surge in the growth and power of unions, which became a core of the New Deal Coalition that dominated national politics for the next three decades.

The NRA, symbolized by the Blue Eagle, was popular with workers. Businesses that supported the NRA put the symbol in their shop windows and on their packages, though they did not always go along with the regulations entailed. Though membership to the NRA was voluntary, businesses that did not display the eagle were very often boycotted, making it seem mandatory for survival to many.

Hey, if Obama recreated this, saving us all those nasty choices, bet you it gets through the Supreme Court this time, 5-4. And as an additional bonus, if the NRA proves unpopular, you can always blame it on that evil gun lobby group.

But, what the NRA did was “price fixing”, and didn’t allow industry to adjust to where the market was.
It, and the ideology behind it, extended the Great Depression, which you would know if you had read “The Forgotten Man”, by Amity Shlaes.

There is one counterexample to this, a case where more choice really does paralyze people — and that’s people coming from former communist countries, or third-world countries, to the U.S. If they’re not used to the dizzying array of choices in the grocery store, they really can get decision paralysis. It happens even to Americans who’ve spent significant time overseas: I was talking just a month ago with an American who’d spent several years in Papua New Guinea, where clothes shopping was extremely simple. If one of the 2-3 shirts the store had available fit you, you bought it right then and there no matter what the design looked like — because if you came back an hour later, someone else might have bought it and you’d have missed your chance. So when he went clothes shopping in America, he said he spent 20 minutes just starting around not knowing where to even start.

But for your average American who’s grown up in this environment, the results are entirely different, naturally.

FDR stole the whole NRA concept from Mussolini. Just as he stole the rest of his 100 Days from Herbert Hoover. Unfortunately for the country, what he “stole” was bad ideas and inpractical programs. That is why the Great Depression lasted so long.

I sometimes “want what I want” and will look through a bunch of choices to find it, but I generally don’t like looking through lots of choices if I don’t already have an idea of what I want. I remember distinctly not buying Klondike Bars on sale once because there were so many types to choose from I couldn’t decide. Of course, my decision was complicated by not choosing just for me but wanting something that everyone in the family would like. That leads me to want to buy several types so people at home can have their choice, but that often gets me criticism for buying too much and not being able to make up my mind and settle on something.

So, analyzing all of those factors into predicting my behavior on any given day is about as helpful as trying to determine how much rainfall will fall on 1 square foot of my backyard.
On the other hand, trying to do a broad study can be helpful for the purpose of a store predicting a general response that would be profitable, I guess.

We do most of our shopping at a discount grocery store with limited options and shop at major stores only for specific sale items. If I had to do all of my shopping at a big store with many selections I would get lost.

Then again part of the issue is if a person is buying for themself alone or for many people. I think it is easier for someone to pick one thing out of 30 if they are buying only for themself, trying to buy one thing out of thirty to please all of 4 people, say, is more challenging, I think.

It’s unsurprising that the NYT and the “Freakonomics” idiots and the like love this idea. The whole “capitalism wastes resources creating so many variations of the same product” BS is classic Soviet propaganda. It’s how they explained away the bare shelves in their stores — it was in the interest of efficiency! Never mind that the stores for the apparat were stocked with goods from the Free World.

They’ll chortle over this kind of crap all day, but don’t you DARE tell them they have to shop at Ralph’s instead of Whole Foods.

While replication is key, replication is darn near impossible to truly achieve… partly because of the difficulty in setting up controlled experiments but mostly because one is never sure what conditions are relevant to the experiment. Time of day? Whether there was an NFL game on TV? Weather? The stock market going up or down the day before? Maybe they’re relevant, maybe not, but all anybody knows is that there is no way of establishing even a preponderance of evidence that Result A is due to conditions X, Y or Z.

Given that no one can truly know what is cause, what is correlation and what is coincidence, what is the basis for believing anyone when they say that they really, really, really know the reason something happened? Like, for example, all of you who swear Romney lost because he wasn’t conservative enough?

I read something similar years ago (who am I kidding, it was decades dammit) where the study was about stress and the experiment used toothpaste. They found that the more brands of toothpaste that was on display the stress levels of the subject went up a tiny bit. The more toothpaste choice, the more stress. I don’t recall whether it impacted their buying habits but it was interesting. Maybe that explains why I’m a gibbering mess when I finish shopping. Then again it could be the prices.

From what I can gather, apparently Soledad’s intelligence and deft interviewing skills have failed to resonate with viewers– therefore having the sad result that her show’s ratings continue to be rather disapponting to the network brass.

I like the Freakonomics people ok and sometimes they give good insight. Other times, they give simple explanations to complex phenomenon that do not apply as a general rule. The choice issue becomes big if you have very limited time to make a choice. To process the information, determine which option matches your preference, etc. all take time. Without artificial constraints on time, most people prefer wider choice and can make better ones matching their preferences. Think booking airline flights for example. You don’t care what time, what day, etc. but getting from pt. a to pt. b, you can use something like priceline or hotline. If you care about which carrier, day, time, airport, etc. then you gather information from a multiplicity of sources but it takes more time.

What bothers me is that I suspect they are trying to set the meme that the govt. needs to take over your 401k and 403b plans because you can’t make the necessary choices for your retirement. This load of bs comes directly from Cass Sunstein and his nudge thesis.