kottke.org posts about James Surowiecki

The food is fresh. Natural. Locally sourced. Sometimes even organic. That might sound like your local farmer's market, but it's actually part of a new and growing movement in the fast-food industry. Think Shake Shack, Chipotle, Panera. While we're not exactly seeing tractors in the drive-thrus, the rise of these chains (and the pressure on their predecessors that placed a lot more emphasis on the fast than the food) tell us a lot about economic inequality, the modern workday, and fries. From The New Yorker's James Surowiecki: The Shake Shack Economy.

It seems that con artists, for all their vices, represent many of the virtues that Americans aspire to. Con artists are independent and typically self-made. They don't have to kowtow to a boss -- no small thing in a country in which people have always longed to strike out on their own. They succeed or fail based on their wits. They exemplify, in short, the complicated nature of American capitalism, which, as McDougall argues, has depended on people being hustlers in both the positive and the negative sense. The American economy wasn't built just on good ideas and hard work. It was also built on hope and hype.

There are a lot of lobsters in the sea. You could even call it a glut. Over the past few years, the massive lobster harvests have resulted in a significant reduction in what buyers are paying for a lobster off the boat. So why aren't we seeing major price drops at our local restaurants? Here's part of the reason: A luxury good is considered a luxury good in part because it's priced like one. Cheap lobster could throw the rest of your menu into chaos.

Studies have shown that people prefer inexpensive wines in blind taste tests, but that they actually get more pleasure from drinking wine they are told is expensive. If lobster were priced like chicken, we might enjoy it less.

The truth is that if Reddit is actually interested in using the power of its crowd to help the authorities, it needs to dramatically rethink its approach, because the process it used to try to find the bombers wasn't actually tapping the wisdom of crowds at all -- at least not as I would define that wisdom. For a crowd to be smart, the people in it need to be not only diverse in their perspectives but also, relatively speaking, independent of each other. In other words, you need people to be thinking for themselves, rather than following the lead of those around them.

When the book came out in 2004, I wrote a short post that summarizes the four main conditions you need for a wise crowd. What's striking about most social media and software, as Surowiecki notes in the case of Reddit, is how most of these conditions are not satisfied. There's little diversity and independence: Twitter and Facebook mostly show you people who are like you and things your social group is into. And social media is becoming ever more centralized: Facebook, Twitter, Tumblr, Medium, Pinterest, etc. instead of a decentralized network of independent blogs. In fact, the nature of social media is to be centralized, peer-dependent, and homogeneous because that's how people naturally group themselves together. It's a wonder the social media crowd ever gets anything right.

It's really in the seventh century B.C.E., when the small kingdom of Lydia introduced the world's first standardized metal coins, that you start to see money being used in a recognizable way. Located in what is now Turkey, Lydia sat on the cusp between the Mediterranean and the Near East, and commerce with foreign travelers was common. And that, it turns out, is just the kind of situation in which money is quite useful.

To understand why, imagine doing a trade in the absence of money-that is, through barter. (Let's leave aside the fact that no society has ever relied solely or even largely on barter; it's still an instructive concept.) The chief problem with barter is what economist William Stanley Jevons called the "double coincidence of wants." Say you have a bunch of bananas and would like a pair of shoes; it's not enough to find someone who has some shoes or someone who wants some bananas. To make the trade, you need to find someone who has shoes he's willing to trade and wants bananas. That's a tough task.

With a common currency, though, the task becomes easy: You just sell your bananas to someone in exchange for money, with which you then buy shoes from someone else. And if, as in Lydia, you have foreigners from whom you'd like to buy or to whom you'd like to sell, having a common medium of exchange is obviously valuable. That is, money is especially useful when dealing with people you don't know and may never see again.

In this same vein, this reply on Reddit to "Where has all the money in the world gone?" is also worth a read.

The thing to remember is that all throughout, from the initial trade to this central-banking system, all of this money is debt. It is IOUs, except instead of being an IOU that says "Kancho_Ninja will give one bushel of apples to the bearer of this bond in October", it says "Anyone in town will give you anything worth one bushel of apples in trade."

The money is not an actual thing that you can eat or wear or build a house with, it's an IOU that is redeemable anywhere, for anything, from anyone. It is a promise to pay equivalent value at some time in the future, except the holder of the money can call on anybody at all to fulfill that promise -- they don't have to go back to the original promiser.

The big challenge for any retailer is to make sure that the people coming into the store actually buy stuff, and research suggests that not scrimping on payroll is crucial. In a study published at the Wharton School, Marshall Fisher, Jayanth Krishnan, and Serguei Netessine looked at detailed sales data from a retailer with more than five hundred stores, and found that every dollar in additional payroll led to somewhere between four and twenty-eight dollars in new sales. Stores that were understaffed to begin with benefitted more, stores that were close to fully staffed benefitted less, but, in all cases, spending more on workers led to higher sales. A study last year of a big apparel chain found that increasing the number of people working in stores led to a significant increase in sales at those stores.

The problem -- in Blockbuster's case, at least -- was that the very features that people thought were strengths turned out to be weaknesses. Blockbuster's huge investment, both literally and psychologically, in traditional stores made it slow to recognize the Web's importance: in 2002, it was still calling the Net a "niche" market. And it wasn't just the Net. Blockbuster was late on everything -- online rentals, Redbox-style kiosks, streaming video. There was a time when customers had few alternatives, so they tolerated the chain's limited stock, exorbitant late fees (Blockbuster collected about half a billion dollars a year in late fees), and absence of good advice about what to watch. But, once Netflix came along, it became clear that you could have tremendous variety, keep movies as long as you liked, and, thanks to the Netflix recommendation engine, actually get some serviceable advice. (Places like Netflix and Amazon have demonstrated the great irony that computer algorithms can provide a more personalized and engaging customer experience than many physical stores.) Then Redbox delivered the coup de grace, offering new Hollywood releases for just a dollar.

A similar phenomenon is at work in an experiment run by a group including the economist George Loewenstein, in which people were asked to pick one movie to watch that night and one to watch at a later date. Not surprisingly, for the movie they wanted to watch immediately, people tended to pick lowbrow comedies and blockbusters, but when asked what movie they wanted to watch later they were more likely to pick serious, important films. The problem, of course, is that when the time comes to watch the serious movie, another frothy one will often seem more appealing. This is why Netflix queues are filled with movies that never get watched: our responsible selves put "Hotel Rwanda" and "The Seventh Seal" in our queue, but when the time comes we end up in front of a rerun of "The Hangover."

The lesson of these experiments is not that people are shortsighted or shallow but that their preferences aren't consistent over time. We want to watch the Bergman masterpiece, to give ourselves enough time to write the report properly, to set aside money for retirement. But our desires shift as the long run becomes the short run.

Companies who target the middle of the market (Sony, Dell, General Motors) are losing customers to companies like Apple & Hermes at the high end and Ikea & H&M at the low end. From James Surowiecki:

The products made by midrange companies are neither exceptional enough to justify premium prices nor cheap enough to win over value-conscious consumers. Furthermore, the squeeze is getting tighter every day. Thanks to economies of scale, products that start out mediocre often get better without getting much more expensive -- the newest Flip, for instance, shoots in high-def and has four times as much memory as the original -- so consumers can trade down without a significant drop in quality. Conversely, economies of scale also allow makers of high-end products to reduce prices without skimping on quality. A top-of-the-line iPod now features video and four times as much storage as it did six years ago, but costs a hundred and fifty dollars less. At the same time, the global market has become so huge that you can occupy a high-end niche and still sell a lot of units. Apple has just 2.2 per cent of the world cell-phone market, but that means it sold twenty-five million iPhones last year.

Of course, everyone knows that art and culture help make New York a great place to live. But Currid goes much further, showing that the culture industry creates tremendous economic value in its own right. It is the city's fourth-largest employer, and generates billions of dollars a year in revenue. More important, New York has no real global rival for dominance in the culture industry. Using an economic-analysis tool called a "location quotient," Currid calculates that New York matters far more to fashion, art, and culture than to finance. To exaggerate a bit, if New York suddenly disappeared, stock markets could keep functioning, but we would not be able to dress ourselves or find art to put on the wall. Currid suggests that, in the fight among cities for business, being the center of fashion and art constitutes New York's true "competitive advantage."

New York's cultural economy has reached a critical juncture, argues Ms Currid, threatened by, of all things, prosperity. The bleak economic conditions of the 1970s allowed artists to flock into dirt-cheap apartments and ushered in the East Village scene of the early 1980s. The boom of the past decade, by contrast, has priced budding Basquiats out of Manhattan, pushing them across the water to Brooklyn and New Jersey. Studio flats meant for artists-in-residence get snapped up by bankers. The closure last year of CBGB, a bar that became a punk and art-rock laboratory in the 1970s (and whose founder, Hilly Kristal, died last month) came to symbolise this squeeze.

Ms Currid sees this expulsion of talent as a serious problem. The solution, she argues, lies in a series of well-aimed public-policy measures: tax incentives, zoning that helps nightlife districts, more subsidised housing and studio space for up-and-coming artists, and more.

One of the causes of feature creep in products like consumer electronics is that when customers are making purchase decisions, they'll likely choose the one with the most features. "But, when they were asked to use the digital device, so-called 'feature fatigue' set in. They became frustrated with the plethora of options they had created, and ended up happier with a simpler product."

Tariffs and quotas are extremely hard to get rid of, once established, because they create a vicious circle of back-scratching-government largesse means that sugar producers get wealthy, giving them lots of cash to toss at members of Congress, who then have an incentive to insure that the largesse continues to flow. More important, protectionist rules flourish because the benefits are concentrated among a small number of easy-to-identify winners, while the costs are spread out across the entire population. It may be annoying to pay a few more cents for sugar or ethanol, but most of us are unlikely to lobby Congress about it.

Maybe we should, though. Our current policy is absurd even by Washington standards: Congress is paying billions in subsidies to get us to use more ethanol, while keeping in place tariffs and quotas that guarantee that we'll use less. And while most of the time tariffs just mean higher prices and reduced competition, in the case of ethanol the negative effects are considerably greater, leaving us saddled with an inferior and less energy-efficient technology and as dependent as ever on oil-producing countries.

Maddening. Partisan politics is a not-very-elaborate smokescreen to distract us from this bullshit.

Surowiecki on the difficulty of short-term thinking in business. "It's no wonder that management theory is dominated by fads: every few years, new companies succeed, and they are scrutinized for the underlying truths that they might reveal. But often there is no underlying truth; the companies just happened to be in the right place at the right time."

James Surowiecki fills us in on a new investment opportunity, housing futures. "If housing futures work the way they're supposed to, they will shift risk from those who are less able to bear it (individual homeowners with hefty mortgages) to those who are more willing to (speculators looking for a big upside on their investments). In the process, they will effectively provide a form of house-price insurance."

CEO pay and perks can be a good indicator of how healthy a business is, so it makes sense that investors are interested in just exactly how much chief executives make. "We shouldn't expect to see a dent in executive compensation anytime soon. But in the long run companies that don't balance pay with performance tend to suffer where it matters most -- in the stock market."

Everything we do to "edit" the [Guardian Unlimited] site seeks to keep a balance between editorial instinct and the desires of the audience, and that, in doing that, we may be reflecting the "community" more fairly, both mathematically and ethically, than the likes of digg.

In order for a crowd to be smart, [Surowiecki] says it needs to satisfy four conditions: 1. Diversity, 2. Independence, 3. Decentralization, and 4. Aggregation.

Much of the online media we're familiar with uses a mix of humans and automated systems to perform the aggregating task. Human editors choose the stories that will run in the newspaper (drawing from a number of sources of information as Lloyd illustrated), blog authors select what links and posts to put on their blog (by reading other blogs & media outlets, listening to reader feedback, and sifting through already aggregated sources like del.icio.us or Digg), and the editors of Slashdot filter through hundreds of reader submissions a day to create Slashdot's front page. Google News uses technology to decide which stories are important, based primarily on what the publishers are publishing. Digg and del.icio.us rely almost entirely on the crowd to submit and determine by a simple vote what stories go on its front page.

Some of these methods work better than others for different tasks. The product of 50,000 diverse, independent, decentralized bloggers is probably more editorially interesting, fair, and complete than that of 50,000 diverse, independent, decentralized Digg users, but the Digg vote & tally approach is less time-intensive for all concerned and the information flows faster. A site like Slashdot sits in the middle...it's a little slower than Digg but offers a more consistent editorial product. A hybrid Digg+Slashdot approach (which is not unlike the one used by individual bloggers) would be for Digg to produce a "Digg digest", a human selected (could use simple voting or let the most highly respected community members choose) collection of the best stories of the day that incorporates what was said in the comments and around the web as well. Actually, I think if you wanted to start a blog that did this, it would do very well.

Surowiecki on the economics of textbooks, i.e. why they cost so damn much. "You can often buy the same textbook abroad for significantly less than it costs in the U.S., so students have learned to buy directly from places like the U.K., and a host of small businesses have sprung up to import books."

The wisdom of crowds you say? As Surowiecki explains, yes, but only under the right conditions. In order for a crowd to be smart, he says it needs to satisfy four conditions:

1. Diversity. A group with many different points of view will make better decisions than one where everyone knows the same information. Think multi-disciplinary teams building Web sites...programmers, designers, biz dev, QA folks, end users, and copywriters all contributing to the process, each has a unique view of what the final product should be. Contrast that with, say, the President of the US and his Cabinet.

3. Decentralization. "Power does not fully reside in one central location, and many of the important decisions are made by individuals based on their own local and specific knowledge rather than by an omniscient or farseeing planner." The open source software development process is an example of effect decentralization in action.

4. Aggregation. You need some way of determining the group's answer from the individual responses of its members. The evils of design by committee are due in part to the lack of correct aggregation of information. A better way to harness a group for the purpose of designing something would be for the group's opinion to be aggregated by an individual who is skilled at incorporating differing viewpoints into a single shared vision and for everyone in the group to be aware of that process (good managers do this). Aggregation seems to be the most tricky of the four conditions to satisfy because there are so many different ways to aggregate opinion, not all of which are right for a given situation.

Satisfy those four conditions and you've hopefully cancelled out some of the error involved in all decision making:

If you ask a large enough group of diverse, independent people to make a prediciton or estimate a probability, and then everage those estimates, the errors of each of them makes in coming up with an answer will cancel themselves out. Each person's guess, you might say, has two components: information and error. Subtract the error, and you're left with the information.