I'm David Rosenthal, and this is a place to discuss the work I'm doing in Digital Preservation.

Thursday, August 24, 2017

Why Is The Web "Centralized"?

There is a groundswell of opinion, which I share, in favor of a "decentralized Web" that has continued after last year's "Decentralized Web Summit". A wealth of different technologies for implementing a decentralized Web are competing for attention. But the basic protocols of the Internet and the Web (IP, TCP, DNS, HTTP, ...) aren't centralized. What is the centralization that decentralized Web advocates are reacting against? Clearly, it is the domination of the Web by the FANG (Facebook, Amazon, Netflix, Google) and a few other large companies such as the cable oligopoly.

These companies came to dominate the Web for economic not technological reasons. The Web, like other technology markets, has very large increasing returns to scale (network effects, duh!). These companies build centralized systems using technology that isn't inherently centralized but which has increasing returns to scale. It is the increasing returns to scale that drive the centralization.

Unless decentralized technologies specifically address the issue of how to avoid increasing returns to scale they will not, of themselves, fix this economic problem. Their increasing returns to scale will drive layering centralized businesses on top of decentralized infrastructure, replicating the problem we face now, just on different infrastructure.

"The early days of the internet were built on protocols -- and the power was in the end-to-end nature of things. But with those protocols, people could build their own implementations and software to work with those protocols. The power was thus at the ends. Individuals could choose how they interact with the protocols and they could implement their own solutions without being completely cut off. You could filter out the content you didn't want. But the choice was yours. Over the last decade, especially, we've moved far away from that ideal (in part because there appears to be more money in locked-in, centralized platforms, rather than more distributed protocols). But, opening things up offers some opportunity to allow good things to happen."

But note that he doesn't seem curious about why there is "more money in locked-in, centralized platforms". Without understanding why this is the case (hint: increasing returns to scale), how can we know whether fixing the protocols is a fix for the problem?

"The early internet was designed to be decentralized. It treated all content and all content owners equally. That equality had value in society, as it kept the playing field level and encouraged new entrants. But decentralization had a cost: no one had an incentive to make internet tools easy to use. Frustrated by those tools, users embraced easy-to-use alternatives from Facebook and Google. This allowed the platforms to centralize the internet, inserting themselves between users and content, effectively imposing a tax on both sides. This is a great business model for Facebook and Google—and convenient in the short term for customers—but we are drowning in evidence that there are costs that society may not be able to afford."

From Roger McNamee's must-read Washington Monthly article How to Fix Facebook—Before It Fixes Us. He's right about "easy to use" but that isn't the only reason. Decentralized systems are more difficult to make easy to use than centralized ones, because they have coordination overheads and latencies. And the components are smaller than in a centralized system, so they are more expensive because they lack economies of scale.