NEWSPAPERS AND THINKING THE UNTHINKABLE

When reality is labeled unthinkable, it creates a kind of sickness in an industry. Leadership becomes faith-based, while employees who have the temerity to suggest that what seems to be happening is in fact happening are herded into Innovation Departments, where they can be ignored en masse. This shunting aside of the realists in favor of the fabulists has different effects on different industries at different times. One of the effects on the newspapers is that many of their most passionate defenders are unable, even now, to plan for a world in which the industry they knew is visibly going away.

CLAY SHIRKY is an adjunct professor in NYU's graduate Interactive Telecommunications Program (ITP), where he teaches courses on the interrelated effects of social and technological network topology — how our networks shape culture and vice-versa. He is the author of Here Comes Everybody.

Back in 1993, the Knight-Ridder newspaper chain began investigating piracy of Dave Barry's popular column, which was published by the Miami Herald and syndicated widely. In the course of tracking down the sources of unlicensed distribution, they found many things, including the copying of his column to alt.fan.dave_barry on usenet; a 2000-person strong mailing list also reading pirated versions; and a teenager in the Midwest who was doing some of the copying himself, because he loved Barry's work so much he wanted everybody to be able to read it.

One of the people I was hanging around with online back then was Gordy Thompson, who managed internet services at the New York Times. I remember Thompson saying something to the effect of "When a 14 year old kid can blow up your business in his spare time, not because he hates you but because he loves you, then you got a problem." I think about that conversation a lot these days.

The problem newspapers face isn't that they didn't see the internet coming. They not only saw it miles off, they figured out early on that they needed a plan to deal with it, and during the early 90s they came up with not just one plan but several. One was to partner with companies like America Online, a fast-growing subscription service that was less chaotic than the open internet. Another plan was to educate the public about the behaviors required of them by copyright law. New payment models such as micropayments were proposed. Alternatively, they could pursue the profit margins enjoyed by radio and TV, if they became purely ad-supported. Still another plan was to convince tech firms to make their hardware and software less capable of sharing, or to partner with the businesses running data networks to achieve the same goal. Then there was the nuclear option: sue copyright infringers directly, making an example of them.

As these ideas were articulated, there was intense debate about the merits of various scenarios. Would DRM or walled gardens work better? Shouldn't we try a carrot-and-stick approach, with education and prosecution? And so on. In all this conversation, there was one scenario that was widely regarded as unthinkable, a scenario that didn't get much discussion in the nation's newsrooms, for the obvious reason.

The unthinkable scenario unfolded something like this: The ability to share content wouldn't shrink, it would grow. Walled gardens would prove unpopular. Digital advertising would reduce inefficiencies, and therefore profits. Dislike of micropayments would prevent widespread use. People would resist being educated to act against their own desires. Old habits of advertisers and readers would not transfer online. Even ferocious litigation would be inadequate to constrain massive, sustained law-breaking. (Prohibition redux.) Hardware and software vendors would not regard copyright holders as allies, nor would they regard customers as enemies. DRM's requirement that the attacker be allowed to decode the content would be an insuperable flaw. And, per Thompson, suing people who love something so much they want to share it would piss them off.

Revolutions create a curious inversion of perception. In ordinary times, people who do no more than describe the world around them are seen as pragmatists, while those who imagine fabulous alternative futures are viewed as radicals. The last couple of decades haven't been ordinary, however. Inside the papers, the pragmatists were the ones simply looking out the window and noticing that the real world was increasingly resembling the unthinkable scenario. These people were treated as if they were barking mad. Meanwhile the people spinning visions of popular walled gardens and enthusiastic micropayment adoption, visions unsupported by reality, were regarded not as charlatans but saviors.

When reality is labeled unthinkable, it creates a kind of sickness in an industry. Leadership becomes faith-based, while employees who have the temerity to suggest that what seems to be happening is in fact happening are herded into Innovation Departments, where they can be ignored en masse. This shunting aside of the realists in favor of the fabulists has different effects on different industries at different times. One of the effects on the newspapers is that many of their most passionate defenders are unable, even now, to plan for a world in which the industry they knew is visibly going away.

~~~

The curious thing about the various plans hatched in the '90s is that they were, at base, all the same plan: "Here's how we're going to preserve the old forms of organization in a world of cheap perfect copies!" The details differed, but the core assumption behind all imagined outcomes (save the unthinkable one) was that the organizational form of the newspaper, as a general-purpose vehicle for publishing a variety of news and opinion, was basically sound, and only needed a digital facelift. As a result, the conversation has degenerated into the enthusiastic grasping at straws, pursued by skeptical responses.

"The Wall Street Journal has a paywall, so we can too!" (Financial information is one of the few kinds of information whose recipients don't want to share.) "Micropayments work for iTunes, so they will work for us!" (Micropayments only work where the provider can avoid competitive business models.) "The New York Times should charge for content!" (They've tried, with QPass and later TimesSelect.) "Cook's Illustrated and Consumer Reports are doing fine on subscriptions!" (Those publications forgo ad revenues; users are paying not just for content but for unimpeachability.) "We'll form a cartel!" (…and hand a competitive advantage to every ad-supported media firm in the world.)

Round and round this goes, with the people committed to saving newspapers demanding to know "If the old model is broken, what will work in its place?" To which the answer is: Nothing. Nothing will work. There is no general model for newspapers to replace the one the internet just broke.

With the old economics destroyed, organizational forms perfected for industrial production have to be replaced with structures optimized for digital data. It makes increasingly less sense even to talk about a publishing industry, because the core problem publishing solves — the incredible difficulty, complexity, and expense of making something available to the public — has stopped being a problem.

~~~

Elizabeth Eisenstein's magisterial treatment of Gutenberg's invention, The Printing Press as an Agent of Change, opens with a recounting of her research into the early history of the printing press. She was able to find many descriptions of life in the early 1400s, the era before movable type. Literacy was limited, the Catholic Church was the pan-European political force, Mass was in Latin, and the average book was the Bible. She was also able to find endless descriptions of life in the late 1500s, after Gutenberg's invention had started to spread. Literacy was on the rise, as were books written in contemporary languages, Copernicus had published his epochal work on astronomy, and Martin Luther's use of the press to reform the Church was upending both religious and political stability.

What Eisenstein focused on, though, was how many historians ignored the transition from one era to the other. To describe the world before or after the spread of print was child's play; those dates were safely distanced from upheaval. But what was happening in 1500? The hard question Eisenstein's book asks is "How did we get from the world before the printing press to the world after it? What was the revolution itself like?"

Chaotic, as it turns out. The Bible was translated into local languages; was this an educational boon or the work of the devil? Erotic novels appeared, prompting the same set of questions. Copies of Aristotle and Galen circulated widely, but direct encounter with the relevant texts revealed that the two sources clashed, tarnishing faith in the Ancients. As novelty spread, old institutions seemed exhausted while new ones seemed untrustworthy; as a result, people almost literally didn't know what to think. If you can't trust Aristotle, who can you trust?

During the wrenching transition to print, experiments were only revealed in retrospect to be turning points. Aldus Manutius, the Venetian printer and publisher, invented the smaller octavo volume along with italic type. What seemed like a minor change — take a book and shrink it — was in retrospect a key innovation in the democratization of the printed word. As books became cheaper, more portable, and therefore more desirable, they expanded the market for all publishers, heightening the value of literacy still further.

That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place. The importance of any given experiment isn't apparent at the moment it appears; big changes stall, small changes spread. Even the revolutionaries can't predict what will happen. Agreements on all sides that core institutions must be protected are rendered meaningless by the very people doing the agreeing. (Luther and the Church both insisted, for years, that whatever else happened, no one was talking about a schism.) Ancient social bargains, once disrupted, can neither be mended nor quickly replaced, since any such bargain takes decades to solidify.

And so it is today. When someone demands to know how we are going to replace newspapers, they are really demanding to be told that we are not living through a revolution. They are demanding to be told that old systems won't break before new systems are in place. They are demanding to be told that ancient social bargains aren't in peril, that core institutions will be spared, that new methods of spreading information will improve previous practice rather than upending it. They are demanding to be lied to.

There are fewer and fewer people who can convincingly tell such a lie.

~~~

If you want to know why newspapers are in such trouble, the most salient fact is this: Printing presses are terrifically expensive to set up and to run. This bit of economics, normal since Gutenberg, limits competition while creating positive returns to scale for the press owner, a happy pair of economic effects that feed on each other. In a notional town with two perfectly balanced newspapers, one paper would eventually generate some small advantage — a breaking story, a key interview — at which point both advertisers and readers would come to prefer it, however slightly. That paper would in turn find it easier to capture the next dollar of advertising, at lower expense, than the competition. This would increase its dominance, which would further deepen those preferences, repeat chorus. The end result is either geographic or demographic segmentation among papers, or one paper holding a monopoly on the local mainstream audience.

For a long time, longer than anyone in the newspaper business has been alive in fact, print journalism has been intertwined with these economics. The expense of printing created an environment where Wal-Mart was willing to subsidize the Baghdad bureau. This wasn't because of any deep link between advertising and reporting, nor was it about any real desire on the part of Wal-Mart to have their marketing budget go to international correspondents. It was just an accident. Advertisers had little choice other than to have their money used that way, since they didn't really have any other vehicle for display ads.

The old difficulties and costs of printing forced everyone doing it into a similar set of organizational models; it was this similarity that made us regard Daily Racing Form and L'Osservatore Romano as being in the same business. That the relationship between advertisers, publishers, and journalists has been ratified by a century of cultural practice doesn't make it any less accidental.

The competition-deflecting effects of printing cost got destroyed by the internet, where everyone pays for the infrastructure, and then everyone gets to use it. And when Wal-Mart, and the local Maytag dealer, and the law firm hiring a secretary, and that kid down the block selling his bike, were all able to use that infrastructure to get out of their old relationship with the publisher, they did. They'd never really signed up to fund the Baghdad bureau anyway.

~~~

Print media does much of society's heavy journalistic lifting, from flooding the zone — covering every angle of a huge story — to the daily grind of attending the City Council meeting, just in case. This coverage creates benefits even for people who aren't newspaper readers, because the work of print journalists is used by everyone from politicians to district attorneys to talk radio hosts to bloggers. The newspaper people often note that newspapers benefit society as a whole. This is true, but irrelevant to the problem at hand; "You're gonna miss us when we're gone!" has never been much of a business model. So who covers all that news if some significant fraction of the currently employed newspaper people lose their jobs?

I don't know. Nobody knows. We're collectively living through 1500, when it's easier to see what's broken than what will replace it. The internet turns 40 this fall. Access by the general public is less than half that age. Web use, as a normal part of life for a majority of the developed world, is less than half that age. We just got here. Even the revolutionaries can't predict what will happen.

Imagine, in 1996, asking some net-savvy soul to expound on the potential of craigslist, then a year old and not yet incorporated. The answer you'd almost certainly have gotten would be extrapolation: "Mailing lists can be powerful tools", "Social effects are intertwining with digital networks", blah blah blah. What no one would have told you, could have told you, was what actually happened: craiglist became a critical piece of infrastructure. Not the idea of craigslist, or the business model, or even the software driving it. Craigslist itself spread to cover hundreds of cities and has become a part of public consciousness about what is now possible. Experiments are only revealed in retrospect to be turning points.

In craigslist's gradual shift from 'interesting if minor' to 'essential and transformative', there is one possible answer to the question "If the old model is broken, what will work in its place?" The answer is: Nothing will work, but everything might. Now is the time for experiments, lots and lots of experiments, each of which will seem as minor at launch as craigslist did, as Wikipedia did, as octavo volumes did.

Journalism has always been subsidized. Sometimes it's been Wal-Mart and the kid with the bike. Sometimes it's been Richard Mellon Scaife. Increasingly, it's you and me, donating our time. The list of models that are obviously working today, like Consumer Reports and NPR, like ProPublica and WikiLeaks, can't be expanded to cover any general case, but then nothing is going to cover the general case.

Society doesn't need newspapers. What we need is journalism. For a century, the imperatives to strengthen journalism and to strengthen newspapers have been so tightly wound as to be indistinguishable. That's been a fine accident to have, but when that accident stops, as it is stopping before our eyes, we're going to need lots of other ways to strengthen journalism instead.

When we shift our attention from 'save newspapers' to 'save society', the imperative changes from 'preserve the current institutions' to 'do whatever works.' And what works today isn't the same as what used to work.

We don't know who the Aldus Manutius of the current age is. It could be Craig Newmark, or Caterina Fake. It could be Martin Nisenholtz, or Emily Bell. It could be some 19 year old kid few of us have heard of, working on something we won't recognize as vital until a decade hence. Any experiment, though, designed to provide new models for journalism is going to be an improvement over hiding from the real, especially in a year when, for many papers, the unthinkable future is already in the past.

For the next few decades, journalism will be made up of overlapping special cases. Many of these models will rely on amateurs as researchers and writers. Many of these models will rely on sponsorship or grants or endowments instead of revenues. Many of these models will rely on excitable 14 year olds distributing the results. Many of these models will fail. No one experiment is going to replace what we are now losing with the demise of news on paper, but over time, the collection of new experiments that do work might give us the journalism we need.

Reality Club Discussion

We'd like to speak to our billionaire readers — there must be one or two of you left — about the current buyer's market for prestige. Everyone else, please skip to the next essay.

Getting your name on a top institution used to be pricey. It cost Bill Gates a few billion, with a $36 billion assist from Warren Buffett, to create a first-rate philanthropic foundation. Want your name on a top college? Good luck: even after a market crash, Harvard's endowment is tens of billions.

But now there's a steal to be had.

Clay Shirky notes that grants and endowments will help support the journalism of the future. The price is surprisingly low. ProPublica, a nonprofit with a Pulitzer-winning staff, operates on $10 million a year (or one sixth of the Stanford athletic department's budget). In other words, you could permanently endow a serious news organization for about $250 million.

Your foundation — The [YOUR NAME HERE] Times — could be one of the essential voices of the 21st century. And like the first college founded in America, it would have a decent shot at becoming one of the nation's most admired institutions.

We'd like to speak to our billionaire readers — there must be one or two of you left — about the current buyer's market for prestige. Everyone else, please skip to the next essay.

Getting your name on a top institution used to be pricey. It cost Bill Gates a few billion, with a $36 billion assist from Warren Buffett, to create a first-rate philanthropic foundation. Want your name on a top college? Good luck: even after a market crash, Harvard's endowment is tens of billions.

But now there's a steal to be had.

Clay Shirky notes that grants and endowments will help support the journalism of the future. The price is surprisingly low. ProPublica, a nonprofit with a Pulitzer-winning staff, operates on $10 million a year (or one sixth of the Stanford athletic department's budget). In other words, you could permanently endow a serious news organization for about $250 million.

Your foundation — The [YOUR NAME HERE] Times — could be one of the essential voices of the 21st century. And like the first college founded in America, it would have a decent shot at becoming one of the nation's most admired institutions.

Clay Shirky does a superb job of laying out the difficulty, if not the impossibility, of extrapolating the future of the written-news business from its past or its present. The digitization of text, combined with the rise of the Internet as a cheap and capacious distribution system for digital products, has changed, profoundly, the economics of the production and the consumption of news stories. And the economic changes are continuing. Shirky's right: Everything remains in flux.

But while Shirky makes an eloquent argument that the future structure of the written-news business is unknowable, he does make at least one very big assumption about that unknowable future: that news organizations, as we have known them, will not survive. Not only is the traditional form of the newspaper doomed, but its underlying "organizational form" — a group of centrally managed professional journalists — is also doomed. To suggest that the organization may survive, says Shirky, is to perpetuate a lie.

I'm not so sure about that. In fact, I think there's a very good chance that, say, 20 years from now, much of the news we depend on will still be produced by large and fairly traditionally organized journalism companies. There will, to be sure, be many other sources of news or news-like material, produced in ways that, as Shirky suggests, have yet to develop, but history shows that a centrally managed, formally organized company of professional journalists is a pretty good and quite robust way of producing a diverse array of news, whether packaged as a paper or a web site, that people value. Any declaration of the death of that organizational form is premature.

So let me lay out a different scenario for the future of written news, one that begins with the source of the current crisis affecting the business.

The essential problem with the newspaper industry today is that it is suffering from a huge imbalance between supply and demand. What the Internet has done, beyond making the copying and distribution of articles very cheap, is this: it has broken the traditional geographical constraints on news distribution. As a result, the market has been flooded with stories, with product. Any article published by any paper is, in essence, available to any reader. Supply so far exceeds demand that the price of the news has dropped to zero. You no longer have to go to your local paper for a report on a story. Substitutes are everywhere. As I write this, Google News, on its front page, is offering me 5,850 different article about a proposal to recoup AIG's executive bonuses by imposing a new federal tax on them. That's a hell of a lot of supply.

To put it another way, the geographical constraints on the distribution of printed news required the fragmentation of production capacity, with large groups of reporters and editors being stationed in myriad local outlets. When the geographical constraints went away, thanks to the Net and the near-zero cost of distributing digital goods anywhere in the world, all that fragmented (and redundant) capacity suddenly merged together into (in effect) a single production pool serving (in effect) a single market. Needless to say, the combined production capacity now far, far exceeds the demand of the combined market.

In this environment, you're about as like to be able to charge for an online news story as you are to charge for air. And the overabundance of supply means, as well, an overabundance of advertising inventory. So not only can't you charge for your product, but you can't make decent ad revenues either. Hence: Bad times.

Now here's what a lot of people seem to forget: Excess production capacity goes away, particularly when that capacity consists not of capital but of people. Supply and demand, eventually and often painfully, come back into some sort of balance. Newspapers have, with good reason, been pulling their hair out over the demand side of the business, where a lot of their product has, for the time being, lost its monetary value. But the solution to their dilemma may actually lie on the production side: particularly, the radical consolidation and radical reduction of capacity. The number of U.S. newspapers is going to collapse (although we may have differently branded papers produced by the same production operation) and the number of reporters, editors, and other production side employees is going to plummet. The radical reduction in the supply of news is exactly what we're seeing today, with the bankruptcies and layoffs throughout the industry. And it will continue.

Economics tells us that as supply shrinks, market power begins to move back to the producer. The consumer, or user, no longer gets to call all the shots. Substitutes dry up, the perception of the fungibility of news stories dissipates, and differences in quality become, once again, both visible and valuable. The value of news begins, once again, to have a dollar sign beside it.

Shirky has argued, in an earlier article, that we're "in a media environment with low barriers to entry for competition." That's true of "media" broadly defined, but it's not true of the production of quality reporting. The capital requirements for an online news operation are certainly lower than for a print one, but the labor costs remain high. Reporters, editors, photographers, and other newspaper production workers are skilled professionals who require good and fair pay and benefits and, often, substantial travel allowances. It's a fantasy to believe that the production of all the kinds of news that people value, particularly hard news, can be shifted over to amateurs or journeymen working for peanuts or some newfangled journo-syndicalist communes. Certainly, amateurs and volunteers can do some of the work that used to be done by professional journalists in professional organizations. Free-floating freelancers can also do some of the work. The journo-syndicalist communes will, I suppose, be able to do some of the work. And that's all well and good. But they can't do all of the work, and they certainly can't do all of the most valuable work. The news business will remain a fundamentally commercial operation.

When you radically reduce supply in the industry, the demand picture changes radically as well. Ad inventory goes down, and ad rates go up. And things that seem unthinkable now — online subscription fees — suddenly become feasible. That will require, of course, not just a reduction of production capacity but the imposition of control on the copying of content — the establishment of a new regime of intellectual property rights. Shirky tells of the upheaval that came after the invention of the movable type press. But one important element of that story that he doesn't talk about is the creation of copyright and other intellectual-property protections that ultimately served to constrain what the new presses could print. The printing press dramatically reduced copying costs, but that didn't lead to the annihilation of writing and publishing as profit-making pursuits. Just the opposite. The reduction in copying costs, combined with protections for intellectual property, brought into being the modern publishing industry.

Once the production capacity for news "product" is rationalized for an electronic market, the ability of the remaining written-news organizations to impose controls on the distribution of their stories will grow considerably, whether those stories are printed or published exclusively in digital editions. The economics of supporting such organizations will become sustainable again. We will end up with far fewer professional news organizations and their customers will likely represent a smaller fraction of the population — quality journalism may end up being consumed and supported by an elite audience — but I'd say that it's a pretty good bet that the essential form of the organization is likely to survive.

Chief Technology Officer, Digital Operations, The New York Times Company

Clay Shirky's shock treatment for newspapers executives — "Nothing will work" — is a refreshing rejoinder to the proponents of the latest batch of so-called solutions to the industry's crisis. His words are all the more important given the fundamentalist certainty with which many of these failed or unrealistic strategies are being advanced. But it is by no means inevitable, as he asserts, that all old media institutions will disintegrate as the printed newspaper itself diminishes in importance and eventually ceases to exist. A few newspapers will make the transition to an all-digital future with their newsrooms largely intact. It's just not obvious yet how they will get there.

As Shirky rightly points out, what is really at stake is not the survival of newspapers but of the quality journalism they have come to represent. And the key to that survival is not the triumph of one particular business model over another, but of innovation. The question is: can newspaper companies learn to innovate before it's too late?

There is still time, though not much time. There are many people in the industry who are innovating even as old business models and ideas compete for our attention. The trouble is that breakthrough technological innovations are inherently disruptive. They do not enhance old business models and processes so much as they overthrow them. That makes it difficult, if not impossible, for all but the most visionary of established institutions to pursue strategies that appear to lead to their own demise but may actually be their salvation.

Given that organizational and psychological handicap, it's remarkable how innovative some newspaper companies have become. Newspaper websites have rushed to embrace the social Web, blogs, user generated content, personalization, data visualization, video and all manner of interactivity. At the best sites, the pace of change is astonishing and exciting to witness, and to be a part of for journalists, business people and technologists alike. The recession combined with the long-term secular decline of print has convinced many in the industry to embrace new ideas even as we are naturally compelled to reconsider some old ones. But on balance, digital news sites are getting better — more useful, more enjoyable, easier to read.

That is one reason why I believe an economically sustainable model for a quality Internet-only news organization will emerge far sooner than anyone expects, perhaps as early as two or three years after our current recession ends. It will be primarily based on advertising, with subscriptions and other forms of paid content adding incrementally to the bottom line. Many newspapers will not make it but those that do will have a few things in common.

First, they will have realigned their costs with their revenue, not just by cutting bodies but by intelligently reimagining how they run their businesses in general and how they produce news and information in particular so that they can become efficient, low-cost producers of quality journalism. They will understand how to use technology to automate what can be automated and to streamline processes that must remain manual. They will learn how to use metrics to improve the presentation of news and information, not to cheapen it or compromise their editorial integrity.

Second, they will stay focused on producing the best digital news products for their audience, products that take full advantage of the Internet's unique properties — its ability to combine immediacy and depth, its ability to offer highly personalized experiences, its ability to offer instant access to useful information from obscure public records to mundane event listings, and its ability to form networks so that users may form communities that mirror and extend the ones that exist in the physical world. They will learn how to engage their users to create and contribute content to enrich the experience of reading online, and not overwhelm their readers with a cacophony of undifferentiated noise.

The point here is that newspapers must accomplish online what they have long been able to achieve for generations of readers in print: they must forge an emotional bond with their readers by becoming an essential part of their daily lives. That hasn't happened yet to judge by the level of reader engagement online versus print. But it will — it must, if newspapers hope to make the transition to the digital age.

Third, and perhaps most important (because without this the first two are impossible), newspapers must recognize (as some already have) that technology and journalism are inexorably intertwined. The software developer who never thought of taking a journalism class in college may be the one who invents a new way of telling a story online, or a new and more profitable way of connecting readers and advertisers. And the journalist who never dreamed of writing a line of code may be the one who creates a new user interface that makes it easier for users to read and interact online.

If the Internet is a medium in its infancy, the creative partnership between journalists and technologists is younger still. That partnership needs to be nurtured and encouraged at every turn. The alternative is to cede that ground almost completely to technology companies, some of whom profess a love of newspapers and journalism but whose larger agendas may lead to the destruction of both.

It is easy to forget that the newspaper itself was once a disruptive technological innovation. Technology got newspapers into this mess, and it's going to get them out of it. One way or another.