You Can’t Say That on the Internet

A BASTION of openness and counterculture, Silicon Valley imagines itself as the un-Chick-fil-A. But its hyper-tolerant facade often masks deeply conservative, outdated norms that digital culture discreetly imposes on billions of technology users worldwide.

What is the vehicle for this new prudishness? Dour, one-dimensional algorithms, the mathematical constructs that automatically determine the limits of what is culturally acceptable.

Consider just a few recent kerfuffles. In early September, The New Yorker found its Facebook page blocked for violating the site’s nudity and sex standards. Its offense: a cartoon of Adam and Eve in the Garden of Eden. Eve’s bared nipples failed Facebook’s decency test.

That’s right — a venerable publication that still spells “re-elect” as “reëlect” is less puritan than a Californian start-up that wants to “make the world more open.”

And fighting obscenity can be good for business. Impermium, a Silicon Valley company that helps Web sites deal with unwanted reader comments, has begun marketing technology that identifies “all kinds of harmful content — such as violence, racism, flagrant profanity, and hate speech — and allows site owners to act on it in real-time, before it reaches readers.” Impermium will police the readers — but who will police Impermium?

Apple, too, has strayed from its iconoclastic roots. When Naomi Wolf’s latest book, “Vagina: A New Biography,” went on sale in its iBooks store, Apple turned “Vagina” into “V****a.” After numerous complaints, Apple restored the title, but who knows how many other books are still affected?

True, these books are still on sale. Unlike the good old United States Post Office, which once confiscated “Lady Chatterley’s Lover” and other books it deemed too lewd, Silicon Valley does not engage in direct censorship. What it does, though, is present ideas and terms that have gained public acceptance as something to be ashamed of. Silicon Valley doesn’t just reflect social norms — it actively shapes them in ways that are, for the most part, imperceptible.

The proliferation of the Autocomplete function on popular Web sites is a case in point. Nominally, all it does is complete your search query — on YouTube, on Google, on Amazon — before you’ve finished typing, using an algorithm to predict what you’re most likely typing. A nifty feature — but it, too, reinforces primness.

How so? Consider George Carlin’s classic comedy routine “Seven Words You Can Never Say on Television.” See how many of those words would autocomplete on your favorite Web site. In my case, YouTube would autocomplete none. Amazon almost none (it also hates “penis” and “vagina”). Of Carlin’s seven words, Google would autocomplete only “piss.”

Until recently, even the word “bisexual” wouldn’t autocomplete at Google; it’s only this past August that Google, after many complaints, began to autocomplete some, but not all, queries for that term. In 2010, the hacker magazine 2600 published a long blacklist of similar words. While I didn’t verify all 400 of them on Google, a few that I did try — like “swastika” and “Lolita” — failed to autocomplete. Is Nabokov not trending in Mountain View? Alas, these algorithms are not particularly bright: unable to distinguish between Nabokov’s novel and child pornography, they assume you want the latter.

Photo

Credit
Ben Newman

Why won’t tech companies let us freely use terms that already enjoy wide circulation and legitimacy? Do they fashion themselves as our new guardians? Are they too greedy to correct their algorithms’ mistakes?

Thanks to Silicon Valley, our public life is undergoing a transformation. Accompanying this digital metamorphosis is the emergence of new, algorithmic gatekeepers, who, unlike the gatekeepers of the previous era — journalists, publishers, editors — don’t flaunt their cultural authority. They may even be unaware of it themselves, eager to deploy algorithms for fun and profit.

Many of these gatekeepers remain invisible — until something goes wrong. Thus, in early September, the online livestream from the Hugo Awards, the Oscars of the science fiction world, was interrupted with a cryptic copyright warning, right before the popular author Neil Gaiman was to deliver an acceptance speech.

An error has occurred. Please try again later.

You are already subscribed to this email.

Apparently, Ustream — the site streaming the ceremony — was using the services of another company to determine whether its streamed videos violated any copyrights. The partner company draws on a very large video archive to see, in real time, if what’s being streamed matches anything in its collection. Somehow, the celebratory video that preceded Mr. Gaiman’s speech tripped a copyright match, and the feed was cut off, even though the organizers had all the requisite permissions (and, under the doctrine of fair use, probably didn’t need them anyway).

The limitations of algorithmic gatekeeping are on full display here. How do you teach the idea of “fair use” to an algorithm? Context matters, and there’s no rule book here; that’s why we have courts. From the perspective of sticky, amorphous human culture, semi-automation — pairing up humans with algorithms — beats full automation. Sometimes, gaps are productive. But will profit-driven Silicon Valley ever acknowledge this insight?

Our reputations are increasingly at the mercy of algorithms, too. No one knows this better than Bettina Wulff, the former German first lady who has sued Google for autocompleting searches for her name with words like “escort” and “prostitute.” Ms. Wulff insists that Google’s algorithms spread false rumors about her; Google says that the suggested terms are just an “algorithmically generated result of objective factors, including the popularity of the entered search terms.”

Google’s defense would sound tenable if its own algorithms weren’t so easy to trick. In 2010, the marketing expert Brent Payne paid an army of assistants to search for “Brent Payne manipulated this.” Soon anyone typing “Brent P” into Google would see that phrase in their autocomplete suggestions. After Mr. Payne publicized his experiment, Google removed that particular suggestion, but how many similar cases have gone undetected? What is “objective” about such algorithmic “truths”?

Quaint prudishness, excessive enforcement of copyright, unneeded damage to our reputations: algorithmic gatekeeping is exacting a high toll on our public life. Instead of treating algorithms as a natural, objective reflection of reality, we must take them apart and closely examine each line of code.

Can we do it without hurting Silicon Valley’s business model? The world of finance, facing a similar problem, offers a clue. After several disasters caused by algorithmic trading earlier this year, authorities in Hong Kong and Australia drafted proposals to establish regular independent audits of the design, development and modifications of computer systems used in such trades. Why couldn’t auditors do the same to Google?

Silicon Valley wouldn’t have to disclose its proprietary algorithms, only share them with the auditors. A drastic measure? Perhaps. But it’s one that is proportional to the growing clout technology companies have in reshaping not only our economy but also our culture.

Obviously, Silicon Valley won’t develop or embrace similar norms overnight. However, instead of accepting this new reality as a fait accompli, we must ensure that, in pursuing greater profits, our new algorithmic gatekeepers are forced to accept the idea that their culture-defining function comes with great responsibility.

The author of the forthcoming book “To Save Everything, Click Here: The Folly of Technological Solutionism.”

A version of this op-ed appears in print on November 18, 2012, on Page SR9 of the New York edition with the headline: You Can’t Say That on the Internet. Today's Paper|Subscribe