Nicholas Carr's blog

Terms of endearment, computer-generated

“As for living,” wrote the French symbolist Auguste Villiers de l’Isle-Adam in his 1890 play Axël, “our servants will do that for us.” Silicon Valley seems intent on giving the infamous remark a new, digital spin: “As for living, our computers will do that for us.”

The latest evidence is Allo, the new Google messaging app that uses artificial intelligence algorithms to generate replies on a user’s behalf. “If your friend sends you a photo of their pet,” Google explained when it launched the software two weeks ago, Allo’s “smart reply” feature will suggest a suitable response, such as “aww cute!” Tap it, and you’re done.

As Evan Selinger and Brett Frischmann pointed out, it’s like an autopilot for friendship.

The smart-reply system, which is built into the Pixel phones Google introduced yesterday, has been in the works for a while. Back in 2012, the company filed for a patent on the “automated generation of suggestions for personalized reactions in a social network.” In the application, Google pointed to birthdays and anniversaries as occasions when a person might want a machine to compose a congratulatory message to send to a friend. What with juggling Snapchat, Instagram, Facebook, and Twitter, who has time to pen a personal note anymore?

Some might point to Allo as yet another example of the trivialization of innovation. Now that the smartphone has become our all-purpose mediator of existence, Google is in a competitive war with rivals like Facebook, Apple, and Amazon to corner the market on human attention and agency. No feature is too trifling to exploit as a potential advantage.

But there’s something deeper going on here. Allo’s message-generation algorithm reveals, in its own small way, the strange view of personal relations that seems to hold sway in Silicon Valley. To the entrepreneurs and coders who run today’s massive social networks, our conversations are data streams. They can be tracked, parsed, and ultimately automated to enhance efficiency and remove kinks from the system.

We already use computers to converse, so the next logical step, in this view, is to use software to conduct the conversations themselves. By relying on an AI to compose our messages, we can optimize our productivity in managing our relationships. Call it the industrialization of affiliation.

Last year, in an online question-and-answer session, Facebook founder and CEO Mark Zuckerberg said that he thinks “there is a fundamental mathematical law underlying human social relationships that governs the balance of who and what we all care about.” Stripped to our essence, we humans are just aggregations of data, and it’s only a matter of time before information scientists discern the statistical pattern that defines our beings. At that point, we’ll all be perfectly programmable.

I expect most people would find such a pinched view of the human condition off-putting, if not repulsive. But as we continue to adapt to the digital processing of our thoughts and words, we may find ourselves embracing, without really thinking about it, the Silicon Valley ethos. We already consider it normal to respond to a friend’s message or photo with a quick click on a like button. Is it really such a leap to let a computer dash off a reply?

The German sociologist Theodor Adorno, in his prescient 1951 book Minima Moralia, warned of the dangers of allowing the values of the business world to creep into our personal lives. Behind the push to make communication more streamlined and efficient, he wrote, lies “an ideology for treating people as things.” Allo and its myriadkin would seem to bear out Adorno’s fears.

In its patent application, Google wrote that an “unstated protocol for behavior” often governs conversations between friends. What to a programmer might look like a formal protocol is actually something fuzzier yet much more meaningful: an expression of kindness, affection, care. It will be interesting to see whether we’ll come to draw a line between artificial intelligence and artificial emotion, or just take them as a package deal.