The Visionary

A digital pioneer questions what technology has wrought.

Jaron Lanier, at home with his daughter, believes that social-networking sites devalue friendship.Credit Photograph by Martin Schoeller

One day in June, Jaron Lanier was lounging barefoot in the living room of his house in the Berkeley hills. Stretching back on a worn sofa, he began musing about the connection between Representative Anthony Weiner’s tweeting of lewd photos and Facebook’s controversial deployment of facial-recognition software, which automatically scans uploaded photos and identifies a user’s friends.

To Lanier, a computer scientist and author, the common thread is that the Internet in general—and social networking in particular—has become difficult for the ordinary person to use with any security. “I’ve really been struck that a lot of people have said, ‘Why would powerful men risk so much for some sexual adventure?’ ” Lanier said. “But risk can be very sexual.” He briefly considered the possibility of two alternate Internets: one in which everything was viewable by anybody, and one in which users had absolute control over their private information. In neither case, Lanier said, would Weiner have sent his illicit snapshots. “What makes it erotic is the risk,” Lanier speculated. “If you had either perfect competence or no need for competence, because everything was a hundred per cent transparent, there would be no risk. So, in a way, the whole erotic risk factor of the Internet is being able to use it but not very well.”

He paused to interrogate a tortoiseshell kitten that was dozing in a corner of the sofa. “What’s happening, Starlight?” he cooed. As the kitten peered up sleepily, he added, “We think she’s female, but I haven’t done the most thorough examination.” He paused and said dryly, “If only cats texted, we’d know by now.”

Lanier is often described as “visionary,” a word that manages to convey both a capacity for mercurial insight and a lack of practical job skills. In the nineteen-eighties, he helped pioneer the field of virtual reality, and he is often credited with having coined the term. He has also dabbled in film. In 2001, he advised the writers of “Minority Report,” Steven Spielberg’s film about a dystopian future. Since 2006, he has worked as a consultant at Microsoft Research.

More recently, he has become the go-to pundit for people lamenting the social changes wrought by modern technology. Last year, he published “You Are Not a Gadget: A Manifesto,” a provocative critique of digital technologies, including Wikipedia (which he called a triumph of “intellectual mob rule”) and social-networking sites like Facebook and Twitter, which he has described as dehumanizing and designed to encourage shallow interactions. Teen-agers, he writes, may vigilantly maintain their online reputations, but they do so “driven more by fear than by love.” In our conversation about Facebook’s face-recognition software, he added, “It’ll just create a more paranoid society with a fakey-fakey social life—much like what happened in Communist countries, where people had a fake social life that the Stasi could see, and then this underground life.”

Such objections have made Lanier an unusual figure: he is a technology expert who dislikes what technology has become. “I’m disappointed with the way the Internet has gone in the past ten years,” he told me at one point. He added, “I’ve always felt that the human-centered approach to computer science leads to more interesting, more exotic, more wild, and more heroic adventures than the machine-supremacy approach, where information is the highest goal.”

These arguments have proved popular. The book has received admiring reviews in the Times and (twice) in The New York Review of Books. In the months after “Gadget” was published, Lanier lectured at Harvard’s Shorenstein Center, travelled to Seoul to speak at a major conference about innovation, and made Time’s list of the hundred “most influential people in the world.” At the South by Southwest Interactive conference, in Austin, in March of 2010, Lanier gave a talk, before which he asked his audience not to blog, text, or tweet while he was speaking. He later wrote that his message to the crowd had been: “If you listen first, and write later, then whatever you write will have had time to filter through your brain, and you’ll be in what you say. This is what makes you exist. If you are only a reflector of information, are you really there?”

Peter Haynes, a technology strategist and former U.S. business editor of The Economist, who is currently working with Lanier at Microsoft, says that he sees Lanier’s book as an overdue corrective to the national obsession with social networking. “As I read it, I was thinking, Yes, goddammit, this is exactly how I feel!” he said.

Such enthusiastic reactions have been, for Lanier, both gratifying and disorienting. He relishes the attention, but it also unnerves him. When a major newspaper asked him to write an op-ed about the Weiner scandal, he declined. “I’m not sure I should be the person who’s doing that,” Lanier explained. “I’m trying to stay focussed on the long game, not the item of the week. Because the issues I’m talking about will take a long time to address.”

For the past eight years, Lanier has lived in Berkeley, the mecca of techno-utopianism, in a ridgetop house that he shares with his wife, Lena, who is a child psychologist, and their four-year-old daughter, Lilibell, whom he credits with being his muse for “Gadget.” When I visited in June, Lanier had just returned from New York City, where he celebrated his fifty-first birthday in the lounge of the Bowery Hotel. The event, which began modestly, gradually turned into a celebrity bash. The film director Jim Jarmusch stopped by uninvited, as did the actor Forest Whitaker. As Whitaker recalls it, he and Lanier got into a long conversation about individual empowerment and the Internet. “When I saw him, I was really excited,” Whitaker remembers. “He was sitting with a lot of other guys. I came over and said, ‘Virtual reality!’ I have a lot of respect for him. He has an artist’s soul.”

Mountainously built, with a broad face surrounded by a mossy underbeard and rootlike dreadlocks, Lanier has an imposing presence that nonetheless comes off as oddly fluttery. He tends to talk in breathless bursts, and he often defuses his inflammatory remarks by allowing his voice to rise into a register that is more often reserved for talking to pets or small children. This can give listeners the impression that he is lecturing to a three-year-old while walking up a steep hill.

His house is nearly submerged in clutter; the living-room décor includes a four-foot hookah topped with a rubber Jar-Jar Binks mask, polka-dot curtains, a grand piano buried under papers and adorned with a pink feathered hat, and a homemade cave draped in scarves. The house also contains more than a thousand rare musical instruments, all of which Lanier plays. He will often begin his talks by performing on an esoteric instrument such as a Laotian khene, which sounds something like a harmonica. That afternoon, Lanier ascended the stairs to his studio, picking his way past an overflowing garbage can and a forest of microphone stands, and seated himself before a tall golden harp. He played a dark, plinky composition in what sounded like a minor key. “It’s not really minor,” Lanier said when I inquired. He played another dissonant progression. “It’s not that simple.” He gazed upward and added, “I’m really interested in scales that are harder to resolve.”

In the nineteen-eighties, Lanier came to believe that virtual reality—the creation of computer-simulated environments in which real people can interact—would precipitate an extraordinary revolution in art and communication. In an interview with Omni in 1991, he described the allure of programs that would let you feel as if you were wandering at will inside a Moorish temple or through the chambers of a beating heart. In an early paper, Lanier wrote of the ability of some octopuses to express fear or anger by changing color. In a virtual world, he hypothesized, people would be able to communicate in similar ways. Tom Zimmerman, Lanier’s business partner at the time, recalls that Lanier was taken by the idea of hosting virtual-reality parties, where guests would arrive in strange and exotic forms. “I had this feeling of people living in isolated spheres of incredible cognitive and stylistic wealth,” Lanier explained.

Constructing such spheres of wonder, however, proved technologically difficult, and by the mid-nineteen-nineties the field of virtual reality had largely collapsed. Despite this, Lanier has continued to argue that the purpose of digital technology should be to enrich human interaction. One of his most recent ventures has been to help Microsoft construct a new, joystick-free gaming system, called the Kinect, which uses a computerized camera to match the movements of a player’s body to the avatar in the game—allowing someone to kick a virtual ninja using her actual foot. In an op-ed piece for the Wall Street Journal, Lanier cited the Kinect, which this spring became the fastest-selling electronic device of all time, as an example of technology that could “expand what it means to think.” Unlike more Luddite critics, Lanier complains not that technology has taken over our lives but that it has not given us enough back in return. In place of a banquet, we’ve been given a vending machine.

“The thing about technology is that it’s made the world of information ever more dominant,” Lanier told me. “And there’s so much loss in that. It really does feel as if we’ve sworn allegiance to a dwarf world, rather than to a giant world.”

“My parents were kind of like me in that they had tons and tons of weird, amazing stuff,” Lanier explained. He recalled that, as a boy, he dug through a pile of his father’s junk and found an antique telescope that had once belonged to Commodore Perry. “This thing was just, like, on the floor,” he added. “So this environment of clutter, and interesting objects, is exactly the one that I grew up in—just with different objects. But I came by it honestly.”

Lanier’s mother and father belonged to a circle of artists in Greenwich Village, but they moved soon after Jaron was born—on May 3, 1960—first to Colorado, and then to a spot near El Paso, Texas, on the border with Mexico. The area was desolate and impoverished, and Lanier has speculated that the move was driven, at least in part, by fear. Lanier’s mother, Lilly, a pianist, painter, and dancer, had emigrated from Vienna when she was fifteen, after surviving a concentration camp. His father, Ellery, the child of Ukrainian Jews who had fled the pogroms, worked as an architect, painter, writer, elementary-school teacher, and radio host. When Ellery was seven, a close relative was murdered by a gang of anti-Semitic men wielding swords. A younger sister of the victim, who witnessed the assault but was warned by the attackers not to speak of it, was so traumatized that she spent the rest of her life as a mute.

Not long after Jaron’s birth, his parents abandoned their last name, Zepel, for the less Semitic-sounding Lanier, after Sidney Lanier, a nineteenth-century poet and flutist, whom Ellery admired. “I think they thought, We’ve got a child now, let’s get far away, let’s hide,” Lanier said.

In the desert, Lanier’s mother helped support the family by trading stocks through a broker in El Paso. Educated and bohemian, she taught her son piano on a Steinway she had shipped from New York, and arranged for him to attend a private elementary school across the border, in Ciudad Juárez. Lanier—a self-described “hyper-romantic” child—spent his free hours poring over art books in the school’s library. He recalls being enamored of a folio of paintings by Hieronymus Bosch, which he would sometimes leaf through while listening to Bach’s Toccata and Fugue in D Minor. “The trifecta for me was eating chocolate, listening to Bach, and staring at Bosch,” Lanier said. The combination produced what he remembers as an “almost sexual” rapture.

Lanier was technologically precocious, as well as artistically minded, a mixture of traits that his father tried to nurture by giving him books about Buckminster Fuller. One Halloween when he was in grade school, Lanier modified a television to generate Lissajous waves: shadowy black-and-white interference patterns that, projected onto the walls of a makeshift haunted house, would jump in response to a person’s movements. Lanier found the effect magical—“like being surrounded by ethereal writhing spirits”—and imagined that other children would line up to visit. None did. “I didn’t have any friends at the time, and I really thought this would be my little honeypot—that somebody would love this thing, and want to know me,” Lanier recalls.

When Lanier was around ten, his mother was killed and his father severely injured in a horrific car crash. Immediately afterward, he fell ill with a succession of infections, including scarlet fever and pneumonia, which kept him hospitalized for almost a year. “I just wasn’t ready to go on without her,” he said.

Talking later about the crash, Lanier noted that it had taken place on a freeway overpass, and that no other cars had been involved. On the morning of the accident, he recalled, local bullies had jumped him as he left for school. His mother, who hadn’t finished dressing, had watched from behind the screen door, “screaming and freaking out.” Though Lanier managed to defend himself—fighting back with a baritone horn—he worried that the fight had distracted his mother and caused the crash.

“I performed the calculus that children do, and blamed myself,” Lanier said. “Then, later, when I was a rebellious teen, I wondered if she and my father had been fighting at the time of the accident. And then still later I started looking into it, and I discovered that there was a mechanical flaw in that particular model of car—so it became feasible that it was actually the car manufacturer’s fault.”

Lanier’s mother had recently bought the family a new house, in El Paso. But it burned down before Lanier and his father could move in. Lanier suspects, without any specific evidence, that the fire was set by vandals. Broke and unemployed, Lanier’s father moved the family to an empty parcel of desert in Mesilla, New Mexico.

In Mesilla, Lanier’s father allowed him to design their new home. Lanier, who was eleven, chose a geodesic dome, and with his father’s assistance he drew up blueprints calculating the angles of the frame, plus plans for a squat, cantilevered spire that he envisaged as the entrance. (“Clearly a subconscious phallic expression of some kind,” he told me.) But the project proceeded slowly. “We’d get enough money to pour the foundation for one part of the house, and then, after a few weeks, we’d get enough to do another part,” he recalls.

During the first two years that the dome was under construction, Lanier and his father lived in an unheated canvas Army tent that was stiflingly hot in summer and frigid in winter. Lanier remembers shivering uncontrollably at times, “like I was having a seizure.” The family belongings, which included his mother’s grand piano and her antique furniture, were wrapped in plastic and heaped together on the ground outside the tent. “We sealed the piano in a bag, kind of,” Lanier said. “It must have sat out there for a year.”

In Mesilla, Lanier remained deeply withdrawn. “After my mother’s death, I had such difficulty relating to people,” he recalls. “I don’t think I was able to really have a normal conversation with somebody until sometime in my late teens. I remember feeling a sense of triumph if I could just go into a store and buy something and leave, because I’d actually successfully negotiated these human relationships.”

Lanier enrolled in the local high school, which was racially divided and often violent. He found the experience “terrifying,” and left after a year. Mesilla was near the White Sands Missile Range, and was home to many scientists, including Clyde Tombaugh, who had discovered Pluto, in 1930. As a teen-ager, Lanier took to stopping by Tombaugh’s house, where he would sometimes look through his homemade telescopes. With Tombaugh and other scientists, Lanier found that it was possible to have long conversations about abstract subjects like mathematics “without even being there yourself”—that is, with little emotional connection.

Not long afterward, Lanier began taking classes in math and chemistry at New Mexico State University. At seventeen, he transferred to Bard College, in New York. To cover the down payment on his tuition, he sold fresh milk and cheese from a herd of goats that he bred. But the transition from goat herding to freshman civilization proved harsh, and he soon hitchhiked back to New Mexico.

At nineteen, Lanier fell in love with a girl named Cynthia Peck, a cellist visiting from Pasadena whose mother knew Lanier’s father. Peck recalls that Lanier was both “intensely brilliant” and “extremely needy.” His room, when she first visited, was heaped with dirty clothes. For their first official date, Peck insisted that they visit a laundromat. They stayed for hours, washing load after load while Lanier serenaded her with a Japanese bamboo flute.

When Peck eventually returned to Pasadena, Lanier followed, only to be told that the romance had ended. From there, he caught a ride up to Santa Cruz, where he spent a few months engineering sounds for video games before developing an unusual game known as Moondust—a cult hit that used the motions of the joystick to generate the soundtrack. Moondust led to a job at Atari and then to Tom Zimmerman, a coder in Palo Alto who had designed an electronic glove that would allow the wearer to “conduct” a virtual symphony. In 1985, Lanier, Zimmerman, and a couple of partners founded a company called VPL, with the goal of developing other tools for virtual worlds. Not long afterward, the company helped the toymaker Mattel produce the Power Glove, which could be used in place of a joystick. (In one game, Bad Street Brawler, players made a fist to “punch out” attackers.) VPL also sold a small number of higher-end gloves to I.B.M. and NASA—in one case, to control a robotic arm that mimicked the motions of the wearer—and helped build the first surgical simulator, an abdominal-surgery training program featuring a virtual stomach, gallbladder, and intestines.

Constructing larger worlds, however, turned out to be more complicated. The prototypes that Lanier built relied on stereoscopic goggles so heavy that sandbags were needed to counterbalance the weight. The software was similarly balky. Unable to match the rapid movements of the human eye, the scene through the goggles tended to lag queasily behind any shift in the user’s gaze. In 1992, Lanier was ousted, and the company collapsed soon afterward.

Devastated by the failure of VPL, Lanier moved to New York, where he recorded an album of Asian string and wind instruments, produced by Philip Glass and others, and embarked on what he describes as “a really crazy, hysterical young marriage” that lasted only a few months. “I was just out of control for a bit,” he said.

Eventually, he was hired as chief scientific officer for Eyematic, a Los Angeles company that was developing algorithms that would allow a computer camera to recognize and track human faces. When Google acquired the company’s patents, in 2006, Lanier cashed out. Shortly thereafter, he took a post as a scholar-at-large for Microsoft Research, where he has gone back to developing tools for virtual reality.

Lanier’s return to his original passion isn’t surprising to Peck. Lanier, she notes, never saw virtual reality as simply a useful technology. “It had to do with being able to be in somebody else’s mind with him,” she said. “With creating a kind of ultimate connection and communication.”

Since joining Microsoft Research, Lanier has been involved in more than a dozen projects, most of which are futuristic and only loosely related to the company’s products. One morning in early February, Lanier met with Janet Galore, a director who designs new devices, and Peter Haynes, a senior director in the company’s Advanced Strategies and Research Group. The trio discussed several of Lanier’s proposals, including one that would make a smartphone screen seem much larger, so that oversized digital documents, like a road map, could be viewed full scale.

Although the exact nature of Lanier’s contribution to the project was murky, it seemed to be a combination of freewheeling speculation and niggling detail. Among other things, he complained repeatedly about the team’s use of NUI (pronounced “newey”), as shorthand for “natural user interface,” arguing that the term was used only “inside the Microsoft bubble.”

“What I’d like to say is ‘biorealistic,’ ” Lanier said, adding, opaquely, “I think that’s really addressing the human nervous system on its own terms.”

Haynes mentioned that another researcher had proposed the phrase “computers that are more like us.”

Lanier squinched up his face. “You see, I don’t like that.”

Haynes turned to Galore. “He doesn’t like that.”

“In fact,” Lanier continued, laying his hands flat on the table, and then turning them over to stare at his palms, “I loathe that. And the reason why is that it implies this philosophical relationship between people and machines that gives machines a certain status.”

“That they could become like us,” Galore affirmed.

“Yes,” Lanier said. Then, as though suddenly sensing the abstruseness of his own oratory, he sat back. “You know, it’s fine,” he said, waving a hand. “I mean, it’s really good that there are different points of view here.”

Unlike most polemicists, Lanier has a disarming tendency to conclude forceful assertions with a moment of cheerful self-deprecation. In part, this habit may be rooted in his desire to avoid confrontation even as he provokes it. But it also seems to reflect a worry about falling too deeply under his own spell. During his time as a pitchman for virtual reality, Lanier said at one point, he developed a “hypnotic voice” with which he could “entrance people.” He told me, “I could have set myself up as a guru figure. But I withdrew from it, because I realized it was the wrong thing to do.”

These days, though, a guru figure is something like what Lanier has become. He was among the first critics to argue that social-networking sites like Facebook, which get their revenue from advertisers who want to know as much as they can about every user, have little motivation to protect people’s privacy. That has now become a popular view, as has his argument that companies like Google and Foursquare—a social networking service in which users broadcast their location to friends—resemble “privatized spy agencies” that collect information without giving users an easy way to opt out.

Just as often, though, Lanier merely seems to be saying whatever comes to mind. Among other things, critics have questioned his claims that innovation in popular music has ceased, and that the creativity of Web-page design peaked in the mid-nineteen-nineties. He is also fond of ambitious analogies and, at times, can make simple arguments almost willfully obscure. One chapter in “Gadget,” subtitled “My Love Affair with Bachelardian Neoteny,” includes a lament about the Web version of “Goldingesque neoteny,” or the tendency of online forums to be dominated by bullies.

Perversely, the opacity of Lanier’s critique may account for some of its popularity. Because his pronouncements tend to be oracularly vague, readers can interpret them to reflect their own views—from the classicist who deplores pop music to the vaguely disaffected Web designer, or the concerned parent who finds his children consumed by social media. The fact that Lanier is a genuine technology pioneer only adds to his authority.

Despite all this, Lanier can be almost pathologically sensitive when he feels misunderstood. When I confessed that parts of “Gadget” made me think he was anti-technology, he threw up his hands. “I mean, how loudly do I have to yell to get people to understand what I’m saying?” he asked.

What he actually wants, he says, is to revive the development of software that allows people to be creative and make a living while doing so. He cited two games as examples: Spore—in which players guide the evolution of simulated life forms—and Second Life, a shared online world in which players create elaborate virtual homes, businesses, and relationships. In both cases, Lanier said, players invest creative effort in the imaginary world; in the case of Second Life, they have also created a commercial market for selling goods.

Still, neither game has been a huge commercial success. Spore has sold around three million copies worldwide; its predecessor, The Sims, sold a hundred million copies. Second Life has fared better, but in ways that could hardly be described as representing a civic paradise. Among other things, the game became a hook-up spot for people dressed as animals and trying to have cyber-sex with one another. And experienced players sometimes extorted money from new arrivals—for example, by forcing them to pay for the restoration of their character’s “soul” after it had supposedly been bitten by a “vampire.” Though Lanier sees failures like these as particular flaws of Second Life, they raise another question: whether anyone will want to inhabit the humanistic future that he is so eager to create.

Likewise, part of what Lanier finds most regrettable about Facebook—the way it mediates social contact—is precisely what makes it so appealing to most people. “We use technology this way all the time,” Andy van Dam, a professor of computer science at Brown University, notes. “To create a layer of insulation. We send an e-mail so we don’t have to call someone on the phone. Or we call someone so we don’t have to go over to their house.” Many of us also use technology, he might have added, when we’re too isolated: when someone wants to find a new friend just because he’s feeling alone—or because he’s living with his father in a freezing tent in the desert.

The day after I talked with Lanier at his home, we drove together to the Virtual Human Interaction Lab, a new virtual-reality lab on the Stanford campus. As I settled into the car, a battered Toyota Solara, Lanier apologized for the condition of the front seat, the footwell of which was cluttered with old magazines, half-empty plastic water bottles, and a squashed box of tissues. As Lanier navigated carefully into heavy traffic, I asked why he hadn’t bought a luxury car, since he could presumably now afford one. Lanier sighed. “The thing is, we’d just beat the hell out of it,” he said.

At the campus parking lot, we were met by the lab manager, Cody Karutz, a mellow, collegiate blond in jeans and flip-flops, who escorted us into the lab’s “experimental room”: a squat chamber panelled in gray-and-tan fabric. A thin orange carpet covered a “haptic floor” that can vibrate and judder. (Later, when I sawed down a virtual fir tree in a simulated forest, the ripping crash as the tree fell seemed to shake the ground.)

In one corner of the room, a plastic headset and goggles hung droopily from a long black cable. Karutz clamped them firmly to my head, tight enough to block out the light. When he launched the first simulation, I found myself standing in what appeared to be the same room as before, but there was a deep rectangular pit in front of my feet.

The pit simulation, Karutz explained, can be used to test the degree to which cognitive knowledge—in this case, the knowledge that the floor does not contain a pit—is capable of overriding gut instincts and fear. Because the simulation realistically mimics the visual experience of a fall, many people do topple over, and may even feel their gorge rising, as though they were falling through space. Karutz offered to spot me if I wanted to try stepping off the edge, but, to my bafflement and shame, I found that I was paralyzed. When I admitted this to Lanier, he confessed that he had had the same experience—“I build these things and I couldn’t do it!” Few people, in fact, can. “The pit is a great example of how you can use virtual reality to really get at something deep in how people perceive the world,” he said. “It’s such a richly detailed window into what works and doesn’t work in our own psyches.”

Heading back from the lab, Lanier for once seemed generally satisfied—temporarily unambivalent about the future of technology and his role in it. “In the case of virtual reality, I think what we have has turned out to be even more interesting than what we’d have if we had actually managed to build the perfect fantasy-experience machine. Because this version isn’t just a tautology, a replica of the real world, it teaches us so much more about ourselves.”

Like an innovative painter who alternately courts and scorns the establishment, Lanier often seems torn between embracing and repudiating his newly influential status. As we drove, he mentioned, with some pride, that he had been “banned” from the TED conferences last year, after publishing an essay about the narcissistic nature of the event in a London magazine. (A spokesperson for TED said that Lanier is welcome at the conferences.) He purported to be similarly unimpressed by Davos, the economic conference, which he has attended “a billion times.” “At one point, I was in an elevator with Newt Gingrich and Hamid Karzai,” he said. “There are really only so many times you want to be in that situation.”

Lanier’s desire to shun convention, even while he longs for acceptance, has deep roots. “My dad has sometimes felt that I grew up a little lacking in sufficient eccentricity—in the sense that I’m willing to live as an adult in a house with walls that are parallel to each other, that sort of thing,” Lanier told me. Then he spoke about his mother. “Had she lived, I think I would have been more conventionally successful,” he said. “I think I would be, like, a Harvard Med School professor or something. My dad was more into ‘Be the Buckminster Fuller or the Frank Lloyd Wright’—be the weird outsider who becomes influential. Which is kind of where I ended up.” ♦