In Extending and Remapping Reality, VR Makes Us More Real

Share

Written By

May 07, 2018

Topics

The journey to alien avatars with multiple limbs, odd features, and abilities found on no human body began with an enormous hand. The path meandered through experiments with unusual limb placements and entailed a detailed study of how a two-armed human might control a six-armed lobster avatar. This experimentation, dubbed the study of homuncular flexibility, explores humans’ ability to inhabit novel avatars. It was the start of a VR phenomenon that extends deep into the realm of fantasy. The one constant, and perhaps the most influential figure in the creation of virtual reality, was Jaron Lanier, a man who argues that VR makes us more real—and more aware of the world.

Lanier, who will deliver the keynote speech at The eLearning Guild’s Realities360 Conference June 26-28, is somewhat of a living legend in the virtual reality universe; he’s also a computer scientist, author, composer, musician, and polymath. As a child, he tinkered with a television set to generate wave patterns that would move in response to a person’s movements, a Halloween experiment that foreshadowed his interest in mapping human movements to technology.

About that lobster. In the late 1980s, Lanier’s company, VPL Research, had created the first virtual worlds where people could take on avatars and interact. It was even possible for people to interact with one another within these virtual worlds. During one of these experiments, “a wonderful bug” caused Lanier’s avatar’s “hand to became enormous, like a web of flying skyscrapers,” as he described it in an essay for Edge. Despite the hand’s size, though, Lanier found that he could still use it somewhat normally. That got his mind working on “how weird the body could get before the mind would become disoriented.” A colleague had created a lobster avatar, and Lanier’s team of computer scientists worked with algorithms and practiced various movements. Ultimately, they were able to map a range of movements onto the avatar and control all of its limbs.

Expanding or altering the mental map of our bodies

A homunculus, literally a tiny human, is a mental model of the human body, a mapping in the brain cortex of movements and sensations experienced by all of the body’s parts. It’s somewhat distorted, as areas of the body with more nerve endings or connections to the brain are larger than less-sensitive areas. It’s long been known that the mapping can be altered. For example, if a limb is injured or amputated, its corresponding areas in the homunculus might shift to a different body part.

The study of homuncular flexibility combines ideas from psychology, neuroscience, philosophy, and computer science and theorizes that the homunculus can adapt to bodies that differ from the typical human shape—that is, bodies with extra or oversize appendages or those capable of atypical movements. In other words, we can learn to control imaginative, unconventional avatars in our virtual reality adventures.

Deliberate remapping has been used to address issues such as phantom limb pain in amputees and to explore body schema transfer—the phenomenon that, using visual and/or sensory stimulation, a person can be “tricked” into identifying an object as an extension of herself. One example is an experiment where a person’s hand is hidden from sight, while a rubber hand is visible in its place. The rubber hand and the person’s hand are both stroked. The mind begins to believe that the stroking sensation is coming from the rubber hand, that the rubber hand is part of the body—and the person flinches when the rubber hand is hit with a mallet. The ability of the mind to accept these non-human extensions, items, or images as “part” of the body is what enables participants in virtual reality to identify with their avatar bodies.

VR leapt ahead of technology

Lanier and his colleagues made many of their discoveries about the flexibility of the homunculus and humans’ abilities to control bizarre avatars in the late 1980s. They also created virtual worlds. Why, then, did it take 30 years for VR to capture the imaginations—and efforts—of commercial and eLearning developers? Technology—or lack thereof.

The early headsets were both unwieldy and expensive. In addition, Lanier lacked sufficient computing power to overcome “lag” and allow participants to establish presence. When the virtual environment is not rendered quickly enough to match the participant’s eye movements, most participants become queasy. The experience is unpleasant, not engaging. VR did not take off or become commercially viable in the 80s and 90s, and, in 1990, VPL Research filed for bankruptcy.

The evolution of VR

Lanier didn’t stop exploring human-computer interactions post-VPL. He was a key member of the team that developed the Microsoft Kinect, for example—a technology that represented an extraordinary leap forward. Kinect could, for the first time, translate a player’s body movement to the screen, rendering the player’s actions inside the virtual game environment—without requiring the player to don a cumbersome, geeky “motion capture” suit. Kinect doesn’t put the player into an immersive virtual world, but the ability to move and have the program render those movements on screen was a significant development.

In his writings and in interviews, Lanier points to myriad technologies as evidence that VR never really went away; it has affected everything from cinematic special effects to automobile development and surgical training. As technology has become better, faster, and cheaper, VR has increasingly moved into the mainstream of entertainment and, increasingly, eLearning.

Exploration of unusual avatars also moved forward. Working with researchers at Stanford’s Virtual Human Interactions Lab (VHIL), Lanier participated in studies of both the logistics and the implications of remapping the homunculus and changing the relationship between tracked movements—what the human does; and rendered movements—what the avatar does.

While most participants in the studies adapted quickly—within about 10 minutes—to their extra appendages or oddly shaped avatar bodies, many reported feeling that they “fluently controlled” the avatar, as opposed to feeling as if they had become the avatar. While there might be many reasons for this, including the quality of the virtual environment or how realistic (and human) the avatar or additional appendages appear, the feeling of using an avatar as a tool is very different from the immersive feeling of taking on a virtual identity.

Encouraging “self-presence” or stronger identification with the avatar might be beneficial in some instances and not in others, however. For example, when an avatar is used in physical therapy for an individual who has an injury or amputation, “to the extent that it disturbs the patient, or reminds the patient of his or her injury, a sense of self-presence in a non-normal body might be better left unprompted,” Won, Bailenson, and Lanier wrote.

Several studies have investigated the psychological effects of inhabiting an avatar that is different from one’s own body. Researchers have asked participants to adopt avatars of different genders, races, ages, and even heights than themselves; they’ve also evaluated participants’ behavior before, during, and after the VR immersion. Initial studies indicate the potential for at least short-term changes in behavior; those changes can be pro-social or anti-social, however; additional research is underway.

Lanier and other key figures in the evolution of virtual reality in eLearning will continue to move the conversation—and the technology—forward. Join them in San Jose at the Realities360 Conference in June to become part of the future of VR.

Keeping it real

To Lanier, the key benefit of engaging with virtual worlds is that VR makes us—and our everyday environments—more real. “Virtual reality was and remains a revelation. And it’s not just the world external to you that is revealed anew. There’s a moment that comes when you notice that even when everything changes, you are still there, at the center, experiencing whatever is present,” he wrote in Dawn of the New Everything. “Your center of experience persists even after the body changes and the rest of the world changes. Virtual reality peels away phenomena and reveals that consciousness remains and is real. Virtual reality is the technology that exposes you to yourself.”

In Extending and Remapping Reality, VR Makes Us More Real

Topics

The journey to alien avatars with multiple limbs, odd features, and abilities found on no human body began with an enormous hand. The path meandered through experiments with unusual limb placements and entailed a detailed study of how a two-armed human might control a six-armed lobster avatar. This experimentation, dubbed the study of homuncular flexibility, explores humans’ ability to inhabit novel avatars. It was the start of a VR phenomenon that extends deep into the realm of fantasy. The one constant, and perhaps the most influential figure in the creation of virtual reality, was Jaron Lanier, a man who argues that VR makes us more real—and more aware of the world.

Lanier, who will deliver the keynote speech at The eLearning Guild’s Realities360 Conference June 26-28, is somewhat of a living legend in the virtual reality universe; he’s also a computer scientist, author, composer, musician, and polymath. As a child, he tinkered with a television set to generate wave patterns that would move in response to a person’s movements, a Halloween experiment that foreshadowed his interest in mapping human movements to technology.

About that lobster. In the late 1980s, Lanier’s company, VPL Research, had created the first virtual worlds where people could take on avatars and interact. It was even possible for people to interact with one another within these virtual worlds. During one of these experiments, “a wonderful bug” caused Lanier’s avatar’s “hand to became enormous, like a web of flying skyscrapers,” as he described it in an essay for Edge. Despite the hand’s size, though, Lanier found that he could still use it somewhat normally. That got his mind working on “how weird the body could get before the mind would become disoriented.” A colleague had created a lobster avatar, and Lanier’s team of computer scientists worked with algorithms and practiced various movements. Ultimately, they were able to map a range of movements onto the avatar and control all of its limbs.

Expanding or altering the mental map of our bodies

A homunculus, literally a tiny human, is a mental model of the human body, a mapping in the brain cortex of movements and sensations experienced by all of the body’s parts. It’s somewhat distorted, as areas of the body with more nerve endings or connections to the brain are larger than less-sensitive areas. It’s long been known that the mapping can be altered. For example, if a limb is injured or amputated, its corresponding areas in the homunculus might shift to a different body part.

The study of homuncular flexibility combines ideas from psychology, neuroscience, philosophy, and computer science and theorizes that the homunculus can adapt to bodies that differ from the typical human shape—that is, bodies with extra or oversize appendages or those capable of atypical movements. In other words, we can learn to control imaginative, unconventional avatars in our virtual reality adventures.

Deliberate remapping has been used to address issues such as phantom limb pain in amputees and to explore body schema transfer—the phenomenon that, using visual and/or sensory stimulation, a person can be “tricked” into identifying an object as an extension of herself. One example is an experiment where a person’s hand is hidden from sight, while a rubber hand is visible in its place. The rubber hand and the person’s hand are both stroked. The mind begins to believe that the stroking sensation is coming from the rubber hand, that the rubber hand is part of the body—and the person flinches when the rubber hand is hit with a mallet. The ability of the mind to accept these non-human extensions, items, or images as “part” of the body is what enables participants in virtual reality to identify with their avatar bodies.

VR leapt ahead of technology

Lanier and his colleagues made many of their discoveries about the flexibility of the homunculus and humans’ abilities to control bizarre avatars in the late 1980s. They also created virtual worlds. Why, then, did it take 30 years for VR to capture the imaginations—and efforts—of commercial and eLearning developers? Technology—or lack thereof.

The early headsets were both unwieldy and expensive. In addition, Lanier lacked sufficient computing power to overcome “lag” and allow participants to establish presence. When the virtual environment is not rendered quickly enough to match the participant’s eye movements, most participants become queasy. The experience is unpleasant, not engaging. VR did not take off or become commercially viable in the 80s and 90s, and, in 1990, VPL Research filed for bankruptcy.

The evolution of VR

Lanier didn’t stop exploring human-computer interactions post-VPL. He was a key member of the team that developed the Microsoft Kinect, for example—a technology that represented an extraordinary leap forward. Kinect could, for the first time, translate a player’s body movement to the screen, rendering the player’s actions inside the virtual game environment—without requiring the player to don a cumbersome, geeky “motion capture” suit. Kinect doesn’t put the player into an immersive virtual world, but the ability to move and have the program render those movements on screen was a significant development.

In his writings and in interviews, Lanier points to myriad technologies as evidence that VR never really went away; it has affected everything from cinematic special effects to automobile development and surgical training. As technology has become better, faster, and cheaper, VR has increasingly moved into the mainstream of entertainment and, increasingly, eLearning.

Exploration of unusual avatars also moved forward. Working with researchers at Stanford’s Virtual Human Interactions Lab (VHIL), Lanier participated in studies of both the logistics and the implications of remapping the homunculus and changing the relationship between tracked movements—what the human does; and rendered movements—what the avatar does.

While most participants in the studies adapted quickly—within about 10 minutes—to their extra appendages or oddly shaped avatar bodies, many reported feeling that they “fluently controlled” the avatar, as opposed to feeling as if they had become the avatar. While there might be many reasons for this, including the quality of the virtual environment or how realistic (and human) the avatar or additional appendages appear, the feeling of using an avatar as a tool is very different from the immersive feeling of taking on a virtual identity.

Encouraging “self-presence” or stronger identification with the avatar might be beneficial in some instances and not in others, however. For example, when an avatar is used in physical therapy for an individual who has an injury or amputation, “to the extent that it disturbs the patient, or reminds the patient of his or her injury, a sense of self-presence in a non-normal body might be better left unprompted,” Won, Bailenson, and Lanier wrote.

Several studies have investigated the psychological effects of inhabiting an avatar that is different from one’s own body. Researchers have asked participants to adopt avatars of different genders, races, ages, and even heights than themselves; they’ve also evaluated participants’ behavior before, during, and after the VR immersion. Initial studies indicate the potential for at least short-term changes in behavior; those changes can be pro-social or anti-social, however; additional research is underway.

Lanier and other key figures in the evolution of virtual reality in eLearning will continue to move the conversation—and the technology—forward. Join them in San Jose at the Realities360 Conference in June to become part of the future of VR.

Keeping it real

To Lanier, the key benefit of engaging with virtual worlds is that VR makes us—and our everyday environments—more real. “Virtual reality was and remains a revelation. And it’s not just the world external to you that is revealed anew. There’s a moment that comes when you notice that even when everything changes, you are still there, at the center, experiencing whatever is present,” he wrote in Dawn of the New Everything. “Your center of experience persists even after the body changes and the rest of the world changes. Virtual reality peels away phenomena and reveals that consciousness remains and is real. Virtual reality is the technology that exposes you to yourself.”

Interested in Writing?

Learning Solutions welcomes contributions from members of the community. If you have a great idea or solution to share, send a query to the editor and our staff will work with you to share your solution with the eLearning community.