Exploring a New World: Harvard Uses Video Games to Help the Blind Navigate

Harvard researchers have developed a video game which, according to the release by the university’s Laboratory for Visual Neuroplasticity, allows vision-impaired individuals to learn the layouts of unfamiliar buildings more successfully than they could by exploring the location in person.

Credit: Harvard Laboratory for Visual Neuroplasticity

CNET reported that this program, which takes the form of a fairly rudimentary video game, has huge potential for how vision-impaired and able-bodied individuals are able to learn and enter into unfamiliar spaces. The technology, said Lotfi Merabet, the head of the neuroscience team whose work produced these results, will hopefully ease the lives of the 285 million vision-impaired people worldwide.

Ease of Mobility

The video game-like program allows users to interact with a given layout much more easily and extensively than they would be able to by walking the halls of a building with a guide and memorizing the map mentally.

Merabet’s team tested the tech with two groups of congenitally blind adults or adults who had lost their eyesight during their lives, according to CNET. One group used the new video game method. The program uses audio cues to indicate to the players their location on the game’s map. The map was based on the floor plan of the Carroll Center for the Blind in Newton, Mass. The video game group explored the game’s layout in order to collect the gems scattered throughout the map while avoiding, in true game fashion, the monsters that try to prevent the gem thieves from escaping.

Another group of blind adults explored the hallways of the physical building itself in the typical method for blind individuals to learn a new environment. The purpose of the test was to see which mental map would ultimately be more effective and more complete, exploring the building or exploring the game.

The results were surprising and promising. The video game group was not only better able to navigate the building, but they could also intuit alternative routes through the space. The map created through the incentive drive, video game exploration was, according to the experiment, far more effective for getting a complex, useful idea of a building than actually moving through the building itself.

Better Understanding through a Virtual Proxy

Conceptualizing the building through the context of the game was apparently more complete, according to the study. This branch of research, substituting an actual experience with an augmented stand-in, uses traditional game elements like monsters and treasure to make the building more of an experience than an act of basic memorization. Using simple controls, the Harvard Crimson reported, the participants were able to learn the location of corridors and doorways of the Carroll Center for the Blind. Features like doors are represented in the game by directional audio cues like a knocking sound in the right-hand headphone.

This is the Carroll Center for the Blind. Some participants in the study navigated the halls of the building through a video game, while others learned the building’s layout by physically walking the halls, rather than virtually. Credit: RSSC Architects, Newton, Mass.

Lindsay A. Yazzolino, a participant in the study who then joined the research team, according to the Crimson, described the game as successful because it encouraged a novel, active approach to learning. The activity of learning the space, said Yazzolino, is easier to absorb than route memorization.

“It is not just about walking a blind person through a building, it really encourages problem solving skills,” Yazzolino told the Crimson. “Even now when I think about the building I still have a map in my mind of how the building is laid out.”

The Crimson also described how the video game players developed more innovative patterns around the buildings than their traditional learning counterparts. The participants who simply walked through the layout tended to follow only the paths they had learned during their initial explorations. Gamers used short cuts and inventive paths to get from point A to point B, demonstrating a more extensive, plastic understanding of the layout.

Revisions in Scientific Thinking

The study also proved, according to Erin Connors, co-author of the study’s concluding paper, that even congenitally blind individuals were capable of using all the parts of the brain that sighted people are able to. For a long time, Connors told the Crimson, it was thought that blind people, particularly those who have been without sight since birth, would be unable to use the part of the brain called the visual cortex. This section of the brain is normally, as its name implies, associated with visual planning and perception resulting from optical stimuli.

Connors described that the capability of blind individuals to build mental maps using the video game method proves what researchers have theorized, that neural plasticity—which essentially means rewiring of neurons in response to stimuli—allows even individuals blind since birth to use this section of their brain to build useful mental maps.

This video game is not only a useful tool but an expansive new look at how people without sight are able to use mental visual maps to function in innovative and efficient new ways.