Primates Use Simple Code to Recognize Faces

Researchers could reconstruct the faces a monkey saw from the patterns of neuronal activity in a certain area of the brain.

Jun 1, 2017

Abby Olena

Real faces that were presented to a monkey, together with reconstructions made by analyzing electrical activity from 205 neurons recorded while the monkey was viewing the faces. DORIS TSAOScientists have hypothesized that individual neurons are responsible for distinguishing individual faces. But now, researchers at Caltech have shown that the brains of macaque monkeys instead use a combinatorial approach to facial identity, in which each face-recognizing neuron responds to a particular aspect of faces.

By defining this combinatorial code, the scientists could reconstruct faces simply by observing neural activity. The study was published today (June 1) in Cell.

“People previously had this notion that neurons are coding specific identities,” says coauthor Doris Tsao of Caltech, but “the neurons are doing something more abstract than that. Each neuron is projecting a different axis. It’s simple and extremely mathematically elegant.”

Tsao and postdoc Steven Le Chang used an online face database to create a “face space” with 50 dimensions, half of which corresponded to facial appearance characteristics, such as skin texture, and half to face shape traits, such as eye height and face width. They used the face space to draw 2,000 random faces.

We “generated space where you could describe any face with just 50 numbers, and then we could really get a quantitative handle on how the cells were coding faces,” said Tsao.

Then, the researchers performed electrophysiological recordings on face-responsive neurons in the inferior temporal cortex from two monkeys while flashing the images of the random faces in front of the animals’ eyes. Each face cell was triggered to fire by certain dimensions in the face space, and did not respond to some axes at all. The authors found that one patch of face cells was more responsive to face shape dimensions, while another patch was activated by facial appearance axes.

“We found that a simple code can be used to reconstruct the face the monkey saw using the responses of hundreds of neurons,” says Chang, “and we can also predict the responses of the neurons” to a face using this code.

“It’s a very strong paper [and could] be a milestone in face recognition research,” says Adrian Nestor, a cognitive scientist at the University of Toronto who did not participate in the study. “There’s a lot more work to look at how well it generalizes to a new set of stimuli,” he adds. “A lot of research is focused on faces because we already know so much about faces and because it’s a rather limited domain of stimuli, but to what extent does this work apply to object recognition, to scene recognition, to world recognition?”

Tsao and Chang agree that how the brain recognizes other objects is an important extension of this research. They are also interested in how other aspects of the face, such as expressions, and how memory and imagination might activate this code.

“The last big direction is to understand how this code gets used by the brain downstream,” says Tsao. “We’ve found this very efficient code that can be used for all kinds of tasks, but it’s still a big question, how is it used specifically” for things like recognizing your grandmother or determining whether someone is male or female, she adds.

This study “is very exciting,” Brice Kuhl, a memory and perception researcher at the University of Oregon who was not involved in the work, told The Scientist. “It gets complicated, but the other thing one wonders about is, how similar is this in humans?”