Q: Your research focuses on how neurons in the visual system process different kinds of visual information such as form, color, depth, and movement. What have you found most surprising about how the brain “sees”?

Livingstone: I was most impressed by the fact that major aspects of our vision are colorblind. This is because the evolu­tionarily older “Where” subdivision of our visual system arises from colorblind retinal ganglion cells, but the newer, primate-specific “What” system can use color. The “Where” system, or the parietal part of our visual system, is responsible for carry­ing information about motion, depth, spatial organization, and figure/ground segregation. Lesions in this part of our visual system result in apraxias, or the inability to use visual informa­tion to guide motor behavior. The “What” subdivision is responsible for object recognition, face recognition, and color perception. Lesions in this temporal subdivision of our visual system result in agnosia (the inability to recognize objects), prosopagnosia (the inability to recognize faces), or central achromatopsia (complete loss of color perception). Artists have taken advantage of this subdivision of our visual systems for a very long time through their various discoveries concern­ing the different roles luminance and color play in art.

Q: The image at left represents an optical illusion. What accounts for this?

A: Most people see movement in this static image, which was made by Akioshi Kitaoka, from Ritsumeikan University in Japan. We found that the critical feature for inducing this illusory motion is the luminance relationship of the static elements. Illusory motion is seen from black-blue-white-yellow-black. Compared to an average gray, black has a higher luminance contrast than blue, and white has a higher contrast than yellow. Since cells in the visual system respond faster to higher-contrast stimuli, the timing differences between white and yellow, and between black and blue, induce a motion illusion, with movement perceived in the direction from the higher-contrast element toward the lower (black-blue and white-yellow). Therefore the motion signals generated by the black-dark-gray and the white-light-gray pair are in the direction of the faster response to the slower response. This makes sense because such contrast-dependent timing differences would mimic the sequence of a stimulus that moved from the position of the higher-contrast element to that of the lower.

But the motion signals generated by the blue-white and the yellow-black pairs are in the direction from the slower response to the faster response, which is paradoxical. This paradox may be resolved by considering the fact that dark­gray and white are opposite in sign of contrast from the average gray, as are light-gray and black. Element pairs that produce motion signals in a direction consistent with their timing differences have the same sign of contrast compared to the average gray, and element pairs that generate motion signals opposite to their timing differences are opposite in sign of contrast. Thus the pattern of responses to these static motion stimuli is analogous to the phenomenon known as “reverse phi motion,” meaning that when their contrast is inverted, apparent motion-stimulus pairs appear to move in the direction opposite to their physical motion (Anstis, 1970). All four adjacent element pairs in the illusion generate a motion signal in the same direction, which is why the illusion is so powerful.

Q: Another interest of your laboratory is to use what you’ve learned about vision to explain some aspects of art. What do you think Leonardo da Vinci understood about color and light that neuroscience is just beginning to unravel?

A: Leonardo da Vinci produced an illusory dynamic quality in the Mona Lisa. People love this painting because her smile seems to come and go; her expression is so dynamic that she seems almost alive. When you look at a reproduction of this painting (or better still the original), look at her eyes and observe how much she seems to be smiling; then look direct­ly at her mouth and see if she doesn’t seem to be much less cheerful. Look back and forth between her eyes and her mouth and see if you don’t see a systematic change in her expression.

The art historians said that her smile was blurry (sfumato) and therefore ambiguous, so her expression depended on the observer’s imagination. But I find that her expression is systematically related to how far from her mouth my gaze is. Your central vision has much higher resolution than your peripheral vision; that’s why you move your eyes when you read. You can see tiny detailed things much better than big blurry things with your central vision, but the reverse is true of your peripheral vision. Mona Lisa’s smile is blurry, there­fore it’s much more apparent to your peripheral vision than to your central vision (see image). She seems to be smiling more when you’re not looking directly at her mouth, and she stops smiling as soon as you look at her mouth. This gives the painting a dynamic, even coy, quality.

It is not clear to me whether Leonardo knew this explicitly. He wrote about a lot of his painting techniques and their scientific basis, but he never described this phenomenon explicitly. He was apparently very fond of this painting, and, as far as we know, he never did this again.

Q: You have suggested that stereoblindness—an impair­ment in one aspect of depth perception—might actually be an asset for artists, enabling them to better render 3-D scenes on a flat surface. What evidence have you uncovered that many famous artists might have had deficits in depth perception?

A: We have two eyes that are horizontally displaced, so they see the world from two slightly different perspectives. As Leonardo da Vinci noted centuries ago, these viewpoints produce two distinct retinal images. Try this: hold your two index fingers up with one about ten inches from your nose and the other a dozen inches away, directly behind the first. Now look at your fingers out of one eye at a time and you will notice that the two “scenes” vary significantly. The brain uses these differences between the retinal images, in addition to other monocular depth cues, to estimate distance and gener­ate a rich perception of depth.

This phenomenon, known as stereopsis, is just one important cue for depth perception; others include perspective, shading, occlusion, haze, and relative motion. Our visual system integrates all of these cues, enabling us to navigate through our environment. In paintings, though, only the monocular static cues can contribute to the illusion of depth; stereopsis and relative motion reveal that the canvas is actually flat. So the next time you find yourself looking at a painting rich in depth cues, stand at arm’s length and try closing one eye; you may experience more of the illusion of depth that the painter was trying to achieve.

The ability of painters to translate the three-dimensional world into two dimensions is remarkable. Perhaps more astonishing, however, is the curious feat our visual systems perform in enabling us to perceive the visual world as three-dimensional in the first place. The brain’s only visual input comes from a pair of two-dimensional images; the retinal images are, after all, flat. Our brains then convert these flat images into a vivid­ly three-dimensional experience by using the same cues a painter employs, plus stereopsis and relative motion.

Our brains convert flat images into a vividly three-dimensional experience by using the same cues a painter employs, plus stereopsis and relative motion.

Just as stereopsis is a hindrance to the viewer who wants to see all the depth the artist put into the painting, it can also be a hindrance to the artist trying to depict three-dimensional scenes on canvas. Art teachers often instruct students to close one eye when viewing a scene in order to flatten it. I have therefore suggested that stereoblindness might prove an asset rather than a handicap to an artist. A person lacking stereopsis might become more sensitive to other (monocular) depth cues, such as shading, perspective, and occlusion—precisely those cues artists can render in paintings.

Stereoblindness is not a prerequisite for artistic talent. Yet the notion that stereoblindness might prove an asset for painters demonstrates the broader possibility that other aspects of brain organization considered detrimental under some condi­tions might offer advantages under other circumstances. Indeed, many talented artists, musicians, mathematicians, and engineers are dyslexic. It is often thought that the over-representation of dyslexics among artists and musicians represents a compensation for failure in conventional academic fields. Yet growing evidence suggests that the correlation may be based, in part, on a positive correlation between dyslexia and extraordinary talent.

Bevil Conway, himself a stereoblind artist, and I have suggest­ed that a number of very talented artists might have been stereoblind, a notion based on looking at photographs of them and finding that their eyes are misaligned. Stereopsis requires precise eye alignment; therefore, people whose eyes are misaligned usually have poor or no stereopsis; they have fine depth perception because they use other depth cues to gauge distance and depth. We recently suggested that Rembrandt was likely to have been stereoblind because he usually por­trays himself as having divergent eyes. Misaligned eyes might seem like a stylism in a painting, except for the fact that the same eye is usually deviated in all Rembrandt’s self portraits, and the opposite eye deviates in his etchings. Since an etching is reversed in the printing process, this mirror reversal of his eye deviation between the paintings and the etchings suggests that the deviating eye was something in his physiognomy that he accurately portrayed.