The "Uncanny Valley" and "The Polar Express"

Japanese robotocist Masahiro Mori made an interesting observation that has come to be known as the “uncanny valley”:

Stated simply, the idea is that if one were to plot emotional response against similarity to human appearance and movement, the curve is not a sure, steady upward trend. Instead, there is a peak shortly before one reaches a completely human ‘look’ . . . but then a deep chasm plunges below neutrality into a strongly negative response before rebounding to a second peak where resemblance to humanity is complete.

In other words, there’s a point just before 100% realism where human models/robots/puppets appear freakishly frightening – the living dead, so to speak. Many of us have experienced the valley when viewing figures in wax museums. There’s just something, well, wrong with the figure. I’ve had this same feeling when viewing CG figures, especially when they are animated. The eyes just don’t look right. Even when the eyes dart and shift in a supposedly natural manner, there’s still something just “off” that gives goosebumps. As production companies strive towards fully realistic CG movies, this will be the biggest barrier to acceptance by the general public.

“The Polar Express” is premiering this week, and as you may know it is a fully CG movie that uses motion capture to record the movements of various actors (Tom Hanks in five different roles), and animates these recorded performances via CG models. The reviews generally state that the movements of the CG figures are amazingly lifelike, but the character’s eyes appear dead, zombie-like, and creepy. Paul Clinton’s review on CNN said:

This season’s biggest holiday extravaganza, “The Polar Express,” should be subtitled “The Night of the Living Dead.” The characters are that frightening.

The technical issue is that the motion capture cameras cannot currently record the movement of the actor’s eyes, as there is no way to attach sensors to the eye (each actor is covered with hundreds of sensors that the infrared motion capture system can detect). I wonder how difficult it would be to implement retinal tracking, perhaps via goggles.