Tintin … make a character too lifelike, and the brain no longer reads it as good animation, but as reality with something wrong about it.

Steven Spielberg and Peter Jackson hope to make Tintin a global household name with their new animated extravaganza, but in the process, they have brought another obscure term into the mainstream: the uncanny valley. The phrase has cropped up a lot in early reviews of The Adventures of Tintin: Secret of the Unicorn, referring to the strange effect created when animated characters look eerily lifelike. As New York magazine put it: "Tintin looks simultaneously too human and not human at all, his face weirdly fetal, his eyes glassy and vacant instead of bursting with animated life." Many others have agreed.

It's a paradox of animation that you can put arms and a face on a spoon, say, or make a deer talk, and it looks cute. But make a character too lifelike, and the brain no longer reads it as good animation, but as reality with something wrong about it. That's the uncanny valley. First identified by robotics professor Masahiro Mori in 1970, it's a phenomenon that has increasingly dogged film-makers, robot designers and video game designers. Some believe we may never cross it.

Most animated films stop at the uncanny valley's edge. Human characters in movies such as Up or The Incredibles are carefully made just cartoonish enough. But like intrepid 19th-century mountaineers, the makers of Tintin instead attempted to conquer animation's final frontier. Building on previous expeditions such as The Polar Express, Beowulf and Mars Needs Moms, the team at Peter Jackson's Weta Digital developed new motion-capture technology to turn live actors' movements into animation. The results are better than ever, but not quite good enough, evidently.

And they never will be, according to Angela Tinwell, senior lecturer in games and creative technologies at Bolton University. "We've come up with the notion of the 'uncanny wall'," she says. "It suggests a viewer's discernment for detecting imperfections will keep pace with technology. As the human likeness increases, so the viewer becomes ever more discerning of little flaws. So when a new, more "realistic" character like Tintin comes along, they won't be rated as more humanlike – previous humanlike characters will actually sink lower into the valley. It's cognitive dissonance; we refuse to be tricked by technology."

Others are not so pessimistic. "You probably already have looked at a film and not realised a character in it is digital," says Chris Lawrence at London's special effects house Framestore. CG characters such as the young Jeff Bridges in Tron Legacy, for example, fooled many viewers. "In that sense I think we've partially conquered it already. In terms of a full photoreal character in a film, though, I don't know. It's a lot of information to have to get right."

All this makes the uncanny valley sound like a new phenomenon, but film-makers have known about it long before it had a name. It's what makes many horror movies tick. Zombies are archetypal monsters from the bottom of the uncanny valley, with their dead eyes and expressionless faces. Likewise the glazed-over doppelgangers in Invasion of The Bodysnatchers or the robotic Stepford Wives, not to mention the legions of dolls, dummies, puppets, waxwork figures and clowns that have struck terror in the hearts of horror fans, from the ventriloquist's dummy in Dead of Night to to Chucky in Child's Play.

There's no accepted explanation as to what exactly makes these near-humans "uncanny". Perhaps the origins are evolutionary. Tinwell suggests correlations between "uncanny" animated faces and traits of human psychopathology. Others see in it something deeper: a threat to our own identity, as humans, living entities, our unique selves. Freud suggested the "uncanny" stemmed from the revelation of repressed primitive desires. On the other side of the valley, presumably, lies a perfect realm of visual sophistication where we can't tell the difference between what's animated and what's real – an idea that's potentially more terrifying than all the zombies, Chuckies and glazed-over Tintin characters put together.