Most studies of facial cues rely on a set of stylized, recognizable expressions—perhaps made by actors in photographs. The actors make expressions meant to be obvious enough to translate across cultures: anger, disgust, fear, joy, sadness, and surprise. But these stylized images don't necessarily reflect the expressions that people make in the real world, says Hillel Aviezer, a neuropsychologist at who is now at The Hebrew University of Jerusalem and lead author of the new study, published online today in Science. Moreover, when emotions get particularly extreme, people undergoing fleeting peaks of intense pain, joy, grief, or anger look surprisingly similar, Aviezer says. From the face, at least, "when you compare extreme pain to extreme pleasure, you really can't tell them apart," he says.

And yet most people are rarely confused about whether someone is experiencing grief or joy. To figure out what tips us off, Aviezer and his colleagues showed photos of professional tennis players to 45 Princeton University students, randomly divided into three groups of 15. Each tennis player had just won or lost an important match, and the participants rated the players' contorted facial expressions from negative to positive on a scale from 1 to 9, with 5 marking the neutral midway point. One group of participants looked at head-to-toe photos of the players, the second group looked at only the players' bodies, and the third group looked at only their heads. Only the final group had trouble making the correct identification, suggesting that facial expressions alone didn't tell them whether the players were joyous or in despair.

With the help of photo-editing software, the team then switched the winning player's heads with those of losers. To keep participants from noticing the trick, they shuffled the doctored photos with similar images. Participants still labeled winners or losers according to the players' posture, not their facial expressions. In interviews conducted after the study, the researchers learned that cues such as whether a hand was open or clenched were more important than facial cues in interpreting expressions. Yet, in a separate experiment that asked 20 participants to state whether they would use body language, facial expressions, or both to evaluate emotion, 80% believed that they could judge the full-body photos by facial expression alone, Aviezer says. That result underscores our bias toward faces, and how little we credit body language, he says.

To see if bodily gestures were more expressive in other contexts, the researchers performed a similar experiment with photographs of people in other high-intensity situations: crying at funerals, winning extravagant prizes on reality TV shows, getting their nipples and ears pierced, and having orgasms. Again, without body language to provide context, viewers struggled to correctly read facial expressions. In fact, they rated isolated faces displaying positive emotions more negatively than faces displaying negative emotions.

David Matsumoto, a psychologist at San Francisco State University in California, has his doubts about Aviezer's techniques for classifying emotion, however. For example, he says, his own research suggests that the face made by athletes when they win is a signal of competitive dominance—not necessarily a "positive" emotion.

The results could potentially help people who have difficulty recognizing facial expressions, Aviezer says. "Maybe we should zoom out from isolated faces when we teach people how to read emotion." First, look at what's happening in the surroundings, he says, "then look at the body. Then, look at the face."