Previously I showed an example of action-perception couplings. In another post about power posing, I talked about how there’s a powerful connection between your brain and body, which means that your body language shapes thought. Now I’m going to give an example of what happens when the feedback loop between body and brain is disrupted.

Facial Feedback Helps People Perceive Emotion

When you are talking to someone and they make a facial expression, you tend to mimic that. And there are social reasons for that—you want to show empathy, understanding —but it turns out that there are perceptual reasons too. You are better at judging their emotions if you are mimicking them. If you prevent this mimicry (via Botox) you are objectively worse at judging emotions that other people are making.

A person’s beyond-the-brain body plays an important role in their cognitive processing and knowledge acquisition. The sub-field of psychology called “Embodied Cognition” provides a framework for collecting “You think of with your body and your brain” research findings together. One of the cleverest studies on embodied cognition measured how the social intelligence of people decreased after they had Botox injections. This research has important implications for people who are building experiences or creating narratives in VR because it shows the limitations of cognitive processing alone. Designers should be considering how to harness the power of the body when they are creating experiences, particularly for head mounted displays (HMD) that could affect how people experience their facial muscles.

How Facial Feedback Changes Cognition

There are basically three steps to understanding how the process of embodied cognition, specifically in how facial feedback improves perception of emotions in others. First, mimicry. You are automatically perceiving and mimicking facial expressions in those around you. Next, this copying means that your muscles are contracting, which means that there is muscle feedback being conducted from face to brain. Lastly, this feedback helps you experience and understand the emotional meaning behind the expressions that you see. The Botox study here is unique because it’s one of the first ones to demonstrate that people can use feedback from their own muscles in order to more accurately understand the emotions of the people around them.

This research is unique because it isolated the exact role of facial feedback in study participants’ effort to interpret the expressions of others. The way that the researchers measured accuracy in emotions is via a social intelligence task called Reading the Mind in the Eyes Test or RMET. Adults are shown a photograph of a pair of eyes and then answer a multiple choice test on which emotion the person in the photograph is feeling. Here are two examples:

An example photograph from the RMET. What emotion is this male displaying? Choose one of the following: Serious (correct), Ashamed, Alarmed, Bewildered.

Another example from the RMET. What emotion is this female displaying? Choose one of the following: Reflective (correct), Aghast, Irritated, Impatient.

When a participant takes the RMET, they view 36 pairs of eyes like the ones above and have access to a glossary to read any definitions of emotions that they wanted. Adults recruited from the general population accurately reported the emotion 72.7% of the time. The researchers compared them to a Cambridge student population who performed better, correct 77.8% of the time, but there was no statistically significant difference between the two groups.

The RMET was developed by psychologists at Cambridge and their 2001 paper only documents how well adults do on this sort of social intelligence test. The Botox study utilized RMET in order to study the body-brain feedback loop, and specifically what happens to performance when that feedback is distorted. David Neal of USC and Tanya Chartrand of Duke recruited one group of participants who got Botox and another group who got Restylane (another cosmetic facial filler) within the previous 1-2 week window. While the participants were not randomly assigned to receive Botox or Restylane, they were well matched; they did not differ in age, ethnicity, or social economic status. The big difference between the two cosmetic fillers is that Botox freezes the facial muscles in place and Restylane does not. Botox paralyzes expressive muscles by blocking the release of acetylcholine, a neurotransmitter, at the neuromuscular junction. This results in a paralysis in the facial muscles, meaning that there is less feedback conducted from the Botox-ed muscles to the brain. In contrast, Restylane is simply a dermal filler and has no impact on muscle or neurotransmitter function. It is interesting to study these two groups because if facial feedback from a person’s body to their brain actually helps them mimic and subsequently read the emotions of the people around them, then the Botox group would perform worse.

That’s exactly what happened. The Botox participants performed poorly compared to the Restylane group. People with Botox accurately stated the emotion of the people in the photographs 69.9% of the time. The Restylane group were correct with 76.9% of the faces, which was a statistically significantly difference from the Botox group. When compared to the groups of adults from the original development of the RMET, the Restalyne group performed in the same range as those controls.

The reduction of muscular feedback due to Botox meant that the participants had a worsened ability to perceive the emotions of others, wheras the Restalyne group was unaffected. The implications for VR designers is that participants have different emotional capabilities for reasons outside of your control. That could lead to less immersive storytelling. This is something to consider for all VR designers - not just experiences made for Botox injectors. People with any type of disruption of the conduit between facial muscles and brain may come away from your experience with a different feeling than those who don’t have that disruption. It’s also an open question - does the mere presence of HMD affect the facial feedback loop?

In my next post, I’m going to talk about a different study conducted by the Botox researchers. In it, they put a gel on people’s faces that made it difficult for people to move their muscles and gave them the same social intelligence task.

The Extended Mind is a Portland-based research firm that applies behavioral science to increase user engagement in immersive technologies. Jessica Outlaw, the founder of The Extended Mind, is also a winner of the 2017 Oculus Launch Pad competition.