A Study on the Dynamics of a Smile

We have had the unique privilege of interviewing Dr. Geoffrey Gerstner, Associate Professor in the Department of Biologic and Materials Sciences at University of Michigan. His recent paper, co-authored with Angela Lin, Tom Braun and James McNamara titled Esthetic evaluation of dynamic smiles with attention to facial muscle activity was published in the journal American Journal of Orthodontics and Dentofacial Orthopedics. The paper studies the dynamics of a smile, an element that can contribute to overall smile esthetics.

What is your background and your research interests?

I’m interested in oral motor control of the face and mouth. I have been rather broadly trained, my formal graduate training was in neuroscience, with a clinical emphasis on oral pain and function; however, my expertise has been in neuroethology. I’ve done a lot of field work with mammals, looking at relationships between chewing variables and body sizes and lifestyles in mammals. I see my research as a mirror image of what almost everyone else does–whereas most scientists use animal models to study human diseases, my interest is in mammals and field work, using human studies as ‘controlled experiments’ where it is impossible to do such experiments on animals. Of course I hope that my controlled experiments shed significant light on health issues in humans, and this is where Angela’s graduate research project, which is the work represented in this paper, fits in.

What led you to do this type of research study?

The human face is distinct among mammals. The face presents a highly sophisticated motor system that can relay (or hide) considerable information about the internal physiological states and traits of a person or organism. How it fits in with my other projects is as follows. Feeding is a real problem for mammals, because we have to eat a lot (being warm blooded). Because of the poor design of the throat (everyone’s choked on swallowed food–wouldn’t it be better if the nose-trachea connection and the mouth-esophagus connection were completely separate?), and also because our metabolic heater requires food in the lower GI to have a high surface-to-volume ratio for absorption, we have to chew a lot. But chewing wears teeth and, unlike reptiles, which get dozens of replacement teeth, we only get one set. So the mammalian strategy appears to be to make every chew count. To make every chew count requires really fine control of the jaw, lips, tongue, soft palate and throat muscles. Such fine control results in a system that is ‘ready-made’ for additional sophisticated behaviors such as speech, communication and facial expression. So, it is my overarching hypothesis (and interest) that the complexity of food acquisition in mammals is directly linked to our complex communication skills and that this skill involves motor control of the entire head (if not entire body). So, that’s my angle on the problem. Being in a dental school puts me in contact with clinical graduate students with slightly different interests and goals from mine. I was introduced to Angela by Jim McNamara, a co-author on this paper who was interested in smiles, and the three of us, along with Tom Braun our statistician, Rick Gracely, who helped us design the assessment methods for rating smiles, and Clayton Gerstner, who did all the computer programming so that the raters could view smile videos and rate them in real time, hammered out this project, which represented a neat mix of mutual interests.

What were your findings?

As you know, still photos often don’t capture what really happens. We’ve all seen a lame picture that catches us in the act of blinking or with our mouths half open taking a bite of food. We recognize that we don’t really look like that. This is because the behaviors we produce occur through time. In fact, nothing in the real physical universe in which we live, happens or is perceived by us instantaneously. So, a photo of a forced smile is really an incomplete set of data on which to base clinical judgements and outcomes regarding facial esthetics. A photo is convenient, but incomplete. Our argument was that, real smiles occur through time–a person develops a smile, holds it for some time, and then releases it. Posed smiles used in photographs don’t happen this way and the photographed image certainly looses the temporal dimension of the smile. Hence, it seemed important to us to test whether the temporal dimension included important information that might be clinically useful. Furthermore, it seemed important to determine whether relatively natural smiles were rated differently from posed smiles. The conclusions were that the temporal dimension of smiles did in fact contain important information, and that those smiles that most closely approximated spontaneous smiles were most esthetically pleasing. First, raters who watched our four female models smile were able to identify and discriminate accurately between four different smiles (by different, I mean we trained our models to move different groups of facial muscles to produce slightly different smiles). Second, the raters rated the physical attractiveness of our models higher when they were rating the videotaped smiles that involved all the muscles used to produce spontaneous smiles as opposed to those smiles that used fewer muscles to produce smiles. This was an amazing finding, because the difference in the four smiles was very subtle. Finally, raters ranked the esthetic attractiveness of our models higher when they were observed videotapes of the smiles than when they were rating photographs of our models smiling.

How can these findings be used?

Clinical orthodontists should consider using videotapes of patients’ smiling to document progress. They, their patients and their patients’ families will likely find this to be a more satisfactory assessment of esthetics. With things like YouTube and digital video as available as cell phones, this is less problematic in this day compared to a decade ago when videotapes were more difficult to make and store.

What do you plan to do next?

Immediate plans call for studying the kinematic differences of the smiles made by our models. I still cannot believe that the subtlety of the smiles was discernable by the raters. It was a robust finding, but I’d really like to know what it is that the raters were keying off of. Secondly, I’m also working with a group of kinesiologists and statisticians to use statistical methods that capture the “shapes” of movements through time; the methods are known as functional data analysis. The promise here is that it will be possible to identify the information that exists in time that our brains key in on as we observe the world around us. Most data and statistical tests being performed today are very much like photographs–measurements are taken at instances in time and used in statistics that look only at the snapshots of data. These new statistical tools we’re using will actually be able to identify information in the continuous ‘signal’ and this will have broad implications for other fields besides ‘smile research’. Ultimately, we will also be interested in looking at how the face, mouth and body are coordinated during behaviors such as communication. I mean, we mammals are really interesting in that we rarely sit stock still like reptiles. Think about someone talking on a cell phone in a car–they often cannot keep two hands on the wheel, because they’ve got to be gesturing with at least one arm, despite the fact that the person they’re talking to cannot see them. Why is that? Why do our brains produce so many energy-using body movements that seem superfluous? Are they superfluous?

What are ways to increase awareness around this?

If you’re asking how clinical and scientific awareness can be increased, then I’d like to see ethologists and people who study motor behavior in less controlled settings be able to make a greater impact on those experimental studies that are performed under highly controlled settings. Control over subjects and the subjects’ environments may make an experiment more convenient in terms of setup and interpretation, but it creates other problems that I think all too often aren’t fully appreciated. I’ve referred to these problems as the ‘uncertainty principle’ of motor behavior–there is an inverse relationship between the amount of control required in a study of behavior and the likelihood that the observed behavior reflects an ecologically relevant condition. If this is the case, how can we move “from benchside to chairside” with clinical studies without intermediate steps in which phenomena we study are more realistic? And it seems to me that an important, relatively easy intermediate step is to capture and analyze full facial and body movements produced under more realistic conditions.

On the other hand, if you want your readers to gain an awareness of the significance of this work, I’d tell them to watch a movie on TV or on their computer, push the pause button when the movie stars are in the middle of a conversation or when there’s a close up of a human face, and take a look at what’s paused. You’ll often catch an attractive movie star in a very unflattering snapshot. But this is exactly how clinical data and a lot of experimental data are sampled; shapshots are taken, quantified and analyzed. I think it’s important to ask what’s the quality or significance of the data in the snapshot? What were the criteria used to use one snapshot of data and not another? I’m speaking metaphorically of course, but I think it’s really important to ask, if we’re going to collapse out a continuous phenomenon into a single data point, how do we know that the data at this point in time are representative or significant?

Please tell us a little more about your department

My department is the research arm of the University of Michigan School of Dentistry. It consists of people studying cellular and molecular bases of neural development, of cancer, of developmental defects and of stem cells, tissue engineering and regeneration, and the lifestyles and pathogenesis of bacteria in the mouth. It’s a diverse group, extremely well-funded (relatively speaking in this day and age). We are also together with the prosthodontists (crown, bridge and implant specialists) in the school as a department.

We have had the unique privilege of interviewing Dr. Geoffrey Gerstner, Associate Professor in the Department of Biologic and Materials Sciences at University of Michigan.
His recent paper, co-authored with Angela Lin, Tom Braun and James McNamara titled Esthetic evaluation of dynamic smiles with attention to facial muscle activity was published in the journal American Journal of Orthodontics and Dentofacial Orthopedics. The paper studies the dynamics of a smile, an element that can contribute to overall smile esthetics.

What is your background and your research interests?

I’m interested in oral motor control of the face and mouth. I have been rather broadly trained, my formal graduate training was in neuroscience, with a clinical emphasis on oral pain and function; however, my expertise has been in neuroethology. I’ve done a lot of field work with mammals, looking at relationships between chewing variables and body sizes and lifestyles in mammals. I see my research as a mirror image of what almost everyone else does–whereas most scientists use animal models to study human diseases, my interest is in mammals and field work, using human studies as ‘controlled experiments’ where it is impossible to do such experiments on animals. Of course I hope that my controlled experiments shed significant light on health issues in humans, and this is where Angela’s graduate research project, which is the work represented in this paper, fits in.

What led you to do this type of research study?

The human face is distinct among mammals. The face presents a highly sophisticated motor system that can relay (or hide) considerable information about the internal physiological states and traits of a person or organism. How it fits in with my other projects is as follows. Feeding is a real problem for mammals, because we have to eat a lot (being warm blooded). Because of the poor design of the throat (everyone’s choked on swallowed food–wouldn’t it be better if the nose-trachea connection and the mouth-esophagus connection were completely separate?), and also because our metabolic heater requires food in the lower GI to have a high surface-to-volume ratio for absorption, we have to chew a lot. But chewing wears teeth and, unlike reptiles, which get dozens of replacement teeth, we only get one set. So the mammalian strategy appears to be to make every chew count. To make every chew count requires really fine control of the jaw, lips, tongue, soft palate and throat muscles. Such fine control results in a system that is ‘ready-made’ for additional sophisticated behaviors such as speech, communication and facial expression. So, it is my overarching hypothesis (and interest) that the complexity of food acquisition in mammals is directly linked to our complex communication skills and that this skill involves motor control of the entire head (if not entire body). So, that’s my angle on the problem. Being in a dental school puts me in contact with clinical graduate students with slightly different interests and goals from mine. I was introduced to Angela by Jim McNamara, a co-author on this paper who was interested in smiles, and the three of us, along with Tom Braun our statistician, Rick Gracely, who helped us design the assessment methods for rating smiles, and Clayton Gerstner, who did all the computer programming so that the raters could view smile videos and rate them in real time, hammered out this project, which represented a neat mix of mutual interests.

What were your findings?

As you know, still photos often don’t capture what really happens. We’ve all seen a lame picture that catches us in the act of blinking or with our mouths half open taking a bite of food. We recognize that we don’t really look like that. This is because the behaviors we produce occur through time. In fact, nothing in the real physical universe in which we live, happens or is perceived by us instantaneously. So, a photo of a forced smile is really an incomplete set of data on which to base clinical judgements and outcomes regarding facial esthetics. A photo is convenient, but incomplete. Our argument was that, real smiles occur through time–a person develops a smile, holds it for some time, and then releases it. Posed smiles used in photographs don’t happen this way and the photographed image certainly looses the temporal dimension of the smile. Hence, it seemed important to us to test whether the temporal dimension included important information that might be clinically useful. Furthermore, it seemed important to determine whether relatively natural smiles were rated differently from posed smiles. The conclusions were that the temporal dimension of smiles did in fact contain important information, and that those smiles that most closely approximated spontaneous smiles were most esthetically pleasing. First, raters who watched our four female models smile were able to identify and discriminate accurately between four different smiles (by different, I mean we trained our models to move different groups of facial muscles to produce slightly different smiles). Second, the raters rated the physical attractiveness of our models higher when they were rating the videotaped smiles that involved all the muscles used to produce spontaneous smiles as opposed to those smiles that used fewer muscles to produce smiles. This was an amazing finding, because the difference in the four smiles was very subtle. Finally, raters ranked the esthetic attractiveness of our models higher when they were observed videotapes of the smiles than when they were rating photographs of our models smiling.

How can these findings be used?

Clinical orthodontists should consider using videotapes of patients’ smiling to document progress. They, their patients and their patients’ families will likely find this to be a more satisfactory assessment of esthetics. With things like YouTube and digital video as available as cell phones, this is less problematic in this day compared to a decade ago when videotapes were more difficult to make and store.

What do you plan to do next?

Immediate plans call for studying the kinematic differences of the smiles made by our models. I still cannot believe that the subtlety of the smiles was discernable by the raters. It was a robust finding, but I’d really like to know what it is that the raters were keying off of. Secondly, I’m also working with a group of kinesiologists and statisticians to use statistical methods that capture the “shapes” of movements through time; the methods are known as functional data analysis. The promise here is that it will be possible to identify the information that exists in time that our brains key in on as we observe the world around us. Most data and statistical tests being performed today are very much like photographs–measurements are taken at instances in time and used in statistics that look only at the snapshots of data. These new statistical tools we’re using will actually be able to identify information in the continuous ‘signal’ and this will have broad implications for other fields besides ‘smile research’. Ultimately, we will also be interested in looking at how the face, mouth and body are coordinated during behaviors such as communication. I mean, we mammals are really interesting in that we rarely sit stock still like reptiles. Think about someone talking on a cell phone in a car–they often cannot keep two hands on the wheel, because they’ve got to be gesturing with at least one arm, despite the fact that the person they’re talking to cannot see them. Why is that? Why do our brains produce so many energy-using body movements that seem superfluous? Are they superfluous?

What are ways to increase awareness around this?

If you’re asking how clinical and scientific awareness can be increased, then I’d like to see ethologists and people who study motor behavior in less controlled settings be able to make a greater impact on those experimental studies that are performed under highly controlled settings. Control over subjects and the subjects’ environments may make an experiment more convenient in terms of setup and interpretation, but it creates other problems that I think all too often aren’t fully appreciated. I’ve referred to these problems as the ‘uncertainty principle’ of motor behavior–there is an inverse relationship between the amount of control required in a study of behavior and the likelihood that the observed behavior reflects an ecologically relevant condition. If this is the case, how can we move “from benchside to chairside” with clinical studies without intermediate steps in which phenomena we study are more realistic? And it seems to me that an important, relatively easy intermediate step is to capture and analyze full facial and body movements produced under more realistic conditions.

On the other hand, if you want your readers to gain an awareness of the significance of this work, I’d tell them to watch a movie on TV or on their computer, push the pause button when the movie stars are in the middle of a conversation or when there’s a close up of a human face, and take a look at what’s paused. You’ll often catch an attractive movie star in a very unflattering snapshot. But this is exactly how clinical data and a lot of experimental data are sampled; shapshots are taken, quantified and analyzed. I think it’s important to ask what’s the quality or significance of the data in the snapshot? What were the criteria used to use one snapshot of data and not another? I’m speaking metaphorically of course, but I think it’s really important to ask, if we’re going to collapse out a continuous phenomenon into a single data point, how do we know that the data at this point in time are representative or significant?

Please tell us a little more about your department

My department is the research arm of the University of Michigan School of Dentistry. It consists of people studying cellular and molecular bases of neural development, of cancer, of developmental defects and of stem cells, tissue engineering and regeneration, and the lifestyles and pathogenesis of bacteria in the mouth. It’s a diverse group, extremely well-funded (relatively speaking in this day and age). We are also together with the prosthodontists (crown, bridge and implant specialists) in the school as a department.