Towards Automated Classification of Emotional Facial Expressions

Lewis Baker, Rutgers University

Vanessa Lobue, Rutgers University

Elizabeth Bonawitz, Rutgers University

Patrick Shafto, Rutgers University

Abstract

Emotional state influences nearly every aspect of human cognition.
However, coding emotional state is a costly process that relies on proprietary
software or the subjective judgments
of trained raters, highlighting the need for a reliable, automatic method of
recognizing and labeling emotional expression. We demonstrate that machine
learning methods can approach near-human levels for categorization of facial
expression in naturalistic experiments. Our results show relative success of
models on highly controlled stimuli and relative failure on less controlled
images, emphasizing the need for real-world data for application to real-world
experiments. We then test the potential of combining multiple freely available
datasets to broadly categorize faces that vary across age, race, gender and
photographic quality