Turning the inside out: Can facial expressions of emotion contribute to predicting experienced emotions in music?

In my PhD project I am trying to shed some light on one of the probably most frequently asked questions in music psychology, namely whether music can induce emotion, and if so, whether these emotions are equivalent to utilitarian and basic emotions we experience in everyday life, or if they are rather aesthetic and epistemic emotions, as for example Scherer and Coutinho (2013) suggest.

While in the context of perceived emotions in music the application of one or another form of self-report can be discussed to be an efficient way to gather judgements about the overall expressed emotion in the music, this doctoral project focuses on introducing an approach for studying music-induced emotion that takes the dynamic and temporal nature of both music and emotion into account. By applying a multi-method approach using physiological measurements (skin conductance, heart rate, blood volume pulse), automated face analysis, and continuous as well as retrospective self-report, a better understanding about emotions experienced in response to music and their characteristics can be achieved. This approach will help to understand whether and to what extent basic emotions and aesthetic emotions are experienced in response to music, and further, how these subjective experiences relate to embodied components such as changes in physiology and motor expression.

In regards to facial motor expression, previous studies have focused on the activation of two facial muscle groups that merely give information about positive or negative valence of an emotion. Instead of limiting facial motor expressions to activation of the zygomaticus major and corrugator supercilii muscle regions I am applying automated face analysis software to get a broader picture about the distinct emotions involved. This method will allow me to get more specific information on the emotion beyond valence, and as opposed to facial electromyography this is a non-invasive and non-obtrusive approach.

At a later stage of the project, this analysis will be expanded to the activation of action units (i.e. specific facial muscles rather than patterns) and the question to what extent these activations predict emotion ratings will be investigated in this project. In addition to basic emotions, the focus will be expanded to aesthetic emotions as measured by the scale.

In the first stage of this project, individuals will be tested in a controlled environment, whereas in the second stage audiences will be tested during a live performance.

One of the many questions that have not yet been addressed in a systematic way is, to what extent the combined changes in activation of the autonomic nervous system (i.e. changes in heart rate, skin conductance, blood volume pulse) and somatic nervous system (i.e. facial motor expression) predict continuous and retrospective emotion ratings made in response to the subjectively felt experience.

Main Contribution:

If this facial motor expressions can be used to successfully predict subjective experienced emotions in response to music, this approach could potentially not only be used to study emotions in music without relying on self-report, which could be subject to demand characteristics and researcher bias, but could also facilitate studying experienced emotion in groups or individuals that are not able to verbally express their subjective experiences.