Jan, 2017
Paper Accepted to ICASSP 2017 : UNSUPERVISED LATENT BEHAVIOR MANIFOLD LEARNING FROM ACOUSTIC FEATURES: AUDIO2BEHAVIOR
Haoqi Li, Brian Baucom, Panayiotis Georgiou
In this work we exploit the slow varying properties of human behavior. We hypothesize that nearby segments of speech share the same behavioral context and hence share a similar underlying representation in a latent space. Specifically, we propose a Deep Neural Network (DNN) model to connect behavioral context and derive the behavioral manifold in an unsupervised manner. We evaluate the proposed manifold in the couples therapy domain and also provide examples from publicly available data (e.g. stand-up comedy). We further investigate training within the couples’ therapy domain and from movie data. The results are extremely encouraging and promise improved behavioral quantification in an unsupervised manner and warrants further investigation in a range of applications.

April, 2015
Paper Accepted to ICASSP 2015 : REDUNDANCY ANALYSIS OF BEHAVIORAL CODING FOR COUPLES THERAPY AND IMPROVED ESTIMATION OF BEHAVIOR FROM NOISY ANNOTATIONS
Md Nasir, Brian Baucom, Panayiotis Georgiou, and Shrikanth S. Narayanan
Assessment and quantification of behavior is an important research objective in the recently developed field of behavioral signal processing. This paper focuses on the estimation of behavior from noisy human assessment. It aims to address the redundancy of behavioral descriptors for couples therapy by introducing a lower-dimensional representation of the behavioral space. We present an improved method for estimating the ground truth of behavioral ratings from assessment by multiple experts or annotators. The results show improved estimation performance using the proposed method and provide an insightful analysis of reconstruction error and decorrelation of annotator bias in the reduced behavioral space.

April, 2015
Paper Accepted to ICASSP 2015 : A LANGUAGE-BASED GENERATIVE MODEL FRAMEWORK FOR BEHAVIORAL ANALYSIS OF COUPLES'S THERAPY
Sandeep Nallan Chakravarthula, Rahul Gupta, Brian Baucom, and Panayiotis Georgiou
Observational studies for psychological evaluations rely on careful assessment of multiple behavioral cues. Recent studies have made good progress in automating the psychological evaluation, which often involved tedious manual annotation of a set of behavioral codes. However, the current methods impose strict and often unnatural assumptions for evaluation. In this work, we specifically investigate two goals:
(1) Human behavior changes throughout an interaction and better models of this evolution can improve automated behavioral annotation and
(2) Human perception of this evolution can be quite complex and non-linear and better techniques than averaging need to be investigated.
For this purpose, we propose a Dynamic Behavior Modeling (DBM) scheme, which models a spouse as undergoing changes in behavioral state within a session, and contrast it against a Static Behavior Model (SBM) which allows only a constant session-long behavioral state. We use Negativity in a couples therapy task as our case study. We present results and analysis on both models for capturing the local behavior information and predicting the session level negativity label.

April, 2015
Paper Accepted to ICASSP 2015 : QUANTIFYING EDA SYNCHRONY THROUGH JOINT SPARSE REPRESENTATION: A CASE-STUDY OF COUPLES’ INTERACTIONS
Theodora Chaspari, Brian Baucom, Adela Timmons, Andreas Tsiartas, Larissa Borofsky Del Piero, Katherine Baucom, Panayiotis Georgiou, Gayla Margolin, and Shrikanth S. Narayanan
The co-variation degree between individuals in their physiological signals can reveal insights about the quality of their interaction as well as their personal characteristics. In an effort to capture the amount of synchrony between Electrodermal Activity (EDA) streams occurring in parallel during dyadic interactions, we propose Sparse EDA Synchrony Measure (SESM), an index derived from the joint sparse representation of EDA ensembles. Sparse decomposition is performed using Simultaneous Orthogonal Matching Pursuit (SOMP) from a knowledge-driven dictionary of tonic and phasic atoms, capturing the slow-varying trends and high-frequency signal fluctuations, respectively. At each iteration the atom having the maximum average correlation with the residuals is selected. We compute SESM as the negative natural logarithm of the joint reconstruction error and evaluate it with data from interactions of married and young dating couples participating in tasks of varying emotional intensity. Through statistical analysis and multiple linear regression experiments, our results indicate that SESM depicts significant differences across tasks in both datasets considered and can be associated to individuals’ attachment-related characteristics.

June, 2014
Paper Accepted to Interspeech 2014 : Predicting Client's inclination towards Target Behavior Change in Motivational
Interviewing and investigating the role of laughter
Rahul Gupta, Panayiotis G. Georgiou, David C. Atkins, and Shrikanth Narayanan
Motivational interviewing (MI) is a goal oriented psychotherapy that facilitates intrinsic motivation within a client in order to change behavior in a dialog setting. The Motivational Interviewing Skills Code (MISC) is a manual observational coding method used to quantify and evaluate the quality of MI sessions using their audio-visual recordings. However, this coding method is both labor intensive and expensive. We present an approach towards automating MISC assignments in MI involving addiction cure. Specifically, we focus on predicting valence for "Client Change Talk" (ChangeTalk) utterances, which indicate a client's attitude towards a "Target Behavior Change" (Target). We further study the effect of incorporating counselor behavior in the model. We observe that our best model achieves an unweighted accuracy of 50.8% in a 3-way classification of positive vs negative valence ChangeTalk vs no ChangeTalk. Furthermore, we study the effect of including non-verbal behavior, specifically laughters, in our model. Information regarding location of laughters improves the unweighted accuracy of our model to 51.4% and our experimental results suggest prosodic differences in laughters belonging to ChangeTalk utterances with different valences.

June, 2014
Paper Accepted to Interspeech 2014 : Modeling Therapist Empathy through Prosody in Drug Addiction Counseling
Bo Xiao, Daniel Bone, Maarten Van Segbroeck, Zac E. Imel, David C. Atkins,
Panayiotis G. Georgiou, Shrikanth S. Narayanan
Empathy measures the capacity of the therapist to experience the same cognitive and emotional dispositions as the patient, and is a key quality factor in counseling. In this work we build computational models to infer the empathy of therapist using prosodic cues. We extract pitch, energy, jitter, shimmer and utterance duration from the speech signal, and normalize and quantize these features in order to estimate the distribution of certain prosodic patterns during each interaction. We find significant correlation between empathy and the distribution of prosodic patterns, and achieve 75% accuracy in classifying therapist empathy levels using this distribution. Experiment results suggest high pitch and energy of the therapist are negatively correlated with empathy. These observations agree with domain literature and human intuition.

June, 2014
Paper Accepted to Interspeech 2014 : Unsupervised Speaker Diarization Using Riemannian Manifold Clustering
Che-Wei Huang, Bo Xiao, Panayiotis G. Georgiou, Shrikanth S. Narayanan
We address the problem of speaker clustering for robust unsupervised speaker diarization. We model each speaker homogeneous segment as one single full multivariate Gaussian probability density function (pdf) and take into consideration the Riemannian property of Gaussian pdfs. By assuming that segments from different speakers lie on different (possibly intersected) sub-manifolds of the manifold of Gaussian pdfs, we formulate the original problem as a Riemannian manifold clustering problem. To apply the computationally simple Riemannian locally linear embedding (LLE) algorithm, we impose a constraint on the length of each segment so as to ensure the fitness of single-Gaussian modeling and to increase the chance that all k-nearest neighbors of a pdf are from the same sub-manifold (speaker).