Automatic Multimodal Descriptors of Rhythmic Body Movement

1 University of Cambridge, United Kingdom
2 Institute for Creative Technologies, University of Southern California

International Conference on Multimodal Interaction 2013

Abstract

Prolonged durations of rhythmic body gestures were proved to be correlated with different types of psychological disorders. To-date, there is no automatic descriptor that can robustly detect those behaviours. In this paper, we propose a cyclic gestures descriptor that can detect and localise rhythmic body movements by taking advantage of both colour and depth modalities. We show experimentally how our rhythmic descriptor can successfully localise the rhythmic gestures as: hands fidgeting, legs fidgeting or rocking, significantly higher than the majority vote classification baseline.
Our experiments also demonstrate the importance of fusing both modalities, with a significant increase in performance when compared to individual modalities.