To link to the entire object, paste this link in email, IM or documentTo embed the entire object, paste this HTML in websiteTo link to this page, paste this link in email, IM or documentTo embed this page, paste this HTML in website

MODELING NONVERBAL BEHAVIORS FOR VIRTUAL AGENTS
by
Jina Lee
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Ful llment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(COMPUTER SCIENCE)
May 2012
Copyright 2012 Jina Lee

Virtual agents are autonomous software characters that support face-to-face interactions with human users. They are capable of understanding human input (i.e. speech, text input) and automatically generating responses that are adaptive to the context of the interaction. By communicating through verbal and nonverbal channels (i.e. behaviors without words), virtual agents can support meaningful social interactions with human users. ❧ One of the main goals in virtual agent research is to emulate how humans interact face-to-face. While traditional human-agent interaction was mainly accomplished through speech or text, with the emergence of better graphical representation and control over the virtual agent's embodiment, communication through nonverbal behaviors has become an active research area. However, nonverbal behavior is complex, involving many different behaviors, such as facial expressions, arm gestures, and gaze movements. There is also a complex mapping between nonverbal behaviors and their impact on communication (the communicative functions) and this creates a great challenge in the task of behavior authoring for virtual agents. ❧ The central goal of this research is to explore ways to derive models that automatically generate nonverbal behaviors and can thereby greatly facilitate behavior authoring. In this research, two major approaches to provide computational frameworks for generating nonverbal behaviors are explored: a literature-based approach and a machine learning approach. The former approach encodes the findings of psychological research into a set of rules, which is validated and prioritized through additional video analysis. The framework developed from this work has been incorporated within a growing number of different virtual agent systems. The machine learning approach focuses on learning the patterns of speaker behaviors, including head nods and eyebrow movements, as well as the behaviors of people with different characteristics (i.e. different job roles and behavioral traits). The objective evaluations show that the probabilistic models learned on a subgroup of people achieved better learning performance and the subjective evaluation study shows that the characteristics of the subgroups of people learned by the models can be carried through their generated behaviors. ❧ This research contributes to the development of virtual agents by facilitating the behavior authoring process, providing comparisons of different approaches for modeling nonverbal behaviors that has not been studied extensively before, and understanding the discrepancies between the behavior modeling process and the perception of the behaviors through evaluation studies with human subjects.

The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

MODELING NONVERBAL BEHAVIORS FOR VIRTUAL AGENTS
by
Jina Lee
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Ful llment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(COMPUTER SCIENCE)
May 2012
Copyright 2012 Jina Lee