Start here!

Not sure what the AAAC is all about? Overwhelmed by the amount of information available? This page gives you a structured entry point.

Emotion-oriented computing is a broad research area involving many
disciplines. The AAAC (Association for the Advancement of Affective Computing) grew out of the EU-funded network of excellence HUMAINE. This project was making a
co-ordinated effort to come to a shared understanding of the issues involved,
and to propose exemplary research methods in the various areas, as explained below.

This overview
page presents a proposed “map” of the research area, distinguishing core
technologies from application-oriented and psychologically oriented work.
Current research issues in the various areas are briefly outlined, and
references for further reading are given.

The following figure is our proposed map of the thematic areas
involved in emotion-oriented computing.

The central column represents the areas where
purely technological challenges loom largest. Detection and synthesis are
distinguished because the background technologies used are very different.
‘Planning action’ involves modelling action
patterns that might be expected in a particular emotional state, either for
driving an artificial agent or for anticipating a human’s action tendencies
in a given state.

The left hand column deals with issues where
application is most obviously of concern. Emotion-related usability issues are more difficult to
address thantask-oriented ones, because emotional responses are subtle and
easily disrupted by interventions that are meant to measure them. Iterative
user-centered design methods are used for tuning a system to non-rational
preferences and dispositions in the user. Work on emotion in complex media is treated separately
because the perspective towards applications in the relatively near future
requires a different approach than the core technologies in the central
column.

The right hand column contains the sub-areas with the
strongest roots in psychology. We distinguish theory and empirical data, because
existing theory is informed by different kinds of data than what seems
relevant for emotion-oriented computing. As a result, there are creative
tensions between that kind of data collection and existing psychological
theory. Similarly, psychological theory can use technology to test its
accuracy and completeness, because the actions of artificial agents can be
controlled with a precision that is impossible with humans.

The task of synthesising agents that can interact
emotionally is at the center because it summarises the state
of the art – it cannot be done well without satisfactory progress in all
the others.

Emotion-oriented computing needs tractable ways of describing the states
that matter to it. Familiar schemes pull people to think in terms of pure
fullblown emotion rather than pervasive emotion-related phenomena like
friendliness, trust, distress, sincerity and mixed or time-varying emotions.
Research in this area is clarifying the states most likely to matter to
emotion-oriented computing, and adapting ideas from psychology such as soft
coding, dimensional representation, and appraisal theory to provide
representations that are more tractable than list of irreducible
categories.

A joint understanding requires clear working definitions of jointly used
terms. The term “emotion” has notoriously been used with very different
meanings. Psychological theory has proposed working definitions of a range of
affective states, including (fullblown) emotion, mood, attitude etc. These
definitions are currently being refined in HUMAINE. Different emotion models
propose various analyses of emotional processes.

Comparing such theoretical models with computational models provides new
insights of what is actually required for an affectively competent agent.
Human behaviour is a natural reference for artificial systems, and as such it
needs to be properly understood. It can provide a benchmark, but it is also
important to understand individual and situational differences. One highly
relevant aspect of this is research in the types of emotional and
emotion-related states that are typically experienced by people in their
daily lives.

Progress in most areas depends on good primary records, with appropriate
annotation, of people interacting emotionally with each other and machines.
There is a need for both generic material (to drive fundamental research) and
application-specific (to achieve tuning to particular settings). Records also
need to reflect differences between people related to their gender, culture,
and individual characteristics, and the context in which they are set.
Techniques for both collection and annotation have been developing, and are
currently being exemplified in the collection of a pilot database.

Research has explored many of the channels that people use to form
impressions of each other’s emotions – facial expression, paralinguistic,
gesture, choice of words and actions. Physiological correlates of affect also
exert a special fascination. High recognition rates can be obtained with
acted or carefully elicited data, but the field has moved on to deal with
naturalistic material. There it is difficult to exceed 80% success in a
binary distinction. Multimodal integration seems the likeliest key to real
improvement.

As in perception, the existence of multiple channels is critical. Early
‘Embodied Conversational Agents’ (ECAs) tried to convey emotion using
analyses of static faces showing fullblown emotions. The results are
recognisable, but disconcerting. Research has moved on to study the rich
range of signals that transmit emotion-related information in interactions,
and the ways they are co-ordinated and dependent on the other party’s
actions. That raises topics such as eye movements, backchannelling, gesture,
and ‘idle movements’. Co-ordinating such behaviour is a precondition for
believable interactions.

An agent cannot engage emotionally unless it has a kind of empathy, i.e.
it can understand at some level what a person’s emotional state might
dispose him or her to do, and how that disposition might be affected by
different actions that the agent might take. Hence interfaces need to include
models of central states and processes in the user that incorporate emotion.
At present, several very different types of model are available – AI (using
propositional representations); neurally inspired; and artificial life. Each
has strengths, but it is difficult to combine them, and finding ways to do
that stands out as the immediate priority.

Emotion can be expressed and influenced not only through basic channels
established by evolution, but also through music, colour, typography, and
above all language. Within language there are many ways to express and
influence emotion, including choice of argument, lexical selection,
politeness, and humour. Work is in progress integrating all of these into the
theory and practice of emotional communication.

Translating theory into product poses special problems in the area, not
least because emotional aspects of response are singularly difficult to
measure without changing the experience. Innovative usability tests and
user-centered design methods are needed to gauge the kinds of innovation
people may want and the way they respond to prototypes, to deliver
appropriate information to designers, investors, and users, and so on. Viable
products depend on combining these streams with traditional research.

Influencing people’s emotions raises ethical questions, but
over-reaction to the issue could stifle thoroughly desrirable developments.
The field urgently needs an ethical framework that distinguishes between
benign and suspect kinds of development, allied to appropriate monitoring
systems. In HUMAINE, a framework based on Principlism has been proposed. It
is important to consolidate that kind of framework and ensure that it is
generally accepted.