An alternative visual input is also being evaluated. Face modeling
can be used to recover subtle facial detail (beyond blob tracking) for
a more convincing interaction. A system which automatically detects
the face and tracks it has been implemented
[26]. It is capable of tracking the 3D rotations and movements
of a face using normalized correlation coupled with structure from
motion. In addition, at each moment in time, it computes an
eigenspace model of the face's texture. This texture description is
used to infer corresponding 3D deformations statistically
[27]. This system generates a real-time temporal sequence which
includes XYZ translations, 3D rotations as well as a set of texture
and deformation scalar values (in an eigenspace).
Figure 10.1 depicts the face tracking algorithm and
examples of the temporal sequences being output in real-time.

Figure 10.1:
3D Face Modeling and Tracking

To synthesize an output, a 3D renderer reconstructs a 3D facial model
in real-time using estimated deformation coefficients, texture
coefficients, rotations and translations. The sample output is shown
in Figure 10.1(d). The data representing each static
frame can again be a time series (with 50 dimensional features) and
the above ARL system analysis is currently being applied to this
platform.