Pulling Faces

Face Generator (unreleased, 1980s)

Back
in the Last Days of the Atari ST, one of the first things I prototyped
with my shiny new 386PC was a face generator program.

The
idea was that you'd input exact measurements of certain proportions of
the human face, and the program would them "cartoon" them as lineart.
Once you had a small library of face metrics, you'd be able to get the
program to generate additional faces as tweened versions, randomised
versions of particular populations, or exaggerated versions of the
difference between given faces.

Since the program was only
generating lineart (and a little bit of eyeball shading), it
was easy to produce real-time changes,
and the program quickly got a few manually-programmed expression
algorithms.
If you wanted the character to look right, or up, or
down, it wouldn't just be
the eyeball alignment that that would change, parts of the
mesh overlay for the eyelids would be "pulled" in sympathy.

The
idea was that the program would have alternative versions
of the same face showing different expressions, and would
isolate
and save the
differences as "expression" files, and
then it'd accumulate a library of expressions that could be applied by
varying amounts, to any of the
other
faces. I was also working on finding the absolute minimal set of
additional features that would be useful for 3D. The idea was that
if you
were, say, a comicbook artist drawing the same character over
and
over again, then the program would give
you they key features of that face, from any given angle,
showing any given expression or combination of expressions. The artist
could then put in the surrounding profile details themselves.
It'd
basically be doing for the face
what those wooden artist's mannequins do for body poses. It'd
also
help people who couldn't really draw to produce
half-decent comics. :)

The
prototype worked satisfyingly well, but I ended up abandoning it
because
it didn't have an obvious market other than people
who needed help keeping comicbook characters fresh but consistent, or
as the basis of
a "talking head" for PC-based news delivery.
Long-term I was
playing with the idea that I could perhaps also produce a speech
synthesis
library, and have a PC download the daily news as a streaming
text
feed
with embedded gestural information, with the character of your
choice as a virtual newsreader in the corner of your screen. With more
advanced hardware than was generally available in the 1980's
one might be
able to package a mapped high-resolution face image along with its
facial metrics, and have the program morph the photographed face
real-time to match the spoken text. Choose your own Newsreader! But it
was all a bit
nebulous,
and the project was sucking up a lot of R&D time without an
obvious
financial payoff. So I pulled the plug.

Singular
Inversions' FaceGen

Anyhow, doing a bit of googling on the subject in
2008 for old times sake, it seems that there are
now a few different companies that've written custom
face-generating programs, which now output
fully-rendered 3D
images
and/or 3D data for
modeling packages. There's now a market for developing computer gaming
characters or online avatars that didn't exist when I was tinkering
with the idea.

The program that I just had a little play with was
one
called FaceGen, which has a free export-disabled
evaluation
version.

For
serious work, you're supposed to feed in front and side views of a
face, mark a few key points, and it'll generate a corresponding 3D
shape. However, FaceGen is smart enough to make a pretty good guess
using just a single front-facing (or roughly front-facing)
image.
Subsequent manipulations of the eye-mesh assume that the subject was
looking directly forwards when the photo was taken.

Here's
what it managed to do with a few single images of distinctive celebrity
faces. Some of the profile views aren't so convincing, but
that's
going to be at least partly because I didn't give the
program "profile" photos to work from (as you're
supposed
to).

Albert
Einstein

FaceGen did a decent job from
a badly-colorised source
picture. The tip of the nose should have been a little
more bulbous,
and the shape of the head looks wrong (to me), but a finished
image
would normally have
hair, so no biggie.

Angelina
Jolie

Britney
Spears

Any
webpage on computer graphics and faces HAS to include at least one
attempt at Angelina Jolie. It's The Law.

I tried two
different Britney
pics and both default FG models were a bit grim, for different
reasons. But a 50:50 average of the two "Mutant Britney" files, using
the FG "tween" function, worked
surprisingly well, and managed to distill out the common
features of both photos.

Danni
Minogue

Jennifer
Love Hewitt

A
few manual
parameter tweeks
helped this one no end. For this test, I switched the texture
off, saw how close I could get to a recognisable stylised
representation, and then switched the texture back on again.

If
you were
doing this
properly, you'd want to erase the
overlapping hair in the source image first. Other than that,
it came out pretty well.

Zhang
Ziyi

Janet
Jackson

FG
doesn't
ask you to identify
the eyebrow line, it works
it out for itself.
In this case, FaceGen
got thrown by the veryheavy eye
makeup, and
decided that the
top of Janet's eyelids were actually her eyebrows. The composite image
looks adequate, but if you switch the texture map off ... ew!
It's gone horribly wrong ...

The
result of using a rough, bitty
image, with teeth
showing. Unless there's some trick that I've missd, FG doesn't seem to
like teeth in an image, and doesn't have an obvious way of letting you
set a
lip line "edge" (see earlier comments about not being able to manually
define eyelid cutouts).

I
thought that the "Borg
eyepiece" might confuse FG, but it cleverly ignored it.

Here are a few "before
and after" pictures of what FaceGen does. On the left is a small
thumbnail of the original source image, then we have a view of the
output
FaceGen face, a smoothed view of just the surface, the surface plus the
source mesh grid, and the mesh grid with the "texture" overlay derived
from the photograph. The file for these images is here. As before,
click on the "bordered" images to see larger versions.

Charisma
Carpenter

source
image

FaceGen
output

smoothed
surface

mesh
without texture

texture
and mesh

FaceGen then lets you morph, tween,
tweek, and shift a face by age, or
ethnicity, or a variety of other averaged trends, and can also
apply some facial expressions.

The
program seems to be hampered by the lack of a decent expression
library, but I guess that if you're producing faces that'll be rendered
as a tiny piece of screen in a shoot-em-up, perhaps there's not so much
justification in developing a proper set of convincing expressions,
beyond cartoon shock and "Grrrrr!". It also has "phoneme-specific"
expressions, presumably for people who need to animate talking
faces.

FaceGen is a commercial program, but
the free demo version is okay for
"play" as
long as you don't mind losing its ability to export .obj
object files. It'll still export "[.fg"
files in its own format, which consists of the face parameter set,
manual parameters, and the texture map. Some games can apparently
import fg files, so if you're a game enthusiast and want to
create your own face for your online avatar, or you think that it'd be
funny to have the cast of Casablanca shooting aliens or the panel of X
Factor running amok in Grand Theft Auto, then FaceGen might be a
reasonably fast way to get results. If you want to export
your [.fg files to more generic computer-modelling mesh files so that
you can
use them in "pro" 3D graphics apps, then you'll probably need to get
the full commercial version.

Looxis Faceworx

Another freebie is FaceWorx from
Looxis. This one does
export object files, but it's only free for non-commercial use.

Looxis
have more of an emphasis on 3D portraiture, and they have booths that
let you sit down, be photographed, and have your head computer-modelled
and
laser-etched into a glass block. Consequently their program is
designed more as a front-end for generating accurate 3D face
surface data, for
more conventional
computer-modelling applications.

FaceGen vs Faceworx

The two companies are coming at the problem from
different
angles. Looxis
are focused on generating standard, accurate 3D files for export to
other
programs, and the Looxis program lets you spend a lot of time
fine-tuning the details of a face by moving guide marks. If you want
to, you can use a lot of guide marks to produce a
very faithful 3D
mesh-map of the given face. It'll tween and smooth the data, and
project the original photograph over the surface as a texture map, but
if you really want a truly convincing rendering of a face's contours,
it's really up to
you to spend some time clicking very accurately.

The emphasis of the SI
programme is more on automatic analysis and
recognition of facial structure. FaceGen
already knows a
lot about faces. Although
it renders 3D maps and textures in the same way as Faceworx, FaceGen stores
faces as combinations of physiological parameters (nostril tilt, nose
length, etc.). FaceGen's "photofit" function only needs a very
small number of points - sides of the nose and mouth, centre
of
the eyes, etc -- and it then tries to recreate a
face by
analysing the photo(s) provided and tilting, sizing, and randomising
its model until it finds a combination of metrics that it thinks might
produce the best match
to those photographs, when appropriately angled and lit. It
then
generates
a mesh that is texture-mapped in a similar way to Faceworx. The
computer's doing all the serious work, but all that
analysis means that if your computer is slow and doesn't have a
graphics
accelerator, it might take fifteen minutes for it to analyse a face.
Once FaceGen has finished, you'll get a
realistic-looking face model,
but it might veer slightly away from the original in terms of any
unusual
aspects that are difficult to replicate using that stack of
parameters. The texture map created from the original photo will
usually do a decent job of putting these missed details
back, but
there
might be a few unusual shaping subtleties that the software
can't
quite mimic by varying its standard parameter set. The use of
a
fixed
parameter set means that extreme "caricature" variations can
still look reasonably
natural, but it won't always give enough variables to perfectly emulate
the precise line of a distinctive nose, or an eyelid crease.
FaceGen also looks
as if it's trying to "rotate" eyes by distorting a mesh
rather than by implementing proper swivellable eyeballs as spheres
behind the
face.

Summary:

Faceworx allows you to
use front and side images to manually place accurate points, and
so it tends ot produce crisper detail and
hopefully a more recognisable, more accurate final 3D contour
map of
the
subject ... but since it's relying on you to place
all those
control points, it takes a certain amount of skill, and any lack of
accuracy on the user's part can result in
a face that has subtle wonkiness or jagginess, fracturing
the
illusion. And once you've generated your static 3D model, there's not a
lot you can do with it other than light it and look at it from other
angles, or export it into another program for more powerful processing.

FaceGen does almost
all the work for you, and can make a
brave attempt at a 3D map from even a
single image ... but some aspects of the resulting face
profile might be a little "generic". It's good at
matching a
lot of aspects of bone structure, and will tend to get the
right
sort of jawline and head shape, but if you zoom on on an
eyelid, you'll find that there don't seem to be
enough "eye
shape" parameters for it to always be able to recreate a
"distinctive"
eye properly within the 3D model, without the help of the
overlaid
"photographic" layer ... if you're modelling a "known" face, you'll
tend
to find that the contours of the eye region in the output file are
slightly
standard-looking, and
that the distinctive detail is instead being superimposed by the
projected
texture map. If you take a recognisable face, and switch that texture
map off,
so that you're just looking at a pure set
of
computer-generated contours, the "known" face isn't
always recognisable. And because it's relying on the projected
texture map, some features can look a bit fuzzy and imprecise. It
manages some subtle features surprising
well, but other aspects like lip line, eyelid detail or the
fleshy tip of a nose are more difficult to parameterise.

Disclaimers

I
don't have any relationship with any of these other companies or their
programmers. I just thought that the programs were cool.

I
used celebrity faces in these "tutorial" examples because a lot of
people will be familiar with them, and unless you're familiar with a
face, it's difficult to judge how well a program works. Educational
use. This doesn't mean that it's okay to use other people's faces in
commercial products or advertising or promotions. It isn't. If you
start using one of these files to produce materials in a way that would
normally require the subject's permission for the use of an image, and
you don't have that permission, you're still liable to get
sued by them.

FaceGen
and Faceworx are both primarily aimed at applications in which the
operator
has full control over the posing and lighting of the original
photograph (front and side view, eyes front, even lighting, blank
expression). Some of the other photographs that I tried, with side
lighting or
strongly tilted faces, didn't work as well as the images above. If
you're thinking of buying FG, try the free version first,
with some material that's representative of what you'll be
working
with.