Forewords

Shape modeling is to represent generic object geometry by a number of models
which account for the regularity and variablity of natural objects. Shape
modeling is the fundation for object recognition under change of pose,
deformation, and varying lighting coditions. However, this is an extremely hard
problem, and most of the mathematical studies appear to be totally irrelevant
to natural object shape description.

Key questions are:

What are the domain or coordinates for shape description? In what mathematical
spaces do natural shape live?

How do we define a meaningful metric (distance measure) in such space?

There is no distinct boundary between shape and texture, can we construct shape
models living in a continuous spectrum of texture models?

The recent study of
texture theory indeed shed light on shape modeling, and we feel these
problems can be answered in near future. In our group, we made four attempts to
study shape models, mostly on 2D outlines of objects. We are planning for the
5th project on shape sketch which works on both contour and inner curves.

FORMS: a Flexible Object Recognition and Modeling System

1). Compute the medial axis of a shape in a bottom-up and top-down loop.
Because the medial axis is ill-defined it must be regularized against some
prior shape models. This is accomplished by a set of graph editing operators
below. It is a graph matching with graph editing process.

2). Then the shape is decomposed according to the computed medial axes. We
assumed a hinge joints for all parts: For example: a dog shape is decompsoed
into 7 parts

Decomposition of a fish into rectangular and circular parts.
Each of these parts are assumed to be flexible deformation from two primitives:
a rectangle for elongated limes and a circle for short parts. Such as the fins
of the fish. Then the deformation is characterized by principle components
analysis, learned from data, the PCA for the two types of parts are shown
below.

Based on an attributed graph matching, it can be used for object recognition.
This work is a truly generative model of shape.

Stochastic Medial Axis and Gestalt Laws in Markov Random Field

The previous FORMS project is a generative shape model. Motivated by the success
of texture modeling, we made the second attempt to modeling shape by a
descriptive (Gibbs model). Two papers reported this project:

Some of the observed shapes as training examples, from which some shape
statistics are extracted along the contour (co-linearity and co-circularity)
and cross the medial axis for (proximity, parallelism, and symmetry). These are
the popular Gestalt features. Then histograms of these features are accumulated
across the data set.

By maximum entropy, we construct a shape model which reproduces the observed
statistics. This figure shows three stages of the Markov chain sampling of this
descriptive shape model. At the beginning, the sample (left) is very irregular.
After adding the contour-based statistics, the sampled shape (middel) becomes
smooth but bloby. After further adding the region-based statistics, the sample
shape (right) has symmetric and elongated limes, just like the natural shapes.
This descriptive model does not know parts or joints.

More examples from the Markov chain random walks, look, how much they resemble
the spirit of those animal shapes in the training set !

A Linear Additive Model for Shape Modeling

Now with our match better understanding of the texture and texton theory, we are
making a third attempt to model natural object shapes. This time, we try the
linear additive model. A shape is a linear sum of a number of "shapelets" (like
wavelets for images). This is done by a MS student Alex Dubinskiy.

For each shape, we construct a set of linear bases represented by ellipses in
the top row. The bases with the same color add up to a part (wings, tail, and
head etc.) When they are all added up, they yield the whole shape (see the
dashed curves). Thus we call the bases graph up the "shape
script". A desciptive model can be learned on this shape
script to generate random animal shapes.

Hierarchic Shape Modeling and Sketch

After three trials, we believe we got some clue !! This is to integrate both
generative model and generative model, integrate both region based parts and
contour based parts. It becomes a shape sketch following artists footprint. It
will be a hierarchic model which capture both the shape contour and the
internal boundary (texture or sketches). Here we go !