Wojciech Matusik

Details in mesh animations are difficult to generate but they have
great impact on visual quality. In this work, we demonstrate a practical
software system for capturing such details from multi-view
video recordings. Given a stream of synchronized video images
that record a human performance from multiple viewpoints and an
articulated template of the performer, our system captures the motion
of both the skeleton and the shape. The output mesh animation
is enhanced with the details observed in the image silhouettes. For
example, a performance in casual loose-fitting clothes will generate
mesh animations with flowing garment motions. We accomplish
this with a fast pose tracking method followed by nonrigid deformation
of the template to fit the silhouettes.

We accurately capture the shape and appearance of a person's hairstyle. We use triangulation and a sweep with planes of light for the geometry. Multiple projectors and cameras address the challenges raised by the reflectance and intricate geometry of hair. We introduce the use of structure tensors to infer the hidden geometry between the hair surface and the scalp. Our triangulation approach affords substantial accuracy improvement and we are able to measure elaborate hair geometry including complex curls and concavities. To reproduce the hair appearance, we capture a six-dimensional reflectance field. We introduce a new reflectance interpolation technique that leverages an analytical reflectance model to alleviate cross-fading artifacts caused by linear methods.

Commercial motion-capture systems produce excellent in-studio
reconstructions, but offer no comparable solution for acquisition
in everyday environments. We present a system for acquiring motions
almost anywhere. This wearable system gathers ultrasonic
time-of-flight and inertial measurements with a set of inexpensive
miniature sensors worn on the garment. After recording, the information
is combined using an Extended Kalman Filter to reconstruct
joint configurations of a body. Experimental results show that even
motions that are traditionally difficult to acquire are recorded with
ease within their natural settings.

We present new hardware-accelerated techniques for rendering surface
light fields with opacity hulls that allow for interactive visualization
of objects that have complex reflectance properties and elaborate
geometrical details. The opacity hull is a shape enclosing the
object with view-dependent opacity parameterized onto that shape.
We call the combination of opacity hulls and surface light fields the
opacity light field. Opacity light fields are ideally suited for rendering
of the visually complex objects and scenes obtained with 3D
photography. We show how to implement opacity light fields in
the framework of three surface light field rendering methods: viewdependent
texture mapping, unstructured lumigraph rendering, and
light field mapping.

We present a system for designing novel textures in the space of textures induced by an input database. We capture the structure of the induced space by a simplicial complex where vertices of the simplices represent input textures. A user can generate new textures by interpolating within individual simplices. We propose a morphable interpolation for textures, which also defines a metric used to build the simplicial complex. To guarantee sharpness in interpolated textures, we enforce histograms of high-frequency content using a novel method for histogram interpolation. We allow users to continuously navigate in the simplicial complex and design new textures using a simple and efficient user interface.

Video matting is the process of pulling a high-quality alpha matte and foreground from a video sequence. Current techniques require either a known background (e.g., a blue screen) or extensive user interaction (e.g., to specify known foreground and background elements). The matting problem is generally under-constrained, since not enough information has been collected at capture time. We propose a novel, fully autonomous method for pulling a matte using multiple synchronized video streams that share a point of view but differ in their plane of focus. The solution is obtained by directly minimizing the error in filter-based image formation equations, which are over-constrained by our rich data stream.