We present a method for recovering the reflectance properties of
all surfaces in a real scene from a sparse set of photographs, taking
into account both direct and indirect illumination. The result is a
lighting-independent model of the scene's geometry and reflectance
properties, which can be rendered with arbitrary modifications to
structure and lighting via traditional rendering methods. Our
technique models reflectance with a low-parameter reflectance model,
and allows diffuse albedo to vary arbitrarily over surfaces while
assuming that non-diffuse characteristics remain constant across
particular regions. The method's input is a geometric model of the
scene and a set of calibrated high dynamic range photographs taken
with known direct illumination. The algorithm hierarchically
partitions the scene into a polygonal mesh, and uses image-based
rendering to construct estimates of both the radiance and irradiance
of each patch from the photographic data. The algorithm computes the
expected location of specular highlights, and then analyzes the
highlight areas in the images by running a novel iterative
optimization procedure to recover the diffuse and specular reflectance
parameters for each region. Lastly, these parameters are used in
constructing high-resolution diffuse albedo maps for each surface.

The algorithm has been applied to both real and synthetic data,
including a synthetic cubical room and a real meeting room.
Re-renderings are produced using a global illumination system under
both original and novel lighting, and with the addition of synthetic
objects. Side-by-side comparisons show success at predicting the
appearance of the scene under novel lighting conditions.

This algorithm is useful in image-based modeling and rendering,
such as image-based visualization of a real scene and integration
between virtual world and reality. It can be used for rendering a real
scene with virtual objects, under novel lighting conditions as well as
moving specular highlights to the right places for novel viewpoints
under original illumination.

Before testing on a real room, we did a test on a simulated cubical room
with 6 tiny spherical light sources, and compared the recovered parameters
with ground truth. The results are shown in the paper.

Some examples of the input radiance images of a real room to our algorithm.

The recovered geometry of the real room: an example of the input geometry to the algorithm.
The white dots are recovered camera positions.

We set up 3 concentrated globular light bulbs in the room to help get better estimation of specular parameters.

On the left is one of the images for the whiteboard.
There are a lot of specular highlights on it.
On the right is the recovered diffuse albedo map for the whiteboard.
It is almost uniform besides the markings.

The diffuse albedo maps of three identical SIGGRAPH'99 posters.
Although they were placed at different places inside the room,
the recovered albedo maps look almost the same, which shows
reflectance is lighting-independent.

On the left is an image with color bleeding effect on the white wall.
On the right is the diffuse albedo map of that part of the wall with
color bleeding removed.

The first row shows three real photographs of the room.
The second row shows three corresponding synthetic images
at the same viewpoints.

A synthetic panoramic view of the room under the original lighting

A synthetic panoramic view of the room under a novel lighting condition

A synthetic panoramic view of the room with virtual objects.

Movie 1: visualization of the hierarchical triangular mesh and the average color of each triangle from photographsMovie 2: synthetic rendering under the original lighting conditionMovie 3: synthetic rendering under a novel lighting conditionMovie 4: synthetic rendering with virtual objects