This work is supported by NSF grant #0413312, "3D Perception of Specular Surface".

Our goal is to establish the phenomenology of human perception of reflective materials. We seek to uncover the “qualia” of a reflective surface: Why do we see specular reflections as such rather than textures attached to a surface? Other questions we would like to address are: to what extent do humans perceive qualitative and metric shape differences? Do they make use of any knowledge (metric, statistical) about ambient structure? Are some ambient structures more informative then others?

Why do we see some surfaces as reflective?

Why do we see the surface of a polished car as a specular reflection of the surrounding environment rather than as a texture property of the car itself? In addition to highlights, other surface properties might contribute to perceived specularity. This study explores whether deformations of the reflected scene and the nature of the scene itself contribute to perceived specularity.

Human observers viewed pairs of images of mirrored surfaces reflecting different surrounding environments and judged which image was more specular. We used a CAD package to render images of mirror-like objects reflecting a range of surrounding scenes. The objects were composed of shape primitives and the scenes were either real-world or synthetic environments (e.g., ellipses on the hemisphere). Surprisingly, objects reflecting artificial environments produced a much weaker impression of specularity than did those reflecting natural environments – reflected synthetic environments were more likely to be seen as the surface texture. Critically, the nature and magnitude of the deformation of the reflected environment was virtually identical across synthetic and natural scenes.

Additional experiments showed that the phase component of the spectral decomposition of the scene plays an important role: reflections of a phase-randomized scene (using an algorithm based on RISE) weakened perceived specularity. Phase randomization weakens the spatial structure of scene, further suggesting that deformation cues based on reflected scene structure affect specularity perception. Yet reflections of phase randomized scenes were still seen as more specular than reflections of synthetic environments. All this suggests that the perception of specularity is strongly affected by the statistics of the surrounding environment - natural statistics yield a stronger impression of specularity. Thus, deformations acting on the reflected scene appear to be a necessary but not sufficient cue.

3D shape can be perceived from a single image using contours, shading, and other cues. For shiny objects, an additional cue is available: the surrounding scene is reflected, and the deformation of this reflection varies as a function of the shape of the object's surface. Do human perceivers use this cue? Previous research has produced conflicting results (Savarese et al., VSS03; Fleming et al, VSS03), although the discrepancy might result from differences in the reflected environments.

If the reflected scene is composed of objects in close proximity to the mirror surface, then the resulting reflections tend to be uncorrelated to surface shape’s principal curvatures. In contrast, when the same scene objects are located further from the mirror objects, the resulting reflections tend to flow along the directions of minimal surface curvature and occluding contours, potentially providing additional cues about the mirrored object’s shape. Does this geometric difference correspond to a difference in the ability to use deformation cues to perceive object shape? We tried to answer this question using 35 distinct, computer generated shapes reflecting 28 environments. The environments consisted of a range of objects randomly located in space at increasing distance from the mirror surface. As a control, we used a subset of shapes with no reflective properties but with surface texture.

We found that mirror reflections were a weak cue for surface shape for human observers, regardless of the environment and the distance. Further analysis is needed to determine whether other aspects of the objects or environment can contribute to the use of specular reflections in shape perception.