There is currently extensive debate about how the visual system derives the material properties of surfaces. The majority of work has attempted to solve this problem by identifying features in the unprocessed images that provide diagnostic information about the material composition of surfaces. However, we previously showed that it is possible to induce transformations in perceived material properties by manipulating the bounding contours that border otherwise identical luminance gradients. Here, we show that manipulating either the motion parallax or stereoscopic depth of sparse dots can cause dramatic changes in the perceived material properties of surfaces containing identical image gradients. In one class of displays, we constructed a series of vertically modulated luminance gradients (which appear as a set of horizontal bands). An array of sparse dots was superimposed on the gradients, and their binocular disparity or velocity was varied in a manner consistent with one of two 3D surface geometries. When the surface geometry was consistent with a luminance gradient on a slowly curved surface illuminated along the viewing direction, the surface appeared strongly metallic; but when the surface geometry was consistent with a rapidly curving surface with grazing illumination, the surface appeared dull and matte. We also show that these results can be extended to doubly curved surfaces. Our results reveal that the visual system exploits internalized physical constraints that relate 3D surface geometry to the rate of change of luminance to compute the material properties of surfaces.