Abstract

We analyze multi-bounce propagation of light in an unknown hidden volume and demonstrate that the reflected light contains sufficient information to recover the 3D structure of the hidden scene. We formulate the forward and inverse theory of secondary scattering using ideas from energy front propagation and tomography. We show that using Fresnel approximation greatly simplifies this problem and the inversion can be achieved via a backpropagation process. We study the invertibility, uniqueness and choices of space-time-angle dimensions using synthetic examples. We show that a 2D streak camera can be used to discover and reconstruct hidden geometry. Using a 1D high speed time of flight camera, we show that our method can be used recover 3D shapes of objects “around the corner”.

Figures (9)

Forward Model. (Left) The laser illuminates the surface S and each point s ∈ S generates a energy front. The spherical energy front contributes to a hyperbola in the space-time streak photo, IR. (Right) Spherical energy fronts propagating from a point create a hyperbolic space-time curve in streak photo.

A space time transform on a raw streak photo allows us to convert a 4 segment problem into a sequence of 2 segment problems. The toy scene is a small 1cm×1cm patch creating a prominent (blurred) hyperbola in thewarped photo. Backprojection creates a low frequency residual but simple thresholding recovers the patch geometry.

The top left figure shows streak images being generated by near field sources. On bottom left we see effect when this sources travel farther away, The rightmost figure depicts how we can analytically predict single sources using multiple sensor laser combinations. Notice how the accuracy is effected if lasers shift.

Simulated reconstruction using CoSAMP. While CoSAMP promises to perform far superior on a perfectly calibrated system, it is outperformed by backprojection on the current data due to calibration inaccuracies.

Reconstruction of a scene consisting of a big disk, a triangle and a square at different depth. (Left) Ground truth. (Middle) Reconstruction, front view. (Right) Reconstruction, side view. Note that the disk is only partially reconstructed, and the square is rounded of, while the triangle is recovered very well. This illustrates the diminishing resolution in directions parallel to the receiver plane towards the borders of the field of view. The blue planes indicate the ground truth. The gray ground planes and shadows have been added to help visualization.

Reconstruction of a planar object in an unknown plane in 3D. (Left) The object. (Middle Left) 2D Projection of the filtered heatmap. (Middle Right) A 3D visualization of the filtered heatmap. (Right) Reconstruction using sparsity based methods. The gray ground plane has been added to aid visualization.

Depiction of our reconstruction algorithm for a scene consisting of two birds in different planes. From top left to bottom right: (a) Photographs of the input models.(b) 9 out of 33 streak images used for reconstruction. (c) The raw (unfiltered) backprojection.(d) The filtered backprojection,(e,f) after taking a second derivative. 3D renderings in Chimera.

The laser beam (red) is split to provide a syncronization signal for the camera (dotted red) and an attenuated reference pulse (orange) to compensate for synchronization drifts and laser intensity fluctiations. The main laser beam is directed to a wall with a steering mirror and the returned third bounce is captured by the streak camera. An Occluder inserted at the indicated position does not significantly change the collected image.