Computer Science > Computer Vision and Pattern Recognition

Title:
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Abstract: Inspired by the recent success of methods that employ shape priors to achieve
robust 3D reconstructions, we propose a novel recurrent neural network
architecture that we call the 3D Recurrent Reconstruction Neural Network
(3D-R2N2). The network learns a mapping from images of objects to their
underlying 3D shapes from a large collection of synthetic data. Our network
takes in one or more images of an object instance from arbitrary viewpoints and
outputs a reconstruction of the object in the form of a 3D occupancy grid.
Unlike most of the previous works, our network does not require any image
annotations or object class labels for training or testing. Our extensive
experimental analysis shows that our reconstruction framework i) outperforms
the state-of-the-art methods for single view reconstruction, and ii) enables
the 3D reconstruction of objects in situations when traditional SFM/SLAM
methods fail (because of lack of texture and/or wide baseline).