Beyond Point Clouds:

Scene Understanding by Reasoning Geometry and Physics

Abstract

In this paper, we present an approach for scene understanding by reasoning physical stability of objects from
point cloud. We utilize a simple observation that, by human design, objects in static scenes should be stable with respect to gravity. This assumption is applicable to all scene
categories and poses useful constraints for the plausible interpretations (parses) in scene understanding. Our method
consists of two major steps:

2) Physical reasoning: grouping the unstable primitives to physically stable objects by optimizing the stability and the scene prior.

We propose to use a novel disconnectivity graph (DG) to represent the energy landscape and use a Swendsen-Wang Cut (MCMC) method for optimization. In experiments, we demonstrate that the algorithm achieves substantially better performance for i) object segmentation, ii) 3D volumetric recovery of the scene, and iii) better parsing result for scene understanding in comparison to state-of-the-art methods in both public dataset and
our own new dataset.