Abstract

Modelling the physical properties of everyday
objects is a fundamental prerequisite for autonomous robots.
We present a novel generative adversarial network (Defo-Net), able to predict body deformations under external forces
from a single RGB-D image. The network is based on an
invertible conditional Generative Adversarial Network (IcGAN)
and is trained on a collection of different objects of interest
generated by a physical finite element model simulator. Defo-Net inherits the generalisation properties of GANs. This
means that the network is able to reconstruct the whole 3-D
appearance of the object given a single depth view of the object
and to generalise to unseen object configurations. Contrary to
traditional finite element methods, our approach is fast enough
to be used in real-time applications. We apply the network
to the problem of safe and fast navigation of mobile robots
carrying payloads over different obstacles and floor materials.
Experimental results in real scenarios show how a robot
equipped with an RGB-D camera can use the network to predict
terrain deformations under different payload configurations
and use this to avoid unsafe areas.