Surface Physics explores new ways of supporting multi-touch and whole hand interaction on digital surfaces. Our aim is to bring the digital to ‘life’, allowing interactions with digital objects to simulate the physics of the real-world. We use an advanced physics-engine to add dynamics to digital objects, allowing them to behave more like real objects when we touch them, with notions of inertia, collisions, friction, restitution and so forth. Our contribution in this work is a novel technique for modelling the data sensed from such surfaces as input within a physics simulation. This enables a variety of fine-grained and casual interactions, supporting finger-based, whole-hand, and tangible input without the need for explicit gesture recognition.