malleable screen

I was working in a project which creates a relationship between material and image. The project consists in a screen made of a material capable to be distorted, and at the same time to produce a distortion on the image that the user see on it.

According to this I made a screen using lycra, where I projected in real time the image of the people who is in front of it. To generate the distortion I started to programming based on a Processing code of an ITP alumni that using the position of the mouse generates a distortion on the image captured by the web cam of the laptop.
The first prototype that I made was using foam and pressure sensors, then I changed the values of X and Y of the mouse by the values of the sensors to produce the change on the image.

The next step was starting to use the Kinect as a sensor to map the depth and the position where the material were distorted, also I used a webcam to capture the face of the user and project from behind over the screen at the same time that the kinect is mapping the distortion, using as a reference the plane of the screen when is not distorted.