Digital Touch: Rainbow Membrane

Rainbow Membrane, by Leap Motion developer Isaac Cohen, is an experimental audiovisual video game that seeks to blur the boundaries between the virtual and physical world. Whilst viewing Isaac’s Rainbow Membrane world through a pair of Oculus Rift goggles, users are fooled into thinking that they can actually feel and touch with their hands, what they see. In fact, they are really just grasping on to thin air.

How is this possible? As the user drags their hands around the silky walls, and along the face of the man they see, unique sound cues are created, corresponding exactly with the users movement through the field. This convinces the brain that you can feel the virtual space around you.

Although this particular game is difficult to play at home without the use of Leap Motion, Oculus Rift, and MozVR to hand, more of Isaacs fascinating interactive games can be found on his website, cabbi.bo.

We found out more Isaac’s ‘Interactive Audio Toys’:

The Plus: What exactly is the sensation that is felt when playing with Rainbow Membrane?
Isaac Cohen: People describe the sensation of touching the box, saying that it felt like ‘the burnt milk on top of a cup of hot cocoa’ or just ‘flesh’ -I feel like the goal of the project was accomplished.

TP: You used spherical environment mapping, could you explain what that is?
IC: Its a specific lighting technique that uses an ‘enviornment map’ to get the shininess / look of a material. You can find out more about it here.

TP: What was the most difficult part of the technical process?
IC: For this project, I used a technique I’ve been working on for a while, which takes an arbitrary mesh and converts it to something that has a sense of ‘Physics’. In this demo there are about 120,000 vertices, and each one of them has its own position, velocity, acceleration, and forces applied to it. That’s a lot of math, and making this perform in real time (rendering 60 frames a second) is extremely difficult.
TP: What kind of impact do you wish to have on the viewer/user?
IC: I hope that I can create some sort of emotion. I think a lot of people think of technology in terms of efficiency (how to we browse twitter faster), or money (how can me make the next ‘viral’ mobile app), but rarely do we ask how technology can make us ‘feel’. Being able to use the web to try to create emotions, whether they are melancholy or joy, is to me the most exciting thing I can do. If I could help people meditate, or feel a sense of peace, after experiencing one of my projects, I would be very happy.