The Sixth Sense

Sixth Sense is a wearable gesture interface that was invented by Pranav Mistry from the MIT media lab. As the name suggests, the technology integrates the physical world with digital information, thus allowing us to interact with the information with physical movements.

Data has always been stored in some form like paper or in a digital form on a screen. To interact with this data on the screen one had to use a computer or some other digital device. We were not able to intuitively manipulate the data with our five natural senses like we do in the physical world. Sixth Sense attempts to bridge this gap. It allows us to physically manipulate our data which is stored in the digital form, using physical gestures. There is no need for an intermediate interface. Sixth Sense attempts to integrate your digital world with the physical world.

The Sixth Sense prototype as of now comes with a project projector, mirror and a camera. The hardware components are very small and can be carried easily on you. The components are integrated together in a mobile device. The function of the projector is to use objects in the physical world like walls as a projectable medium. It projects the data onto the physical medium thus serving as a visual interface. The camera’s function is to track and observe the movements of the color markers on the users hand(fiducials) . This is achieved using computer vision algorithms. As the user interacts with the physical medium and moves his or her hands, the projector displayed the updated world. Basically, you are able to use the physical medium as a monitor and your hands act as input interfaces (mouse) with the system. The system also supports multi touch technology as you can wear multiple fiducials.

Here in this picture we can see how the entire prototype comes together. The camera keeps a track of the color markers over the projector, thus allowing us to manipulate digital content.

Here, the user is using the prototype to manipulate google maps. Observe how the wall behind him is used as a projectable medium. Also using his hands, he is able to interact with the projection.

The convenience and the simplicity of Sixth Sense are clearly at display here. The user is able to use his hand as a projection for phone’s number pad. From here, he can call someone with just a couple of gestures with his fingers.

From the above examples it is clear how useful and convenient this prototype is. It is very intuitive to use and fits perfectly into the digital world. It also takes our interaction with the digital world to a whole new level. Though this technology is still in the prototype stage, I think it has a lot of potential and can end up revolutionizing human computer relations.