Projection-based input seems to be all the buzz of Microsoft Research this year as a number of such projects were shown off not too long ago at the 20th anniversary of the research group. Although KinectFusion, OmniTouch and HoloDesk got the most press coverage, another research project does a really good job at bringing all the different technologies together.

If I may use a cartoon analogy, “Augmented Projector” is like the Captain Planet equivalent of projection-based human-computer interaction. In essence it seems to combine the results of other experimental projects – Kinect-based environment modelling, projection-based gesture interaction and spatial-aware projections into one super project.

At the center of this project is a handheld projector which provides a small window for the user into a digitally augmented environment around them. Used like a torch, users can see and interact with virtual objects in 3D space. In one example, a user was able to kick and move blocks around them which bounced around the environment realistically. In another, a user could write in the air and see the virtual text cast realistic shadows onto the wall.

Even though the prototype is quite primitive I can see a lot of potential of this technology in medical and entertainment scenarios where the physical immersion of the environment is important. I anticipate with better projection technology and perhaps fixed projectors, the virtual augmentation can be much more immersive. Demo embedded below for your viewing pleasure.