iPhone 4 & iPad 2 Head Tracking via Front Camera: It’s 3DTime!

I’m sure most of you have seen Johnny Chung Lee’s Wii head tracking demo. But while Lee had to use a Wiimote and IR diodes (and a PC) for his hack, Jeremie Francone and Laurence Nigay of the Grenoble Informatics Laboratory were able to pull off the same trick on the iPhone 4 and the iPad 2 using only the front-facing cameras on the devices. Plus some magical code of course.

Lee had to use IR LEDs because that’s the only thing that the Wiimote can detect. But Francone and Nigay’s program seems to be able to visually tag an object to follow, which I think is quite similar to how the Kinect works. Note that only the front-facing camera is used, and not the built-in accelerometers. Here’s the hack on the iPad 2:

And here it is on the iPhone 4:

The practical applications of such a feature are of course another matter altogether. Perhaps it could be used in games – perhaps a new version of FixPix or an evade-bullets-Matrix-style game – or combined with the rear-facing camera to create more interactive AR apps.