0

One of the main limitations on touchscreen interfaces these days is that all you can do is poke at them. We do all kinds of things with our hands, but when it comes to screens, we just poke at them all day. UIs are doing all right, since our phone OSes still mimic mouse-based desktop OSes to some extent, but Microsoft is looking to ways to integrate more natural hand gestures incorporating more than just a “click” derived from a fingertip.

It’s a very natural extension of how you interact with a surface, since really, that’s how you interact with most surfaces. Right now “Rock and Rails” only includes support for three gestures: a fist, which holds things down, a straight hand, which sets a line to which UI elements can move perpendicular, and a “curved rail,” which sets a pivot point. I’m sure you can imagine a few more, but at the same time, limiting the “vocabulary” of a UI is part of making it usable. Nobody wants to remember a dozen different gestures.

Speaking of surfaces, the video shows the tech being demonstrated on a Surface, which uses a different detection method than your average smartphone or tablet — it can detect shapes far more easily. And the new Surface units, as we learned at CES, have thousands of pixel-sized cameras that can even detect patterns and text. So don’t expect this kind of special recognition to come to iPads any time soon, although it would be similarly unwise to underestimate developers who might want to make it happen.

The whole paper is being published later, so unfortunately this video is pretty much all there is right now, but if you head over to Hrvoje’s page, you can keep track of this and his other projects.

OverviewIn 1991, Microsoft Corp. became one of the first software companies to create its own computer science research organization. Microsoft Research has since developed into a unique entity among corporate research labs, balancing an open academic model with an effective process for transferring its research to product development teams. Today over 1,100 world-renowned scientists and engineers work in …