Leap Motion Tech Added to OSVR Dev Kit

In 2013, I reviewed Leap Motion’s controller—a steak-fry-sized $80 USB gizmo that let you control PC games, reading and creativity apps, and other software by waving your hands as if you were Tom Cruise in Minority Report.The open-source OSVR virtual reality headset is getting one of its first peripherals in the form of an optional faceplate that brings hand tracking to virtual reality projects via an embedded camera.

Meanwhile Razer’s OSVR headsets, which are shipping to developers in June, will offer an experimental option that completely replaces game controllers with natural movement. In the absence of polished consumer units, the only real hardware language available to hobbyists is a pidgin of devkits and kludgy peripherals; VR meetups and developer jams are still a chaotic sea of cobbled-together controllers. Powered by ‘s sensors and software, the add-on will support projects that previously took built on the Leap Motion SDK using the company’s Controller.

Today, Leap Motion announced that it would be adding its hand-tracking technology to the OSVR Hacker Development Kit, a piece of open source hardware that will be released this summer. An announcement has been made by the group that suggests Leap Motion is fully onboard – supporting the initiative and preparing their motion tracking equipment to ship with the first developer-aimed hardware later this year. But it was also obvious that the technology would fail or flourish based on whether third-party developers came up with uses for it that had wide appeal. It’ll also let developers pass video from the camera on to the display, making it easier to give users some idea of the space around them while still wearing the headset.

OSVR is a partnership between various VR and gaming companies, but it’s better known for its headset, a rougher, cheaper, and more modular alternative to the Oculus Rift development kit. The sensor, which allows for nearly latency-free hand- and finger-tracking in 3-D space, has been a longstanding favorite among VR developers for its elegant controller-free solution—so much so that last fall Leap Motion began shipping a $20 mount that could attach its sensor to the front of an Oculus Rift devkit. Why this matters: The whole point of virtual reality technology is to make it feel like you’re in a totally different world, be it a game or a virtual movie theater. But by becoming an official partner of the OSVR’s modular headset, Leap Motion hopes to clear the playing field a bit. “Being embedded is just a better experience,” says Leap Motion CEO Michael Buckwald. “For developers, there’s no fragmentation, and it’s better for users.

That’s always the end goal.” Leap Motion’s sensor is already embedded in laptops and desktops from both Asus and HP; this is simply their first official partnership in the VR space, where input is still very much an open question. This is fundamentally a more sophisticated version of a mount that Leap Motion released last year — one that’s been smoothed out and redesigned for a single product. If Leap Motion’s hand tracking works well enough it could make interactions in virtual reality feel more natural without the distraction of greasy thumbs stumbling over the X and Y buttons. The virtual and augmented reality markets—together, let’s just call it “immersive tech”—has seemingly trifurcated into three primary categories: desktop PC, mobile, and console.

Given the OSVR’s $199 price tag, grabbing both will probably cost less than purchasing a $350 Oculus Rift developer kit and Leap Motion’s controller and headset mount if you’ve got an idea for a VR experience that could benefit from precision hand gestures. On the console side, PlayStation has made its Project Morpheus prototype compatible with two of its existing controllers, the PlayStation Move and the Dualshock 4 gamepad. On the other hand, the Rift’s display supports a 75hz refresh rate compared to the OSVR’s 60, so those prone to motion sickness should probably go the Oculus route.

Things are muddier in the desktop segment; while Valve software is creating custom controllers for the Vive headset it produced in collaboration with HTC, Oculus still hasn’t announced any input solutions for the forthcoming consumer version of its Rift. (While Oculus purchased Leap Motion competitor Nimble Sense last year, it hasn’t yet confirmed whether the fruits of that acquisition would appear in the Rift.) And Samsung’s Gear VR—currently the most full-featured mobile solution, and one seeing a full consumer release this year–utilizes a small touchpad located on the headset, while also allowing the use of Bluetooth Android-friendly game controllers. The same sensors in its USB controller will be embedded in the faceplate, allowing for motion control similar to what Leap is also doing with Oculus, except without the stand-alone controller, mount, and additional cable. What all of these solutions are missing, of course, is Kinect-on-steroids experience of using your hands in VR—for navigation, selection, even control of three-dimensional objects in three-dimensional space. “We very much think the answer is going to be hands, and that we’re going to be providing that tracking,” Buckwald says.

OSVR, which is spearheaded by gaming PC company Razer among others, is an open-source platform that aims to become an Android-like force for VR: foundational technology that many companies can use to build products. And because the sensor’s computational strength lies in its software rather than its hardware, the company can push frequent updates to keep its capabilities cutting-edge long after the headset goes home with you.

Otherwise, “we’re essentially dooming developers to a world of fragmentation” as they pick between everything from Xbox controllers to exotic hardware like the omnidirectional treadmill. According to Leap Motion: “The OSVR partnership is the first in a future lineup of head mounted displays that will feature Leap Motion’s technology built in.” During a recent visit to Leap Motion’s San Francisco office, I chatted with the company’s founders, David Holz and Michael Buckwald, and tried out some virtual-reality demos using both an OSVR headset and Oculus Rift. Not only might this nudge developers towards using its system, it will be on the radar if any major manufacturer decides to use OSVR’s kit as a reference design. “If they get excited about a full hardware product with input, they can have all the hardware done, and they just need to talk to us about the software,” says Buckwald. I nudged spheres and cubes around with my fingers; controlled a torrent of stars a la Mickey Mouse in The Sorcerer’s Apprentice; and manipulated controls such as pushbuttons and sliders.

They were just examples of things which Leap will help developers accomplish, not full-blown VR applications, and the experience wasn’t quite eerie in its realism. (Sometimes my fingertip would poke inside a cube, and I had to get used to the fact that I couldn’t feel the objects I touched.) But it sure felt more natural than using a conventional game controller while wearing a VR headset. Buckwald says the company has been talking to “pretty much all the major players” in the space. “Our goal is to be embedded everywhere,” he says. “This is not an experiment for us—we want to be the primary input for VR.” Buckwald says there’s no hard timeline, but he thinks there’s a market there as well — he imagines a game where players use a controller as a gun in one hand, but throw physical punches with the other. And the larger future, to him, seems bright. “We want people to feel like they are touching something digital and believe it is physical in five years,” he says. “I think with VR, we’re either there or very, very close to there.”