Primary Navigation

GesturePak 2.0 Alpha for the Kinect for Windows v2

Description

Today's we close out our Kinect v2 week with a project from the one and only, Carl Franklin. He's taking his cool GesturePak project forward to support the future, the Kinect for Windows v2 device. The really great news is that he's going to be releasing the source with this new future release too!

When the Microsoft Kinect for Windows team sent all of it's MVPs (myself included) the new Kinect Sensor and access to the Developer Preview edition of the Kinect For Windows SDK v2, it didn't take me long to refactor the GesturePak Matcher to work with the new sensor.

The difference is amazing. This new sensor is so much more accurate, so much faster (lower latency) and can see you in practically no light. Clearly, there will be a demand for robust and easy gesture recognition software.

Background

I wrote GesturePak to make it easier for developers and end users to create and recognize gestures using Kinect for Windows. You essentially "record" a gesture using movements, the data is saved to an xml file, and you can then load those files in your code and use the aforementioned GesturePak Matcher to tell you (in real-time) if your user has made any of those gestures.

GesturePak 2.0

GesturePak 1.0 was fun to write. It works, but it's a little clunky. The device itself is frustrating to use because of lighting restrictions, tracking problems, jitters, and all that. The biggest issue I had was that if the Kinect stopped tracking you for whatever reason, it took a long time to re-establish communication. This major limitation forced me into a design where you really couldn't walk away from tracking to edit the gesture parameters. Everything had to be done with speech. Naming your gesture had to be done by waving your hands over a huge keyboard to select letters. Because of this, you had to break down a gesture into "Poses", a set of "snapshots" which are matched in series to make a gesture.

For version 2.0 I wanted to take advantage of the power in the device to make the whole experience more pleasant. Now you can simply record yourself performing the gesture from beginning to end, and then sit down at the machine to complete the editing process.

...

GesturePak File Format v2 ...

POSE is now FRAME ...

Recording a Gesture ...

Editing your Gesture ...

Using the GestureMatcher in your code ...

Source will be included in v2 ...

The price has not been set, but I plan to ship the C# source with GesturePak 2.0 at some level. You will be free to modify it for your own apps and use it however you like. You will get the source code to the API, the recorder/editor, and the tester app. The recorder/editor can be modified and included in your own app if you want to give your end-users the ability to create their own gestures. If you have code to contribute back to GesturePak, I would welcome it!

Get the bits!

Do you have the Kinect for Windows Developer Preview SDK and the Kinect Sensor v2? Would you like to take GesturePak 2.0 for a test run? Send me an email at carl@franklins.net with the subject GesturePak 2.0 Alpha and I'll gladly send you the latest bits. I only ask that you are serious about it, and send me feedback, good and bad.