Description

With the latest SDK release, the Kinect platform has become pretty robust – skeleton tracking, gesture support, and access to the raw infrared stream all come “out-of-the-box”. Which is great, because if you’ve ever used any of the opensource equivalents, things can get pretty hairy.

Hands and skeletons are cool and all, but unfortunately there isn’t an *easy* way to do blob or generic object detection.. Right now, we’re working on a fun project and ultra fast blob detection is one of the major tech hurdles we’re working through.

After googlin’ around for blob-tracking with Kinect I came up empty handed. So I decided to look into OpenCV for doing blob detection. Many curse words later, I finally got things working the way I wanted. Along the way I noticed a handful of forums with people looking to do similar object tracking with Kinect, so I figured I would share how I cobbled things together.

Demo Overview & Source Code

The demo below is showing a couple of key things:

Consuming the raw Depth image from the Kinect sdk

Slicing the Depth image to a particular range

Using OpenCV to perform object/blob tracking

Rendering OpenCV output with WPF

...

Ok, so what?

I know what you’re thinking…so you have some blobs, big deal. And you might be wondering what else you can do with blobs. Well, let me tell you. By using generic blob detection you can not only detect when objects are present, you can also track their size, shape and rotation. For example, its possible to track an open hand transitioning to a closed fist for a grab gesture, or a user holding a wand or prop. Or in our case, we want to very efficiently capture the position of small, fast moving objects that penetrate an “invisible” IR plane. There are a ton of uses for blob detection including surveillance, card counting, and even real-time projection mapping.

What’s Next

While I’m pretty happy with the performance of this solution, I’m curious if it would be better/faster if I rewrote everything with the C++ flavor of the Kinect sdk and coupled it with DirectWrite or XAML to render the UI. Anybody out there do anything like this yet? If so, shoot me a tweet or email.

We’re currently rolling this stuff into a fun side project that I’ll post about shortly…stay tuned.