Preventing Kinect Flickering

Lately, I’ve been playing around with real-time matching using 3D point clouds taken with the Kinect sensor (read my previous post for details). When visualizing the stream of data, I detected some kind of flickering, which expresses itself in drastic change of the number of valid points between two frames. Browsing the web I found multiple references to folks reporting the same kind of behavior. Up to this date it remains unclear to me of what causes the flickering, but I suppose it has todo with any of the following points

Data flickering becomes a real world problem, when doing real-time object tracking, where one is interested in stable point cloud sizes between subsequent frames.

We tackle the flickering with a simple statistical approach that classifies a given frame as flicker or not-flicker. It works like this: accumulate the samples (a sample corresponds to the number of valid points in a frame) using a rolling mean. If the sample deviates significantly from the mean, classify it as flicker. Otherwise, classify it as non-flicker. Update the accumulator with the query sample to accommodate for changing environments. We’ve found that a window size of twenty samples works well in practice.

Like this:

Related

5 thoughts on “Preventing Kinect Flickering”

Hello Christoph, I am doing a mildly similar thing with Kinect data and the Boost Accumulator libraries – running averages along the axis to help with gesture recognition, ran into a snag and ran across this blog entry. I was wondering if you found the Accumulator libraries to actually release the out of date sample data by using the rolling window? My test implementation seems to not release the old samples, resulting in an egregious memory issue. Cool stuff in this blog entry, btw!

by the time of writing i was using boost 1.46 and outdated samples were dropped by the accumulator. Did you specify the window size correctly? I still have the implementation around and will ensure my claims by today or tomorrow.

Hi Christoph,
No worries, don’t intend for you to be Boost technical support! :)
I was running a barebones test case but the memory leak was caused by not properly dereferencing the return value. I was monitoring the Accumulator itself by adding the Count stat to the Accumulator template, and count(acc) does indeed count the number of samples pushed in, not the total samples stored – constant in a circular buffer, which threw me off.
I truly hope you didn’t spend much or any time on this!
Cheers,
Steve