KinEmote is an easy-to-use application that allows Windows users to navigate the menus of Boxee and XBMC media portals using nothing but hand gestures that are captured by MicroSoft's Kinect camera. It's built around the OpenNI framework and NITE Middleware provided by PrimeSense Inc.

Yes i will eventually rewrite it to something usable under linux as well, but for now i use C# since i have most experience with that. That way i can rapidly prototype different techniques for controlling xbmc. The swipe gestures look neat, but are absolutely unusable in a library with more than 10 movies. I'm currently looking into a virtual slider solution that enables you to scroll at different speeds, but maybe a push and swipe technique would work even better. Also, context sensitivity has to be integrated, when playing a movie you want different actions than in your movie library.

Ideas on navigation techniques are very welcome, as very people have created something like this before. Xbox kinect dash & the opensense media center demo are my only references right now.

alex84 Wrote:Can i buy an kenetic kit and get this working on my asrock 330ht ?

I'd have to test the performance, but as long as it runs windows it _should_ work.

Johnsel Wrote:Ideas on navigation techniques are very welcome, as very people have created something like this before.

I don't know how possible it is, but the obvious one would be trying to emulate the touch-swipe motion of smartphones. You'd probably need position, velocity and acceleration (with the latter two being derivatives of position of course). I think this would involve the entire hand and maybe some use of the forearm/arm.

Practically, I think it would get tiring waving your arm around that much though. Assuming the user is close enough to the Kinect so that it can distinguish individual fingers, an entirely different route could be this for basic navigation (which I think I like more):

Have the nominal state of scrolling (e.g. no movement) occur when the user has a clenched fist (thumb pointing upwards). I think this makes sense because it's relaxed and natural

Point the index finger out to the Kinect and have this be one keystroke of 'down arrow'

Hold this for, say, .5 seconds to start inducing an accelerated scroll (easily done in EventGhost, not sure about XBMC, but probably that as well)

Seperate finger tracking is another interesting technique, will look into that too. Only problem with the vanilla kinect driver is the low resolution (640x480), which obviously complicates tracking very slight changes in hand position. I did see an example that increases the resolution to 1280x1024 at the cost of framerate, which goes down to 10fps iirc. Don't know whether that would be enough.

I was thinking about how to go about implementing scrolling last night. I came up with as follows:

To initiate scrolling the user would put their hand out and show their palm to kinect.
This would start the scrolling functionally by setting the hands current position as a start point.
The user would then move their hand up,down,left,right depending on the required direction still pointing their palm at kinect.
The further the user moves their hand from the start point the faster scrolling would go probably using an exponential scale so that it would be easy to move one or two items at a time without overshooting.
The user would then close their hand to stop scrolling.

haven't really considered how to implement other features but I'll have a think

That was actually one of the first things i tried. Scrolling that is, not the palm gesture. I've thought about it though, but in a grab and "continuous swipe" style. I guess i'll wait with releasing a binary until i've create a demo app that lets you choose between a couple of different control techniques.

In this video I control XBMC using my hand as a cursor. Push and pull are respectively click and go back. It seems that using the xy coordinates with a subtle smoothing is very accurate, but the push used to click will almost certain move the cursor too much. I think the best solution for this is speed (and maybe z axis) dependent sensitivity.

I am not entirely sure what information you are able to get or are using from the kinect but after watching your video these are my control scheme suggestions:

My idea for a control scheme would be to have your palm facing the screen and rotating it counterclockwise to scroll left and clockwise to scroll right with the degree of rotation determining how quickly it scrolls. With this added to the single swipe gestures from your first video for more precise navigation I think it would be very easy to quickly get to an area of your Video Library with my gesture then select the exact movie with yours.

Johnsel Wrote:In this video I control XBMC using my hand as a cursor. Push and pull are respectively click and go back. It seems that using the xy coordinates with a subtle smoothing is very accurate, but the push used to click will almost certain move the cursor too much. I think the best solution for this is speed (and maybe z axis) dependent sensitivity.

I remember reading an article about the Dance Revolution game and how they spent months trying to get something that worked AND wasn't insanely hard to accurately select something with. I'm guessing the mouse is just never going to be the best kind of kinect controller. But I'd love to be proven wrong.