Kilobuck Open Kinect Project Prize

Full of video and audio sensors, the newly released Kinect is Microsoft’s answer to Nintendo’s Wii MotionPlus and Sony’s PlayStation Move. Now there is money up for grabs to hack it. Adafruit is offering up a one thousand dollar prize to open source the driver for the Kinect. What do they want this driver to do? They want RGB and distance values. We’re excited to see the hacks that will come around because of this product, and now that prize money is involved, everything has been turned up a notch.

﻿Update: The bounty has been raised to $2000 after a Microsoft response to CNET:

But Microsoft isn’t taking kindly to the bounty offer. “Microsoft does not condone the modification of its products,” a company spokesperson told CNET. “With Kinect, Microsoft built in numerous hardware and software safeguards designed to reduce the chances of product tampering. Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant.”

It would be pretty amazing to have such a powerful and inexpensive device at our disposal. It’s just that $1000 may be nothing compared to the effort it could take. The Kinect uses about 10% of the XBox’s processing power. I’m guessing MS optimized the crap out of that 10%. If that’s all gesture recognition, then probably no big deal, but if the XBox handles the ranging, I don’t know…
I remember reading MS had all kinds of trouble developing this. Also, I’m sure they want to use the technology in other applications. I don’t know how hard it would be to recreate all that development, and I bet MS has taken steps to thwart our attempts.
Very nice of Adafruit though. I do think it will happen. It always does, but sometimes it takes a really long time.

@NatureTM
If you read the description on the adafruit website, it says “the Kinect sensor outputs video at a frame rate of 30 Hz, with the RGB video stream at 32-bit color VGA resolution (640×480 pixels), and the monochrome video stream used for depth sensing at 16-bit QVGA resolution (320×240 pixels with 65,536 levels of sensitivity)”

But Microsoft isn’t taking kindly to the bounty offer. “Microsoft does not condone the modification of its products,” a company spokesperson told CNET. “With Kinect, Microsoft built in numerous hardware and software safeguards designed to reduce the chances of product tampering. Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant.”

I love this, they upped the bounty to $2k because M$ bragged abut the security in the device and their lawyers xD

Update: We’ve increased it to $2,000 – why? We just read this at CNET…

But Microsoft isn’t taking kindly to the bounty offer. Bounty offered for open-source Kinect driver – “Microsoft does not condone the modification of its products,” a company spokesperson told CNET. “With Kinect, Microsoft built in numerous hardware and software safeguards designed to reduce the chances of product tampering. Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant.”

Seems silly they would be worried about people using it on things other than the Xbox. I could see worrying about someone modifying the xbox/camera with a virus or something of that nature, so that its basically broadcasting a camera feed to the internet. But this is just giving people a reason to buy a microsoft product, who otherwise might not. I don’t own an xbox, but this gives me a reason to buy an xbox product. Sounds like a good deal for both sides.

Why would they want to stop people from hacking the Kinect? I wonder how many extra Wiimotes were sold because people were hacking them. Hell if someone hacks the Kinect and gets facial recognition and audio working, we’re talking HAL. Put a couple around the house and you’ve got a system that will do whatever you program it for.

When I heard this uses time of flight, my immediate reaction was that all of the underlying technology is heavily patented. That’s the reason why a SICK sensor still costs $5k. My theory is that primesense’s licensing requirements are very strict to keep this product out of the research/robotics domain. Thus, I wouldn’t be surprised to see a fully encrypted path between the kinect and the xbox. If true, this is very bad news for hacking. Due to the absence of any processor, the encryption would have to be built into the primesense IC which would make sniffing for keys really challenging.

I can understand Microsoft wanting to protect its intellectual property as far as the software running on the Xbox that is able to do skeletal tracking from RGB and depth info, that’s fair in my book.

But even a dumb sensor that only returns a raw depth map is a huge advance to robotics.

Considering Microsoft bought out the only two companies that developed affordable gated time-of-flight depth sensing cameras, them taking measures to prevent people from purchasing and using that hardware for their own purposes is just bloody mean.

Kudos to Adafruit for offering the bounty. I hope this gets cracked wide open.

@Ice Brain
“The Kinect could be sold at a loss to drive up game sales, in which case buying it without the intention of buying XBox games would be bad for them.”

Ah yes, I totally forgot about this style of economics.

As far as their intellectual property, wouldn’t this be more of creating something new? Unless the people extract the software out of the xbox and copy paste it into their driver. It would seem to me all this is doing is creating a new interface to an existing product’s data stream. Like hooking up a vcr to your cable tv line instead of a cable box or something like that, not sure, am not a lawyer, just seems like this shouldn’t be that big of an issue, if nothing is extracted out of the xbox.

Probably be easier to build your own. I’ve never worked on an xbox before, no idea if you can run programs on it, otherwise wouldn’t this require a hardware USB sniffer? Those are pretty expensive from what I’ve seen. Not sure if you could build some pieces of equipment, a device and a host, to trick the kentic and xbox into giving up enough information to get the ball rolling on reversing the data.

@whatnot You may remember the DARPA sponsored autonomous land navigation challenge a couplathree years ago. MS got the guys that placed 2nd and they developed the Natal/Kinect. Like Han says, “She’s got it where it counts kid.” It will be a nice piece of interface gear (minus the size I guess but ya mount it). Like others say, I’m excited but feel $1000 is too low a bounty unless they just want to get the first scoop on the hack which is what this reads more like. I shake my fists at the bloated God gizmodo and her phallic tentacles. User mileage may vary.

“The Kinect could be sold at a loss to drive up game sales, in which case buying it without the intention of buying XBox games would be bad for them.”

This is exactly what MS did with the orignal Xbox. They sold the hardware at a loss, expected to make up for it in games and xbox live. So they went extreme lengths to secure it. They had a dummy bios on the mobo, and actually hid the bios inside the custom nvidia gpu. Since it was basically a legacy free PC, it meant people could buy it, hack it, install linux on it, and have a cheap ass pc.

Why would they want to stop people from hacking the Kinect? I wonder how many extra Wiimotes were sold because people were hacking them. Hell if someone hacks the Kinect and gets facial recognition and audio working, we’re talking HAL. Put a couple around the house and you’ve got a system that will do whatever you program it for.

—————————-

1. Hacking works the other way too. Instead of increasing sales this reverse engineering could be used to make Kinect clones.

2. Perhaps the R&D and cost means they’re relying on game to break even, something PC users wouldn’t be buying.

Lately, MS has been saying that the kinect uses less than 10% of the xbox’s CPU power. If that’s true, do you really think it’s simply sending RGB distance values to the xbox? I bet it’s doing much more on-board processing, like real-time skeletal modeling to produce x, y, and z values (or difference vectors) for specific joints.

I don’t think MS did the hardware, or at least I think they ported someone else’s reference design and used someone else’s RGB-D CCD. Going from there to a commercially viable product is a lot of work, especially since a lot of the magic is in the software that goes with it. But much of the “R” cost was elsewhere, while MS picked up the “D” portion related to what they wanted to use it for.

What does microsoft think Visual Studio does. it allows people to use their windows computers for something they were not designed to do. I guess if you are willing to pay $700-1000 for their software then its ok. When will these companies realize that if they make the code and hardware open then many more people will buy their products and probably buy multiples of them. I can’t even begin to guess how many PS3’s were bought just because they could run linux and how mad they were when they suddenly stopped working

From looking at the iFixit teardown its obvious that there is some kind of CPU or MCU in there

As for why MS doesnt want hacking, I am inclined to believe the “sell at a loss to make up money on games” and the “sell far below what similar systems would cost as long as its only used for games”

As for reverse engineering, as long as its not using encryption and as long as no (C) MS/(C) PrimeSense code is used, it should be legally clean.
Even if its using encryption, IIRC there are legal precedents (I cant find any right now though) where cracking encryption for interoperability like this is not a DMCA violation

I am thinking M$ is wanting to use this device for passive monitoring of the fools that connect this thing up. Oh don’t hack it or you may find that code that sends your life to our Big Brother Network. Yeah I know conspiracy nut. BUT it does look like a fun project just to thumb a nose at Billy with.

I don’t think the $2000 is going to be taken anytime soon. I would be surprised if at least some of the Primesense IP was not running on the XBox’s processor, so I would expect the comms channel from the cameras to be well encrypted, and even then it might not contain readily intelligible data.

Perhaps the extra microphones are for doing beam forming for locating the voice in the room. I don’t recall the Primesense eval kit doing that and that IP certainly is in the public domain. Even so, this would make an expensive USB microphone.

Microsoft bought 3DV Systems a while back. They were the ones who developed the camera technology. If anyone has drivers for the camera that they were going to release as a webcam, they are probably pretty close to what is used in the Kinect.