Inside the race to hack the Kinect

Over a million people bought one of Microsoft's Kinect game controllers in the 10 days after its US launch on 4 November. Most eager customers rushed home, plugged it into their Xbox console and began experiencing a whole new type of gaming – one in which keyboards, mice and multi-button control pads are no longer needed.

But a few of the purchasers had no plans to use it for its intended purpose. For months they had been drooling over the technology in the device, which includes a sophisticated depth-sensing camera and infrared scanner.

Radical applications

For these elite hackers, the Kinect's release was a rare opportunity to take a piece of big-name consumer tech and see what it could really do. If the device could be made to work with any computer, radical new applications were sure to follow.

"The day it was announced we were like, 'We're going to reverse engineer this'," says Kyle Machulis, a hacker based in Berkeley, California. "We just love doing this."

What happened next took the hackers by surprise. Adafruit Industries, a New York-based producer of DIY electronics kits, announced on the day of the launch that it would give $1000 to the first person to get a Kinect running on Windows, or another operating system.

Unintended consequences

Microsoft also provided an incentive, albeit an unintentional one. Just hours after the bounty announcement, the firm said it did not condone modification of the Kinect and would "work closely with law enforcement… to keep Kinect tamper-resistant". It was a red rag to the hackers, and that evening a note appeared on the Adafruit blog: "Ok fine, the bounty is now double, $2000."

"Suddenly it was a race," says Machulis.

But he and his friends were stuck. Reverse engineering is like code breaking. As hackers listen in on the conversation between a device and a computer they eventually pick up the language the two use to communicate. Unfortunately for them, the Kinect produces a fire hose of data from its camera, four microphones and depth-sensing system.

Reading and recording this data requires a specialist piece of hardware, known as a USB analyser. Adafruit and others had ordered one, but it was not due to arrive for several days.

Hackers' honour

While they waited, others charged ahead. By Saturday 6 November, a hacker who goes by the nickname AlexP had gained control of the Kinect's motors. Microsoft tried to quash the news, saying the Kinect had not been hacked "in any way". The hacking community took it as an affront. "This is silly, so now we've made it $3k," wrote Adafruit.

Two days later, AlexP posted a video that demonstrated his control of the Kinect's video and depth-sensing system. The prize was AlexP's to claim, provided that he made his code public – something that is standard practice among hackers. But AlexP had other ideas. The following Tuesday, he announced that he would release the code if the community contributed $10,000 to fund his work.

It was a breach of hacker etiquette. AlexP had won the race, but his peers felt he was now holding his discovery hostage and demanding ransom. "We were frothing," says Machulis.

Data flood

Finally the analyser arrived, and on the Tuesday evening, Adafruit posted reams of data recorded from the Kinect. Machulis and others set to work, swapping progress updates in a chatroom.

Someone worked out the command used to activate the Kinect. Another hacker cracked the code that controlled the motors. They called it a day during the wee hours of Wednesday, confident of finishing the job and claiming the prize the following day.

They had, however, reckoned without Héctor Martín. Wednesday was European launch day for the Kinect. At 10 am Spanish time, as Machulis and company's heads were hitting their pillows, Martín bought a Kinect at his local electronics store near Bilbao. A hacker and undergraduate computer scientist, he had been up all night working on the data that Adafruit had provided. "I tried to sleep but couldn't," he says.

Soon after 11 am Martín had his computer talking to the Kinect. An hour later, video and depth information was streaming from the device onto his screen. He uploaded a video by way of proof, sent a tweet announcing his success and went to sleep.

Victory confirmed

When the US hackers woke up, all that remained was for Martín's claim to be verified. Appropriately, it was Machulis who downloaded the code and confirmed that it worked. "He's incredible at this," says Machulis. Adafruit awarded Martín $3000, which he pledged to distribute to other hacking projects. AlexP even sent Martín the $457 that had been donated to his effort.

The hackers' success has unlocked what promises to be a revolution in robotics research. At robotics company Willow Garage in Palo Alto, California, researchers have bought around 20 Kinects. "We're losing count," says engineer Ken Conley. He and colleagues have shown that multiple Kinects can be combined without generating interference and are currently integrating the device into the company's PR2 robot.

If you would like to reuse any content from New Scientist, either in print or online, please contact the syndication department first for permission. New Scientist does not own rights to photos, but there are a variety of licensing options available for use of articles and graphics we own the copyright to.

Combining Kinect with a computer program can distort a user's own 3D image in real time (Image: flight404)