Microsoft has unveiled its latest gadget in the “controller wars” a motion-sensing device codenamed “Project Natal” which promises to give hands-free control to a wide variety of games. The product demonstration from creative director Kudo Tsunoda at E3 certainly seemed to impress the crowd.

At first glance, the device – a black box that sits by your TV and watches your movements – could be likened to a next-generation cross between Sony’s “Eye Toy” and Nintendo’s Wii controller. Making use of state-of-the-art 3D motion-capture technology, Natal can translate the user’s movements into those of an on-screen 3D avatar, with no need for wearable components and no user manipulation of a physical controller of any kind. Microsoft even boast that Natal can handle multiple players at the same time using facial recognition technology to identify individual players in a room and even log them into Xbox Live automatically.

While publicly stating that all of the Natal software was developed in-house, Microsoft has been more coy about where the hardware comes from. However, as Microsoft recently purchased a 43% stake in 3DV Systems, an Israeli technology outfit with cool 3D motion-sensor technology, it doesn’t take a genius to join the dots. Indeed, there had been plenty of previous speculation that Microsoft might be interested in 3DV precisely in order to create a Wii-busting next-gen control system.

The Natal hardware incorporates several key components: an RGB camera, an infrared camera and a multi-array microphone. The two cameras combined give the system the ability to identify players and track their movements. The microphone is there to allow voice commands or other audio input without the need for a headset. Whilst this seemed to work fine in the demonstration, one has to wonder how well this might work in a noisy environment with background music or other sounds in the room.

The crowd certainly seemed impressed and the technology does indeed seem to be a real step forward from existing console control mechanisms. Tsunoda was keen to stress that the system is pretty fault-tolerant and will be able to handle occlusions (e.g. someone walking in front of the sensor and obscuring part of a player), changes of player (by recognizing the shape and face of the new player) and varied lighting conditions. It sounds a little too good to be true, but if Microsoft can pull it off, they could have a winner on their hands. Feedback from journalists who tried the device at E3 has been largely positive; the message seems to be “believe the hype”. Tsunoda indicated that 3rd party developers have just received their Natal development kits, but so far there is no word from Microsoft on when we should expect to see the final Natal hardware, or any Natal-capable games available for sale.

Reader Comments

Robo Bob

Where the hardware and software came from doesn’t really matter anymore. Software and hardware are kinda like cars now. And to be honest, the user base and public isn’t going to care either. They just want the product to work as advertised. Definitely sounds ground breaking if they can pull this off. The problem with the Wii is the requirement to hold the controller which can cause property or physical damage depending on the circumstance. Seeing a relative of mine play baseball on the Wii had me worried about him breaking his TV screen. I agree that hopefully the mic will be able to filter back ground noise but I wonder if you can just wear a wireless headset as well. Other applications could be to link this to your TV, Tivo, and other media hardware. It would be nice to “tell” your Tivo what to record and when.

lennard

I am looking forward Natal

Venger

The problem with this for gaming is that we don’t currently know how precise and how small a gesture the system can register, for example nothing in the E3 demo suggested it could read specific finger gestures, which limits the range of controller inputs that are possible. If the system can’t read you making a gun shape with your hand and use your thumb movement to fire how would you play a shooter (can it even accurately figure out where you are aiming from the position of your hand and arm)? Tell the system to shoot every time with a verbal command? How accurate is the verbal control, will it misunderstand words and accents? While the technology is very cool, and I’ve heard rumors of playing Burnout on the controller, (I assume it must understand the gesture of miming a steering wheel and watch your feet for peddle controls, maybe even your head movements to generate views, that sounds pretty tiring after a long session) I’m having a hard time seeing how they can actually use this technology for a wide range of games. To be successful it needs good games, and people must want to use it over traditional controllers. I think that paint demo actually does a pretty good job of explaining the pros and cons of Natal, it wasn’t even remotely close to precise, best for broad gestures and control, I’ve heard a lot of people talking about delays between movements and responses. The Milo demo promised even more impressive technology though there’s a lot of talk now about how much of that demo was real, Molyneux himself in the video admits they used tricks to make it work and hope to one day bring it all together. But again, is Milo really a game that people will want to buy? While intuitive and simple control schemes work for some games and appeal to some gamers they aren’t universal. I’m too old and too fat to be leaping around my living room, and if there isn’t some other more traditional control set up available those games won’t be bought by me or by players like me. Unlike the unschooled masses that Microsoft aimed that demo at I’m not intimidated by a controller. Ultimately the success of Natal is going to come down to games. I’ll be interested to see what kinds of games they can actually make and if people really prefer to play them this way.