Monday, June 1, 2009

If you've been wondering why my project blog has been pretty quiet, I can finally say it is because I have been helping Xbox with Project Natal. If you haven't seen the vision video, it is definitely worth checking out:

Now, I should preface by saying I don't deserve credit for anything that you saw at E3. A large team of very smart, very hard working people were involved in building the demos you saw on stage. The part I am working on has much more to do with making sure this can transition from the E3 stage to your living room - for which there is an even larger team of very smart, very hard working people involved. The other thing I should say is that I can't really reveal any details that haven't already been made public. Unfortunately.

Speaking as someone who has been working in interface and sensing technology for nearly 10 years, this is an astonishing combination of hardware and software. The few times I’ve been able to show researchers the underlying components, their jaws drop with amazement... and with good reason.

The 3D sensor itself is a pretty incredible piece of equipment providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost. Depth cameras provide you with a point cloud of the surface of objects that is fairly insensitive to various lighting conditions allowing you to do things that are simply impossible with a normal camera.

But once you have the 3D information, you then have to interpret that cloud of points as "people". This is where the researcher jaws stay dropped. The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision in this domain. The sophistication and performance of the algorithms rival or exceed anything that I've seen in academic research, never mind a consumer product. At times, working on this project has felt like a miniature “Manhattan project” with developers and researchers from around the world coming together to make this happen.

We would all love to one day have our own personal holodeck. This is a pretty measurable step in that direction.

Xbox and Microsoft deserve an enormous amount of credit for taking on such an ambitious project. It’s one thing to say “Wouldn’t it be cool if…”, but it’s another thing entirely to say, “let’s dedicate the resources to really make it happen inventing whatever needs to be invented along the way.” I have to say it's pretty neat building the future.

Cool in a lot of ways, but I'm always suspicious of the big corps. vs academia. I mean, we're potentially talking a revolutionary set of techs for the advancement of useful functional AI and robots, and it's being used as a proprietary marketing tool for a games machine in order to attempt to compete with another games machine.

I'm sure it connects via USB and I'm wondering if microsoft is going to limit functionality to games on the Xbox, or more positively open it up to open PC/hobbyist/Academic developement with an available SDK containing all those algorithms. I'm guessing the former. Imagine that one day we create computer sentience and a new life form is copyright Microsoft. Still, I'd rather it comes from MS than DARPA. I think.

On another note, on the one hand I'm impressed they've made an affordable time of flight camera that captures 3d. On the other hand I'm depressed they've made an affordable time of flight camera that captures 3d. Why? Because a whole bunch of smart people are now going to ask what the point is anymore in making better stereoscopic vision systems.

Johnny, that's awesome. I love your blog and projects, and I got a little vicarious thrill for you when I saw that you were working on something so (seemingly) perfect for you. I hope it's a ton of fun.

Wow. This is truly amazing--I have followed the various DARPA challenges for years, always wondering how long it would take for someone to finally develop the technology that can efficiently do what the human visual system does. I studied neuroscience alongside engineering, and I always marveled at how capable our visual system is at object detection and identification. What you guys are doing is nothing short of revolutionary. Here's to hoping that y'all will have the opportunity to develop these systems further. I'm just imagining all of the dangerous places we could take people out of and replace them with expensive (but replaceable) machines. Bravo, sir!

wow.. incredible!This is the interface of the future (now!). I can imagine this kind of interface in all of hardware that need an interaction with us.. mobile phones, computers, cars?, houses? I try the wii-pc but I don't like the result.. I can't wait to test Natal :)Thank you for your work!!

I don't know if this video was just a promotional one, showing only ideal scenarios, but still, is one of the most impressive things I ve seen last years. I am also in the Computer Vision field and I can really REALLY understand what a difficult task this is.

However, I have two concerns.

1) How robust the system is and2) as far as the technologies are concerned, are they generically applicable or you assume pretty much "static" objects, confined in a specific space?

This product makes me laugh! While I very much appreciate the programming and hardware that goes into Natal, I have to ask: Do you think Wii Fit users will give it up for Natal?

I ask that particular question because that is inevitably the only (gamer) market for such a device. Bear with me ...

How many gamers do you know?

Okay, how many gamers do you know that enjoy aerobics?

Okay, how many gamers do you know that will give up their rumble seats and VR headsets to jump around the room, waving their arms and ducking?

I think you get my drift. Comic Book Guy from The Simpsons certainly ain't gonna buy one of these devices. Same goes for every hard core gamer I know or have ever been exposed to. This is simply too sloppy, and always will be, given the time delays inherent in motion to pixel translation. It's only for slow games that do not rely on reflex. In other words: The boring games favored by the ladies (love ya, but you don't buy many games) and the wee children (same).

This (admittedly cool) version of this type of tech (around since at least 1990) will appeal to a limited gamer audience because it involves much more physical activity than hanging out, drinking Rockstar and wasting bugs does.

It really feels like Microsoft spent a bunch of resources trying to unseat Wii ... a little late, once again.

I predict that the true end game for this product is not going to be gaming. Maybe something along the lines of remote communications (controlling avatars) or something. It won't be for remote controlling machinery, because it's far more efficient to manipulate a hardwired prototype, so it won't be for medicine or security work, the traditional targets of biomechanical interfaces. Hard to tell where this will end up.

It would be good for Microsoft to stop worrying about following the leaders and to remember that gamers prefer sitting on the couch to jumping behind it.

James Butler there doesn't realize that people are much more willing to change their habits than it may seem at first. Who will want to miss out on this? Besides, you saw the car racing game, you can play while sitting on the couch!

And why am I still stuck with only one camera?! I hope this thing is very hackable, like the wiimote :)

Someone linked me this video on Facebook the other day. It's obviously a concept demo, something an expensive marketing team put together rather than a group of tinkerers.

Just now, I saw your name in a headline and clicked-through to find a Natal on-stage demo at E3. As a developer for Google's just announced Wave system, I can't help but wonder how much of this live demo was faked. There's got to be someone driving offstage picking those colors as the guy's talking. There are probably plenty of other things that are baked in to the demo but should be set in real-time.

As you are evidence of yourself, Microsoft has some amazing researchers doing really cool things. I hope more of this demo is factual than cheated. That concept vid makes it look like vaporware, but I trust you guys are working on some really cool stuff under the covers.

I must wonder though, at the end of the day, how many parents are going to want their kids shouting colors at a TV? ;-)

Very exciting stuff. Games are cool, but the potential for completely immersive training experiences, and communication is really exciting. Can't help but be a little jealous that you get to work on such cutting edge tech, but even more jealous that you understand it!

This is not just for games. This very natural device for inputting 3d cloud data. Whatever is in front of the camera gets mapped. The Photosynth and Seadragon technologies are making an appeance hear and my mind just can't imagaine the potential applictions this is going to unlock. This Natal part with a body language inteface is just one potential application. I like the idea of the camera being able to recognise things I move around in front of it. Object search for Bing - find me another one of these. :)

This makes me wonder what the game corporations will invent next... a street? a football field? a shopping mall? Everything starts to resemble real life more and more, so why just not live the original real life instead? Get out instead of becoming integrated with your couch!

Following James Butler's comment, I feel it's quite unfortunate that this is directed to a very small target group of people. But then again, being a computer vision scientist myself, I know what great potential the technology itself has (and it is quite impressive, I agree). So let's hope this will get out of the reach narrow-minded, short-sighted, for-profit managers can envision.

Johnny Lee , It is very cool when knowing that you are one member in Project Nathal .

Maybe Microsoft will be a winner in this nextgent console war .

I have some theories which I posted on nui forum http://nuigroup.com/forums/viewthread/5886/ want to share with all :

Anyway , I wonder that if Microsoft want to take cover all licenses on Time of Flight tech ? I know that Microsoft bought 3DV (one small company has Time of flight tech) http://en.wikipedia.org/wiki/ZCam . Big M$ also bought licensees from GestureTek on Time of flight tech , in this webpage http://www.gesturetek.com/3ddepth/introduction.php , look at demo , it is the same Project nathal , exactly , and scrool to the bottom of the webpage you can see EYETOY and XBOX360 logo . So , what can I say ? I think sony bought licensees for ps2 eyetoy , but that gesture tech version is the old one, just 2D gesture . In the case of project Nathal , It can sense 3d object , so this gesture version is the new version . And I think big M$ bought exclusive licensees for long term (maybe 10 years ..) , adn they also bought 3DV company . So, Micrsoft will be the winner in next generation console war . Oh man ! who know . Sony and Nintendo can't purchase Time Of Flight tech due to exclusive licensees .

My cautious optimism has gotten slightly more optimistic. I'm glad someone of your stature is involved with this seemingly ground-breaking project. Obviously, it will be a year or more before this tech lands in our hands, but the strides we have made in this console generation are astounding. Good luck and Godspeed to you guys!

Also, for people who don't quite believe in this tech yet, here is a hands-on preview from a reputable, unbiased source: Brian Crecente from Kotaku played Burnout: Paradise (not a game designed for Natal) using Natal controls. http://kotaku.com/5276789/from-stick-figure-to-mind+blowing-controls

I have to say I was meh on Project Natal, but seeing how you're involved in the project now has me excited. You've done some amazing things with the Wiimote, and to see even a fraction of those ideas translated to Project Natal would make it above and beyond what's on the market now.

Too many people are just assuming this is the same as the Vision/Eyetoy cameras available now. I really think Microsoft must make it crystal clear that this is very much the next generation of interaction.

Johnny, any time I'm in the mood to completely blow away anyone (techie or otherwise) with the future of videogames, I show them your VR head-tracking video. I don't doubt that you have many other involvements with Natal and head-tracking may not even be one of them, but I really hope to see something similar to your technology in games sooner than later. Please???

Not to be a downer, but there are a number of things wrong with this idea. Not everybody has a 40+" screen to play on, nor do they have fifteen square meters of floor space to occupy. People aren't going to want to have to rearrange their furniture to play a game when they know damn well they could be holding a controller. Further, its still going to be a gimmick, much like the Wiimote was for the first while. It had its moments (Trauma Center!) but it was mostly waggling (Any game with a sword). Besides which, if the wiimote couldn't do Red Steel properly (gestures were used instead of button combos to input moves, rather than direct translation) then what are the odds of this working properly? If I'm driving, will it actually accept handoverhand movements, or consider the steering wheel to be on the opposite side because of the orientation of my hands vs. the invisible virtual wheel (like MarioKart - soon as you turn past 180degrees, your wheel goes from all-right to all-left, and you lose control) Could it determine between me holding a sword, and me spinning said sword (which would appear to be me using nunchuks, without a physical object that the camera can recognize)? How would we alter the camera angles ingame without an extra stick? Where is the Big Green X button going to be?

All of this tells me that despite the wonderful and fantastically life-changing technology (just like the wiimote five years ago, remember) we will still be playing our games with a controller. Because god knows that I am simply incapable of doing any kind of high kick with my actual body - this is why I play games in the first place.

We are talking about a device with the additional add-on of an earpiece and some object recognition. It could recognize a walk sign from a don't walk sign. Keep a person on a sidewalk and between the lines in a crosswalk. It wouldn't be easy at first being overwhelmed with constant audio info about objects and distances being constantly relayed to the ears but we humans have the amazing ability to adjust to new circumstances.

To all the fellows who say they're jealous of Johnny Lee -- just study harder! :)

"This makes me wonder what the game corporations will invent next... a street? a football field? a shopping mall? Everything starts to resemble real life more and more, so why just not live the original real life instead?"

Arguments of this time are easily countered by the fact that in real life you can't really fight aliens. (Or can you?... No, you can't!)

This is simply amazing. Being one of the hardcore gamers I'll have to admit, it will be difficult leaving the controller behind. But also being athletic (like several of my friends on xbox live) I think it will be well worth it.

However I do have concerns:1. How accurate are the vision sensors and recognition algorithms?It could be real frustrating in the middle of a fight to have to redo the same command because it wasn't recognized the first time.

2. How durable is it? I've noticed lately that things have grown far more fragile than they were in the past when that certainly should not be the case. I wouldn't want this device to get knocked of the t.v. and bust due to all the jumping around you will most likely be doing.

3. Will gaming be reduced to a bunch of cartoony characters and childish mini-games? Will the leading game producers be able to incorporate this new device into their games easily and effectively?

I'm guessing based on the wikipedia page that they're actually using IR light sources with polarizing lenses to project a grid, then using the deformation of that grid to calculate 3d coordinates. The technology is actually very old but there hasn't been much done with it. I have a robotics book from the 90s that talks about it.

Anyone who doesn't think this can appeal to hard core gamers, obviously hasn't ever met an *actual* hard core gamer. Uber-nerds have been using things like the TrackIR for years. These are the same people that spend $700 on joysticks. If MS can bring that sort of functionality to the merely "serious" gamer, it would be incredible.

Now add a few party/sport games to grab the casuals, and boom, you've appealed to the entire gamer market, top to bottom.

Here's a little video that demos that TrackIR thing. Natal should be able to do similar things without breaking a sweat: http://www.youtube.com/watch?v=9wXx3vMy_AQ

I'm curious as to why everyone assumes that, because there's this new "camera" controller, that your regular controllers just vanish. I forsee games using the stock XBox 360 controller along WITH the camera system, if appropriate. Awesome to see you on the project... even more excited now!

I don't think this wouldn't work well with television gaming. It will replace some game types but this wouldn't revolutionize the game industry. Some people wouldn't buy this because those people wants to just relax, not move around and get tired. lol

Target and Sell this to fat people who wants to lose weight!

I might enjoy seeing people kicking their television and breaking them in YouTube videos. lol

But this is a great invention to revolutionize way we control computers, this might extinct mouse controllers. lol

any chance for a pc port of this?im drooling at the thought of modifying it for pc use.Constantly I see mocap and video games with live mocap done in 3rd person with no developement for first person, ie. if you can do cost affordable live mocap using a device like this, this means all you have to do is put on glasses,re-orient the view from 3rd person to 1st (computer camera view inside characters face, like a fps view) and you have a websuit similiar to that used on the movie the lawmowerman, only without the websuit...

glasses would be vr glasses like used in the movie the lawmowerman, this would give the sensation of actually game emmersion, ie when you move your hand, youd see your characters hand move as your own.if you havent seen the lawmowerman with jeff fahey and pierce brosnan, def worth a lookat.

> The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision in this domain. The sophistication and performance of the algorithms rival or exceed anything that I've seen in academic research, never mind a consumer product.

Really cool video and work. You could build a whole new world of applications that you could interact with on the XBOX. Imagine games like interactive World of Warcraft, where people will actually have to fight to win. The possibilities are endless. Great work!!

This will clarify some recent discussions surrounding GestureTek in the 3D space. GestureTek is the inventor, pioneer and world leader in video gesture control technology for both 2D and 3D cameras. Our software works with 3DV, Mesa, Prime Sense, Canesta and other 3D depth cameras. For more than 20 years, we have been creating 3D tracking technology and evangelizing video gesture control as the next step in computer human interaction. With more than 4000 public installations worldwide, we have a robust library of video gesture control technologies, applications and patents. GestureTek has licensed these patents and technologies in various ways to a multitude of consumer electronics providers, including for PlayStation, Xbox 360, Hasbro, etc. GestureTek is proud of the trail we’ve blazed in the area of 3D depth tracking and control and our current leading position in the market today.

Here’s more on GestureTek’s 3D tracking and control software: http://www.gesturetek.com/3ddepth/introduction.php .

Here are some examples of work that GestureTek was showing more than a year ago, before Natal:

+1 for more info on the hardware and software solutions presented here.

I'd sincerely hope MS Research give back to the Computer Vision community as much as they've received from it.

The video was awesome, but was I the only one to spot the Natal system had miraculously obtained details of the underside of the 'skateboard that was scanned' that were occluded by the rider's fingers? Ok - I guess they could assume Markov Random Properties and texture fill the gaps...somehow I just get this feeling it was more a staged video effort instead. I would LOVE to be proved wrong though since it does look very very good.

If you play boxing and get punched, can you feel the impact of that punch but not be hurt?If you get shot can you feel like you've been shot but not be hurt?If you play tennis can you feel the impact of the ball on the racket but not smash a window?If you kick a ball can you feel what it's like when you really kick a ball but not have to go and fetch it?Are any of the experiences offered by this "new" technology really any different or just a different way to experience the same "old" technology.If the former, I'm in.If the latter, I'm out.And I'm 31, cash-rich and bored!!

It seems like the holodeck is "invented" by the videogame people every few years. The holodeck is a fantasy, which reality can only asymptotically approach. WoW, which uses none of these "technologies", has paradoxically achieved a much greater immersion factor than any game on the Wii or VirtualBoy or what have you. At the same time, Mr. Lee is awesomely cool and I wish him, and Microsoft, the best of luck with this product, which, let's face it, will be pretty hard to sell.

"The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision in this domain... ...At times, working on this project has felt like a miniature “Manhattan project”..."

Because eventually, robots will use this technology to identify and destroy all humans. :)

I dug around your VITA but could see anything, but is this closely related to active appearance modeling by Cootes? I'm a grad student that is slowly getting more and more into the HCI world and that's where my focus is leading me

For those begging for more info, I hate to say it but you probably ain't gonna get it - any disclosure at this point would likely invalidate patent applications (it's a reasonably common thing to see researchers rush off to present and/or publish papers only to discover that they've just destroyed their IP protection).

I suspect, given MSoft's buying up of 3DV, that this is going to be handled as a submarine patent to gain trade secrets protection too - filing of a provisional application and then repeated extensions to prevent that patent from being completed (and publicly available) until threatened by a rival technology. If that is the case, then SDK information will likely be through an intermediary interface.

Pure speculation, of course, but given the money that MSoft seems to have thrown in this direction they'd be rather stupid not to protect their investment.

Hey johnny, its nice to get a bit of technical info on Natal, I'm also in the image processing field kinda fell into by accident on my way to other topics.

I presented a paper at the ICPR (international conference on Pattern Recognition) conference (which to my knowledge is one of the premier computer vision conferences) in dec last year.

I didn't see anything remotely close to what natal claims to be capable of, there is no way you would able to do that with normal cameras. The use of a special camera that generate those point sprite image does make it seem a lot more feasible.

I'd love to get more info on the camera, I'm sure a lot of the computer vision researchers would kill to get their hands on it.

Even so, I still think to a large extent the system is still a little over hyped, I would love to see a real demo of it with people walking behind/in front of the players and see if the system will still maintain its target points.

Just to clear things up, industry is usually miles ahead of academia in regards to real world applications, and most of the research they do never gets released due to NDAs and so on.

at the end of the day the proof is in the pudding as they say, I'll be interested to see it in action, and not some "semi-faked" closed door presentations.

very, very skeptical. while me and my fellow nerds will no doubt enjoy the system in it's final stage, it will not create the family atmosphere that is shown in the commercial. the very first time it fails to recognize my mother or father or pretty much anyone who is not a Nerd they will never try it again. keep that in mind when you make statements like "this will revolutionize gaming."

I suppose that 3d camera and chip were the last piece of the puzzle. Maybe the most important one even, since voice and facial recognition were in development at least for a dozen years and works well nowadays even on PCs. In any case, MS is probably the only company in the world who have the resources and talent to put all pieces together.

This is some sort of black magic. I'm at the same time conspiracy level suspicious, and childishly optimistic. If this comes to fruition at the same level of immersion as it's being presented with, then that may be the first time that a product has reduced me to tears of joy.

I love comments about how people won't like Natal and there are no good uses for it and living rooms aren't big enough, etc.

Why can't you just appreciate the opportunity for new creative possibilities and use your brain power to build instead of just raining negativity. It is so small minded to categorically say that people won't like it.

Why don't all of you nay sayers get together and create an experience where all you do is rank on things. That should be fun. And please close the door so we don't have to listen to it.

It will work even in absolute darkness. Those 3d cameras are based on lasers and receptors.The camera is not even an important component here, the important is the chip that gives out 3d map in real time.

Congratulations! I've been following your blog for a while as an enthusiastic Wii user and been consistently impressed with your experiments with that tech. Clearly someone at M-soft noticed it too - congrats on landing an exciting (and well-deserved) position. :)

I saw your video on head tracking with the wiimote (dude awesome) you have been a geekhero of mine since then and to see your name pop up with this project awesome. I am amazed that the software is able to track a person in what seems like a split second. I saw a pic of the demo with Jimmy Fallon and the red jump suits was that just a gag or was that used so the camera wouldnt slip with all the bright lights.

Dude you keep rocking, I am not a code guy but I do understand the wonderment that you guys can create and whatever project you are apart of I am putting money down on it because I know it is going to be great. I am sorry I am kind of star struck you da man.

As a gamer: I am very excited to see the possible applications that this technology can deliver to enhance my experience. I, among many others, play games to escape from the real world from time to time, and this will (possibly/hopefully) further immerse me into my imagination.

As a game designer/developer: The possibilities are limitless. While some of the projected uses are "neat", in the short amount of time since the announcement of this project I have already envisioned a vast array of applications for this technology. I, for one, will most definitely push hard for further development with/for this tool for my future projects, hopefully inspiring others to also find unique and practical uses for it.

As a technologist: While the technology itself is not new, Johnny says it the best himself: "... providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost." This will undoubtedly make this type of technology more accessible in a casual market. Even if an SDK isn't provided "right away" its only a matter of time before its reverse engineered. I wouldn't be surprised if in 10 years (or less) this technology is completely mainstream, where businesses can set up displays to usher customers into their establishment without having to hire extra help.

In the posted video, the first scene shows the teen interacting with an AI that follows his movement around the room, "talking" to him. Using this same sort of application, I immediately envisioned walking through a mall and seeing a television displayed in a shop window. While walking by the window, the virtual usher greets me with an advert, or possibly a clever or witty hook, all while "following" me. This experience of course feels very personal as it seems the interactive AI is talking to me and not a generic community. With this in mind, the AI could react to my interaction to its introduction, such as if I walk into the shop I would be welcomed in kindly. Or, if I chose to ignore the introduction, the AI would respond differently, perhaps something that may incite emotion (i.e. guilt). And because of this project, this type of advertising may become very popular, not only because the team found a more cost effective way to do this, but also because of the popularity of video games in this generation will spark interest among marketing teams to invest their resources to make this happen.

The applications are endless, in the video game market or otherwise. Awesome job to Johnny and everyone else working hard to bring this to life.

I've seen comments about Natal all across the net, from "Xbox vs. Wii and PS3" to "It's a shame that this tech is being wasted on games that even gamers won't want to play".

I shouldn't be, but, I'm completely surprised at the limited scope of these commentators.

Project Natal, including the games (Project Milo, etc.) that will eventually support it, is a way to perfect this tech, recoup the R&D expenses, push the Xbox platform, and get the population accustomed to interacting with software in ways that most of us would currently find uncomfortable if witnessed by even our loved ones.

To think that Microsoft doesn't have plans outside of gaming for this tech is absolutely ridiculous. Take what they learn from Project Milo 5-10 years into the future. A personal database that you interact with visually and vocally, whose avatar is completely customizable, that you could put on multiple devices, that's tied to Bing and MS's other services. If you can't envision the possibilities that this would allow just trust others when they say it's mind-boggling.

Natal, as a gaming device, is the perfect cultural Trojan Horse for Microsoft to get us ready for what lies ahead..

2d tracking was too hard, so now they have moved to 3d tracking to more easily separate objects, but it's still got a looooong way to go. I suspect that, like the wiimote, it will only sorta-kinda work.

How can you walk and look around (a must for, for example, an FPS) without a controller? you can't expect people to walk in place or do a moonwalk thing to walk through the game... For looking around you could make the screen scroll when you're pointing your﻿ hands/fingers to the sides of the screen but then you'll loose the realistic effect because then the game doesn't do exactly what you're doing anymore and you won't be able to shoot accurately.also how do you run, jump, roll, move the camera, and do things like controlling speed, or play a game like mirrors edge?what if you're para-jumping in the game? How do you control that? Or when you have to swim or dive somewhere?even when you have your own Holodeck you won't be able to swim and para jump etc. with the same movements.I can't think of a way how you'd solve this problem without taking something away from the game experience.. or of course using some kind of controller anyway..

You share valuable information and excellent design you got here! I would like to thank you for sharing your thoughts and time into the stuff you post!! Thumbs up. Please come visit my site Long Beach Yellow Page Business Directory when you got time.

I was thinking of looking up some of them newspaper websites, but am glad I came here instead. Although glad is not quite the right word… let me just say I needed this after the incessant chatter in the media, and am grateful to you for articulating something many of us are feeling - even from distant shores. Please come visit my site Sacramento Yellow Page Business Directory when you got time.

You may have not intended to do so, but I think you have managed to express the state of mind that a lot of people are in. The sense of wanting to help, but not knowing how or where, is something a lot of us are going through. Please come visit my site Santa Ana Yellow Page Business Directory when you got time.

Hmmmm. Spatial and visual depth and positioning determination is one aspect of the Human to Machine interface. What is still missing is the tactile pressure feedback or resistance to movement which humans would meet in real life. Until this is incorporated it is still "virtual" reality, the ability to feel the pain (via controlled electrical stimulus perhaps) would make it less virtual and more realistic allowing each individual to achieve independent personal limits (not those pre-programmed by a young techie geek). If a "holo deck" is the ultimate goal then the introduction tactile feedback is a must to ensure physical barriers can be detected and determined ie walls, doors stairs etc. It would be interesting to see a device that can stimulate a human response to an action to "virtualize" climbing stairs and present the gamer with "tired" legs and exercised muscles after the climbing event.....now that would make games sell.

Hmmmm. Spatial and visual depth and positioning determination is one aspect of the Human to Machine interface. What is still missing is the tactile pressure feedback or resistance to movement which humans would meet in real life. Until this is incorporated it is still "virtual" reality, the ability to feel the pain (via controlled electrical stimulus perhaps) would make it less virtual and more realistic allowing each individual to achieve independent personal limits (not those pre-programmed by a young techie geek). If a "holo deck" is the ultimate goal then the introduction tactile feedback is a must to ensure physical barriers can be detected and determined ie walls, doors stairs etc. It would be interesting to see a device that can stimulate a human response to an action to "virtualize" climbing stairs and present the gamer with "tired" legs and exercised muscles after the climbing event.....now that would make games sell.

Project Natal, if similar to the tech demo shown at E3, will revolutionize user experience all across the board.

Didn't get to read half the comments, but the possibilities are astounding.

As shown in the demo, it would be great if a game were developed (though more of a athletic simulator) which acted more like a personal trainer for working out (with/without weights)... or perhaps a martial arts game were developed to help teach people self-defense or meditation techniques while monitoring the person's form.

Though it would be great if a person were scanned and projected within a game. It would be neat if the scanner also acted as a projector to project the simulation onto a wall surface. (case someone doesn't have a big tv) or the scanner were used in conjunction with virtual goggles to create a truly 3d virtual simulation.

Above all, we all look forward to your groundbreaking research at Microsoft and the products that come forth.

I recently came accross your blog and have been reading along. I thought I would leave my first comment. I dont know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.

I am not really sure if best practices have emerged around things like that, but I am sure that your great job is clearly identifed. I was wondering if you offer any subscription to your RSS feeds as I would be very interested and can't find any link to subscribe here Please come visit my site Norfolk Business Directory when you got time. Thanks.

I enjoyed reading your work! GREAT post! I looked around for this… but I found you! Anyway, would you mind if I threw up a backlink from my site?Please come visit my site Chesapeake Business Directory when you got time. Thanks.

north face jackets,north face jacket. Here is a black The North Face Women's mountain jacket .

4) Women's Sky Blue The North Face Original Twinset Parka/Jacket

This is a style of sky blue The Women's North Face outdoor and mountaineering apparel .The North Face offers advanced fabrics and technologies for all-weather performance and protection during demanding outdoor action. The design adds darts at the elbow, allowing for easier arm movement and a full range of motion. The internal wind skirt effectively keeps snow and wind out.100% Omni-Tech!100% SATISFIDE YOU!

data entry india refers to a service where data from different fields of entry is extracted and converted into electronic formats. This electronic data can be stored in a secure location and accessed from multiple locations. Our forms processing services can help you store large volumes of critical data efficiently and securely.

I enjoyed reading your work! GREAT post! I looked around for this… but I found you! Anyway, would you mind if I threw up a backlink from my site? Please come visit my site Prince Edward Island Directory when you got time.

Subscribe To

About Me

I am currently a Technical Program Lead in the Advanced Technology and Projects (ATAP) group at Google. Previously, I worked at Google X, and Microsoft Applied Sciences as core contributor to the Xbox Kinect. In 2008, I graduated from Carnegie Mellon University with a PhD in Human-Computer Interaction. My research interests are in exploring novel interface technology that can influence the lives of many people. My main website can be found at johnnylee.net