Where is my *%$#! virtual reality display?

For years I’ve been asking, loudly and frequently, where’s my F*&#$%ing virtual reality? I realize that many of us, depending on what influenced our dreams of VR will have different definitions, so let me elaborate. When I was a kid, I played Dactyl Nightmare at the Union station in St. Louis. You stepped inside this “stage”, donned a massive clunky headache inducing HMD, and brandished a floating joystick of doom to challenge other players and a flying Pterodactyl. I was awe struck. My 12 year old brain decided that this was the future.

Fast forward 20 years and the thought of VR is relegated to niche builds in peoples homes. Graphics hardware has progressed considerably, with games that look absolutely breathtaking becoming the norm. When you sit down to play an FPS style game at a PC and move the mouse around, it seems that it would only be a tiny step to replace that with some nice head tracking. I’ve actually seen people just tape gyro mice to their head to achieve their goal. I’ve seen people strap entire monitors to their head too! I’ve always wondered why there isn’t a better commercial solution though. Are there really so few of us that want this?

Now, with 3d movies seeming to take hold, and 3d televisions appearing in peoples homes, I am reminded again of the grandeur that a proper hmd could supply. My brother has a 3d television and tossed in some boxing game for the PS3 that is 3d. It was amusing, but not really immersive. Don’t get me started on the inaccuracy of those stupid motion controllers I had to use either. The Kinect is a big move forward in terms of how to track body movements, I’ve seen that added to the mix in peoples homes as well. Frankly, I’d settle for a decent HMD.

As it turns out, [John Carmack] is also on the quest for VR. He was at E3, talking about a new headset kit being built by PalmerTech called Oculus Rift that blows anything else out there away in terms of resolution, response, and cost. This kit will be DIY style and supposedly cost around $500, though you still have to add tracking. The kit looks pretty cool, but what is cooler is that he describes what is really holding most progress back. Apparently, it is the latency involved with the entire loop of motion->computer->display.

Are you listening to this [Jeri Ellsworth]? You paying attention [Jeff Keyser]? Valve needs to make me an HMD, even if I’m the only one on the planet that will enjoy it. Just think of it as a hat.

All head mounted displays do this in essence…they throw an image focused at infinity at you with a relatively limited fov per display. Which effectively looks like a large theatre screen in front of you. To take the image of the small lcos and transform it into the ~160 degree of fov per eye would require significantly more expensive optics, and thus it becomes best to use many small displays instead of one large one (like sensics displays)

Also, expanding 1080p like that would make whatever area you are focusing at relatively grainy. (think getting up next to your monitor until it fills your fov)

This unit looks like a great compromise between all of these things, and it will be the first affordable display that is high resolution…if it shakes out to work well.

The original ones in the mall didn’t fit me very well, so I had trouble with the depth perception, and the pterodactyls always kicked my butt really fast.

Dual-eye gargoyle goggle have been around for years, but they’ve been intended for the gamer market or for watching movies, so until recently most of them haven’t had a high enough resolution to read significant amounts of text (and any that did were ridiculously expensive for hobbyist/gamer use.) Nice to see they’re starting to get good ones, and accelerometers are now cheap enough to do good head motion detection, though Kinect or Leap may also add some value.

I played a Pac-Man clone on a similar setup to Dactyl Nightmare, as a kid at the XS New York arcade during a vacation around 1996 or so. While the game was primitive, I thought it was really cool how my head and body moving directly influenced my vision in a natural way. A head-mounted display is like being outside, versus a monitor being like peering through a window at the outdoors.

There are very few actual VR HMDs available right now. They are largely low resolution and completely lacking in any software support. What Carmack is showing off is very much newer and higher quality than any actual VR HMDs out there. The software support within the game is essential too.

True and this is a big problem (the chicken and the egg.) Developers don’t put software support if there is no installed hardware user base. Hardware developers won’t make it (and that won’t create the user base) until they see enough software support. Lose-lose scenario for those who want this tech. I’ve been toying with stereoscopic since the early 90s and I gave up on it for the lack of game support.

Maybe now that Carmack is into it more people will start following. We already have the tech, someone needs to take the leap, put it together and mass market HMDs. It is good that Carmack listed what he wants and the problems he sees in current display technology. Hope the big manufacturers are paying attention. There needs to be a true “game mode” that completely bypasses all that DRM crap they put on TVs today. Current TVs don’t bypass all of it.

Software support..? Aren’t basically ALL games 3D these days? Doesn’t that mean that you could use 3rd party software to get the correct images for both eyes from any fullscreen 3D application?

I remember 10-15 years ago my dad had a graphics card with a pair of shutter glasses. The graphics card driver managed the convertion, meaning any Direct3D/OpenGL game (if memory serves right) could be viewed in 3D on my dad’s awesome 17″ monitor.

So, generating the images isn’t a real issue. At least, it wasn’t a decade ago.. As for the feedback: gyroscope setup doing headtracking, communicating with the computer as a generic mouse. After all, just about every FPS-type game produced in the past two decades uses the mouse to look around in-game.

I might have missed out on something here but if someone like Sony told their dev team to come up with a pair of LCD glasses (possibly with a “1 pixed black/white LCD behind the main displays for optional transparent “AR” viewing) that had at least 720p resolution, covered a good part of the users field of vision, and sported a gyroscopic bluetooth “mouse”… Are you saying they couldn’t do it?

Sony, if you’re reading this: Please start production right now! It can be done, it could be made to work with any 3D computer game made in the past 10 years, and it would sell like a goddamn iPhone!

You don’t need accelerometers for head tracking. Just use the Leap Motion device to track your head and hand movements (coming out later this year). Cheap at $70, and almost zero latency. Now you just need a good HMD and you’re golden.

There are a lot of interviews with Carmack about this. But one of the things he mentions repeatedly is that the latency between moving your head and updating the screen is more important that you might think.

Even professional systems typically introduce too much lag and needs to be modified. Wireless systems like Bluetooth introduce too much lag and are right out (typicaly BT lag is 100ms). He comments that a fair lag between moment to “photons”/screen update needs to be in the 20ms range and that’s very fast.

He also mentions that it’s faster to send an IP packet from the US to Europe than to draw something on a screen from a PC next to it. So this is not an insurmountable problem. It’s just something computers and displays today are not really designed for.

I’d pay the 70, but I don’t believe in ‘on-paper’ stuff scheduled for 7 months later, I’ll see it when it exists and how it pans out.
Look at a huge company like intel and what became of their lightpeak in the end to see an example of the reason of my concerns.

Not necessarily. Most people see the HMDs by sony and such and think they would work well. While they might get you by, the field of view is pretty narrow. Like looking at a theater screen. The optics in the VR helmets actually cover your peripheral.. or they should.

I was under the impression that VR technology like this died out because people who used it extensively started having problems with depth perception in their every-day lives – some story about a bunch of engineers at Boing having permanent depth-perception issues after using VR to design aircraft.

My technological solution would be eye-tracking + Z-buffer + electrowetting liquid lens. Adjust the lens focus based on the distance to the object they’re looking at. Just looking from my screens to a far wall, there’s a noticeable focusing delay for my eyes, so I don’t think speed is a problem. Problems are probably going to be can you get a liquid lens of the right size, and can you gaze-track accurately enough to not piss off the player who’s trying to focus on an object behind a stand of bamboo. Maybe with a very very good eye tracking camera, you can actually get some hint about the focus of the eye itself, and adjust based off that.

The question isn’t where the HMDs are. The question is, where are YOU?

Several have come and gone already. There was one maybe five years ago (I don’t recall the name) that I think went for $500; when no one bought it, the NTSC-ready LCD displays and lens assemblies were available super cheap on electronics surplus sites. Was a golden opportunity for anyone interested in HMDs.

Several more still exist. Try a Vuzix Wrap 920 for $210, though you’ll need to add your own head tracking. If you want the head tracking built-in, there’s the Z800 3DVisor for $1,800. And that’s just what a mere two minutes of Googling turned up.

So if you really want it, stop ranting and get one already. ;)

But HMDs are not for everyone. The weight of the HMD fatigues the nose and ears. 3D without true depth fatigues the eyes. And the more realistic the display, the more it’s likely to cause severe motion sickness and headaches. Even on a regular screen, first person shooters do that to me; it takes at least a week of short play periods before it goes away.

I wonder if the issue of focus-based depth perception could be addressed using pupil tracking like HaD featured the other day using OpenCV, coupled with the pseudo-DoF effects you see in a lot of next-gen titles today. Can’t imagine it would be perfect, but I’d be interested to see what perceptual difference it might make.

On a side note, in the past I’ve had a bit of a love/hate relationship with John Carmack, but I found this interview incredibly inspiring as a budding hacker. I really admire his cross-disciplinary solutions to make what looks like a pretty interesting product.

DoF is only going to force areas of the screen to be fuzzy no matter what. What you need is something like an electrowetting liquid lens that can change focus extremely quickly, based off of the z buffer depth of the game. Once you have that you would either want everything sharp so that when you change what you’re looking at it still is in focus, or to do DoF blurring based on the gaze-tracking camera’s measurement of the eye’s current focus, if that’s even possible. Maybe infrared lasers…

The first thing I wondered from this video was whether the chips and firmware he received were updated beyond current ITAR restrictions. The PS3 and Wii controllers have extremely high res accelerometers but they are purposefully limited so bad people can’t use them in missile guidance. Don’t forget, one of the companies that Carmack works for is Armadillo Aerospace. He may have access to higher frequency/resolution hardware than regular citizen can get without a license/background check.

Second, the $500 project is not Carmack’s but a guy named PalmerTech and it is called Oculus Rift. The website is http://oculusvr.com. There is supposed to be a Kickstarter starting up any day now for the kits.

I highly doubt this. There was some kerfluffle about the cellular microprocessors at the time of the gulf war, but every single component in these things is manufactured in or readily available in china.

You know, the guys buying oil from Iran, and who just signed a big integrated trade deal with Russia, and who keep North Korea stocked with pickles, bullets and booze. Who does that leave that can’t get them? Dr. Evil? Somali Pirates?

I guarantee that the accelerometers in the console games are neither high resolution, nor more accurate than commodity off the self units, nor fast by any definition of the word, and that they’re not even close to being able to deal with greater than 3G forces.

If you want inertial reference, you need reasonable quality gyros plus accelerometers… and the Gs are pretty significant for anything badass. Anybody capable of building an ICBM is going to be able to source the parts through intermediaries. But even if they do, it probably won’t work. Rocket Science is hard.

ITAR went right out the window by the time Reagan was elected, and is more of a selective policy hammer than anything else.

Trust me, the folks keeping us safe (whatever that means) are about three decades ahead of whatever duct-tape gadgets some patsy can put together to grab the headlines for a few days.
Heck, it seems like lately they even help those patsies by supplying cash, encouragement, materials, and logistics.

A few people make smallish rockets and loud noises and accomplish very little, other than inviting retribution by lobbing large firecrackers over a wall.

When you can’t get food, electricity and medical supplies, the odds of putting together a useful sub-orbital space program are pretty slim. And even if you do, you might notice your launches never make it out of boost phase for some reason. oops.

But really, even a few mushroom clouds won’t cancel Fox news, stop us from eating at Olive Garden or McDonalds, or even close a single Walmart. They are highly adapted creatures.

We might lose a few more freedoms, but nobody seems to be able to make a list of those – and they’re mostly for decorative purposes anyway.

Basically, we have WAY more to fear from diabetes, lack of exercise and poor judgment than we ever will from whatever monster is threatening to eat our children tomorrow.

What would really be nice is a HMD/VR/AR product. Take your pick. A removable cover on a VR product would make it also an AR product. I think the hardest part in current products is probably the FOV limitations.

We have _tons_ of “virtual reality”. If you’re using a PC, get a stereoscopic display and grab your favorite 3D massive multi-player title. My favorite is Tron 2.0; I’ve used the environment with both a lenticular stereoscopic display and an 8 foot wide video projection. The head mount displays are still sub-par. Maybe with independant head tracking and an _vastly_ increased field of view, I would go that way. My preference was the video projector; I could lay on the floor below the image with my head slightly raised, arms laying extended at about 45 degrees, keyboard angled at one hand and mouse at the other. When all was set just right, I would move so little and be so comfortable, that I truly felt immersed in the towering images. The environments in Tron 2.0 are huge; they deserve to be experienced in that specific way! Try it!

The Dactyl game ran on Amiga systems. I used an amiga to build my own pseudo system. I took apart a mouse to make a head tracking unit (based on head position relative to shoulders, using the mouse wheels). I used a small black and white handheld tv as the (all too) heavy video display. The goal was to send each scan line to a different eye piece but it never got that far.

I tried to talk to the guys from Bethesda back when they had a wireframe 3d flight game for the Amiga to make a 3d version that I could use with this, but their objection was that they had tried similar and the planes were just tiny dots, so it didnt work out.

Still the head tracking was fun but it ended up in a box somewhere, haven’t seen it in years.

I always thought the hardest problem with VR is dealing with user motion. For a really great experience you should be able to walk around in your virtual environment, but unless you’re standing on some kind of nutty trackball, how do you allow for that?

I’ve been asking the same question for years, and with each passing year I add another expletive to it….

I’d love to see someone crazy enough to bring something like that to the home market (like Nintendo or, at a push, Apple) actually do it. Surely enough time has passed that these things can be mass-produced for a fraction of what they used to cost?> I think it’d be a true next generation console step.

I’d be happy standing/sitting in one place, using a button on a controller to walk forwards, but if they could just work out the kinks in the headset tracking so that it can track your movements faster than you can move your head, and the visual part so that your eyes focus at the right distance that it actually FEELS like you’re IN this virtual world, I’d be throwing money at them right now.

I’ve tried the Sony system which is AFAIK the current system with the highest FoV you can get, and that’s 50 degrees diagonal.

When you look into a system like that it’s a lot like watching a big screen in a room. But it doesn’t go outside your normal field of view because that would defeat the purpose of them. They are designed to be used to watch existing content like movies or games and that requires that it does not completely immerse you in the experience.

Most systems which have screens with smaller FoV than the Sony (such as the Carl Zeiss and Vuzix) are more like watching a screen at the end of a black hallway. Not a pleasant experience, and certainly not “VR”.

I hope they re-worked the optics for that, I have the first version of the Cinemizer LCD glasses and quite frankly the optics are shit, which surprised me considering they’re branded Carl Zeiss.

Of the various consumer-grade LCD video glasses I’ve owned/own, most are disappointing overpriced toys, Sony Glasstron & Olympus FMD stood head & shoulders above the others in terms of optics and the Glasstron 700E’s have a very good image.
I haven’t tried the eMagin z800’s or the Sony HMZ-T1 because I haven’t been to anywhere that has them for demonstration and the price is a bit too high for a gadget that frankly I won’t use that much (I rarely use the various LCD glasses I do have because your eyes can’t handle them for more than an hour or two)

I could definately see that something like this could be something like a next gen mobile console. Like the PS Vita or Nintendo 3DS. So you’re holding a controller which is also the console. And you have a pair of glasses to play the games.

At least that way there would exist more of a reason to get a mobile system like that as you could get a different experience than from your home system.

I’ve said it before, I’ll say it again..”iOptik by Innovega”.
It’s a Contact Lens that ONLY works on TINY , Extremely Close , Polerized Screens.
With these awesome contacts all you would need is glasses with embedded transparent HD OLED screens and you would have 120 deg FOV. It doesn’t need to be OLED, as long as it’s polarized it will work of course.

With these contacts you essentially remove all mirrors, lens, focus rings, from today’s VR/HMD’s.

If all goes well it should be here by 2014. PLEASE make sure you Check out there site, lots of really good info about the future of HMD’s and there new tech.. I can’t wait.

I’ve been told my vision is 15/20 so I need contacts anyway. Might as well make it a “Hybrid, Marco Vision, Polarized capable” kinda lens. Throw in some OLED glasses that automatically dim and become sunglasses when I step out.

120º FOV , Optics so light and small you wear them on you pupil , does not interfere with normal vison , instant 20/20 vision improvement with RX, contact lens tech is mature enough to trust, OLED is made for this kinda stuff and just coming to market. I’ll take two 1″ screens please.

That’s cool. There are several other vision systems being displayed as well so I think it’s fair to guess that the technology to make the “Google Glasses” possible will be available fairly soon.

And I think it’s fair to say that the Google Glasses are not specifically about the technology to see super imposed information. It’s about what information to display and how to generate it. So really, they are symbiotic in nature. :-)

Personally I think that the headset latency problem can be solved by using eye tracking. Only the part of the picture that you are looking at should be sharp and crisp. Your brain won’t notice that the rest of the picture is of lesser quality.

Another thing is that the computer doesn’t have to map the entire surroundings if you make the HUD partly transparent. Only the game characters have to be rendered. The rest of the picture could simply consist of you “natural” surroundings. Please let me know what you think…

Seems like a space that Microsoft’s Kinect Glasses/Project Fortaleza could have a huge impact. Assuming they’ll wind up being just as hackable as the Kinect itself was, I predict a revolution in homebrew VR worlds!