September 29, 2015

How Virtual Reality Works

Gaming as we know it is about to change. New visual computing advancements and gear such as Oculus Rift engage players with immersive, interactive and intense sensory experiences. It’s like super realistic science fiction playing out right before your eyes.

“Virtual Reality (VR) is a set of technologies that are aimed at fooling your senses into believing that you’re in a different environment than the real world,” said Kim Pallister, Director of Content Strategy for Intel’s Visual Computing Group, explaining how virtual reality works.

Edge of Nowhere is just one example of a new wave of computer games that will use the Oculus Rift headset in the coming months. In this case, the game brings players to a monster-ridden version of South Pole.

In six months, such immersive games will be available to everyone thanks to the release of virtual reality headsets. The Oculus Rift headset, which features large, padded goggles and headphones that are mounted to a person’s head with side and overhead straps, is designed to provide visual fidelity that enhances movie watching, game playing and world exploring.

Rather than looking at a flat screen ahead, in virtual reality, players look through headsets that bring a small image for each eye, then your brain combines the two into a three dimensional experience.

Since the lenses magnify the image, players aren’t looking at a rectangle in front of them in the same way they would a TV screen.

Instead, the image fills your field of view. The screen stays strapped to the player’s head, so the picture moves with the person.

While television typically broadcasts at 30 frames per second (FPS) there are games today that go as high as 60 FPS. VR displays moving images at 90 or even 120 frames per second, brings smoothness that makes the world feel more solid.

“The first and foremost thing is presenting a stereo 3D view,” Pallister explained. “Right alongside that is letting you change that view along with changing the position and orientation of your eyes and your head.”

It’s as if the wearer actually rotates his head in the virtual world, allowing him to look up into a virtual sky or look back on a virtual motorcycle.

“When you tilt your head downwards to look at the floor, the view of the world you are seeing looks down toward the floor in a way you expect,” Pallister said. “You forget you are wearing a head-mounted display.”

“I usually start off by telling people that this generation of VR hardware is more like the first generation holodeck,” said Denny Unger, Creative Director of Cloudhead. “The problem then becomes proving to them just how apt that description is and the only way to do that is to get them in a headset, in a walking volume and with hand controls.”

The first VR headsets will have screens with resolution similar to HD televisions, but soon there will be greater resolution, bringing even greater details.

Industry experts expect that within the next few years, the screens will jump from HD quality to 4K or even 8K display capabilities, and this will require ample computing performance to work smoothly.

Some headsets take immersion further by using external sensors like a camera to track how the player’s body moves. Now, wearers can duck to see what is on the floor or lean out a car window to watch the scenery.

The screens aren’t a static image. They are a view into an artificial reality.

The sensor also tracks an area of a room, so players can even walk around a room and the image reacts like they are walking in the virtual world.

The software to make VR images believable is tricky, according to Unger. It requires a distortion of the images so the lenses can fit the visuals in front of the eye properly. It has to present the image smoothly, so the moving graphics aren’t jerky and don’t make people dizzy.

Players travel around a world, but now, they are actually walking around a scene, leaning around the puzzles and using Vive’s motion controls to handle the objects involved.

It’s profoundly different from statically clicking a cursor on a computer screen.

Pallister compares this to the visual information provided by a game or movie on a computer monitor.

“They say, ‘We will do our best to tell a story, but you have to suspend your disbelief,” said Pallister.

“With VR, they say, ‘What are the things that cause people to realize they are not really in that environment? Let us try to fool those senses, one by one, until they can no longer tell.’”

Like the evolution of most successful consumer electronics, VR advance with new features and capabilities. Eventually things like tracking eye movement, facial expressions and even lip movement will be common, according to Unger.

Virtual worlds will be presented more realistically, and players will be more accurately represented in them. Such fidelity will bring more compelling experiences, and like cellphones, adoption will become more widespread.

“‘Good VR’ is undeniably compelling and consumers from a broad cross section will have a high appetite for it,” Unger said.

“The applications are so far reaching that it’s clear this medium will dominate in entertainment, education, training, travel, research and socialization. It’s one of those technologies that will have a profound impact on our lives.”