July 23, 2015

The Technical Challenges of Virtual Reality

Imagine walking around feeling dizzy and nauseous whenever you moved your head. This was a common occurrence in the early days of virtual reality.

It just felt wrong — what the virtual reality industry calls “simulation sickness“— and is the result of using a headset on a computer that can’t run the software smoothly.

With quality virtual reality headsets due to be released later this year, the tech world is abuzz with the promise of better technology. The hope is that users won’t just see fantastical games, but they will be able to walk around in them.

That sensation of getting lost in a game has led to the rise of another buzzword: presence.

Oculus, the company at the forefront of the virtual reality revival, described presence as “The unmistakable feeling that you’ve been teleported somewhere new. Comfortable, sustained presence requires a combination of the proper VR hardware, the right content and an appropriate system.”

Believable experiences in virtual reality come at a price. “The reality is that VR is more demanding than any other type of gaming we’ve ever had,” said Palmer Luckey, Oculus founder. It has taken Oculus years to get the headset and its accompanying software just right.

To understand why these specs are necessary, it’s important to know how virtual reality is different than normal software. Picture a game running on an HD monitor; this has a resolution of 1920 by 1080 pixels.

For a game in virtual reality, that screen would be cut into two halves, each brought up close to an eye, with a combined resolution of 2160 by 1200 pixels. This picture occupies most of your peripheral vision, which means it shows more virtual objects than a normal game.

And it is a stereographic image that runs twice, once per eye.

The other issue is that VR has to be smooth, so when a player moves his head, the picture reacts quickly and believably.

While television shows run at 30 frames per second (fps) and some console games run as high as 60 fps, the Oculus Rift runs at 90 fps to make the virtual world feel seamless.

“When you are trying to do 90 frames per second, stereo 3D, over-1080p resolution, and have a very wide field of view at the same time? It’s just a lot of things to stack on top of one another,” Luckey explained.

While a computer runs these extreme graphics, it also processes data from the movement of the user’s head and body (picked up by a camera sensor) and 3D audio that adjusts to the user’s movement. That’s all while processing input from controllers, too.

The result is indeed an immersive game, but it is taxing for a computer. And if users don’t have a PC that’s up to the task, the picture jumps around, resulting in simulation sickness.

“If you are doing a simulation on a 2D monitor and you are bringing it to VR, you are going to up the performance requirements by a factor 1.4 to 2x,” said Kim Pallister, director of content strategy for Intel’s Visual Computing Group. “It varies a lot based on the headset.”

Oculus and its Rift headset will have some competition. From HTC’s Vive to Razer’s OSVR, some will be more powerful than others.

When using a headset, the content of the actual games also matters. In virtual reality, players can lean over and look at something up close. Such scrutiny requires better graphics to be believable.

Players have to be able to see the hairs on a dog. Also, graphical tricks used in the past — like using flat 2D images to provide set dressing like grass — become obvious when viewing a 3D picture up close.

Developers have to avoid shortcuts and make virtual reality worlds quite detailed.

Also consider that the specs that Oculus released are only the bare minimum requirements; some games will push the graphics even further, requiring PCs that scream.

But it isn’t all doom and gloom. Computer prices eventually go down.

Oculus is working with graphic card makers Nvidia and AMD to improve virtual reality image processing.

Microsoft has tweaked Windows 10 for VR, with Oculus Rift being recognized without any hassle. Previous versions required a good bit of tinkering.

And Intel is doing research into how some of the heavy lifting that the GPU does can be done by the CPU.

“For that first round of VR products, graphic cards are going to be one of the most obvious things in terms of system requirements,” Pallister said. “But this is also an experience that is going to couple with a pretty high-end CPU and that you won’t be able to skimp there much.”

VR is just starting. Just as cellphones get annual releases, companies will likely improve models every year or two.

Resolutions will increase to 4K or more, motion controllers will come and, eventually, the frame rate will be increased beyond 90fps.

Much of this processing will require a more powerful CPU. PC hardware will have time to improve in both power and cost.

But with its high requirements, the first headsets released will be more priced for early adopters.

“It’s just the reality of things,” said Luckey, noting that those who balk at the price aren’t likely to buy into the first round of virtual reality equipment.

“VR is going to go down in price. Computers today that cost thousands are going to be Black Friday specials for $200 in a few years,” he continued. “But that’s not today.”