What's The Big Problem With Developing Oculus Rift VR?

Answer: It's so awesome that developers will need to rethink the way they design games. "People ask all the time, how long does Rift integration take?" says Nate Mitchell, one of Oculus VR’s founders and the VP of Product. "The integration for a senior developer doesn't take long. But designing an awesome experience is so much harder." The simple, natural interface is a win—but now we just need to totally reinvent gaming paradigms in order to use it.

"It's like I am a painter and a new color is invented. If that did happen, every painter would be like, 'I've been spending so much time training myself to paint and there was this certain set of things that I understood; and now, all of a sudden, there's this thing that just fundamentally changes the craft.' That's exactly how this feels."

These words, from Paul Bettner, the creator of Words with Friends, sound like hyperbole, but for many people affordable and effective virtual reality is a paradigm shift for how games are played and how developers make them.

We have talked about Oculus VR before, but here is a digest: Oculus VR was a company set up after a prototype virtual reality headset called the Oculus Rift made a big splash at last summer's E3, the annual video game conference. After the company was founded, they launched a project on Kickstarter in August and raised $2 million. Since then, they actually have produced a $300 dev kit, an advanced prototype for game developers to try out and to start making games with. They started shipping those in March, and have shipped about 4,000 of an expected 20,000 or so units.

"So people get the dev kit and they just start racing through all of the demos and seeing what everything's like. Even then, you are going to run up against a wall pretty quickly," said Nate Mitchell, one of Oculus VR’s founders and the VP of Product. "There's just a lack of real experiences. People are happy to have Team Fortress 2, but it's really intense."

Team Fortress 2 is the first game to support the Oculus Rift dev kit. TF2 is a fast-paced first-person shooter where teams play against each other. It is made by Valve Software, creators of such hit series as Half-Life and Portal, as well as the company behind the enormous digital game store Steam. Valve has supported the Rift from early on and updated TF2 to add Rift support. "We wanted to learn as much about virtual reality as we could and Valve has always found that the best way to learn about something is to build something in that space and get it in front of customers," said Valve's Joe Ludwig, one of the programmers behind the Rift implementation.

Ludwig found that the biggest challenge to playing in a virtual world were the controls. "Figuring out how to combine mouse and keyboard input with head tracking was not very well explored territory, so we had to break some new ground there," said Ludwig. Ultimately, TF2 would support the ability to choose between eight different control schemes, changing the behavior of the game's controls and the Rift's headtracking feature to fit your game style. Do you position the crosshair where your head looks, or with the mouse? Does your character turn with your head, or the mouse?

Adjusting how a player controls their character in a virtual world will remain a concern with games that were made for flat monitors and not virtual reality. BJ Wooden is an indie game designer, founder of Cymatic Software, who decided to fiddle with the Rift as he designs his own VR game. He garnered some interest on YouTube as Cymatic Bruce for posting videos of existing games adapted for the Rift. He used third-party software called Vireio Perception to get the games running, since the actual Oculus software isn't supported by these games yet.

Vireio allows games not made for 3-D headsets to deliver stereoscopic images, and then allows you to tweak the convergence point so that it "pops" correctly. And then you have to assign movement to the headtracking and modify the game’s controls since that piece of movement has been reassigned. And even after all that you may have to tweak things in the game, turning off shadows or reflections, so that the 3-D graphics converge correctly. BJ Wooded said, "VR is a new experience and a new field. And I think firsthand experience is the best way to get that information."

Those who watched the videos came to see how much VR does deliver to their favorite titles, like the aforementioned Portal series, the award-winning roleplaying game Skyrim, or the parkour action-game Mirror's Edge. Wooden found user interfaces an issue. "You can't have your information on the outskirts of the screen. You have to have it near the middle, where the person can actually see it. Or you have to have it someplace where the person can look on the player's character model," said Wooden.

These games ported to the Rift revealed some of the challenges inherent to seeing first-person into a virtual world, especially when trying to tell a story. "Taking control of the camera has to be very carefully done. In my Skyrim playthrough, it was very jarring when the guy is kicked over. The computer takes control of the camera and tosses you on the side. I had to turn my head to the side with it, to avoid being disoriented," said Wooden. "Is it an absolute that you don't take control of the camera at all? Is it you can only move at this velocity and at this many degrees from the horizon before the player hates it? A lot of stuff is undefined territory."

Oculus VR is aware of these issues with simply modding games. That was one reason why they released the dev kit ahead of a consumer version, so developers can play with and modify current games, to figure out some of these issues before creating new games. "The mods don't really work. A lot of the experiences are not ideal for VR. There's all these things we talk about that make for a good VR experience. A lot of those are totally missing, especially from the mods," said Mitchell. "People ask all the time how long does Rift integration take. The integration for a senior developer doesn't take long. But designing an awesome experience is so much harder. That's what takes time."

Some developers are tackling this problem head-on. Denny Unger is the founder of Cloudhead Games, and he's been following the Rift from the beginning. Since Cloudhead is a small, indie developer, Unger believe it gives them an edge on VR. Unger said, "Indie developers can take on a lot of risk. We have a very low operating margin and we can jump into this and go, 'What does a real proper virtual reality game demand? What kind of control schemes? What kind of experiences?' I think major publishers are going to have big problems with that because they're locked into the pre-established, console/PC market where they are defined by the hardware."

With the Kickstarter project for their game successful, though they are continuing to accept funds and give rewards on the website, Cloudhead is diving into true virtual reality. The game is called The Gallery: Six Elements and it is an exploration/puzzle game, not unlike the '90s megaseller Myst. Such a game is perfect for a virtual world. "We chose something that is slower paced, that is exploration based, that's fantasy, and a fun, engaging experience," said Unger.

Unger knows it won’t be a simple transition for gamers. "We are trying to ease people into the experience. You give people a virtual body they can see, arms and hands that they can actually use in a natural way. By doing that, you give them a surrogate body to enter into this world. And you give them the same rules they use in everyday life, the same rules of how fast you can walk and how fast you can jump in real life," said Unger. "That's a big difference between what you see in modern first-person games where players run around at 30 mph, whipping themselves around at these superhuman values that don't work in VR because it will make people sick or they won't connect to the experience they are having."

Oculus Rift

It’s more than today’s games’ exaggerated physics that Unger wishes to avoid. "Just throwing someone into a first-person killing simulator is a sad statement of what the possibilities are with this new technology. It will become apparent that VR can actually give people enlightening experiences that you couldn't do on a monitor," said Unger.

Paul Bettner agrees. "A gun, for better or worse, is a tool that can reach out and make change happen all around you. When you are confined to a TV and all you have is a gamepad, that's a metaphor that really works," said Bettner. "But when you are in an immersive VR world where you can dribble a basketball or pick up a paintbrush, that need to simplify controls down to a gun evaporates. You can really create a game about anything."

Bettner left Zynga to form a new studio called Verse. His drive was to create games for exciting new platforms, like the Oculus Rift. Such indie companies like Verse and Cloudhead are turning to the Razer Hydra to provide natural controls. The Hydra is a pair of hand controllers made by PC peripheral maker Razer, in partnership with motion control company SixSense. It is not unlike the PlayStation Move or the motion controls for the Nintendo Wii. It tracks your hands in 3-D space as you move the unit in each hand around. And this allows for some truly virtual gaming.

Oculus VR worked with Razer to create a virtual reality demo featuring a house and a variety of objects to pick up, examine, and move. "People just pick up beachballs and throw it outside and it goes soaring. Seeing your hands, being able to interact with the game in new ways," said Mitchell. But such simplicity, combined with the natural interface, really works. Bettner played that demo with great interest. Bettner said, "You pick up a basketball and walk up to this little house, and you can throw it against the wall. You can play this game of bouncing a basketball back and forth and that one little thing is the most compelling thing! It's so natural. You don't have to think about it because it's something you do in the real world."

Cloudhead is supporting the Razer Hydra in The Gallery, making it likely the first truly virtual game that will be released for the Oculus Rift. And BJ Wooden has been using the Hydra. "It's attached to your hand in 3-D space and you can move an object in 3-D space, just like you would move your hand naturally. And your brain accepts that," said Wooden. But there is a step beyond that. "Instead of having the Hydra controller sitting in for your hand, I just want to use my hand! I just want to move my arm and my hand and have it be there in virtual space. That's going to be key."

Since you can simply turn your head with the Rift, simply moving your hands to interact with a virtual world makes sense. Microsoft has moved in that direction with the Kinect, which senses your whole body’s movement. Leap Motion, will be shipping their product in July, a three-inch sensor that tracks hand movement. "The Kinect hints where this stuff is going to go, or the Leap Motion Controls. There are problems with all those, but it will get ironed out," said Bettner. "In a year or two, you will have something that can track where your hands are and you can see your hands in the virtual reality world without any gloves or wires or anything else. That's just going to unlock new types of games that we can't even imagine right now."

So once developers adapt their games for a virtual world with headtracking, and adopt the Razer Hydra or possibly even a camera-based interface, what is the future of VR games? Bettner intends to find it. "Words with Friends couldn't have existed before the iPhone. It needed a networked, handheld, portable, game device that was always with you for that game to even exist. There are virtual reality games that are yet to be made that will be like that, that you couldn't even make it work on a TV," said Bettner. "I hope that the games that we are building end up being like that. We want to build the experience that could only exist in VR."

Unger’s The Gallery may be the closest thing to that. "You are an urban explorer spelunking a modern sewer network and you come across this gallery. You can physically enter into these painted worlds and experience different things," said Unger. "In the Water world you will be able to enter a small submersible and dive down under the sea. Or in the Air world, you're actually in the middle of a beautiful night sky and clouds below, and you are on these platforms precariously balanced in the middle. These areas show what's possible with virtual reality."

Wooden’s work-in-progress will also take advantage of VR’s strengths. "I am working on a level that is inspired by the Holodeck from Star Trek. So far I have an empty Holodeck and I'm able to walk and look around. I want to expand that into a hub with several different experiences," said Wooden. "Maybe you find yourself on stage at a rock concert, your Hydra controllers are now drumsticks and you have to play a drum solo. Or you are in a dark alley, with a seedy club there. You have to do a secret knock on the door, using the Hydra in the right hand positions. And so on and so forth, just moving between these different experiences that are unique to VR."

Razer Hydra

Mitchell, as one of the founders of Oculus VR, is waiting for these truly virtual games. "The best experiences are going to come from the indie community, in areas where we don't even know what they are yet. It could be some scuba diving game that ends up being the best. Or a hang gliding game where you are flying over an island and you look around," said Mitchell. "You could imagine these neat experiences where VR gives you the ability to go anywhere and do anything, jumping on the moon or something like that."

And Oculus VR is doing everything they can to get there. They are talking to developers about what they like and don't like about the developer prototype of the Oculus Rift headset. The company will be increasing the resolution of the display from 720p to 1080p, for a more detailed picture. They will also likely add positional sensors, so that the headset can track not just the rotating and tilting of your head, but leaning forward, ducking, and sidestepping. Without that, the head movement doesn’t feel quite natural and may actually cause nausea due to the disconnect between a player’s inner ear and what they are seeing.

And when people other than developers be able to use this tech? "The commercial version of the Oculus Rift is rumored not to be out until the third quarter of 2014," said Unger. Then the public will have inexpensive virtual reality, shaped by developers. And from there? Mitchell wants to see more than games, "Something that is going to be compelling is Skype in VR. Or if there's a Wikipedia for VR: You walk around exploring a ruin, being able to explore the Colosseum. Those are some of the things I am most excited to see."

Unger also believes accessible virtual reality will move beyond entertainment: "This isn't just a gaming tool. It's potentially a training tool. It's something that can be used for treating mental disorders. There's a vast implication of what this is going to do for society."

Kevin Ohannessian

His work has also been published by Tom's Guide, Tech Times, MTV Geek, GameSpot, Gamasutra, Laptop Mag, Co.Create, and Co.Labs. Focusing on the creativity and business of gaming, he is always up for a good interview or an intriguing feature.

I'm really looking forward for this .. might even pick up a dev kit ..one thing i don't get is when in fps, why not make the mouse control your cursor/shooting arm wasd control; your legs and the Orift control your where you look ..it seams like a simple solution to the problem of getting the Orift to work with keyboard and mouseor is it not that simple?