Imagine that your good friend has recently purchased a Virtual Reality (VR) headset and invited you over to their place to experience it. "Come on over," they said; "it'll be fun" they said. You arrive, only to find your friend hogging the VR headset. Boring. You can't exactly see what they're doing. All you see is them wearing the goofy goggles waving their arms around like a buffoon.

What if you could experience what your friend was experiencing? What if you could see your friend in the virtual world, as before, but also see what they are seeing as if you were able to be in that world with them? Wouldn't that be a much more compelling experience? Here at the Imaging Research Center you can now do that.

We have assembled an Observational VR lab space. Observational VR, otherwise known as Mixed Reality, allows people to watch subjects in both the real and virtual worlds simultaneously. To accomplish such a feat, we have set up our VR computer in a green screen room and added a video camera. The video camera then records a subject using VR in front of the green screen.

In the virtual worlds that we create we add a third person virtual camera where it’s movement, position, and rotation are tied to tracked VR controller. That tracked VR controller is attached to our video camera. Then using a software program such as OBS Studio we combine the video feeds from the virtual camera and video camera into one where the green screen is keyed out and replaced with the virtual world. Now the subject can be seen in context, interacting directly with the VR environment, rather than looking like someone making funny movements for no reason.

HTC Vive controller affixed to a camera.

Of course, our primary purpose in creating this space is to further research into interactive virtual environments. Do you have ideas about using this new space? Questions about observational VR? Do you want to see it in action? Come join us at the IRC’s Observational VR event: