Immersive News In Virtual Reality

Immersive journalism is a form of non-fiction storytelling that gives users a first-person experience of news events. By making use of Unity 3-D, a body-tracking system, natural sounds (recorded by witnesses at the scene) and an Oculus Rift, users are transported to a virtual recreation of the event.

But why is immersion important to the news?

It’s an important tool for story-telling in the same way that its an important tool for video games – immersion brings the user one step closer to the event, and being one-step closer to the event means that the audience is one-step closer to the sights and sounds (and even the feelings and emotions) which accompany it. The goal of well-crafted journalism has always been to elicit a connection between the audience and the story, and what better way could there be than using badass virtual reality to recreate the feeling of actually being there.

Hunger in Los Angeles – created by Nonny De La Pena – is a great example of modern, immersive journalism. The simulation puts you in the shoes of an eyewitness in line at a food bank at the Los Angeles First Unitarian Church. Coupled with live audio captured from the scene, users are put through the trauma of witnessing an elderly citizen have a diabetic attack (they survive). The end result is a fully immersive, simulated news scene where participants can suit up, walk around and interact with other characters in the story.

Nonny has been creating experiential and body-tracked virtual reality for six years. A graduate of Harvard University and USC, her work on Hunger was collaborative USC’s famed MxR Lab. The HMD used was a prototype created by a then-lesser-known MxR member by the name of Palmer Luckey. It was demoed at SunDance 2012, and the queue was 3 hours long.

“I still use the HMD Palmer worked on all the time”, Nonny told me. “This review in the verge gives you an idea of how powerful that HMD can be when it is coupled with the Phasespace trackers. Palmer was instrumental in other ways as well. He even offered to drive the truck back from Sundance with all of the equipment! To be honest, Hunger was my first real proof of concept that immersive journalism could even work and that I wasn’t completely crazy. I took surveys at Sundance and people overwhelming said the scene felt “real” at times.”With the Oculus Rift, people now see it can reach a wide audience.”

This has powerful implications for the way we, as human beings, make sense of the news.

When we digest a news story – or any story – we do so as an outsider, objectively. To make sense of it, we practice moral judgement and then come to a conclusion based on our own beliefs as an outsider. We draw conclusions based on what we believe would have been the most moral action to taken, were we physically there ourselves. This dis-junction between abstract moral judgement and actual real human behavior is a discrepancy that can only really be studied in the field – in real environments – because who can really know what they’ll do unless they’re actually there making decisions and witnessing it first hand?

By employing IVR (immersive virtual reality), these situations can be recreated virtually, and because users can walk around and interact with the characters in the scene, they are forced to participate and make choices. This is the news of the future!

Research conducted by UCL’s Department of Computer Science has shown that IVR often leads people to think about and respond to virtual events as if these were real, which makes them an excellent tool in the study of human social behavior. Immersive journalism is an important subject because its results provide laboratory-controlled data in a field where available data is either opinion based, or based on historical record.

“After Mel Slater (Professor of Virtual Environments at UCL) let us into his lab to build an embodied piece about being a detainee in a stress position using an HMD, I never wanted my pieces to be a “sit in the audience” experience again”, Nonny explained. “Not long after, it dawned on me that using VR spaces to tell news stories was a no-brainer.”

The simulation that Nonny is talking about is a recreation of the events surrounding Detainee 063, Al Khatani’s imprisonment in Guantanamo Bay in 2002 and 2003. The simulation makes use of information provided in FOIA-obtained transcripts which document his interrogation. It involves sitting in the same stress position that he was held in, whilst the sights and sounds of a Guantanmo Bay cell immerse you.

The goal of the piece is clear – to stir up feelings of empathy in the participant by figuratively putting them in ‘his shoes’. This is something that on-the-ground war reporting tries to capture, but without being immersed in the environment completely, it’s impossible to trick our brains into feeling like we’re there.

“A full bodied experience can be amazingly powerful!”, Nonny told me. “One caveat – if the audio sucks, it won’t matter how good the graphics are – it’s unlikely people will feel like they’re “there”.

And that’s one of the biggest things about VR – the display is just the start – its enough to trick our brains a little bit, but for a truly immersive VR experience, we need just a little bit more tricking before it lets us believe that we’re really there. Hence the need for high quality sound, hence the need for omni-directional treadmills and (someday soon) haptic bodt suits/ neuro-sensors. We need to really feel like we’re there, and its got a lot more to do than just putting on a Rift and picking up a controller. And its definitely the way I want to receive my news from now on wherever possible.

So, what’s next for immersive journalism?

“I just won a grant from Tribeca and the AP/Google Technology Scholarship that will let me build my next piece called “Use of Force Protocol.” Using cell phone video and audio depicting the border patrol murdering a migrant, (the San Diego coroner’s office ruled it a homicide), I will recreate an eyewitness account for the audience to experience. I hope it will be meaningful as it was uncovered by good investigative reporting and includes a moment when one witness bravely tries to intervene. Anyway, I never seem to make anything too gentle…”

As soon as Hunger is released onto the rift this fall, I’ll post up a link.