The Why, Where, How, What and Who of Virtual Reality Filmmaking

Why waste your time with something that was made up by bored science fiction and Simpsons writers?

Never before have we had access to the stuff dreams are made of, and given its current trajectory, a shared Virtual Dreamscape on the scene wouldn’t be all that surprising.

Virtual reality as it stands has excellent gimmick factor – its the most immersive platform of entertainment available today. It’s easily accessed through our smartphones, our Facebook and origami goggles google have dubbed ‘Google Cardboard’ for that total Virtual Reality experience that is both cheap and cheerful. (WARNING **Cheerfulness not guaranteed)

It’s an excellent marketing tool and in the filmmakers line of work and everyone knows that is where the money is. Facebook have even created heat maps for their videos – tracking where their users most look during a session. If you master this art and accept that Big Brother is watching you – you can make a pile of cash.

Where can Virtual Reality go?

Literally everywhere you can shove a VR camera! (Or 16 GoPro’s attached to a 360×180 degree rig – giving you complete freedom to create an equi-rectilinear 360 video and no nadir blind spot) Virtual Reality promises real life experiences that your life might not allow you to access. It’s the perfect premise for a generation of millennials who can’t afford to buy a house but they have the more power in their phones than the last shuttle sent into space – which incidentally would be a perfect idea for a VR Video!

Live streaming VR will allow you to sit in the front row of a Beyonce concert! Or if you were so inclined – Coldplay (the only obvious downside being that your boos would go unheard – but that’s what twitter is for).

Although a lot of the equipment such as rigs, light stands and higher quality cameras have been developed you will face a lot of trial and error. With this micro industry, “fix it in post” will inevitably become something of a mantra, but hold on to your fierce filmmaker’s optimism and you could become a pioneer with experience in something only a handful of people in the industry have today. The only limits lie in the adaptability of your technical and engineering prowess or your threshold for general squeamishness.

How to make your own Virtual Reality

I know what you’re thinking – how can I, a filmmaker, tell a story through the medium of song/Virtual Reality (song optional). Well here’s some top tips to make your musical virtual reality dreams come true!

1) Blocking and Direction

The most important thing to remember when writing is that you won’t be able to control where your viewer is looking – an unfortunate side affect of the total immersive experience. There’s a temptation to fill the frame with exciting props and light and interesting things, and in a VR piece without story that would be possible to explore like a virtual easter egg hunt. However too much can be distracting and in some cases nauseating. This includes movement; slow and steady is the best way to avoid causing motion sickness.

What works best is having an interesting setting – but keeping action concentrated in one or two quarters of your 360° circle of fun. Splitting your storyboards and scripts into quarters allow for
every part of the VR experience to be planned out in as much detail as possible. For example…

Think of it like dumping your audience in the middle of a play or a musical; the whole stage is filled but there is always a place to focus.

2) Cuts

In traditional filmmaking the cuts can be the make or break of a piece – however it can create the sensation of teleporting. Visual and audio queues however can both train where you want people to look and build up to a potentially disorientating cut. The MINI 360 advert “Real Memories” uses audio queues effectively.

3) Camera positions

Camera position in VR determines how tall your audience member will be.

On top of that – VR offers an interactivity that traditional film often doesn’t (aside from Hardcore Henry but that’s like the only one). Like gaming you become the silent character that everyone ignores, or everyone talks to; creating either a passive or empathetic experience. Gaming VR is of course very successful; boasting interactive VR that allows you to interact between headsets.

Like gaming, successful narrative VR suspends disbelief through world building and transporting you into new bodies of different ability, gender, age or none of the above – depending on whether or not you decide to give your audience a body. At this stage, being experimental and wild with your camera positions can only help hone your craft. The same applies with post. VR is more stimulating the more weird and wonderful the world you’re creating is. Put music behind it and you’ve got one hell of a gimmick to promote that band.

FOALS’ “Mountain At My Gates” is an excellent example of VR world building that is both simple and exciting to look at. You’re standing stationary in the middle of a courtyard, being serenaded by four FOALS, knitted together with a helluva lot of compositing, that includes birds, leaves and several Yannis Philippakis-es in one pretty far out video.

What do you need to start Virtual Reality filmmaking?

First you’re going to need a GoPro. Then you’re going to need 15 more. It’s easy! Just use that limitless supply of cash you independent filmmakers seem to have! Forgo eating for a month or so! Have sleep for dinner!

Okay here’s the deal with cameras.

Products on the market right now (even rental – which is preferable) are subject to two debilitating factors – “Stitch” lines and Resolution. Imaging your frame of 360° as a world map – this can be viewed as either a sphere from within (or Globe if you will) or one long flat rectangle (or Atlas), the latter of which you would deal with in post

The above shows one ‘Atlas’ (right) and a ‘Globe’ split in two, this is owing to the fact that this was shot on a Ricoh Theta S; which utilises 2 x 180° cameras to create a 360° image which is then ‘stitched’ together.

On desktop VR at first you can only see an aspect (ratio) of the atlas. This severely reduces the quality – for example if you record in a long 4k atlas frame, when the video is viewed after being stitched together and seen through YouTube you’ll more or less be zooming in on a quarter of that 4k.This lowers the resolution and makes the videos look soft. The answer of course is to film in higher quality, and higher quality cameras will do that at the expense of your wallet.

However having more cameras, like the 16 GoPro rigs, can improve the quality, but again you are at the mercy of “stitch” lines.

Imagine each camera is filming a certain size of box; those boxes have edges and with enough of them wrapped around a rig, and the edges of those boxes will over lap. To turn this into your ‘atlas’ – which is required to do any kind of finished VR post work – you must stitch these boxes together. You can make use of markers such as corners of rooms or props to align stitch lines. Where they meet however is not always perfectly lined up at the right angle, so like a punch in the face – these fractions of difference can cut off a mans leg as he cycles past.

Dealing with these is where your production becomes post heavy. People who can stitch are in short supply – learn to do it well and you will have VR productions queuing to work with you. There are things you can do during filming to ease the post production pains; its important to let your actors know where the stitch lines are, and if they cross them they can’t get too close to the camera. The use of a digital flash, enveloping a room or scene in a bright white light that will take up the same frame can replace a clapperboard and help sync the timelines of recordings. Clapping also can create an audio reference. VR is the perfect partner for binaural audio; recording sound in 3D that will allow sound (with the right headphones) to pan and replicate how living ears work, and is recorded by this freaky thing.

There are others but this one makes me feel pretty.

Like any edit there is no set way to get the job done. Nuke is the industry standard software for this kind of work utilising a plugin called CARA VR, although there are plugins available for Adobe After Effects by the name of ‘Metal Skybox’, which is familiar as well as more cost effective. Less cost effective but still another way to go is ‘360VR Toolbox’ which is compatible with AE, Premiere Pro and Final Cut Pro, this software allows headset monitoring while editing in full 360°.

Who’s who of Virtual Reality

If you like this article, tell your friends, if you’ve truly let it inspire you – you’ll find these people and hound each and every one of them until they give you your first job in Virtual Reality.

eleVR – Like anything; practice makes perfect – but there are those out there who have done some legwork already. Elevr is a website dedicated to the science of VR, playing with different VR
filmmaking methods and improving them any way they can. Remember the only difference between goofing off and science is writing down what happens!

“You can build any building you want if you know how to lay bricks!” – Elliot Grove whoever that is.

Felix and Paul – Félix Lajeunesse and Paul Raphaël are responsible for a Jurassic park Virtual Reality documentary that is off the chain. Their content is available on the oculus store.

The Foundry – has cornered the market of VR in post houses in London. (If they’re not in London – I don’t want to hear about it!) They have a research team “dedicated to solving the big challenges that our industry faces with new technologies” (Andy Whitmore CPO). If you’re looking to learn VR post, going through this posthouse might just be right up your alley.

Chris Milk of Samsung MilkVR has a studio vrse.works, his portfolio offers a range of live and composited VR experiences, as well as pioneering Virtual Reality sound for Samsung VR. MilkVR specifically is an upload site for the US that is worth pitching to.

Kaleido VR – a community for Virtual Reality creators is funding projects and content. Many other festivals, not just Raindance are taking up the VR mantle and hosting competitions; South by Southwest; Tribeca and Sundance to name a few. If you can find a localised event – it’s a good idea to go to them and pitch, you never know what may come of it.

Other good names to know are wevr.com who are curators of content and honeyvr.com, who are investing in CGI VR content.

Lytro are looking for worthy candidates to use their LightField Camera; this is a camera that can focus on literally everything in range and can measure light even around corners!

But I know what you’re thinking avid blog reader, where can I; a filmmaker; get hands on experience with VR equipment as well as up to date industry insight?

Johnny Johnson is a VR evangelist who embraced the world of 360° filmmaking at the very start of its current incarnation in 2014. Having initiated his career by studying at Westminster Film School, he went on to become a freelance camera assistant on a wide range of commercials, movies and TV dramas. In 2007 he formed a facilities rental company and has continued to work in production facilities and media business development for a range of broadcast and media organisations since. Alongside this work, he as produced a number of shorts, commercials and a horror feature. He currently operates SpeedVR as a resource for the 360° VR filmmaking community.