Content Creator’s Guide: Making 3D Audio and Interactive Sound

As a content creator, I’m always asking how I can tap into the next technological tsunami. And the next wave on the horizon is head-tracking 3D Audio for VR. It allows us to interact with audio in real-time. When I heard this technology was going to be at CES earlier this year, I had to hear it for myself to know if it would be a groundbreaking technology or just another fun idea like the never-to-be-mentioned Nintendo Virtual Boy. So how is this all being transferred to virtual reality and gaming? Sounds usually come through our senses in a stereo sense, or through two channels. When we are out in the world, our brain does this thing called localization where it places the sounds where we think they are, in a particular place. We haven’t been able to record sound like this until recently, and more and more people are getting a handle on it and using it to their advantage. The real trick is getting head-tracking headphones and mics so that they move with our heads, instead of us having to try and do it manually.

Beyer Dynamics Headzone

3D Sound One Module by 3D Sound Labs

Head-tracking headphones aren’t new; Beyer Dynamics put their set, called Headzone, on the market in 2006. Only issue was the sadly-prohibitive $2,000 price tag. People have also used the WiiRemote as a headphone gyroscope to do the trick back in 2010 – hardly practical, but it had the right idea.

Then 2015 happens and in-swoops 3D Sound Labs like a raging Phoenix, introducing their bluetooth version called Neoh at CES in 2015. The $300 price tag is so much lower because it’s using the consumer’s smartphone to do all the head-position processing. Their new $99 product is just the head-tracking module which can attach to your own headphones! Now we’re talking…

If the source audio is 5.1 or 7.1 surround sound, it makes magic happen. After I bought their headphones, all I wanted to do was figure out how to make my own 7.1 mix. Thankfully, with the right tools, it won’t cost you anything. I installed the Reaper audio workshop on my laptop, and then I used a free surround sound rendering plug-in called Surroundizer.

I recorded my own sounds, and suddenly I was walking around my room, interacting with the soundscape that I custom made. Next, I made a 7.1 mix using video game sounds from Bioshock… Let me tell you – it put the fear of God in me. It’d been quite a while since technology satiated my craving to feel like a kid again.

That’s all sunshine and roses, but the future of VR sound isn’t 7.1 – it needs to be 360. So, the question remains, “How can I create 360 content of my own?” Well, capturing 360 video is a no-brainer with a simple device like the Ricoh Theta M15 for $230. However, capturing audio in 360 (ambisonic sound) is kind of like catching lightning in a bottle… and kind of expensive.

You can get true ambisonic sound capture with this device from Brahma: a 360 microphone and recorder in one. However, it’s out of most folks’ price range – at around $1,000. So, what about those who don’t have an extra grand lying around?

Binaural microphones are much more affordable, and they have a direct application to VR! Products like the Hooke Verse can be portable, durable and worn on the head making it a snap to record my sound wherever I am or whatever I’m doing because it’s wireless. With binaural recording, you are directly transported into whatever world and wherever you were when you recorded your sounds. And that’s basically what VR is doing with sound to make it sound like you are really in the space–they are using their knowledge of sound localization that your very own brain uses to help them place sounds where and when they need to be, so that virtual reality feels just that more real and interactive.

Thankfully, binaural recordings can be natively converted into VR because companies like Oculus and Samsung already support Quad-Binaural source audio. You can use Oculus’ downloadable SDK, or use an app called Milk VR for use on the Samsung VR.

All you need to do is record the scene four times – once while facing North, once while face East, then South, and West. Import the 4 tracks, and the software does the rest. Finally, a way to do this without breaking the bank…

If any of that went over your head, allow this to console you: I didn’t know a single thing about 360 sound until I went to CES this year. I did a lot of Googling… and I’m pleased with what I’ve found. It’s inspiration fuel.

I hope my research empowers those who have ideas and stories to tell in these unique new ways so that they can be shared. Our ears are open – bring it on!

Join the discussion One Comment

360 video and/or game dev sound designers, it is highly recommended that you do not upmix your masters to 5 .1 or 7.1. There are exceptions, but that’s a good general rule of thumb. If your master is quad-binaural, the deployment platform will crossfade your 2-channel stereo tracks dynamically according to head tracking if it’s set up to do so. Upmixing any binaural signal to surround will result in massive loss of spatial clarity. The only time you should do it is if your deployment platform of choice is offering no other option.