Hey! Listen: Sound Design with Unity & FMOD – Part 1

Series Overview

In the first part of this sound design tutorial series we will be learning how to integrate FMOD Studio with Unity and setup a few basic audio events using only the scripts and features that FMOD provides. In the rest of the series we’ll be looking at calling and tweaking FMOD events via code, doing some cool stuff with occlusion and physics sounds using filters and parameter changes at run-time. We will also be looking at adding the Google Resonance plugin, creating super-realistic audio environments for use with AR & VR games and applications.

How does this all help you? Well for starters, FMOD Studio integration makes it really easy to setup most of the audio logic you’ll ever need in your games with just a few clicks! For a little bit of time spent setting up a Unity project with FMOD, you will save heaps in the long run. And it’s all free!

Now, I’m writing this tutorial series with the assumption that you are familiar with the Unity editor because there’s a few things to learn if you’ve never tried FMOD. Also, I’d like to touch on a bit of theory as well as practical skills. First up, let’s talk about why sound design is worth studying and how we can get better before we start on the technical stuff.

Sound Design Inspirations

Where can you gather sound design inspiration? YouTube let’s plays. Really. I usually only use YouTube to access discussions on game design theory but I don’t think anything has helped me with sound design as much as blind gamer Terry Garret playing The Legend of Zelda: Ocarina of Time (OoT). Truly-inspirational stuff, not just for sound design considerations! It’s no wonder OoT is so adored for it’s ‘rich, atmospheric environments’ there’s a lot of sounds in the 3D world that Terry can use to gather information. I know this game is quite old and there are many better examples for complex sound in modern 3D games (such as Overwatch) but for such a huge topic it’s best to nail the core basics first.

Watch a bit of the embedded video below, one of Terry’s let’s plays, and think about this: which sound events are helpful and why?

OK that was a trick question, all of the sounds are helpful! When it comes to sound design for accessibility, more is more! But the ‘why’ does help us to remember the ‘how’ that we will be covering in detail ahead.

We should be designing for accessibility always and in multiple passes. Can we play with just the sound? Can we play with just the image? If you ask yourself these questions, your game won’t just be more accessible, it will be that much more immersive and involved for the many that can access both sight and sound. Your game world is only as rich as either the audio or the visuals, so if you want a good review for your latest hard work, make sure you’re giving love to more than one of the player’s senses!

So I’ve watched quite a few of Terry’s OoT let’s plays but I’ll attempt to make a rough list of the most important sound design considerations that I think could be gathered by studying any one of them. I’m going to describe these considerations under the following headings: ambiance, persistence and feedback.

Ambiance

Ambiance describes the particular qualities given to a sound by the space in which that sound exists. In this example I am including all the audible ways that the environment reacts to the player character’s presence and actions. Footstep sound changes, wall collisions and familiar audio effects such as reverb heard in empty spaces are all examples of ambiance. In OoT, Terry determines where he is by the sound his character’s footsteps are making, he knows he’s hit a wall when those footsteps slow down, and he knows what the wall’s made of by hitting it with his sword. Doors make a certain sound when struck and this sound tells Terry that he’s within striking distance but he might just need to realign to access it.

Persistence

The persistence of a looping audio clip might be considered ‘ambiance’ per the more-common use of the word, and it kind of is… But the use of persistent audio in accessible games is super-important so it should get it’s own heading! The right amount of persistent and distinguishable sound effects can enable sight-impaired players to navigate effectively without relying on cues that they’ve run into walls. Persistence is vital for hazards or enemies to gauge their location and potential movements. In OoT Terry also uses the persistent sounds of; a torch by one of the doors in a dungeon room, the humming from an interactive crystal object, and the trickle of water that marks the river running through Hyrule field to navigate through the game world.

Feedback

The bare minimum sound category required for a user, with feedback, more is more. There are so many examples here and much of it is this: if you press any button, you should know what it has done (or not done) without seeing the screen. Feedback communicates affordances. In OoT, feedback tells Terry that he is near enough to interact with a sign or a character, that he can target an enemy or interactable and know which one it is (thanks to Navi’s alerts). Again, you can never have too many feedback sound effects! If something happens or, in some cases, doesn’t happen then it should have a sound effect!

These above headings will help us understand and categorise the FMOD components we will be integrating with Unity and the logic for our sound events within FMOD Studio.

Setting Up FMOD Studio & Unity

After you’ve installed FMOD Studio and downloaded the FMOD Unity integration package let’s setup Unity. In an existing or new Unity project, go to Assets > Import Package > Custom Package… In the popup window, navigate to where you saved the FMOD Unity integration package and open it.

Once that package has been imported you should see the below errors in your Unity console. If you ever get this error in the future, you may have just changed some file folder structures. We just need to point Unity towards an FMOD Studio Project folder!

So lets just open FMOD Studio, make a new project and then save it by going to File > Save As. I recommend saving it in the Unity project that this Studio project will be linked to so that the project folders are not split up accidentally in the future.

IMPORTANT! Saving your FMOD Studio Project is important, but none of the logic you implement there will be seen in Unity unless you hit Build! This builds all the logic and sound into something that applications like Unity can access easily.

Now that we have imported the integration package into a Unity project and we’ve created an FMOD Studio project we can point Unity to the FMOD Studio file in that project folder. In Unity, Click FMOD > Edit Settings and you should see the FMODStudioSettings data in the Unity Inspector panel. It should look like the screenshot below. If you don’t see the FMOD button at the top of your Unity window then you may have forgotten to import the FMOD Unity integration package (see above).

Next click the Browse button next to the Studio Project Path, navigate to your FMOD Studio project folder and open the FMOD Project File.

The Inspector should look as follows.

Also take note that there is a new folder called “StreamingAssets” in your Unity project folder. This folder now contains the file that your game build will access at runtime, it contains all the sound and logic from FMOD Studio. If you lose it you can just rebuild your FMOD project.

Now we’re all setup! If all’s gone well you can now hit Clear in your Unity Console panel and you shouldn’t see anymore path reference errors pop up.

The Game & Assets

In this tutorial we will be setting up a little indoors level to be played in first-person perspective. The setting will be a large office building and we’ll be using the persistent sounds of the office equipment, collisions and footstep changes to help our players navigate. The player will need to find a specific computer in the office and interact with it. I know this seems pretty boring, but on the plus side you might be able to record all of the sounds you need in your own home. Otherwise I suggest you grab some CC0-licensed sound from freesound.org. Please note that you actually don’t need to import these sounds into your Unity project, it will happen when we make an event in FMOD Studio. Just leave the sounds in your Downloads folder or something. You might also want to grab some textures from opengameart.org to help you distinguish between the different floor types – the footstep sounds I’ve used are for wood and concrete. I’ve made a zip file with the sounds, textures and 3D models that I’ve used in this tutorial – click here to download .

Creating Events With FMOD Studio

There are a few ways to create Events in FMOD Studio and my favourite is the ol’ drag-and-drop. Simply drag an audio file from your operating system’s file explorer into the Events tab in FMOD Studio. Let’s start by adding some persistence to the computers that, along with the desks, will function as obstacles and potential interactives to sort through in search of the correct one. Let’s drag the computer_hum.wav file into Events, leave 3D Event selected in the popup window and click Create.

The above step has done two things: made a new 3d Event and added a new file to your FMOD Studio Audio Bin. We can check out the Audio Bin in FMOD Studio by clicking Window > Audio Bin.

The Assets tab in the Audio Bin shows us all of the working files in this FMOD Studio project folder. Dragging files into either the Events tab or the Audio Bin Assets tab makes a copy of that file in the project folder. You can go ahead and drag all your sounds into the Audio Bin, or you can drag one at a time as we create more Events later.

Our next step is to add some simple logic to our computer_hum event that will suit a persistent sound effect for multiple identical objects in proximity. First though, let’s make this Event loop. Right click the audio track and then click New Loop region. Notice the blue bar that appears above the audio track, this is the Loop Region. You can pull the handles left and right to shorten or extend the Loop Region into or past the full length of the track. I’m just going to leave it looping from the start to end of the track because I don’t want any gaps or cuts and I know this track loops nicely in full.

Next we will address the phasing issue. Phasing happens when we play multiple identical audio tracks at the same time, this causes the volume to increase and we don’t want that. To solve this we are going to add an offset to the start time of this Event and then randomise it. Left click on the audio track to select it, then click on the little white triangle to extend the Trigger Behavior tray at the bottom of the FMOD Studio window. Next adjust the Start Offset dial by left-clicking and dragging your mouse up. Leave it at about 50% – this will make the Event seek to half-way through the audio track when it is called in Unity. Now we add what’s called Modulation to randomise the Start Offset. Hold the Alt key (Option for Mac) on your keyboard and click and drag on the Start Offset dial, notice that a thick green line expands from the pointer on the dial. Lastly, click the Loop icon in the top-right corner of the Trigger Behavior tray to ensure that it loops back to the start of the audio track as soon as it gets to the end. Otherwise we’d have a lot of empty space because of the Start Offset.

So to recap, this random range of Start Offset is applied when the Events are called and *should* be different for each instance if you give a large range. To be safe you could add Modulation to the full range of the Start Offset. If you want you could also add a very tiny Modulation to the Pitch dial to help distinguish separate sounds, but don’t go overboard because pitch is affected by some 3D sound filters and movement. When a persistent sound moves towards you (or vice versa) the pitch increases, the opposite is true when moving away. This is the Doppler effect, and is simulated in most every 3D game these days. It’s setup by default in Unity’s audio system and as long as we setup these FMOD Events in 3D we will get the effect, which helps a player know if a sound is moving towards or away from them.

Almost done, we just have to assign our finished Event with all it’s audio and logic to a Bank. Banks allow us to tweak the properties of a collection of events all at once. We might, for example, want to drop the volume of all the persistent background sound effects while a character is talking – so we would put all the ‘background’ Events into a similarly-named Bank of their own. But for such a small project, just the one Master Bank is fine. Right click computer_hum in the Event tab, click Assign to Bank > Master Bank.

You can check out which Banks reference which Events in the Banks tab.

Level Setup in Unity

Now that we understand how Events work in FMOD Studio we can start creating our actual game in Unity, placing models and calling Events from the scripts that came with our FMOD integration package! Start by quickly blocking out a large office space. I usually start with cubes (right click Hierarchy > 3D Object > Cube) for walls because a cube is 1 by 1 Unity units which I think of as metres. So if you scale it by 4 on the Y axis then you have a 4 metre high wall and a starting reference for the overall size of your level.

For the floor I’ve used two planes, one for the wooden floor and one for concrete.

Another thing that helps be gauge the size of the level is dropping in a character controller and just walking around. We will need the Characters package from Unity Standard Assets. Go to Assets > Import Package > Characters. You may want to uncheck the audio files though because all our audio will be contained in the FMOD folder. If you want to use the default Unity footstep audio you could just drag them into your FMOD Audio Bin folder after unpacking it in Unity.

Drag the RigidBodyFPSController prefab into your scene. If you started with a default Unity scene you will get an error stating that there are 2 audio listeners, just delete the Main Camera that came with the scene and clear that error.

All we need to finish setting up the player controller is to add an FMOD Studio Listener component to the Camera. This is all we need to do, don’t remove the Unity Audio Listener component that was already on the Camera.

If you downloaded my project files zip you should drag the models and textures in now so we can finish setting up the graphics of the level.

Now let’s texture our ground planes. Did you know – If you drag a texture onto a mesh that only has a default material applied to it, Unity will make a new material in a “Material” folder named after that texture. Pretty cool.

Next let’s make one computer and desk setting. If you used the Google Poly models I did, they’ll probably be huge in your scene. The desk was not too bad so I just scaled it down in the Scene window but the Computer was giant!

Let’s shrink that Computer model. Click on the model object in your Project window and in the Inspector, under the Model tab, you’ll want to change the Scale Factor to something like 0.01 to have a more manageable size.

Another problem with the Computer model is the origin point, it’s waaaay off the centre of the geometries! These Google Poly models sometimes man… We could fix this in Blender but a quick fix is to select all the child objects of the model’s prefab and use the Move Tool in the Scene view to bring them left on the X axis and forward on the Z axis roughly centered on the origin.

Playing FMOD Events

With a complete desk setup we can add the script for our first FMOD Event. With the Computer parent object selected, click Add Component in the Inspector tab and search for Studio Event Emitter. This is the script that will call Events from FMOD.

For this object, we want the persistent audio to play when the object is enabled – so select that in the Play Event drop down. It also makes sense then to stop the event when the object is disabled. I like to pool objects in my scene by disabling them when they are ‘destroyed’ and re-enabling them whenever I want to spawn a new one. If you like to instantiate and destroy your game objects then use Object Start and Object Destroy to start and stop your events. To search your FMOD project for events, click the little magnifying glass next to the Event text box.

If you don’t see your Event in the search box, you may have forgotten to Build in FMOD Studio after you made it. It’s a good habit just to Build whenever you make changes to Events in FMOD Studio, right after you save – Remember that Build hotkey (F7 on Windows).

Now duplicate the desk setup a couple of times so that we can pick one to be the goal, I made the Computer game object a child of the Desk to make it easier.

Before the next step press play in Unity and have a walk around. Notice how the offset makes identical looping audio tracks sound distinct and seperate?

Now lets setup some feedback for the player when they step in front of the correct computer. I chose a computer startup chime so it’s like the computer wakes up when the player gets near. Let’s go back to FMOD Studio and create a new event. We don’t need any extra logic, just assign it to the Master Bank, save and Build.

Next, let’s attach a Sphere Collider to one of the Computers and make it a trigger. Now add a second Studio Event Emitter for the computer_startup Event and set the Play Event to Trigger Enter and the Collision Tag to Player. Don’t forget to set the tag on the actual Player controller.

Well that’s the basics and the end of Part 1. In Part 2 we will be using custom scripts to change the footsteps sounds when the player is on different surfaces, add occlusion effects and Google Resonance zones for truly immersive spatial sound.

Now that you’re familiar with playing Events why don’t you try adding a couple of extra persistence and ambiance sound effects. Maybe a radio playing on one side of the office and collision sounds when the player walks into the desk.