The Layers of Sound

Cinema may have begun its life in a soundless world with no dialogue, but the idea of having recorded sound accompany the moving pictures was there from the very start. In fact, there was a showing of such a film as far back as 1900, but practical limitations would keep us in the silent era for a couple more decades. No problem. Filmmakers innovated quickly, adding various sound accompaniments like a live orchestra to their otherwise silent films. Music was always seen as an essential part of films from the start, but the same can’t be said for sound.

The Jazz Singer was released in 1927. Presented as the first feature-length talkie, crowds flocked to the theater to get a taste of this new exciting experience. The introduction of talkies would quickly put the silent era to an end, but many filmmakers were hesitant to adapt to this new method. Many people saw talkies as a passing fad that would soon fade away. Some even saw it as a dismantling of everything cinema stood for. They thought the inclusion of recorded dialogue in a film would take away from the specific and unique qualities that cinema offered. If films were ‘talkies’, how were they any different from the radio at that point? None of these reservations or concerns could stop the steam train that was the talkie though. Never had so much money been rolling into theaters around the world and talkies soon became an international phenomenon that became the ultimate death of silent films.

General audiences may have been thrilled by this exciting new trend in films, but for the filmmakers, to just put it bluntly, sound was a pain in the ass. Back then, cameras were really noisy. The very process of capturing an image was ruining any recording of dialogue that was being taken. It’s hard to capture sound when you have today's equivalent of a rickety air conditioner constantly blowing right next to your ear. The solution was to take the camera and put it in a box. The problem with that is you can’t easily move a camera that’s sitting in a giant soundproof box. Solution to that problem? Shoot with more than one camera. Sound stages were built to have more control over the on-set environment for recording. It goes on and on from there for years. To sum it up, sound introduced more problems in production than any innovation before it. Filmmaking is the art of problem-solving, perhaps more-so here than ever before or again.

The evolution of sound has a long and varied history, but like everything else, it just got better over time. Today we see recorded sound as an essential part of filmmaking and sound design is a key part of the entire process. The Jazz Singer may have come out in 1927, but the actual title of ‘sound designer’ wasn’t used regularly until 1979. The first person known as a sound designer was the legendary Walter Murch, a title given to him by Francis Ford Coppola for his work on Apocalypse Now, which was also one of the first films to be released in the modern surround sound format of 5.1 (even though most theaters weren’t equipped for it).

Since then, computers have brought us into another age completely and the tools used for sound design have increased exponentially. Today, a film’s soundtrack (not the music) consists of manufactured layers that are built on top of one another to create a fully fleshed out world. If you didn’t already know, almost all of what you’re hearing in a scene is a carefully manufactured illusion. Perhaps you see a waiter come up to a table and drop off a few plates and napkins. Chances are that what you’re hearing was recorded months after that scene was actually shot. In fact, sound in films is a little weird if you start paying close attention to it. It doesn’t really operate in reality at all but in the illusion of reality. Of course, the biggest compliment to a good designer is “I didn’t notice the sound at all”. If everything sounded perfectly natural, that’s a job well done. I’m going to run through the individual layers that make up the final soundtrack of a film.

Backgrounds/Room Tones/Ambience

Laying down a background track is often the first step to tackling a scene in the post-production process. Are we in a crowded kitchen? An empty apartment? A cement basement? All of these environments sound completely different from each other. Without background noise, things just don't sound right. Think about it. Have you ever been in an environment that was absolutely silent? The answer is no unless you’ve been in space with your helmet off. Even in the quietest places on earth, the smallest sound is still some sound. Background sounds are the most monotonous layer, droning on in the back without much variation, but they’re critical to creating a convincing world. If possible, these backgrounds should always be recorded at the filming location for the best result.

The only time you ever get rid of a background track is for emphasis or if you’re going for a realistic portrayal of space. Even with famous moments of silence like in Raging Bull, there’s often still a background track providing a minimal amount of noise. No doubt you’ve seen a film that does briefly drain of sound entirely though. An example of the complete removal of backgrounds can be found in Under The Skin, although these scenes are NSFW. A perfect (SFW) example of this can be found in Star Wars: Episode 2 - Attack of the Clones.

In the clip linked above, you can hear the background noise fairly clearly if you ignore the dialogue and sound effects. It sounds like the wind passing by a ship. Kind of weird to have in space, but it works, doesn’t it? At about the 9 second mark right before the bomb detonates, all sound is briefly pulled out of the mix including the backgrounds. This is to create emphasis for the following explosion. Dynamic range is a huge part of the final film and can basically be summed up by “there should be quiet parts and loud parts”. We apologize that sometimes you feel like you have to turn up the volume for dialogue and turn it down for explosions, but trust me, you don’t want everything to be at the same level. By having something loud follow something quiet, it will have that much more impact. A micro version of that is the scene linked above.

Foley

Foley is possibly the strangest part of sound. It would no doubt draw strange glances if people happened to walk by a Foley artist at work. The job of a Foley artist is to re-create and record sounds for the film after it's been shot. Remember that scene of the waiter dropping off a couple plates? Yeah well, somewhere in a studio, a Foley artist would be playing around with plates next to a microphone. Is a character wearing a leather trenchcoat? Go grab some leather and start rubbing it near the microphone. Maybe she’s wearing boots and walking through a snowy field? Yup, grab a pair of boots and get to work in some cat litter. And that’s the normal part of it.

It gets creative fast and Foley artists are always cheating in strange ways. Really bloody scenes have found their sound through the use of goo and a toilet plunger. Hay in a barn can be shredded paper from the office upstairs. A crackling fire can be beads being rubbed together. Someone falling can be a sack of sand being dropped. It goes on and on, I’d just encourage you to watch the video linked below from The Lord of the Rings to see it in action. Foley artists work in real time with the film while it’s being played on a screen in front of them.

Note that the video is absent of all the other layers of sound so you can really focus on what Foley is supplying to the mix. You can also revel in the glory that is temporary ADR which I will talk about next.

Automated Dialogue Replacement (ADR)

Also called looping, there’s really nothing automated about it. If dialogue isn't captured on set, either because it was never recorded or just isn’t salvageable for whatever reason, it must be recorded after the fact. The actor comes into a studio and the scene they need dialogue for is played on a screen in front of them. They then attempt to say the needed lines perfectly in time with the movement of their lips on screen. Here’s where the looping term comes into play. This line will play over and over again until the actor gets a good take as it’s difficult to get it on a first or even second try. Sometimes it can take dozens. So in practice, you’d be watching someone saying the same thing over and over again repeatedly as he watches a video of himself saying the same thing.

It sounds really simple on the surface, but actors don’t get nearly enough credit for this part of the job and it’s hard to understand just how difficult this can be until you try it for yourself. It’s tough to sound convincingly scared or give a blood-curdling scream while you’re standing in a sound booth. Some actors are very talented at doing this successfully, and others…are not. I’m sure you’ve heard plenty of lines that just don’t quite sound right for whatever reason. This can either be a performance issue or a mixing issue, which I won’t get into right now, but I’ll link a video down below from The Great Gatsby which displays a wide range of problems including some pretty bad ADR. Of course, there are no hard and fast rules in making a film and everything can be used creatively. David Lynch is one filmmaker who has used poor ADR in a creative way to enhance a thematic idea. I’d cite Mulholland Drive as a great film that uses bad ADR to achieve a creative purpose. Along with The Great Gatsby clip, I'll link a short video of Hugh Jackman performing some ADR so you can see it in action.

Sound Effects (SFX)

SFX is perhaps the hardest layer of sound to talk about in a concise way as it’s something that’s dealt with differently across projects. The overlap between SFX and foley can be pretty large, but I’ll talk about this layer a little differently than how I talked about Foley.

The first thing that must be mentioned is sound libraries. These are giant collections of pre-recorded sounds which can range from literally anything to everything. The dream goal for a big production is to record everything you need for the film yourself, but this just isn’t always practical or possible and sound designers will be using sounds from libraries more often than they care to admit. It’s hard to even explain how massive these libraries can be with folders within folders of sound effects. They are incredibly extensive, often containing hundreds of sound effect files for one thing.

I’ll run through one example. Let’s say you need a gunshot. You got some recordings from a live shooting range, but you’re just not happy with the sound you recorded from a pistol. So you go to the guns folder and disregard all the folders for the other types of guns (shotguns, sniper rifles, automatic weapons, revolvers, etc). So you go into pistols. Time to narrow down the search again. It’s not a silenced pistol, it’s not an automatic pistol, it’s not a flintlock pistol, and it doesn’t have multiple barrels. OK, so we go to the right folder and now we can choose between the 278 sound effects of regular pistols shooting different types of ammunition in different environments. Here’s a screenshot for a basic sound library you can purchase.

A lot of the time though, professionals are focusing on recording their own sounds for the film, and they usually have their own libraries of sounds that they’ve recorded over time. Getting these sounds can range from the basic foley stuff to conceptually difficult tasks that have no basis in our world. If someone came up to you tomorrow and said “I need a velociraptor noise”, how would you go about getting it? You’re certainly not going to go down to the velociraptor petting zoo. If you’re Gary Rydstrom, who worked on Jurassic Park, maybe you’ll use the sound of tortoises having sex. The Rancor from Return of the Jedi was a slowed down chihuahua. The main element in a T-Rex roar is a baby elephant. A Star Wars blaster is a metal cable being struck with a hammer. So this brings out the conceptual side of sound design, meaning you must create things that don’t exist from things that do. It’s also the most rewarding, exciting, and frustrating part of the entire process. One out of place sound effect can ruin everything in a moment.

There are endless techniques to how people go about getting unique sounds, but I’ll mention a few. Record a sound and slow it way down, speed it way up. Weird things can happen and it quickly stops sounding like something from our world. Microphones can record sounds that we can’t hear. What if there was a high pitch sound that you couldn’t hear, but you recorded it and brought the pitch down? Take Godzilla (2014) for instance. A plastic knife on a tire produced sound outside of the human hearing range, but when pitched and slowed down it produced some great cavernous monster noises.

The Final Mix

The last stage of this is to take all these layers and effectively combine them together into a cohesive whole. The four main points of consideration during the final mix are speech (dialogue, ADR), backgrounds, SFX, and music. Every sound must find a proper home in some combination of six channels(speakers). There’s the center channel which is positioned behind the theater screen, the front left and right, the rear left and right, and the LFE (Low-Frequency Effect). Generally speaking, most dialogue and a lot of sound effects are going to be mixed solely into the center channel. The front left and right channels are going to add that stereo presence and the rear left and right channels complete the “surround” aspect of surround sound. Together, these 4 channels are going to allow the world to feel alive. They create space that works in tandem with what you’re seeing on screen. Humans can hear sounds all around them, so the surround aspect adds that important layer of immersion. The LFE is responsible for vibrating your seat and skull with a powerful low-end presence, aka Michael Bay’s favorite channel. It’s also the channel that should be used the least, and a lot of the time nothing at all will be going through it. Unfortunately, a lot of films tend to overuse the LFE which robs it of its potential impact.

Mixing is a hard process, but a lot of it is rather self-explanatory and even more of it is filled with technicalities and jargon that I won’t lay on anyone. The process is all about finding balance. Turning things up and down and taking things in and out until it starts to sound like it makes sense. It’s easy to find yourself in the final mix and realize there is either way too much going on and you are just being drowned in sound or there is just not enough going on and it sounds flat and lifeless. Either way, you need to edge it in the right direction and start making those final decisions. One of the most common questions is ‘how much will music dominate this scene?’ As disheartening as it is to wash away hard work, sometimes music just needs that extra blast until you can barely hear anything else. It’s about finding what works for the story. Everything comes together here and that Oscar for Best Sound Mixing (won by some guys you’ve never heard of for that one war film) is won here as well.

The final scene I will link down below is a scene from Gravity, best experienced in full surround sound, but stereo will also do. It’s a scene that I think embodies the entire process really well and you can hone in on the specific layers at play, or even the lack of specific layers. Love or hate the film, this is some top-tier sound design that is both creatively and realistically motivated that creates tension without bending to cheap tactics. Listen to the space of the sound and how it’s balanced. Listen to how the scene escalates throughout that entire space from left, to right, and behind you.

Radio chatter is used ingeniously as a background layer to substitute in for the void of space. Chaos is created through amount rather than just loudness. There’s no explosions or other loud sounds to lean on as a crutch for tension so they increase the amount of sound happening, having it attack you from left, right, and behind to create disorientation and tension. Dialogue dominates the mix at the start of the clip, but music begins to take up more and more space as the conflict escalates. Don’t forget to pay attention to the ADR performances. There’s a lot going on here and it’s a great example of how sound work can be done with limitations. And it’s got some excellent mixing.

This is a lot of generalized information on a topic that is vast and seems never ending. Delving into specific details on even one of these layers can take a long time and it's an interesting field because of how close the marriage between being technical and being creative is. Also remember that each production can deal with sound in different ways and that the only rule in filmmaking is that there are no rules. Everything said in this post has countless examples of films that have done it differently. Thanks for reading!