The evolution of audio in videogames

Musician and gamer Daniel Scarratt traces how sound in videogames has evolved through the decades.

Sound design and music play an important role in the overall experience of modern videogames. Game audio can help designers to create tension, add emotion, build immersion in the game world and even solve design problems. However, many of the audio techniques we benefit from today are only possible because of the ingenuity of game composers and sound designers of the past.

The earliest attempts to create games on computers happened in the 1950s when people seemed really into jukeboxes. The games built during this period were made by scientists who tested them on extremely huge computers that took up almost the whole room. People were lining up to go and test... Tic Tac Toe? Wait, that doesn’t have sound, as well as being really boring by today’s standards.

Twenty years on in the 1970s, people are finally making commercial games for people to enjoy. Games are still very simple by our standards but they've started to employ sound to enhance the experience for players, like in the early arcade classic Pong (1972).

Sounds like it's just a bunch of bleeps and bloops right? Well, there's actually a little more to it.

With sound capabilities far worse than a modern washing machine, the creators of Pong varied the pitch and duration of the beep to differentiate between the ball bouncing off a wall, being hit by a player’s paddle or when a point was scored. With just three simple bleeps and bloops, the actions on screen were differentiated from each other, adding more tangibility to the events of the game.

Sound design for games seems pretty cool now, right? Well by the 80s, videogames were as popular and complicated as a nice mullet and leg warmer combo. People weren't just playing games in arcades anymore, the decade saw the rise of home computers that play games on floppy discs as well as consoles like the Nintendo Entertainment System (NES).

There was one challenge bigger than Donkey Kong for game composers of the 80s – memory. Did you know that the average total size of an NES game was 384kb? Last year I got a 4GB thumb drive for FREE from my dentist. That thumb drive could have stored over ten thousand copies of an average NES game, but all I got was some program called “Teeth.exe” that I’m too scared to open. You also need to remember that the 384kb of these game cartridges had to store everything needed for the whole game, which meant that the memory dedicated to music and sounds was only a fraction of that.

To deal with the memory constraints of games at the time, 80s game composers created music using basic, computer-y sounding waveforms. They also couldn’t have too many sounds playing at one time, so their music generally has only four or so different "tracks" for instruments or sound effects to play on. However, it’s these very limitations that will create the quintessential "videogame music" sound – let’s take a listen.

Check out this excerpt from the 2017 survival/horror game Resident Evil 7: Biohazard

It’s dark, scary, atmospheric – it sounds like the score to a scary film. Why? Because by 2017, there’s nothing stopping a game composer from hiring an orchestra and using the same techniques that film composers use to scare us in cinemas.

So what if a composer wanted to make scary music for a game built for the NES? Due to the device's limitations, you would never be able to match an orchestra's dynamic range and variety of sounds and textures.

Game composers had to get clever. They had to create songs that gesture in the direction of other scary music that we’ve heard by using one of the greatest compositional tools available to them – melody.

Listen to a bit of this excerpt from Toccata & Fugue in D minor by composer Johann Sebastian Bach.

A pretty sinister piece of music that has long been associated with the horror genre due to its extensive use in films.

Now check out the intro to this piece from Castlevania II: Simon’s Quest for the NES (1987)

Can you hear the melodic similarities?

The composers of Castlevania II use a melody that invokes our association with the horror genre and blend it with the rock music of their present-day – resulting in a spooky tune that also makes me want to pump my fist in the air every time.

It’s no coincidence that so many themes from 80s game franchises have been reimagined time and time again – the strong melodies that were a necessity of the time are extremely memorable, and have become as iconic as Mario’s overalls. Almost anybody can hum the theme from Super Mario Bros – will you remember that excerpt from Resident Evil 7 in even a weeks time?

By the 90s games were becoming 3D, and thanks to advances in technology, music in games could be made with real instruments. Some composers continued to make the melodic music that we've come to expect from games, but others were starting to get creative with the possibilities. This would eventually lead to the use of "real world music" in the form of in-game radio stations or as a backdrop to the action, as well as the use of live instruments and orchestral music in games.

With the technological limitations of the past lifted, game composers and sound designers could now take the ingenuity which they had used for so long to emulate other forms of music and turn their attention to solving bigger design problems.

One notable example is Foley – the creation of sound effects for media. Games of the past had to settle for electronic noises (like Pong) or generic, multipurpose sampling. Sound designers can now record real-world audio and create libraries of sounds to be used in their game. This technique was used in the game Never Alone (2014). The sound designers travelled to snowy environments to record audio that would then be used in the game. There is a whole blog post about the techniques that they used here.

Another notable staple of modern game audio is the use of dynamic music. Games have become complex and no longer need to be played linearly or have entirely scripted events, meaning that the best game music would need to react to the events in the game, whatever they may be. Perhaps the most well documented use of this process can be found in the game Red Dead Redemption (2010). The music in this game was made so that sections of music, or "stems", could come in and out of the score based on what was happening on screen.

As we move further into the future, the role of sounds in games will continue to play a huge part in shaping our experiences. So what can we expect? With virtual reality games growing rapidly, I'm excited to see how composers and sound designers tackle the challenges of 3D audio.

ACMI acknowledges the Traditional Owners, the Wurundjeri people of the Kulin Nation, on whose land we meet, share and work. We pay our respects to Elders past and present, and extend our respect to Aboriginal and Torres Strait Islander people from all nations of this land.