Seeing Sounds

My mother, who lost her hearing aged thirty-five, once told me she missed the sound of birds singing and the rain on the roof. This was my inspiration for my year-long conceptual research project that sought to use design to visualise the experience of hearing so that the hard of hearing could reconnect with lost sounds.

Seeing Sounds City Mural AR Animation

Seeing Sounds is a series of interactive augmented reality (AR) murals that visualise soundscapes around Wellington city. Waves crashing at the beach, birdsong in the forest, or the chatter of conversation — translated into visuals so that those who cannot hear can once again ‘listen’… with their eyes.

The ‘City’ mural shown above depicts the conversation and footsteps that walk past as you sit in the heart of Wellington’s CBD.You can see the AR animation here.

Seeing Sounds Forest Mural AR Animation

The forest mural shows the twitter of birdsong and the rustling of leaves of the botanical gardens.

To see the murals, simply hold up your mobile phone with the Seeing Sounds app open and watch the sounds come to life!

In my designs, used synaesthesia as a framework to design sound visualisations from a human-centred point of view that is perception-led rather than scientific. I also studied Wong’s visual language theories. By building on the simplest elements of design I could combat subjectivity.

Subjectivity was a big challenge. The way I might visualise sounds could be completely different to how you would. So I designed a co-design experiment, based on the research methodology created by designer Orlagh O’Brien.

This experiment was made to see if there were similarities in the way people visualise sounds, or if they were all different. First I asked participants to tell me the emotions and memories they associated with the five different sounds, which I later used to inform my designs.

Despite all linking different feelings and memories to sounds, people intuitively picked similar or the same shapes to show them. This proved that emotional sound visualisation could be done despite the subjective nature.

I then took the most common answers and used them to inform my designs so that the visualisations were not a reflection of my own subjective experiences but reflected many different real humans experiences.