SoundMesh

SoundMesh is an audio application that implements distributed surround sound using Web Audio.

The idea is that if you have a lot of people in a given location, each will select their approximate location, and then we can use every one of their phones as speakers. Using this, we can create effects such as making a car race around the room, or having each phone play a different instrument, all in sync. We have also used the Leap Motion to control panning sound not only left and right, but in 2D around the room.

We used web sockets and a time sync protocol to ensure that we have the same reference time on all devices even on networks with non-ideal latency, so that they can play different audio segments at precise intervals.

Each "performance" is encoded as a JSON structure which is then multi-unicasted to the correct individual devices, giving them instructions on how and when to play audio segments. The possibilities are endless!