Play Sound on Hover – Web Audio API Edition

I got an email from a Notary Sojac who saw my tutorial on playing a sound on hover. In that tutorial we had to resort to some fairly hackish behavior to get the sound to react nicely to our events. Notary Sojac had an update to make the existing code work better, but more importantly, was looking into the Web Audio API for doing stuff like this. I didn't even know that was a thing. Turns out it's not controlling HTML5 audio elements, it's deeper level access than that.

The following is a guest post by Notary Sojac explaining all that.

We're all impressed with the capabilities of HTML5 canvas. The idea of creating a standardized and logical way of performing 2d drawing operations opened the door to an amazing new world where one programmer could learn one language and develop amazing products that could be deployed to all relevant platforms! Think about how critical this is for a moment. Now think about who would want an amazing graphics application that has a buggy, glitchy sound system that only makes sounds moments after you clicked and sometimes forgets to make sounds all together? That's the kind of product you get (especially in mobile platforms) if you go the HTML5 <audio> element route. It's better in desktop browsers, but unfortunately will never be perfected due to reasons beyond the scope of this article. But as of now, for the desktop, there is still hope. And there is hope in the future for mobile applications as well. That hope is named...

Web Audio API

The web audio API is designed for high precision and low level access. You can literally write bits of data to the samples. I'm not 100% sure what a sample is, but I think it has to do with air pressure against a microphone (aka, the microphone's driver position) at a single moment in time. That's pretty low level if you ask me. In Google Chrome, all audio functions are handled by a separate thread to ensure non-wonkyness.

OMG you are right! We will be able to play any synthesized sounds, like organs, bass guitar, whatever, right from the keyboard. The possibilities are endless with MIDI. and the keyboard has so many keys it will be pretty cool. Someone will run with this and make a name for themselves.

I`m afraid that`s really not the case. Controlling audio in real time is not possible without a significant delay. The flash audio api is around for a couple of years now with full MIDI support with the same issue. That`s why we still need low-latency(ASIO) sound cards.

Happened to be browsing with Safari, clicked the demo button and got the fall back message (“Web Audio API is not supported on this browser. HTML 5 Audio Elements will be used instead.”), only there is no audio. Not sure why. In Firefox, which also uses the HTML 5 fall back, the audio is there.

Only the very latest Safari browser supports the Web Audio API. It seems as though Apple is withholding that software from users who haven’t bought the latest version of Apple. This makes it a lot harder for linux and windows developers to work on solutions for that platform, and more difficult for Apple users who don’t have time to keep track of where to buy the latest update from and whether or not their system is eligible. The good news is that the Web Audio API will be allowed on recent mobile platforms, so kudos to Apple there.

It seems mysterious (but true) that apple is only supporting the .mp3 format with their web browser. The decision only makes sense to me if they’re receiving payments FROM the mp3 patent holders rather than paying them out. FF will not behave in this manor when it’s ready (it’s still in the progress of adopting the technology). And obviously Chrome’s browser is designed to work with open standards like .ogg and webm.

I managed to get out to an apple store and look things over (my libraries only have PCs) and as I suspected, changing the code to load an mp3 instead of an ogg resulted in a success. It seems like a petty task for Apple to have me do so I’ll gripe about it with them for a while and then update the framework to support fallback formats in a clever and logical manor. Right now I’m switching my OS to Linux Mint so I have a lot of development environment stuff to set up and work needs me to do bacula maintenance.

A general reason why no audio elements should be used on websites at all: They are just and simply annoying! The very same is true for smart phones that sound on touch or cameras that do sound as preset. Ok, that’s a personal opinion, but if you use public transportation you know what is behind my statement…

It sounds like there are a lot of technical issues with getting sounds to play just when you want them to. Considering that, my suggestion may not be technically possible, but why doesn’t CSS cover sounds that should play on hover? CSS handles the presentation of the markup, but pretty much only visually. Why doesn’t CSS handle that kind of audio presentation? If you want a button to change colors on hover, you use CSS. Why is it that when you want a button to beep on hover, that’s not CSS? For that matter, why doesn’t CSS handle a sound you want to play on :target? What about at the very end of a CSS animation like a z-index animation?

I’ve seen sounds played on websites for many years now, (such as when hovering an element). I’m surprised by what few website owners actually do this since its so cool and unique. (goes beyond the “look” of the website and easy to do).

Interesting ideas; and as you point out, the number of innovations coming into HTML5 / Canvas is amazing – but I really hope that this doesn’t lead to a proliferation of noisy websites, as one of my number one pet hates when browsing sites is ‘unsolicited noise’.

Any suggestions on how to make the sounds to stop playing when the mouse stops hovering over the element? The soundbites I’m using are 5 seconds long and it gets pretty crazy when the user accidentally hovers over multiple elements, lol.

Hey Mel, glad to see you’re thinking outside of the box on this new technology. When I initially laid out the framework, I had one thing in mind, play many sounds at once and let them overlap as needed (echoing peacefully off into the background before finally reaching /dev/null, their origin). This entailed rapidly spinning off new sounds without saving a reference to them in an array or anything, so in the main branch’s current form it wouldn’t have been so easy to program a stop() method.

Luckily the code is still fresh in my head, so I started a new branch on github where I’m testing the code out that allows for playing sounds single file, one at a time without any overlapping of the same sound.

It’s getting really late at night, but checkout the “stop-sound-effects” branch. The code and demo there should allow you to do .stopSound() just fine, and it’s rigged not allow sounds to overlap at all thanks to a new syntax being used under the hood.