Exploring HTML 5's Audio/Video Multimedia Support : Page 2

The <audio> and <video> tags were among the first features to be added to the HTML 5 specification. Find out how these elements enable the browser to work with both types of media in an easy-to-use manner.

by Kurt Cagle

Nov 16, 2009

Page 2 of 2

Audio, Video, and the DOM

Both the <video> and <audio> elements use the same DOM interface, based on the abstract HTMLMediaElement (i.e., there is no formal <media> element). You can then use this interface to control the various video and audio streams within the page. Listing 1 shows the IDL for this interface.

From the IDL, the API for the media elements breaks cleanly into the following tasks:

Controlling the network retrieval of resources

Controlling the buffering

Controlling the playback

Setting the attributes of the various controls.

The src property is for setting the @src attribute on the media, but changing src by itself doesn't automatically change the current video. Instead, after you change src, you then need to call the load() function to load the element with the new media, and then invoke play() when the video has loaded and finished buffering. In a purely synchronous world, this particular task would be relatively easy. If you're pulling media off a local file with a locally running web page, this code will in fact work:

However, media by its very nature is a time-spanning process. You're working with both the need to load and cache audio and video files and the network connection, which itself can prove to be a challenge that's usually not a factor for most non-temporal resources. For these reasons, when dealing with video and audio on the web, if you want to take control of the process in any way, you have to work asynchronously, tying into the various events that are passed back to the client from the media-serving website. Table 4 (taken directly from the HTML 5 documentation) provides a full listing of the various events and when they are dispatched.

The user agent can resume playback of the media data, but estimates that if playback were to be started now, the media resource could not be rendered at the current playback rate up to its end without having to stop for further buffering of content.

Either the volume attribute or the muted attribute has changed. Fired after the relevant attribute's setter has returned.

Of all the events in Table 4, perhaps the most useful are the canplay and canplaythrough events. The first, canplay, will fire when enough data has been loaded for the video player to start actually rendering content constructively, even if not all of the data has been loaded. The canplaythrough event, on the other hand, will fire when the data has essentially completely loaded into the browser's buffer, making it possible to play the video all the way through without the need to pause and retrieve more content.

Listing 2 provides an example of this: a person enters the URL to an OGV movie then presses the "Load" button.

When the video reaches a point where it has enough to play, it intercepts the oncanplay event, which enables the Play button. In turn, when the user presses the Play button, it will actually play the video just downloaded.

At least that's the theory. In practice, the <video> and <audio> tags are not fully implemented even in Firefox. As such, many of the events are not being passed back up to the JavaScript level yet. This means that, as crude as it is, calling load() and then play() still seems to be the best way of playing externally loaded video resources. This area is very likely to change as the implementations (and the XHTML standards) become more fully fleshed out.

Browser Vendors Very Interested

By all indications, the browser vendors view the multimedia aspects as perhaps the most crucial in the developing HTML 5 standard. Given the complexity of both types of media and the prospect of being able to better promote video and audio usage within web browsers, it's hardly surprising. However, before HTML 5 multimedia becomes ubiquitous, the state of the art for these browser implementations still has a ways to go technically.