MediaSource Extensions v0.2

Draft Proposal

Status of this Document

This document is a draft specification proposal with no official status. Send comments to Aaron Colwell. It is inappropriate to cite this document except as a work in progress.

Abstract

This proposal extends HTMLMediaElement to allow
JavaScript to generate media streams for playback.
Allowing JavaScript to generate streams facilitates a variety of use
cases like adaptive streaming and time shifting live streams.

Table of Contents

1. Introduction

Historically <audio> and <video> have only allowed media playback from container file formats. Support for use cases like live have required
hacks to the container format, like 0 size segments in WebM, or required the browser to support manifest formats like HLS. Supporting new adaptive streaming proposals like 3GP-DASH also require significant browser changes and the adaptation algorithms cannot be changed once the browser is released. The file container model also puts constraints on on-demand playback. Efficient seek support requires an index to be transfered to the browser which results in either delayed startup, or delays during the first seek. All of these issues stem from the fact that the browser is expected to manage all media fetching and do so based on the information contained in a single URL.

This proposal introduces a new object called MediaSource that allows JavaScript to control how media data is fetched. This will enable significant flexibility for media fetching without requiring browser changes. Manifest formats like HLS & 3GP-DASH could be handled in JavaScript. Indexes for on-demand playback would not be needed because JavaScript could make an AJAX call to a web service to find an appropriate seek point. Adaptive streaming algorithms could be implemented and evolved over time without needing to upgrade the browser. Fetching media from multiple CDNs or splicing in media from the local file store could also be implemented without browser changes. Allowing JavaScript to control media stream construction opens may avenues for experimentation and innovation.

2. Media Element Extensions

We extend HTML media elements to allow media data to be streamed into them from JavaScript.

End of stream status values:

EOS_NO_ERROR (numeric value 0)

The stream ended normally without any errors.

EOS_NETWORK_ERR (numeric value 1)

The stream ended prematurely because of a network error. If the JavaScript code fetching media data encounters a network error it should use this status code to terminate playback. This will cause the media element's error handling code to run and the error attribute to be set to MediaError.MEDIA_ERR_NETWORK

EOS_DECODE_ERR (numeric value 2)

The stream ended prematurely because there was an error while decoding the media data. If the JavaScript code fetching media data has problems parsing the data it should use this status code to terminate playback. This will cause the media element's error handling code to run and the error attribute to be set to MediaError.MEDIA_ERR_DECODE

The webkitSourceEndOfStream(status) method must run the following steps:

This seeking algorithm assumes that the data passed to webkitSourceAppend() is packetized media with timestamps (ie Ogg pages or WebM clusters). This allows the "new playback position" to be determined by the timestamps in the data. If we want to support formats that are not packetized, like WAV files, then we will need a different mechanism to communicate the "new playback position" when a seek occurs.