We can’t change history, but we can change the future.
Be nice to each other.
@robertnyman

The video element in HTML5 – great possibilities, but also codec and licensing problems

Published on Monday, January 25, 2010

Man has always been inspired by things moving around and giving away noises, so it was just a matter of time before video content showed up on the web. For a number of years, Flash was the de-facto standard of showing video, but now, with HTML5, the video element has made its way into our lives.

The video element

The video element is as simple as can be:

<video src="swedish-flag.ogv"></video>

Then you can add a number of attributes for if controls should be visible, if it should autoplay when the visitor loads the containing web page etc. The applicable attributes are:

autobuffer

Loads the entire file if this attribute is enabled, but is ignored if autoplay is set.

autoplay

Video starts playing as soon as it is ready.

controls

If video controls should be displayed – looks dependent on web browser.

What is also nice is that web browsers won’t automatically download the entire video file, but just a part of it to be able to display the first frame. Also, you could write your custom controls and control the video via script – this is more described in Using audio and video in Firefox.

Different treats for different web browsers

This all sounds good and well; just specify the source of your video file and you are good to go! Well, it could have been like that, but unfortunately isn’t. Just as before in history, it comes down to video codecs – yes, exactly that hassle we got around with having video in Flash.

Firefox supports the Ogg/Theora encoding.

Google Chrome supports the H.264 and theOgg/Theora encoding.

Safari supports the H.264 encoding.

Opera will support the Ogg/Theora encoding (no final release with video element support yet).

So, what the video element offers us is to specify various video sources for different web browsers, like this:

And, instead of the “sorry”-phrase above, we could also go to the length of offering QuickTime and Flash fallbacks instead for web browsers not supporting the video element at all.

Codecs and licenses

I think it’s fair enough that we can offer content to please every web browser, but at this point it becomes cumbersome. Maintenance, it’s a nightmare, since we need to maintain and store at least three formats for each video to be able to offer it to a vast majority of our visitors. The reasoning behind codec support is that Apple has invested a lot of time and money into H.264 so they want to push that, Google Chrome has bought licenses for H.264 (which means the only version of Google Chrome that supports it is the one downloaded from the Google web site – Chromium and others don’t support that license) and support Ogg/Theora as well, while others like Mozilla (with Firefox) and Opera believe that the open standard element video should naturally support open codecs and therefore only support Ogg/Theora.

What we don’t want with a new standard/element and de-facto way of doing is to go into the situation of GIF again, but instead avoid closed formats that requires a number of expensive and limited licenses. I believe Robert O’Callahan explains it beautifully in Video, Freedom And Mozilla just why H.264 is not a viable option, and why it is not on par with the open web and web standards.

And, to make matters worse, in my eyes, Google just started offering the HTML5 video element on YouTube and Vimeo announced HTML5 video element support as well. Great news, but the big big problem with this, though, especially being two of the most visited video web sites on the web, is that they only support the H.264 encoding. This means that only Google Chrome and Safari users (and, oh, those three IE users in the world with Google Chrome Frame) can see it, excluding a vast portion of the web browser market with Firefox and Opera (the initial complaints were also that fullscreen video wasn’t there – well, Firefox 3.6 supports fullscreen video, so if they would have chosen another format that would have been there too).

What is bothering me with the H.264 approach, and is quite saddening, is that Google Chrome supports the Ogg/Theora codec as well, so they could have chosen the open route instead. Their move of introducing the video element on YouTube is a very important decision, and I believe it’s great to move away from Flash and into HTML5 elements. But as long as they do it with closed-in licensed codecs, it has been all in vain, if you ask me.

Conclusively…

The whole reason video on the web took off in the first place with YouTube and others is because it just worked as long as you had Flash installed, no matter the web browser and no matter the platform. If just end up in another codec war again, the video element will never take off – don’t waste our time with web browser-specific solutions and proprietary attitudes. If we ever truly want to use the video element on the web, it has to be open. Period.

17 Comments

On one hand I really hope Firefox and Opera can stick it out and make Theora the standard on the web. On the other hand H.264 is the better codec and the license/patent issues only affect the US.

What is bothering me with the H.264 approach, and is quite saddening, is that Google Chrome supports the Ogg/Theora codec as well, so they could have chosen the open route instead.

Obviously they picked H.264 because that's what they were already using (no rencoding and extra storage required). If the competition have to pay licence fees (five million USD I believe) before they can start a similar service that's in their interest too.

There seems to be 3 issues with Ogg Theora that prevents Google and Apple from embracing it.

1) No hardware acceleration, so no go for mobile devices.

2) Poor efficiency. Google has stated that they can't support Ogg Theora on YouTube because it would cause their bandwidth to skyrocket. Some have disputed this, but I think the YouTube people should know thing or two about encoding video.

3) Fear of patent infringement law suits. The Ogg Theora camp claims that it's clean as a whistle, but I'm sure Apple has a better legal team than Xiph and if anyone is going to get sued it's Apple if they roll out Ogg on the iPhone and iPod.

Prediction: HTML5 video will get adopted quickly with H.264 since site owners will want to publish video in a format that works on the iPhone (and Pre, Android). Some sites will offer Ogg too, but most won't care. Firefox, Opera and IE will get the Flash fallback option. Apple and Google will be happy because this will give Safari and Chrome another feature advantage.

The tag is in a similar situation. Firefox and Opera don't support MP3 and AAC, only Ogg and WAV (PCM).

Sidenote: This comment preview thing is slow as molasses, I had to type this in TextMate and paste it in. Using Safari 4.0.4.

Regarding patents, there's more on that topic in H.264 patents outside the US; basically, there are a lot of patents and things that might apply elsewhere too.

Competition-wise, and in conjunction with what YouTube to adapt their videos for the iPhone, I do understand the current reasoning, but I don't believe it's a good one on the long run for the web.

Jasper,

As for now, in our own web sites, sure that could be an option. Personally, though, I'm overall not for exclusion.

Henrik,

1) True, but I guess it would be in there soon if another format was chosen.

2) Well, while I'm sure Google knows their performance and encoding, I think to bear in mind that they are also very partial in this question, because of what they have already invested in H.264 encoding to accommodate the iPhone.

3. Absolutely, it's a good reason for fear, and the toughest one to cater to.

And maybe you are right in your prediction, although, if true, I see it as a very sad way of the web to move "forward" (rather, just exchanging some patents and closed-in approaches for others).

The preview tends to be that, especially in Safari. I tweaked so it should work sufficiently fast now.

Henrik,

It would actually be very nice if Microsoft chose the Ogg path. 🙂 Maybe not likely, but like you say, at the same time they would do anything to compete with Apple and Google.

What most people tend to forget is that flash video is just a wrapper around h.264 video. That's the reason why it's so easy to offer native h.264 support: all you need to do is get rid of the flash wrapper & point to your video directly, done. Any other browser can still get the flash version.

I'm all for open standards, but I have the feeling that Apple will resist this as much as possible (similar to not having Blue Ray drives in their computers). Google has lots of h.264 content, so will definitely support that. I don't know what Silverlight uses, but my guess is that either it's proprietary MS codecs or h.264. It would be great if someone could clear that up.

IF it is h.264, I think IE will support h.264 as well & Firefox & Opera will have to swallow the bitter pill of buying a license.

It will become an interesting battle. Let's hope that Flash won't be the winner in this one.

[…] IE9 has chosen to use the H.264 codec, and if that remains their only choice and no support for Ogg Theora it is a sad, sad day for open video on the web, and something which will severely cripple the usage of the video element. Read more about this in The video element in HTML5 – great possibilities, but also codec and licensing problems. […]

[…] IE9 has chosen to use the H.264 codec, and if that remains their only choice and no support for Ogg Theora it is a sad, sad day for open video on the web, and something which will severely cripple the usage of the video element. Read more about this in The video element in HTML5 – great possibilities, but also codec and licensing problems. […]

[…] have previously described the issues in greater depth in The video element in HTML5 – great possibilities, but also codec and licensing problems, but suffice to say, Firefox and Opera will never support H.264 and Safari will most likely never […]