HTML5 video in Internet Explorer 9: H.264 and H.264 alone

It's already been announced that Microsoft will be supporting HTML5 video in …

Microsoft has put its stake in the ground and committed to supporting H.264 in Internet Explorer 9. That the next browser version would support H.264 HTML5 video was no surprise (though the current Platform Preview doesn't include it, it was shown off at MIX10), but this is the first time that Microsoft has provided a rationale for its decision. More significantly, this is the first time the company has confirmed that H.264 will be the only video codec supported.

H.264 certainly has some advantages. It's standardized, resulting in wide support in both software and hardware. This also provies a migration path of sorts from Adobe Flash; the same H.264 video file can be played both in Flash and via the native browser support, which allows site owners to target both HTML5 and Flash users with a single codec. But the biggest advantage cited by Microsoft was intellectual property: the IP behind H.264 can be licensed through a program managed by MPEG LA. Other codecs—the blog post named no names, but Theora is obviously the most widespread alternative for HTML5 video—may have source availability, but they can't offer the same clear IP rights situation.

The codec choice is a long-standing issue with HTML5. The current draft specification refrains from mandating any specific codec, and the result is a schism. Firefox and Opera have gotten behind Theora, citing its openness; Safari and now Internet Explorer 9 have gone for H.264. Chrome supports both. It's not impossible that the spec will ultimately be changed to specify a specific codec—and that codec might not be H.264. If that turns out to be the case, Redmond will likely have to revisit the decision.

H.264 has its problems, though. It's currently royalty-free for Web usage, but there are no guarantees that this will continue in the future. In contrast, Theora is perpetually royalty-free.

The status of Theora might change if patent concerns emerge—though designed to avoid patented technology, there are no assurances in the murky world of software patents—and so there is some risk to browser vendors. But the unfortunate reality is that's true of H.264 too. MPEG LA may believe it holds all necessary IP rights to H.264, but one never knows when a patent troll might emerge unexpectedly from the deep. H.264 might be the safer option, but the difference is not as clear as Microsoft makes out.

The decision may also be premature if Google opens up VP8, as is widely expected. If Google does indeed publish the source, and offers perpetual royalty-free usage of the VP8 patents, the codec may offer the same level of freedom and openness as Theora, combined with the corporate backing of Google, and assurances that both Google (and On2 before it) have done their due diligence.

Google would also have the power to promote VP8 in other ways; as owner of YouTube, the company could, in principle, make VP8 one of the most widely used codecs on the Internet. Adoption by the company's own Chrome browser is all but certain, and if the terms are suitable, VP8 could also find its way into Firefox and Opera. Such a move would give VP8 considerable momentum.

Microsoft's decision to support only one codec is also a little surprising when one considers the way in which video support will be implemented. Internet Explorer 9 will use the Media Foundation media decoding framework that's part of Windows Vista and Windows 7—another reason that the browser won't run on Windows XP. Just as with its predecessor, DirectShow, Media Foundation is an extensible framework that allows third-party codecs to plug in to the media infrastructure so that applications can use them automatically.

If Redmond allowed IE9 to use Media Foundation without any restrictions, it would enable the browser to use whatever codecs were installed; H.264 is already included with Windows 7, so it might be a good default, but there's nothing preventing Google from producing a VP8 plugin for Media Foundation, for example. This would provide automatic support to IE9 users for VP8 video.

That said, with the browser locked down in its protected mode sandbox, and the fact that Media Foundation codecs should all be modern (and hence, security-conscious), the risk seems marginal. Morever, there are likely other ways to feed malicious video into insecure codecs (embedding Media Player, for example), so the level of protection this offers isn't clear.

HTML5 video is an important part of Internet Explorer 9, and given the big-name industry support for H.264, Microsoft's decision was not altogether surprising. As welcome as the new capabilities are, the decision to rule out the more open codecs is somewhat disappointing.

Yea. So now instead of running different plugins (flash, silverlight) for video on a given site, I have to run different browsers based on what standard the site supports. Today a site typically supports flash or silverlight, but not both, so I would imagine that the same will be true of HTML 5 video codecs.

While yes, it is removing the plugin pitfall from the equation, feels like a step backwards.

I think h264 is a good bet, personally. It's widely supported and has good quality:compression. And if MPEG decides to change the licensing in 2016 or whatever, I believe it will be 1. unwise and would earn MPEG a huge amount of 'free hate' for little gain and 2. easy to switch to another format in that time frame because computers will have near unlimitedless computing power at that point (compared to what we have today.)VP8 sounds interesting though, and Microsoft should probably allow any Media Foundation codec installed on the system to play html5 videos.

Even if Google decides to back VP8 using YouTube -- and I hope they do! -- I can't imagine they'll stop distributing H.264 in the forseeable future. As for MS, I think everyone saw this coming. Why would they back an open format? Though I'm sure they would have loved to use a format they control themselves.

Notice Apple and Microsoft are a part of that list. Once the license is no longer royalty free, Apple and Microsoft stand to make a lot of money from it. This is a motivating factor behind Microsofts choice to support H.264, and considering its the only codec they'll support, its probably a large part of that decision.

The fact that Flash can also play H.264 bodes well for Adobe. They are not really harmed by this decision, in fact I feel this will be helpful in providing alternate content to devices without Flash support, namely iPhone & iPad.

For more details on the MPEG-LA and various Apple Patents in the HTML5 spec, please check out our article:

I thought the whole purpose for going to HTML 5 for video was so that we could avoid plug-ins and requiring users to install various codecs in order to play videos. Why are we talking about installing codecs? I thought that the browsers themselves would have the necessary codec included so that the user didn't have to do anything else.

I realize that codecs and plug-ins are different, but maybe someone can clarify this for me.

Notice Apple and Microsoft are a part of that list. Once the license is no longer royalty free, Apple and Microsoft stand to make a lot of money from it. This is a motivating factor behind Microsofts choice to support H.264, and considering its the only codec they'll support, its probably a large part of that decision.

I considered that, but if MS were really motivated by license fees, wouldn't VC-1 be a "better" choice?

h.264 won. done, game over. vp8 won't do squat even if google decided to adopt it for youtube (which they probably won't, unless their pre-h.264 library was already in vp7/8). h.264 has critical mass: easily accessible content creation tools for professionals and consumers, hardware accelerated encoding/decoding on portable devices, the support of major content creators and distributors and the fact h.264 allows for far simpler implementation online. one h.264 video file served natively to ie9 and webkit based browsers, a flash wrapper to play the same h.264 video for everyone else. only those people opting to "make a point" are going to jump through the hoops to include ogg/theora and h.264. theora has none of that, vp8 does have some of it, but not all.

and when you look at the patent holders of h.264 the vast majority of them are hardware manufacturers not content creators. it's content creators that look to gain on per-viewing license fees. hardware manufacturers only care about unit sales. they'll make orders of magnitude more money by selling you widget Y two years after you bought widget X than they will on minuscule amount they might gain by enforcing the viewing license fee.

Google is currently supporting some ARM-specific work done on Theora. I think the official line was something like "it's good for there to be a standard lowest-common-denominator".

Also, there's been alot of back-and-forth over the quality of Theora versus H264. As far as I can tell, at the bit rates that are common for today's online videos, normal people are not going to notice a difference. And with higher bandwidth, everything looks great.

What would be terrific is if all browsers simply linked the HTML5 video tag to the collection of codecs installed on the computer. Then it wouldn't matter what codec they standardised on, because new websites would be free to promote other trusted codecs. I realise HTML5 is supposed to move away from this, but there doesn't seem to be a better compromise when the alternative is everyone being forced to use the ticking timebomb that is H264.

I missed some explanations and need some help. Why do browser developers have to pick formats anyway? All of these browsers have used plug-ins and helper programs for many years. I haven't seen why that has to end with the HTML5 video tag.

VC-1 wouldn't necessarily be the better choice. MS lost that battle when more CPU power and hardware decoding eroded VC-1's main appeal. Getting 3 percent of $98 is better than getting 78 percent of $2. Or whatever the acual breakdowns are.

There are many reasons to choose H.264:1) Best HW support.2) Certainly Good Enough compression3) Many many more videos available online are encoded in H.264 compared to either Theora or VP8.

If Microsoft is focusing on the embedded market (e.g., they will be using some form of IE9 in their phone/embedded OS) hardware decoding is a must. Also, going with something like VC-1 would make them the most non-standard of the video formats and give people more reasons to not use IE.

The whole thing is a mess. It's hard to see how things are progressing when there still will not be onevideo format which will actually play in all video-capable browsers/devices.

I guess it'll sort itself out in the end but it's an issue that could've been solved in the next few years (basically, as soon as the older, non-HTML5 browsers finally died out) but now looks like it'll go on a lot longer because people couldn't man-up and agree on a common standard codec that suited everyone's needs.

I can't say I like the idea of the H.264 owners holding us to ransom over undefined future fees (which may even apply to transmitting the videos; not just encoding or decoding them, which seems ridiculous to me). The seem like crack dealers giving us the first hits free like that's doing us a favour. Maybe they'll be nice about the whole thing but when has a group of big corporations ever done that when given sufficient leverage to churn a profit?

The stuff about Theora submarine patents (which should be illegal, but that's another argument) applies to H.264 or any other codec. Large companies have shipped products with Theora in them, now, without being sued. Okay, more have shipped with H.264, but someone could be waiting for the optimal moment to pounce on patents related to any of the codecs.

Points about Theora not having hardware decoders do seem valid, but it's not like they couldn't be built.

Mainly, I just can't really back the idea of adopting a format which we do not yet know the cost of and which appears to be very problematic for certain browsers to support at all (even if they wanted to). Mozilla cannot support H.264 without fundamental changes to the way they build and distribute their code while Microsoft and Apple are choosing not to support Theora.

Oh well, I guess we can use Flash for Mozilla and old-browser (IE6, sigh) support and H.264 for iPhone support, with Chrome and IE9 able to use both. Having to use two video files and put even more crap into the HTML to serve them doesn't exactly feel like progress, though.

(Well, two container files anyway. I know Flash can play H.264 but it doesn't help clean up the HTML code.)

Of course, that is an example of targeting one specific DSP, but it does happen to be a fairly common DSP used in a few mainstream devices. Google could build in support for targeting common DSPs into Android and encourage handset makers to support VP8 or Theora, but that looks unlikely to matter at this point.

If the concern that Firefox & Opera has is the "future" of H.264 after the end of 2016, then why not just commit to supporting H.264 until then? Heck, why not build a time-bomb into the browsers that will disable H.264 playback after that date. It's SIX AND A HALF YEARS from now.... there's loads of time to figure out what to do about video playback, and loads of time for MPEG-LA to clarify whether they're going to enact a cost structure for browser vendors and content viewers.

In the meantime, H.264 is free game, it's a good standard, and people can use it. If we have to switch to something else in 2017, it's really not a big deal. Computers are going to be much more powerful then, so converting the video to whatever will be available then will be no big deal.

I still don't understand why MS and Apple both refuse to include Theora. Even if it isn't a practical replacement for H.264, it's free. The code is already there to be compiled into a library for playback. Really, it's the same with Dirac too.

Yea. So now instead of running different plugins (flash, silverlight) for video on a given site, I have to run different browsers based on what standard the site supports.

No, because sites will use fallbacks to make sure things work. (Except freetard sites which will be Theora-only and will only work in Firefox/Chrome.)

If your fascist payware can't handle some random format, it certainly won't be the fault of it being gratis and libre. If you choose to cloister yourself behind some iron curtain, that's no one's fault but your own.

NOBODY should be dictating codecs. This goes equally well for mozilla as it does microsoft.

Mainly, I just can't really back the idea of adopting a format which we do not yet know the cost of and which appears to be very problematic for certain browsers to support at all (even if they wanted to). Mozilla cannot support H.264 without fundamental changes to the way they build and distribute their code while Microsoft and Apple are choosing not to support Theora.

I suspect if you asked most people here (or anyone who really understands The Video Codec Dilemma) they would love to see a universal, free, efficient, hardware-supported video standard that everyone can use without any ongoing licensing fees. I don't think anyone, short of a MPEG-LA shareholder, would argue that point. But the simple reality is you can't always get what you want, so some people make their compromises based on ease-of-use, and others on ideology. Neither is right or wrong, but the money is ultimately siding with the licensed solution that moves businesses forward.

The alternative is to do what humanity has done for all of recorded history up until a few years ago: not watch video in a web browser. There is always that option, which people seem to forget.

I thought the whole purpose for going to HTML 5 for video was so that we could avoid plug-ins and requiring users to install various codecs in order to play videos. Why are we talking about installing codecs? I thought that the browsers themselves would have the necessary codec included so that the user didn't have to do anything else.

I realize that codecs and plug-ins are different, but maybe someone can clarify this for me.

I guess we are moving from one problem to another . As someone mentioned earlier, the worse, if different browsers support different codecs natively, you would have to switch browsers to play different videos (or) sites.

Quote:HTML5 video is an important part of Internet Explorer 9, and given the big-name industry support for H.264, Microsoft's decision was not altogether surprising. As welcome as the new capabilities are, the decision to rule out the more open codecs is somewhat disappointing.

Wouldn't it make more strategic sense (and help Microsoft regain a good name!) to support Ogg Theora?!

Supporting inferior products just because they're open source is not the way to go.

This is like buying a shitty car just because it was made in your country.

bad example, its buying a shitty car because it comes with all the schematics for its parts in the glove box, so anyone can provide you with replacements or upgrades if wanted.

heck, cars are open source, at least as long as your driving a fully mechanical one. Just observe the number of people that manage to maintain a car from as far back as ford's original in running condition to this day, thanks to being able to basically make replacement parts.

Heck, recently i read that old steam locomotives are easier for enthusiasts to maintain then more recent diesels.

Supporting inferior products just because they're open source is not the way to go.

This is like buying a shitty car just because it was made in your country.

While we're on car analogies,

It's like buying a fuel efficient car instead of a Hummer because down the road everybody will be better off, even though the fuel efficient car is a little worse in every way (replace Hummer with a gas guzzler that suits you).

Notice Apple and Microsoft are a part of that list. Once the license is no longer royalty free, Apple and Microsoft stand to make a lot of money from it. This is a motivating factor behind Microsofts choice to support H.264, and considering its the only codec they'll support, its probably a large part of that decision.

I considered that, but if MS were really motivated by license fees, wouldn't VC-1 be a "better" choice?

*Anything* is a better choice than H.264. In the age of Comcast and FiOS I don't understand the obsession with file sizes. CPU use (which is terrible for H.264-based codecs) is a more important concern.

I still don't understand why MS and Apple both refuse to include Theora. Even if it isn't a practical replacement for H.264, it's free. The code is already there to be compiled into a library for playback. Really, it's the same with Dirac too.

There is a difference between MS' position and Apple's position, however: Safari will use a Theora plugin if installed (and such plugins exist). This piece makes it clear that IE will not use any plugins to play HTML5 video. The reason for not including it is most likely risk minimization: There may be patents out there that cover Theora, even if the risk that they're not included in H.264 is rather small.