A prominent Mozilla developer has proposed implementing support for H.264 …

The HTML5 video element promised to be a game-changer for Internet media publishing. It provided a vendor-neutral standards-based mechanism for conveying video content on the Web without the need for proprietary plugins while offering a path for tighter integration of video content on the Web and broader platform support than has historically been available through plugins.

But the HTML5 video element has yet to live up to its full potential, because a dispute over video encoding has prevented the standard from being implemented consistently across all Web browsers. Mozilla, which has long resisted adoption of H.264 on ideological grounds, is now preparing to support it on mobile devices where the codec is supplied by the platform or implemented in hardware.

The popular H.264 format is widely viewed as the best technical choice for encoding Internet video, but its underlying compression technologies are covered by a wide range of patents. This has raised the question of whether its appropriate for a standards-based Web technology to rely on a patent-encumbered video format that requires publishers and software implementors to pay licensing fees.

The ubiquity of the Web and its strength as a platform for innovation are partly due to the royalty-free licensing model that the W3C mandated for Web standards. As Mozilla and other parties have argued over the past few years, the use of a patent-encumbered video format is antithetical to the principles of the open Web. Critics of the H.264 licensing model have advocated the use of other video codecs, causing a split in the browser landscape.

Apple and Microsoft both support H.264, while Mozilla and Opera oppose the use of patented codecs. Google previously favored H.264, but shifted its position after opening VP8, a codec that the search giant has put forth as a viable alternative to H.264 for Internet video. Google vowed to remove H.264 support from its Chrome Web browser at some undisclosed future date, but has not yet done so.

The lack of universal support for a single codec has proved problematic because it compels content creators to either encode their video in multiple formats or fail to support large segments of their audience. Building consensus around a single codec would remove one of the biggest remaining impediments to widespread adoption of the HTML5 video element.

A change in course

Mozilla's strong commitment to the open Web made it seem as though the organization's position was intractable. Mozilla's resolve on the matter appears to have cracked, however, as the organization confronts the challenge of bolstering its credibility as a mobile platform provider.

Andreas Gal, Mozilla's director of research, announced on a public mailing list today that he wants to proceed with a plan that would enable H.264 decoding on Mozilla's Boot2Gecko (B2G) mobile operating system. The proposed change would allow the video element in Mozilla's HTML rendering engine to rely on codecs that are supplied by the underlying operating system or dedicated video hardware.

In addition to enabling H.264 playback in B2G, the proposed patch would also enable it in the Android version of mobile Firefox. Gal further expressed support for eventually taking similar measures in the desktop version of Firefox, with the stipulation that it would only be practical if the implementation ensured support for virtually all users.

Modern versions of the Windows operating system expose an H.264 codec to third-party software, but Windows XP does not. Gal said that he'd favor supporting H.264 in Firefox on the desktop if a means could be identified for ensuring that XP users (which represent a very significant portion of Firefox's audience) aren't left out. This is a radical change of policy for Mozilla, one that could have significant ramifications for the future of video on the Web.

Despite the pragmatic concession, Gal says that Mozilla's ideological position in favor of open codecs remains unchanged. The organization is still hopeful that an unencumbered codec will eventually prevail.

"We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats," he wrote. "I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience."

The option of using system-provided codecs is an obvious solution that would allow Firefox to play H.264 video without having to ship the code itself. We've discussed (and endorsed) this approach in some of our previous coverage, but Mozilla has historically rejected it on ideological grounds. In the past, Mozilla's position was that it didn't want to take any steps that would legitimize or encourage the use of a patent-encumbered codec. The organization is no longer maintaining that argument.

Google's major investment in advancing its unencumbered VP8 codec gave open Web advocates hope that H.264 could still be displaced, but it hasn't happened. The lack of follow-through from Google on its promise to remove H.264 from Chrome has eroded faith in the search giant's ability to popularize VP8. Gal says that it's no longer feasible to wait for the open codec to gain additional traction.

"Google pledged many things they didn't follow through with and our users and our project are paying the price," he wrote. "H.264 wont go away. Holding out just a little longer buys us exactly nothing."

The proposal to support H.264 in mobile Firefox has generated a tremendous amount of controversy among Mozilla developers. The critics include Mozilla employees and independent contributors. Mozilla's Joe Drew characterized the proposal as "capitulating on Free codecs" and expressed concern that the mobile-centric rationalization amounts to pushing an ideological compromise through the back door.

Firefox developer Justin Dolske also expressed some concerns. He pointed out that the possibility of enabling support for system codecs was discussed once before in relation to Fennec on the Nokia tablet devices and that it was rejected at the time for ideological reasons. He asked that the issue receive further discussion, specifically some clarification about what circumstances have changed that necessitate a reversal of the previous policy.

"The state of HTML5 video started off from a bad place, and to be fair still isn't in a good place. So reassessing Mozilla stance is not unreasonable. But I think if Mozilla is going to do an about-face on open video standards (and it is an about-face), then there should be some serious discussion about it. Certainly more than than a few terse words saying it's hopeless and obvious," he wrote. "We spent a lot of time and made a lot of blog posts about why H.264 was bad for the web. Leaving those who advocated for us suddenly high-and-dry doesn't feel like the right thing to do."

The debate has continued on the mailing list. There is also some preliminary discussion from certain participants in the debate about whether it would make sense at this point to simply license the codecs and ship them directly in the browser. Such a move, which would be a step further than merely supporting external codecs where available, would ensure support for Windows XP users but would detrimentally impact downstream distributors of Firefox code.

The outcome of the debate is unclear, but it currently appears probable that the plan to support system-provided codecs will be upheld and carried out. There are already some patches that have been hashed out, which means it can be practically implemented without much difficulty. The questions about how to proceed on the desktop and whether to license and ship the codecs are more tentative in nature and will likely take more time to be resolved.

159 Reader Comments

Can we PLEASE stop making engineering decisions based on Windows XP? If keeping your userbase happy involves keeping an 11 year old platform alive, your project is already a failure. You're tilting against windmills at that point, as more and more computing is moving away from XP. At some point you have to cut the cords.

So, can we have H264-support in FF for Macs already? The codec and appropriate license has been provided by Apple for any software to use since 2005. Must Mac users suffer for reasons connected to Windows XP?

So, can we have H264-support in FF for Macs already? The codec and appropriate license has been provided by Apple for any software to use since 2005. Must Mac users suffer for reasons connected to Windows XP?

Windows has had the codec and license since 2006 as well, so if we have to suffer because of FF's obstinance, you do too.

I wonder if people complaining about XP will be happy when stuff stats providing Metro-only programs and tells then to upgrade from the legacy.

The time to ditch XP is not yet with us, it's not like Windows 2000 that is as good as dead (though some people still use it - its fast and efficient, and should have been the basis for Windows Phone )

Most people upgrade the OS by way of buying a new computer with the more recent OS bundled.

Also, XP for most is "good enough" by a large margin. This in part because MS oopsed with Vista and also provided quite the upgrade in XP SP2. End result is a OS that has stamina, no matter what tech heads think.

I agree that nobody should use Windows XP this day and age. It makes no sense to use a 10 year old operating system. However, the reality is that it's 40% of Mozilla's user base. That makes it more difficult for them to toss it aside as obsolete and alienate way more users than this codec debate has done.

I favor the move to H.264 as quickly as possible so the web can finally start building some real usage of HTML5 video, but at the same time I respect Mozilla for having fought for the principle of royalty free video for so long. It's just painfully apparent that ideology alone isn't going to win over users, web developers, content producers, AND hardware manufacturers all at the same time. Let's have Mozilla with H.264 now WITH users so that they can fight battles for the open web in the future.

While it's fine to have ideals, one must live in the real world. That means that you have to do what is pragmatic. If Mozilla doesn't support H.264, its browser will become uncompetitive. When that becomes too much of an issue, people will stop using it. It's up to them to decide what's more important, a silly idealism, or going out of business.

I wonder if people complaining about XP will be happy when stuff stats providing Metro-only programs and tells then to upgrade from the legacy.

The time to ditch XP is not yet with us, it's not like Windows 2000 that is as good as dead (though some people still use it - its fast and efficient, and should have been the basis for Windows Phone )

Geesh! They can support more than one codec, or method of video delivery. It doesn't have to be one OR the other.

not too important to me if FF uses it or not, there are other browser choices.

It's important to web developers compelled to support a browser with a huge market share though. Mozilla supporting H.264 in HTML5 video means that I can devote precious server space that would otherwise go towards a WebM version towards providing additional profiles for different bandwidths in H.264.

"Fuck XP" is just as silly as "we're only doing WebM, not h264." It's not pragmatic. There are a ton of XP machines still out there; they have to be supported by companies. Users and businesses alike, because it's good enough.

Most people upgrade the OS by way of buying a new computer with the more recent OS bundled.

Also, XP for most is "good enough" by a large margin. This in part because MS oopsed with Vista and also provided quite the upgrade in XP SP2. End result is a OS that has stamina, no matter what tech heads think.

The biggest problem with Vista, is in their money making heydays, MS, hardware vendors and software houses all agreed that Vista would be their way to make money. MS would start their silly, yet more profitable, segmentation of a thousand OSes, hardware vendors would make money by not writing vista drivers for older hardware and software houses would make money by not updating older software to run on Vista. For many consumers, it was not about hating vista, it was about needing new hardware (printer/scanner/fax/router etc.) and new programs to be compatible with vista.

The thing MS, hardware vendors and software houses underestimated, is the extent to which people who were functioning well with XP, their hardware and software would be hesitant to blow a bunch of money just because vista demanded it. For absolutely no appreciable benefit. Add to this, that the first vista computers were slower due to vista's higher ram needs going unfulfilled and no wonder the "unwashed masses" began to question why they needed all this pain to continue doing what they could already do.

I wonder if part of the issue is that if it is in software (even if it is basically a firmware blob or special shader) the software-provider fronts the license. But if it is set in hardware, then it can be baked into the purchase price of the chips.

Let's hope this gains some traction.If this is what Google purchasing VP8 and opening it up as WebM brought us, I'm glad.Still, I'm afraid if Mozilla and Opera cave too early (and Google never do their part: dropping h.264 in Chrome), there will be no incentive from the h.264 side to pursue this.

I never quite understood why the browser developers weren't simply delegating to whatever the platform's video playback API is. For me they are simply doing work that is better delegated.

I can understand the ideological point of view, and it should be encouraged. At the same time sticking your head in the sand when the market is already moving in a certain direction isn't. My problem with VP8 is the lack of creation or playback tools, it's lesser image quality (compared to H264) and that it really doesn't seem to pose any universal advantages.

I would even like it so that plugins or extensions could extend the video tag. This would provide a way to experiment with more 'complete' video playback implementations to be used, such as ffmpeg and libvlc.

"Fuck XP" is just as silly as "we're only doing WebM, not h264." It's not pragmatic. There are a ton of XP machines still out there; they have to be supported by companies. Users and businesses alike, because it's good enough.

Then support it the same way everyone else is: Let the site designers and builders decide if they want to allow for Flash-based fallback on their pages. If someone browses to an HTML5 site and don't have a supported codec, they can either get an error or load a fallback Flash-based video player option. The website designer gets to decide if they want to support XP then. FF is holding back HTML5 adoption by making the decision for website designers.

It doesn't bother me that much that people are still using a 10 year old operating system, supporting legacy systems to a degree is expected and XP was a solid OS. Don't get all flustered because many people decide to stay with something their accustomed too, or a business that has invested in programming to support an operating system doesn't want to expend the effort to upgrade yet. Hell, even current gen game consoles are hitting their 7 year marks, but just because the console OS has continuous updates and redesigns doesn't make the hardware any newer. Google bought Motorola Mobility which owns a ton of patents, some which surround H.264 and that may be one reason why they haven't made a move to remove it yet (and might I add MS and Apple supposedly have both filed suites against MM because of patent licensing fees or something).

What bothers me the most is that an organization known for its incredible stance on open technologies is swinging because of what seems to be internal pressures (just my observing opinion no fingers being pointed directly). First someone decides they should start a discussion about implementing Webkit css prefixes in FF because 'everyone is developing for Insert-Webkit-Browser-here' and now there's pressure for H.264...whats next?

I see the reasoning behind the discussion for H.264 and it makes sense to have the discussion, it even makes sense to want it(H.264) for obvious reasons. But I also don't think that just throwing in the towel and saying "screw it" is going to help fix the very real problem of patent-induced innovation stifling.

I never quite understood why the browser developers weren't simply delegating to whatever the platform's video playback API is. For me they are simply doing work that is better delegated.

I never quite understood why images aren't decoded by platform codecs, so everyone and their mother can post proprietary raw camera files on the web. Heck, I never understood why HTML parsing isn't delegated to the platform API, they are simply doing work that is better delegated.

You could use system libraries for everything from images/audio/video to XML/HTML/Javascript. Why video and not the rest? Because it's hardware accelerated? JPEG decoding can also be hardware accelerated.

Besides, the Windows/Mac API that decodes h.264 with hardware acceleration probably doesn't even work inside the security sandbox of a browser like Chrome. Are we supposed to let video content violate sandbox restrictions after all the brouhaha of letting Flash run in a relaxed sandbox?

While it certainly could be thought of as "idealism vs. pragmatism", I think just as strong an argument can be made overall that it's "idealism vs. idealism", except that each side has different ideals. On the Mozilla-side, the ideal has been something that is entirely unencumbered by software patents as the highest priority. On the H.264-side, the ideal has been the absolute best technical solution, either dealing with or simply ignoring the patents (since the best H.264 is open source, widely ported and widely available, as a practical matter the vast majority of individuals wouldn't need to give a shit about licensing even if most systems didn't come with it). This sort of conflict, between law and engineering, is hardly a new one.

Quote:

The organization is still hopeful that an unencumbered codec will eventually prevail.

This will happen by definition, depending on "eventually". After all, even though they shouldn't exist period software patents are still patents, which means that they do at least expire. Formal approval of H.264 came in March 2003, and format finalization was a ways before that. So even in the worst case, "eventually" (around 10 years from now) H.264 will itself no longer be covered by anything. Hopefully we can get software patents abolished entirely well before that though.

I don't know if I really care at this point. HTML5 video hasn't seen much of any progress in a year, the whole talk about MPEGLA suing over VP8 hasn't happened (and that was a year ago) or haven't read any updates to it here.

Chrome supports h.264 still but it still plays webm from youtube anyhow.

I never quite understood why the browser developers weren't simply delegating to whatever the platform's video playback API is. For me they are simply doing work that is better delegated.

Except that fixes exactly zero of what the media tags in HMTL5 was supposed to fix, the need to use plugins for video playback while images could be handled natively by the browser.

Never mind that we had the same issue back in the early days of the web, as GIF was used extensively while on questionable licensing grounds. End result was PNG, that sadly MS only recently seems to have gotten proper support for.

The 'business' justification for XP has long passed its expiration date. If a business has a hard-coded reason to stick with XP for internal browsing, then they have forfeited the right to access the external web and demand it cater to their obsolete limitations. Pick one or the other, and stop externalizing your IT costs onto the rest of the world.

If employees need to see external H.264 video for some legitimate reason, they have browsing and security needs that are beyond the scope of XP's capabilities anyway.

Heck, I never understood why HTML parsing isn't delegated to the platform API, they are simply doing work that is better delegated.

Huh? HTML not only didn't exist as part of a system until long, long after development, but it is still a rapidly evolving standard with implementations rapidly evolving in turn. Unlike a static, fixed format that will not be changing it absolutely makes sense to have it being independently developed, which enables iteration at a vastly faster pace.

Quote:

You could use system libraries for everything from images/audio/video to XML/HTML/Javascript. Why video and not the rest? Because it's hardware accelerated? JPEG decoding can also be hardware accelerated.

Images, audio, and video might in theory make sense. The formats are all fixed and not rapidly evolving (primarily bug fixes) on the decoder side (work continues on the encoder side of course, but that's immaterial here). In practice, images are so lightweight in both code and hardware, static, unencumbered, and critical that it's pragmatic to just have them included if only for portability reasons.

What the hell is wrong with these people? Just fucking support H.264 through OS codecs on platforms where it is available, and don't support it where it's not. You avoid licensing/patent issues while still supporting H.264 on platforms where it's possible to do so. It's a win win situation. This is not fucking rocket science.

Mozilla shouldn't worry about Windows XP not having native h264 support. Windows XP usage should dramatically drop after the official 2014 end of life deadline. I am also sure Microsoft could do something about patching H264 through its updates channels, if Firefox is serious about H264 b/c it is in Microsoft's interest to undermine WebM and promote H264.

Mozilla stance towards h264 is seriously undermining the mobile version, and its desktop version gets only gets by ironically because of the dominance of propriety laden Flash plugin.

This generation video codec is largely settled. What Mozilla should be worry about is the next generation and whether the open video codec movement (Google, Mozilla, Opera) has a viable codec competitor to the eventual H265 specification.

What the hell is wrong with these people? Just fucking support H.264 through OS codecs on platforms where it is available, and don't support it where it's not. You avoid licensing/patent issues while still supporting H.264 on platforms where it's possible to do so. It's a win win situation. This is not fucking rocket science.

Except that the whole point of standards is to deliver a consistent experience for everyone. Regardless of the age of the OS, and XP *is* way past its time, it's still 40% of Mozilla's user base. Nobody wants a half cocked solution any more than they want a royalty-free codec that nobody supports.

So even in the worst case, "eventually" (around 10 years from now) H.264 will itself no longer be covered by anything. Hopefully we can get software patents abolished entirely well before that though.

Hopefully we can get them abolished and also have a better standard than H.264 in 10 years

H.265 (HEVC) is well under development, although irritatingly it's also hampered by software patents, albeit in the exact opposite respect from this situation. I think they're aiming for another 40-50%, while not exceeding 3x computation complexity, but I haven't been following it along carefully for a while. It is supposed to scale much farther, all the way to 7680x4320.

However, even then it's not as though H.264 would remain solid and extremely widely used, along with a massive existing library of media that wouldn't be going away or getting reencoded (just as with most popular media formats to date). If we don't manage to get rid of software patents, then that in itself would be competition for H.265 if they aren't careful (which is a good thing), and encoders are so incredibly important that H.264 will probably exceed a future standard in quality and efficiency for at least a year or two, or more depending on who gets to work on it.

This generation video codec is largely settled. What Mozilla should be worry about is the next generation and whether the open video codec movement (Google, Mozilla, Opera) has a viable codec competitor to the eventual H265 specification.

+1. Or that they have the marketshare/weight to lobby for a royalty-free variant of H.265 for the web. Which is something that they can't do if they lose all their users because of lack of H.264 support.