Tech —

Decoding the HTML 5 video codec debate

The HTML 5 video element has the potential to liberate streaming Internet …

The increasingly competitive browser market has at last created an environment in which emerging Web standards can flourish. One of the harbingers of the open Web renaissance is HTML 5, the next major version of the W3C's ubiquitous HTML standard. Although HTML 5 is still in the draft stage, several of its features have already been widely adopted by browsers like Safari, Chrome, and Firefox. Among the most compelling is the "video" element, which has the potential to free Web video from its plugin prison and make video content a native first-class citizen on the Web—if codec disagreements don't stand in the way.

In an article last month, we explored the challenges and opportunities associated with the HTML 5 video element. One of the most significant of these challenges is the lack of consensus around a standard media codec, a contentious issue that has rapidly escalated into a major controversy. The debate has now stalled without a clear resolution in sight.

The HTML 5 working group is split between supporters of Ogg Theora and H.264. Their inability to find a compromise that is acceptable to all stakeholders has compelled HTML 5 spec editor Ian Hickson to "admit defeat" and give up on the effort to define specific codecs and media formats in the standard itself. This is problematic because the lack of uniform codec availability will make it impossible for content creators to publish their videos in a single format that will be viewable through the HTML 5 video element in all browsers.

In an e-mail posted to the WHATWG mailing list, Hickson outlined the positions of each major browser vendor and explained how the present impasse will influence the HTML 5 standard. Apple and Google favor H.264 while Mozilla and Opera favor Ogg Theora. Google intends to ship its browser with support for both codecs, which means that Apple is the only vendor that will not be supporting Ogg.

"After an inordinate amount of discussions, both in public and privately, on the situation regarding codecs for <video> and <audio> in HTML5, I have reluctantly come to the conclusion that there is no suitable codec that all vendors are willing to implement and ship," Hickson wrote. "I have therefore removed the two subsections in the HTML5 spec in which codecs would have been required, and have instead left the matter undefined."

Ogg Theora is an open format that is thought to be unencumbered by patents. The primary reference implementation is distributed under an open source license and it is being developed by the non-profit Xiph.org with funding from Mozilla. Ogg is strongly preferred by the open source software community because it can be freely redistributed without requiring licensing fees.

H.264 is a high-performance codec that is maintained by the ISO Moving Picture Experts Group (MPEG) as part of the MPEG-4 family. It is emerging as the dominant codec for both streaming video and optical media, as it is said to deliver the visual quality of MPEG-2 (used on DVDs) at roughly half the bitrate. The MPEG LA consortium manages licensing of the underlying patents that cover H.264 compression algorithms and other software methods needed to implement the codec. In order to use the format, adopters have to pay licensing fees to MPEG LA.

Patent problems

Patent encumbrance is one of the driving forces behind the HTML 5 video codec controversy. The patent licensing requirements mean that H.264 codecs can't be freely redistributed, making the format a non-starter for Mozilla and most other open source browser vendors. Opera also objects, saying that the licensing fees are too high. Mozilla and Opera strongly advocate Ogg Theora as an alternative because its freedom from known patents could ensure that there are no licensing barriers that prevent ubiquitous adoption.

Apple objects to Ogg Theora, claiming that the lack of known patents on Theora doesn't rule out the threat of submarine patents that could eventually be used against adopters. Apple is also concerned about the lack of widespread support for hardware-based Theora decoding, a factor that diminishes the format's viability on mobile devices. Google shares Apple's skepticism about the potential of Theora in the marketplace. The search giant claims that Theora's lack of quality relative to H.264 will make it an impractical choice for large-scale streaming video services such as YouTube.

Obtaining a license for H.264 from MPEG LA doesn't guarantee complete immunity from patent infringement liability, though. Although it is generally assumed that MPEG LA controls all of the relevant intellectual property pertaining to H.264 implementations, there is still the possibility that a third-party which is not a member of the consortium has a broad patent covering related compression technology that it can independently enforce against MPEG LA licensees.

Although Theora is not known to infringe any patents, critics fear that enhancing it to make it competitive with the most modern and efficient codecs will greatly increase its exposure to infringement risks. Some critics even contend that it's not possible to advance Theora without inevitably hitting a patent wall.

Another licensing issue that is often overlooked is the ambiguity of MPEG LA's future patent royalty collection plans. MPEG LA has established broadcast fees that licensees will be required to pay for distributing free (or ad-supported) streaming video content on the Internet. These fees will not be instated until the end of 2010, when the second H.264 licensing period goes into effect. The language used in the current license treats Internet streaming just like over-the-air television, implying that the licensees will have to pay broadcast fees per-region. That could prove to be extremely costly for Internet video providers who make their content available around the world.

MPEG LA has provided no guidance, clarification, or insight into what the broadcast licensing fees will look like. When asked directly about the issue, MPEG LA representatives say that they haven't even decided yet themselves. The worry is that H.264 licensing for content distributors could potentially become too costly to sustain widespread use for streaming Internet video.

The compression efficiency debate

The viability of Theora for large-scale streaming video web sites is Google's primary concern. Google is committed to shipping both Ogg and H.264 support in its browser, but intends to use the latter to power its popular YouTube video site. The direction that YouTube goes for streaming video will be enormously influential and could, by itself, play a very significant role in determining the outcome of the codec issue.

The extent to which Theora lags behind H.264 is often overstated and the codec is, in actuality, in better shape than is generally thought by many of its critics. Google open source programs manager Chris DiBona is skeptical, however, and articulated the search giant's concerns about Theora's compression efficiency during the debate on the WHATWG mailing list.

"If [YouTube] were to switch to Theora and maintain even a semblance of the current YouTube quality it would take up most available bandwidth across the internet," DiBona said. "The most recent public number was just over 1 billion video streams a day, and I've seen what we've had to do to make that happen, and it is a staggering amount of bandwidth."

DiBona's quality claim was broadly disputed by Theora supporters on the mailing list. Mozilla's Mike Shaver encouraged DiBona to examine the most recent Theora developments, suggesting that the latest improvements have helped to significantly close the gap in compression efficiency.

"I don't think the bandwidth delta is very much with recent (and format-compatible) improvements to the Theora encoders," he wrote. "[Codec improvements] are a big part of what we've been funding, and the results have been great already. I'd like to demonstrate them to you, because I suspect that you'd be a better-armed advocate within Google for unencumbered video if you could see what it's really capable of now."

Xiph's Gregory Maxwell responded to DiBona's mailing list post by publishing a comparison that aims to demonstrate Theora's efficacy relative to H.264 in the context of YouTube-quality streaming video.

"Using a simple test case I show that Theora is competitive and even superior to some of the files that Google is distributing today on YouTube," he wrote. "Theora isn't the most efficient video codec available right now. But it is by no means bad, and it is substantially better than many other widely used options. By conventional criteria Theora is competitive. It also has the substantial advantage of being unencumbered, reasonable in computational complexity, and entirely open source. People are often confused by the correct observation that Theora doesn't provide the state of the art in bitrate vs quality, and take that to mean that Theora does poorly when in reality it does quite well."

Ogg has several high-profile supporters, including popular video streaming site DailyMotion and the Wikimedia Foundation, the organization behind the popular Wikipedia Internet encyclopedia. DailyMotion recently began the process of converting its video library to Ogg which it plans to deliver through the HTML 5 video element. DailyMotion has acknowledged that its Ogg streams have some technical deficiencies compared to its current Flash-based video streaming solution, but is confident that it's the best approach in the long run.

The Wikimedia Foundation, which is a strong supporter of open technology and unencumbered accessibility to information, was already committed to Ogg even before the HTML 5 video element gained multiple browser implementations. The organization is collaborating with Mozilla in the effort to boost Theora quality. In an e-mail on the WHATWG list, Wikimedia Foundation volunteer media contact David Gerard said that the organization is also interested in helping Mozilla to raise general awareness of the advantages that unencumbered video would bring to the Internet.

"I'd also point out that Wikimedia has vast publicity abilities in this direction," he wrote. "And we're watching the progress of Theora and Dirac on a day-by-day basis, for obvious reasons. So if you need large charitable organisations to help you with making this the obvious publicity choice for a happy Internet with cute fluffy kitties, I can tell you we'll be right there!"

A Fluffy Kitty summary of the codec debate

The following images demonstrate what the future of the web might look like, depending on potential outcomes of the codec debate.

The Web, with an unencumbered video codec

The Web, with many competing patent-encumbered codecs

I, for one, welcome our fluffy kitty overlords.

The undesirable middle-ground

A solution that seems logical on the surface is to simply expose each platform's underlying media playback engine through the HTML 5 video element—DirectShow on Windows, GStreamer on Linux, and QTKit on Mac OS X. This would make it possible for the browser to play any video formats that are supported natively on the user's computer.

From a purely technical perspective, this is not an impossible problem to solve as there are already existing libraries that do this and provide a cohesive abstraction layer on top. One prominent option is Nokia's Phonon library. It could also possibly be done by using the Quicktime and DirectShow plugins for GStreamer.

Mozilla strongly opposes this approach because it would heighten the risk of fragmentation. Allowing content providers to use any codec that is available on the user's computer might undermine the advantages of the HTML 5 media element because there would be no consistency guarantee and content would not be able to work everywhere. That is, however, arguably the situation that already exists as a result of the impasse in the codec debate.

Conclusion

Hickson has clearly grown tired of the debate and has no interest in allowing the divisive issue to continue distracting HTML 5 stakeholders from their efforts to push forward the standard. He takes the view that documenting a codec in the standard will achieve nothing unless the browser vendors are willing to conform with what the standard says. Microsoft, which has no plans to implement the HTML 5 video element at all, is also still an impediment to bringing open Internet video to the masses.

It's unfortunate that this debate is threatening to derail the adoption of standards-based Internet video solutions. The dominant video solution today is Flash, a proprietary technology that is controlled by a single vendor and doesn't perform well on Linux or Mac OS X. There is a clear need for an open alternative, but the codec controversy could make it difficult. My inner pessimist suspects that Microsoft will finally get around to implementing HTML 5 video at the same time that the H.264 patents expire, in roughly 2025.

>This is problematic because the lack of uniform codec availability will make it impossible for content creators to publish their videos in a format that will be viewable through the HTML 5 video element in all browsers.

While technically true, this statement is extremely misleading. You need to have two files, one Ogg and one H.264, and then you can support all browsers. You just specify two source files inside the video tag. And if you want to support fallback to IE as well, you can use Video For Everybody ( http://camendesign.com/code/video_for_everybody ) with an additional Flash file.

Yes, it sucks that you'll have to have two video files to support all browsers, but that's only an issue for the largest websites that deal with a ton of video, and in any case, Google does this anyway for the iPhone!

>The extent to which Theora lags behind H.264 is often overstated and the codec is, in actuality, in better shape than is generally thought by many of its critics.

This is problematic because the lack of uniform codec availability will make it impossible for content creators to publish their videos in a format that will be viewable through the HTML 5 video element in all browsers.

Untrue. The video tag allows multiple sources. A website can (and likely will) provide both H.264 and Ogg encodings of the same content, and the user's browser can decide which format is most appropriate for the platform, depending on capabilities (For example, even if both are supported by the same browser, H.264 might be strongly preferred due to the availability of hardware decoding, especially on mobile platforms).

Serving up multiple versions of the source file is something that most sites have been doing for a long time (varying bitrates, etc). Google already handles multiple formats with YouTube, serving up both flash and h.264 encoded video.

quote:

A solution that seems logical on the surface is to simply expose each platform's underlying media playback engine through the HTML 5 video element—DirectShow on Windows, GStreamer on Linux, and QTKit on Mac OS X. This would make it possible for the browser to play any video formats that are supported natively on the user's computer.

...

Mozilla strongly opposes this approach because it would heighten the risk of fragmentation

This is a perfectly reasonable solution. Safari already does this, its video element can play anything QTKit can play. If the user installs a quicktime plugin that includes Ogg support (several are available), then Quicktime Player, iTunes, Safari, etc, all gain ogg abilities. Using platform-specific APIs here also allow you to take advantage of hardware decoding. From a technical point of view (ignoring politics), using the platform provided video playback engines is obviously the right thing to do.

Now, to turn away from facts and towards opinion:

Mozilla's doesn't want to use platform provided video APIs, because they want Firefox to be simply incapable of playing H.264. They're worried that if DirectShow allows H.264 to be used on Firefox/Windows, and QTKit allows H.264 to be used on Firefox/Mac, then Firefox/Linux is left with so little market share that no one bothers to produce Ogg content.

Screw Apple if they don't want to use an Quality / Free / Open Source / Open Standard format that anyone can implement and improve for free.

Proprietary companies who complain about open source/free software are forgetting that they themselves can contribute and make the software the quality that they need it to be, which would benefit all of society.

So, let Apple suffer in this area. We need to begin having all standards become open source/free software to begin to destroy proprietary trash that is holding society back.

A solution that seems logical on the surface is to simply expose each platform's underlying media playback engine through the HTML 5 video element—DirectShow on Windows, GStreamer on Linux, and QTKit on Mac OS X. This would make it possible for the browser to play any video formats that are supported natively on the user's computer.

<snip>

Mozilla strongly opposes this approach because it would heighten the risk of fragmentation. Allowing content providers to use any codec that is available on the user's computer might undermine the advantages of the HTML 5 media element because there would be no consistency guarantee and content would not be able to work everywhere. That is, however, arguably the situation that already exists as a result of the impasse in the codec debate.

This shows how Mozilla is thinking, and why they cannot understand the objections Apple and Google have. Does anyone think that Mozilla will implement GPU acceleration for the three major GPU manufacturers into their code, and keep that up to date with new hardware releases? No, they won't. They'll just run it on the CPU, cause even a P4 is good enough to run that on, right? Sure - but Atom might not be, and even a CPU that can run it will use up more battery than doing it on the GPU. A mobile CPU is obviously out of the question.

Both Apple and Google are focused on the mobile platform here. To them, <video> is all about making video on the web work well on mobile devices without having to do special fixes like Youtube and the iPhone. Theora won't help them do that, so if that is what <video> will be about, why bother. Mozilla, OTOH, is all about free software and what is "architechturally clean".

I'm not saying that Apple and Google are right and Mozilla is wrong, but Apple and Google certainly understand the objections Mozilla is having - both have stated so on the list. Mozilla (and Xiph), OTOH, do not seem to understand the objections against Theora. The fact that there is no ASIC that implements Theora today is waved away as a minor inconvenience, and minor encoding improvements in the latest releases (or maybe not even that) are hailed as mostly erasing the advantage that H.264 has. Not to my eyes, they don't.

I'm afraid that neither Theora nor H.264 are a good solution that will satisfy, but we're out of good options here. The way it's going, Youtube will use H.264 as its primary source, because that's the only thing that will run acceptably on mobile platforms, and use Flash as a fallback, because nothing else will work with IE. This means that Safari and Chrome will run Youtube better and cooler than Mozilla, which will still get Flash. Even sites that do use Theora will run better on Safari and Chrome, once the appropriate plugins have been installed. I just can't see how Mozilla can ever win this.

No, it's not. Why? Because you've freed us from the problems of browser plug-in hell only to give us codec pack hell. That is in no way an acceptable solution, especially to the average user that doesn't even know what a codec is.

Originally posted by bradwjensen:Screw Apple if they don't want to use an Quality / Free / Open Source / Open Standard format that anyone can implement and improve for free.

Proprietary companies who complain about open source/free software are forgetting that they themselves can contribute and make the software the quality that they need it to be, which would benefit all of society.

So, let Apple suffer in this area. We need to begin having all standards become open source/free software to begin to destroy proprietary trash that is holding society back.

On the desktop, Apple doesn't suffer if Theora becomes the standard. Safari simply exposes the entire Quicktime architecture to the <video> tag, so once the appropriate plug-in has been installed, Safari plays Theora clips inside <video> today. Did so before FF, in fact, since Safari 4 was released before FF 3.5. All Apple needs to do is to re-enable the QT codec-finding service and include the Xiph plugin.

The iPhone can't do that, but neither can any other smartphone today. In that sector, we all suffer.

Originally posted by bradwjensen:Screw Apple if they don't want to use an Quality / Free / Open Source / Open Standard format that anyone can implement and improve for free.

Proprietary companies who complain about open source/free software are forgetting that they themselves can contribute and make the software the quality that they need it to be, which would benefit all of society.

So, let Apple suffer in this area. We need to begin having all standards become open source/free software to begin to destroy proprietary trash that is holding society back.

Apple uses open source stuff all the time, where do you think WebKit came from? Do you think it was this good before they got ahold of the code?

The fact is, hardware acceleration is essential with mobile platform video, and everyone is already shipping hardware that can decode h.264, it is the defacto standard for all sorts of stuff.

The real problem here is that the MPEG LA is imposing licensing fees on just about everyone to use the codec, but it is in fact open source, we have an encoder, decoder freely available in 100% software already.

This is certainly one of the best articles on the issue I've read recently. But one thing you failed to mention is that support for Ogg Theora in Safari can be achieved through the use of the XiphQT codec. So with Mozilla and Chrome supporting it natively, Opera having announced its intent to support it, and Safari via a 3rd party codec, that will be 4 out of 5 browsers with Ogg Theora support.

Although Microsoft hasn't made any comment about their intentions for supporting <video>, at least none that I can recall, my educated guess would be that, if and when they do ship support for the element, IE will support any codec supported by Windows Media Player either natively or via 3rd party codecs.

Mobile playback is very relevant to the ongoing discussion, and the article should have mentioned that. There are several commercially available (cheap!) h264 accelerators out there - every modern iPod with a screen has one (even the nanos).

But what does not exist is ogg/theora HW acceleration. You simply cannot play ogg on your iphone even if you wanted to, the processing unit has acceleration only for certain codecs, one of which is h264. There's no way that an ARM by itself could decode h264 or theora in real time while still retaining the notion of battery life.

If someone actually made hardware for decoding theora, it could be an alternative. But as of now, it simply is not. What I have heard, is that making HW decoders for theora is not as easy as it is for h264, but I have no sources to back this up. With really nice web-enabled devices like the Pre and the iPhone gaining traction they need to be taken into account when deciding the direction where the web should be going.

Well Mozilla is just going to have to bite the bullet and use native codecs.

DirectShow in windows, QT in OS X, and Gstreamer in Linux.

I don't see any reason why there should not be multiple video/audio formats anymore then there are multiple image formats. Would it better if there was no PNG or no JPEG? It's very obvious to me that one format is not suitable for all users and uses on the web.

There are very excellent open source codecs for H.264 and most other popular encodings. It's just that you can't allow users to legally redistribute them, which is the one very positive benefit to Theora.

What format will be popularly used will stem down to what tools webmaster prefer to use for setting up their websites and handling video. Very few people care or are qualified to really go into the nuances of various encodings, formats, bit rates, and the like. They will, for the most part, just use the defaults of whatever tools they like using.

This is why you won't have a explosion of different encodings. 99% will use the top couple formats then the rest will be speciality websites that use oddball stuff for specific reasons.

Basically My Recommendations are:

* Mozilla should carefully reconsider using native codecs.

Plus having Ogg + Theora + Vorbis codecs bundled with the Windows and OS X installers is a PERFECT way to distribute support for that format to increase it's popularity with non-Linux users. The installer should test for ogg/theora/vorbis support, and if it's not present then it should install the codecs on the client's machines.

Its a opportunity, not burden.

* People who purchase Apple products need to call up and tell Apple to quit being such f*kers over this whole thing.

Ogg Theora is acceptable, not ideal. It is freely available under favorable licensing terms and most of the work is already done. It's a FREEBEE for Apple support. They don't have to do anything much, it costs nothing, and it will have no negative effect for end users.

There is one reason and one reason alone that Apple refusing to bundle support.. its because they want to see Ogg Theora fail. I am sure that they have their own reasons for that, but it's not complicated to understand the motivations.

Microsoft, which has no plans to implement the HTML 5 video element at all, is also still an impediment to bringing open Internet video to the masses.

Does anyone have a link to Microsoft making an official statement on this, or is it just their de-facto status at the moment?

Microsoft is starting to support bits and pieces of HTML5.

HTML5 is specifically designed to make doing online applications much much easier. Video support is only a small corner of it. So it's very advantageous for MS to support some of it for the purposes of things like Sharepoint or Outlook Webmail.

Like everything else with Web standards you can expect IE to trail everybody else by a couple years. So with any discussion with Web standards Microsoft has successfully excluded themselves, which is unfortunate.

No, it's not. Why? Because you've freed us from the problems of browser plug-in hell only to give us codec pack hell. That is in no way an acceptable solution, especially to the average user that doesn't even know what a codec is.

By the time HTML5 is in common use including <video>, Windows XP should be regarded similarly to how Windows 98 is today: as an ancient relic not worth worrying about. Exposing MediaFoundation to the browser would get you hardware accelerated codecs for H264, MPEG-4 ASP, MPEG-2, and VC-1 on Windows 7, which should be quite common by the time this starts being regularly used. I don't really agree that codec hell is a problem here, except for those pushing Theora because the odds of MS providing that in the base Windows install are pretty much nil.

On another note, I agree entirely with mpat and TSTM. Theora has no commercial hardware to support playback, and that makes it a non-starter in my book. HTML5 isn't just for powerful desktops and laptops, it's also for phones and netbooks, which commonly have hardware acceleration for H264 in place and lack the CPU power to decode video without it (any netbook powered by Ion or Intel's Poulsbo chipsets, which should be quite a few going forward, has H264 hardware decode).

Also, Google's quality complaint holds water IMO. Theora is a VP-3 derivative, and that codec was pretty much garbage to start with. Recent tests with it have not impressed me, and the patent wall sounds very real given how many patents that currently apply to H264 are broad-reaching and could easily apply to an improved theora. Not only that, but at least with H264 there's an organization that claims to have all the patents in order available on RAND terms, Xiph just claims there aren't any on Theora (which might be true now, but with future enhancements to make it competitive with more modern codecs that may not be true). There are also OSS implementations of H264 that work very well, on both the encoding and decoding end (x264 and libavcodec specifically), patents aside.

On another note, I agree entirely with mpat and TSTM. Theora has no commercial hardware to support playback, and that makes it a non-starter in my book. HTML5 isn't just for powerful desktops and laptops, it's also for phones and netbooks, which commonly have hardware acceleration for H264 in place (any netbook powered by Ion or Intel's Poulsbo chipsets, which should be quite a few going forward, has H264 hardware decode).

Which is why there needs more then one encoding.

Even with hardware acceleration options H.264 is unacceptable for the majority of consumer handheld electronics. Websites targeting small devices won't be able to use that any more then Theora.

Also you have the entire Chinese nation to take into account here. There are plenty of chinese electronics out there that only support "alternative" codecs because the per unit licensing is to high of a burden for their target price.

quote:

Also, Google's quality complaint holds water IMO. Theora is a VP-3 derivative, and that codec was pretty much garbage to start with. Recent tests with it have not impressed me, and the patent wall sounds very real given how many patents that currently apply to H264 are broad-reaching and could easily apply to an improved theora. Not only that, but at least with H264 there's an organization that claims to have all the patents in order available on RAND terms,

Patent debates (of this nature) are a red herring. Mpeg won't protect you from submarine patents any more then Xiph will. And it's likely that any that will apply to Theora will apply to multiple other codecs.

I believe that Microsoft has ran into problems in the past with Mp3 and them paying one group of holders, but ended up having a separate group coming after them.

Originally posted by drag:There is one reason and one reason alone that Apple refusing to bundle support.. its because they want to see Ogg Theora fail. I am sure that they have their own reasons for that, but it's not complicated to understand the motivations.

I agree but to me, that seems short-sighted. My solution is that I'd use Vorbis or Chromium on Mac instead of Safari. So they'll lose browser marketshare to protect their H.264 lock-in.

Of course I typed that and then though: "How much do they make on Safari vs. how much they make on iTunes". Oh well.

Yes, I had no problem understanding what side the author was on. A biased and thoroughly worthless piece, when Ian Hickson's original was much more balanced, informative, and useful.

You want Ogg to dominate? Get the System-on-a-Chip (SoC) manufacturers to support it as well as they have H.264. That will give Apple a way to support it on the iPhone, and their reluctance will fade away. They aren't going to support a technology that is not only technically inferior but would also hamstring one of their flagship platforms.

Improving the quality (which is substantially inferior to H.264 at the same bitrate) and clearing up the patent landscape (EFF and similar) would also be beneficial to everyone involved.

Originally posted by drag:Also you have the entire Chinese nation to take into account here. There are plenty of chinese electronics out there that only support "alternative" codecs because the per unit licensing is to high of a burden for their target price.

Name one that supports Theora. A single one. From what I recall not even the ridiculous supports-everything hardware like the Popcorn Hour boxes support Theora.

If you want to talk about Red Herrings, bringing China into the debate is a hilarious way to do it. Typically they'd still use H264, they just wouldn't pay for it. Just like they did with MPEG-1 (VCD format stuff) and MPEG-2 (DVD).

Oh and as for Apple not wanting Theora, the only reason I could think of is that they own patents that may still apply to the mp4 file container. The mp4 container is basically a derivative of the Quicktime mov format, for which they have extensive patents. This also probably makes it a bit cheaper for them to support H264. There also may be a bit of typical-Apple not-invented-here syndrome going on too, but I'm not that sold on the idea that Apple is out to undermine Theora, more that they just think that H264 is a better solution (and they're right, money aside).

Originally posted by drag:* People who purchase Apple products need to call up and tell Apple to quit being such f*kers over this whole thing.

Ogg Theora is acceptable, not ideal. It is freely available under favorable licensing terms and most of the work is already done. It's a FREEBEE for Apple support. They don't have to do anything much, it costs nothing, and it will have no negative effect for end users.

There is one reason and one reason alone that Apple refusing to bundle support.. its because they want to see Ogg Theora fail. I am sure that they have their own reasons for that, but it's not complicated to understand the motivations.

It won't be very free to implement if someone comes after them for patent infringement. And of course there is the lack of hardware support for Ogg which pretty much rules it out for mobile devices. It isn't some sort of vested interest, it is simply that H.264 is much better suited to the areas that Apple and Google work in. Mozilla are really just producing a desktop browser. Apple and Google are not just producing both desktop and mobile browsers, but also entire mobile platforms.

As mentioned in the article and a couple comments, "submarine patents" are a concern for either codec. So really that does not lend weight to either side of the argument.

The lack of HW support for Theora is indeed an issue, but I am 100% sure that it would be resolved if Theora became the standard for HTML5. Yes it would suck for people who have one of the current mobile devices that only support h264, but that is often an issue for early adopters.

Giving the MPEG LA consortium carte blanche over licensing fees on all web video seems to outweigh all of the other issues in my mind.

Originally posted by bradwjensen:Screw Apple if they don't want to use an Quality / Free / Open Source / Open Standard format that anyone can implement and improve for free.

You're wrong because H.264 is not Apple's. Apple played a part in developing the standard yes and it makes sense that they would have a vested interest in getting H.264 as the standard but it is not their standard in any more way than AAC is their standard.

H.264 makes the most sense because it is available everywhere from YouTube on the iPhone to Blu-Ray and various portable movie players.

I've seen the quality of Ogg-Theora video (admittedly much can change in the year that I last looked at the format) and I don't think the quality is that great. YouTube already sucks now because of the pathetic quality that is being distributed under Flash. However, I agree that for those that want to support Ogg-Theora there should be no reason to not allow that support by way of multiple links.

However I fully support Google. They're saying 1 billion videos a day. Think about that. To have to support Ogg-Theora simply because FireFox and Opera don't want to pay a licensing fee would mean that Google would have to have at least 2 billion videos available, one for every browser that supports H.264 and one for every browser that supports Ogg-Theora. It's a logistical thing for Google more than anything and I can see them going H.264 due to the proven streaming capabilities of the format as well as the fact that their video is already all in H.264.

Does anyone know if any work has been made on getting Ogg-Theora decoders and encoders to work well on GPUs or portable media players? However good Ogg-Theora may be, it won't win any more followers unless it resolves the existing concerns. Being smug is no a concern resolving approach.

As to submarine patents, is there no way to get the patent office or the justice department to make a call for patent holders that might have submarine patents to come forward, or forfeit any claims in the future? I know I am probably being a little too optimistic.

Whatever are the merits of the two CODECs, I think we can all agree that Chris DiBona is a complete dick for suggesting that a 5% increase in YouTube bandwidth consumption would bring down the Internet. That is such a dumb thing to say, it would only be said by some corporate PR tool.

Originally posted by drag:* People who purchase Apple products need to call up and tell Apple to quit being such f*kers over this whole thing.

Ogg Theora is acceptable, not ideal. It is freely available under favorable licensing terms and most of the work is already done. It's a FREEBEE for Apple support. They don't have to do anything much, it costs nothing, and it will have no negative effect for end users.

There is one reason and one reason alone that Apple refusing to bundle support.. its because they want to see Ogg Theora fail. I am sure that they have their own reasons for that, but it's not complicated to understand the motivations.

It won't be very free to implement if someone comes after them for patent infringement. And of course there is the lack of hardware support for Ogg which pretty much rules it out for mobile devices. It isn't some sort of vested interest, it is simply that H.264 is much better suited to the areas that Apple and Google work in. Mozilla are really just producing a desktop browser. Apple and Google are not just producing both desktop and mobile browsers, but also entire mobile platforms.

Ya well. Most mobile devices can't support H.264 either.

Remember:

Most Mobile Devices != IPhone.

And I am advocating supporting multiple codecs. There is absolutely no good reason why H.264 and Theora should be mutually exclusive.

quote:

It won't be very free to implement if someone comes after them for patent infringement.

Its just as likely that H.264 is violating unknown patents as Theora.

There is ZERO protection that Mpeg will offer you if a third party decides that their patent covers H.264 or any other mpeg-originated codec.

quote:

Name one that supports Theora. A single one. From what I recall not even the ridiculous supports-everything hardware like the Popcorn Hour boxes support Theora.

In what world does "Not support H.264" means automatically "Supports Theora"?

I never said anything of the sort.

I AM saying that H.264 only does a slightly better job covering the needs of mobile devices vs Theora. Most current devices don't support H.264 either, there are already "standards" for encoding that are supported by almost all Cell phones capable of video playback and the like support.

The chinese have their own oddball codecs that they use... All of them suck, but that's besides the point. The point is that for them they had to go that route because Mpeg is too expensive for them.

The patent argument is unpersuasive. Patents are written to be as broad as possible, and often accepted with light scrutiny. Undoubtedly, some patent covers portions of Ogg. If the Ogg people haven't been sued, it's only because the target wasn't big enough. While it's true that the proposed licensing scheme by the H.264 people is idiotic, no business will pursue a route that will cause them to be dogpiled by heavy-hitters. I've no doubt that under pressure, the terms will rapidly become more reasonable.

If the linked comparison in the article is representative, Ogg's video component is noticeably inferior. Why bother with an inferior codec?

Because it's not that worse and is vastly cheaper to implement.

If your a big player and you have lots of money and bandwidth costs are going to your principle concern then absolutely H.264 has big advantages.

However the trade off is that your under licensing restrictions for the entire workflow. From encoding tools to your browser to anything that must encode or decode the format.

These restrictions after the ability for you to redistribute your software. Thus open source applications suffer hugely... I can take a OSS application and include H.264 support and pay for the licensing, but if you get it from me you are not free to improve the application and/or redistribute it.

Theora gives you a surprisingly big and undervalued feature. This feature is:

"I don't have to give a shit"

You don't have to worry about encoding stuff over twelve minutes, you don't have to worry about your clients.

Remember that currently while video usage is pretty restricted to "just playback" of news and homemade videos (for the most part) the HTML5 standard is designed to make doing web applications easier.

This means that you can inject video as part of your website.

The video tags work in the same way as image tags are. They can be used to create elements of your website. Behavior and appearance can be modified through the use of Javascript, html, and CSS. Things like having your video have user-created objects injected into the video, compositing, modifying colors, modifying look, modifying behavior, elements of games, etc etc.

This is for much higher level of utility then just "youtube" style videos.

-----------------------------------

Also, I have not seen the following point:

H.264 licensing fees and requirements may change at any time. The updated "phase 2" of H.264 licensing were Mpeg starts charging money for people who currently don't have to pay much has not happened yet.

If there is a competing standard to H.264 in the form of Theora then that could have a very favorable effect on licensing pricing.

Its the difference between mpeg having a monopoly status over web video vs mpeg having to compete with a no-cost alternative.

Far too much of the criticism of Theora is either laughably easy to disprove or so vague as to almost qualify as FUD. For instance, Apple is complaining about the potential for "submarine patents" although the techniques used in H.264 are newer and less well-understood, oh and they just so happen to be part of MPEG LA as well.

They're also complaining about the lack of decoder chips, which is a chicken and egg problem that would easily be solved if it was standardized on.

Chris DiBona's quote has ALREADY been shown to be false or at least misleading. Theora nearly matches or beats youtube's implementation of H.264 at the same bandwidth and is still improving.

Also, while we're at it..who all are "some"? Because if "some" say that Theora can't be improved without running into patent problems, it would be useful to know who those some are. Otherwise there is nobody to rebut.

I have never seen DVD quality Youtube videos. Why are they worried about the quality of mobile devices watching youtube? Apple, Google, and Microsoft have too much vested in the mobile market to be free from bias here.

Ideally, Ogg Theora would be the perfect codecs and thats what everyone would use. In real life, however, thats not the case:

1) Little to no hardware support. This is a killer. It means devices like the iPhone cannot display it. Even worse is that there are no SoCs that support it, and I haven't heard of any companies planning on creating SoC's to support it.

2) Submarine patents are as much of an issue for Ogg Theora. The difference is that MPEG LA has hundreds of licencees (including Microsoft, Apple, Google, Dell, etc..) who will all pool in money for a patent fight. No one who has such a patent will want to go to court, but will instead settle quietly in discussions, before ever threatening legal action. That is not currently the case with Ogg, and it will be a while to develop as formidable a support as MPEG LA.

Apple and Google are doing the right thing, as are Mozilla and Opera. The only thing this issue shows is that different codecs are good for different needs, and the whole idea of standardizing on one was flawed to begin with.

I'm with Apple. What's the point in implementing an inferior codec that's useless to most devices? Even – especially – if ogg takes off in desktop browsers, it perpetuates the current situation we have with Flash et al: mobile video isn't possible because everything is done on the CPU. Hardware acceleration is clearly the way forward for mobile devices, so to implement technologies that preclude that, even as a backup, relegates them needlessly to being second-class citizens.

[Edit]: Not to mention, even if claims about ogg's decent quality are true... the lack of quality tools for encoding, hardware support for decoding and playback out-of-the-box on most devices (or even at all, on many) is inescapable.

It's the usual Apple way: something crappy is *worse* than nothing, especially if its adoption now will get in the way of something potentially better in the future.

FWIW, perhaps a pragmatic way to limit the damage from fragmentation of video capabilities would be to require AT LEAST ogg or h.264; that satisfies all parties' demands, and puts a cap of 2 on the number of files needed to guarantee playback in an HTML5-compliant browser. It doesn't improve the status quo, but it prevents any future deterioration.

Originally posted by arcadium:2) Submarine patents are as much of an issue for Ogg Theora. The difference is that MPEG LA has hundreds of licencees (including Microsoft, Apple, Google, Dell, etc..) who will all pool in money for a patent fight. No one who has such a patent will want to go to court, but will instead settle quietly in discussions, before ever threatening legal action. That is not currently the case with Ogg, and it will be a while to develop as formidable a support as MPEG LA.

There is ZERO advantage of H.264 over Theora when it comes to submarine patents.

* There is absolutely no chance that MPEG will pay your legal fees or licensing fess if a third party comes after you.

Microsoft, Apple, et al will NOT fight the patents. They will simply roll over and pay the licensing fees like they currently do. If they are not fighting the licensing fees now it is not likely they would fight additional fees in the future.

The fees, as with all software patent licensing, will cost Microsoft, Google, Apple less then fighting the software patent and winning.

That is if Microsoft and Apple or whoever decide to fight the patent and they win the patent lawsuit and caused the patent to be invalidated... then that means that they would of spent more money then if they just paid the fees in the first place.

This is how software patents work. It's all done on purpose.. the better the patent the more it costs. The lawyers who own the patents are very careful about pricing so that they have very little risk in getting their patents invalidated.

Not to mention that invalidating patents is VERY difficult. Microsoft, I believe, on multiple occasions spent millions of dollars trying to invalidate patents and failed.

* It is VERY unlikely that some unknown codec patent will affect JUST Theora. More then likely a patent will affect mpeg licenses also.

* If Theora is adopted as a standard then Apple, Microsoft, Google, etc etc. will have a investment in Theora also.

This point is pure nonsense. It sounds good on the surface.. that you pay one group money and it should protect you... but it does not. That is simply not how patents work and I contend that paying money to mpeg will not protect you in any way. After you pay the money you are no better off, and no better protected then if you just used Theora from the start.

I believe the real comment on Microsoft's support for video is two-fold.

1) They have not said anything at all, but this is not unusual for them. In fact, they rarely show for meetings, return email or phone calls, according to Ian Hickson, I believe.

2) The video tag would conflict with Microsoft's plans for controlling the internet with Silverlight. In no way will they allow anyone send video and audio over the net without you purchasing something from them.

Of course, Microsoft has always held back the web from moving forward and, on this point, things are no different.

Wait, we're still letting Opera think they're relevant with their miniscule market share? They've gone through many years and many versions and they still haven't gone anywhere significant. Who cares what they think? They'll be dead in five years, ten if you want to be conservative. The market is now between MS, Apple, Google and Mozilla.

Sure, OGV may not be the best codec out there, but it's still scalable to H.264 and I believe by the time HTML 5 comes out OGV will be as good. So, I'm casting my vote for OGG on HTML 5 if they let us vote lol.