Google support aside, WebM carries patent risks from MPEG LA

During the Google I/O developer conference last week in San Francisco, the search giant launched WebM, a new project that aims to deliver a royalty-free multimedia format for the Web. A key component in WebM is VP8, a high-performance video codec that Google obtained in its recent acquisition of video technology company On2.

Google is making VP8's underlying intellectual property freely available under permissive terms in order to promote adoption of the format, but there are still some unanswered questions about its legal status. In a statement given to the Wall Street Journal, a representative of MPEG LA recently suggested that its members and other parties may hold patents on some of the technology that is used in VP8.

MPEG LA is a licensing consortium that has established a patent pool for various multimedia technologies and compression algorithms. The group has a large and diverse roster of members that includes some of the largest hardware and software companies in the world. They set the licensing terms for H.264, a widely used video format that could see its ubiquity threatened if a royalty-free alternative were to gain broad traction.

The underlying principles of video compression are heavily patented by a multitude of parties. In order to build a product, companies have to obtain licenses from all of those parties. Without patent pools, that process would likely be prohibitively costly and impractical. The purpose of licensing groups like MPEG LA is to offer a more streamlined and predictable path for getting the necessary licenses.

Although patent pools are convenient for commercial software vendors, software patents pose some obvious problems for the Internet. Patent-encumbered software isn't freely redistributable, which makes it antithetical to the inclusiveness of the Internet. The success of the Internet and the innovation that it encourages has historically been predicated on openness—a characteristic that is threatened by the kind of undesirable barriers to participation that patents inevitably erect.

In order to address the need for an unencumbered codec that is suitable for widespread use in standards-based Web video, Google opened VP8 and is making the underlying technology available under permissive terms. Unfortunately, it is still unclear at this time whether VP8 infringes on patents held by other parties.

MPEG LA says that it is exploring the possibility of establishing a patent pool for VP8 licensing, much like the one that exists today for H.264. Such a process would involve bringing together all of the companies that own patents that cover the codec and negotiating collective licensing costs. That move would effectively eliminate WebM's status as a royalty-free format, undermining its relevance on the Web.

Some critics of VP8 contend that its design is sufficiently similar to H.264 to warrant concern. One such critic is Jason Garrett-Glaser, a software developer who works on x264, a well-known open source implementation of H.264. In a lengthy analysis of VP8, he attacks On2's claim that the format is superior to H.264 and says that the format's legal status is too dubious for companies to trust.

"With regard to patents, VP8 copies way too much from H.264 for anyone sane to be comfortable with it, no matter whose word is behind the claim of being patent-free," Garrett-Glaser said. "This doesn't mean that it's sure to be covered by patents, but until Google can give us evidence as to why it isn't, I would be cautious."

One specific characteristic of the codec that Garrett-Glaser considers particularly prone to patent risks is its handling of a feature called intra prediction. He accuses On2 of cribbing the technology from H.264.

"VP8's intra prediction is basically ripped off wholesale from H.264," he wrote. "This is a patent time-bomb waiting to happen. H.264's spatial intra prediction is covered in patents and I don't think that On2 will be able to just get away with changing the rounding in the prediction modes."

Despite these concerns, Google says that it has done its own detailed analysis to vet VP8. The search giant is confident that the format will withstand legal scrutiny. Google has had a considerable amount of time to do the legwork since the announcement of the $106 million On2 acquisition last year. It's also unlikely that the company would have put so much cash on the table without doing some relevant analysis first.

That said, it's worth noting that Google's track record isn't spotless. The company's Android mobile operating system is facing patent threats from rival Apple, but Google has not intervened to aid the Android adopters like HTC that are embroiled in costly litigation. There is some risk that WebM's early adopters, including Mozilla and Opera, will be forced to defend themselves if the patent trolls come knocking. This risk could deter some browser vendors and industry players from putting their weight behind Google's open video codec.

Patent holders are not compelled to defend their intellectual property in a timely manner, so the risk could be drawn out for years. It's possible that patent holders will sit back in the shadows and wait for the format to gain significant traction before they start enforcing their IP. If they wait until there is a large audience of heavily dependent users, they can get more licensing revenue from patent royalties.

Unfortunately, it could be difficult for Google to provide any unambiguous assurances about VP8's patent status. Under US patent law, companies can be forced to pay triple damages for "willful" infringement—cases in which it can be demonstrated that the company was previously aware of a patent that it infringed. As such, publicly discussing specific patents can dramatically increase a company's exposure to liability.

MPEG LA's threats at this stage appear to be little more than self-serving saber rattling, but others who have analyzed the technology seem to believe that there could be serious patent risks on the horizon. The challenges that Google faces in attempting to deliver an unencumbered video format are emblematic of the broader problems with software patents, which critics say are detrimental to innovation.

While the x264 devs certainly know their stuff when it comes to videocodecs, Garrett-Glaser's post had about as much intellectual honesty as the Matroksaguy who ripped on Ogg a few months ago.

The post dismisses VP8 for its similarities to H264, and then goes rightahead and dismisses it again for its differences; the basic point isthat VP8 is a bad rip off of H264. But then at the end, we find outthat VP8 actually predates H264, which negates much (though certainlynot all) of the previous criticism. This contradiction is not evenacknowledged in the summary.

As for the legal analysis, it sounds like it was written by a retardedmonkey (as the author claimed VP8's assembly code was). He rips on VP8for not including B-frames, even though he acknowledges that there wasno way to do so without violating numerous patents. And where VP8 makessubtle changes to intra prediction to (hopefully) avoid patents, hestill blindly asserts that it is a "patent time-bomb".

All in all, the post is a great technical polemic, but written from sucha biased perspective that I couldn't really take it seriously...

Without something like MPEG LA, it would be very difficult to develop and market any video standards because it would require that YOU negotiate with every patent holder. MPEG LA allows you to negotiate with a single entity. This does not address what happens if that organization abuses their market position, as Nero is stating in their antitrust suit against the MPEG LA.

"It's possible that patent holders will sit back in the shadows and wait for the format to gain significant traction before they start enforcing their IP. If they wait until there is a large audience of heavily dependent users, they can get more licensing revenue from patent royalties."

This, if anything, needs to change. Too often is this the case where the only way you know you are infringing is by success.

Without something like MPEG LA, it would be very difficult to develop and market any video standards because it would require that YOU negotiate with every patent holder. MPEG LA allows you to negotiate with a single entity. This does not address what happens if that organization abuses their market position, as Nero is stating in their antitrust suit against the MPEG LA.

As for the legal analysis, it sounds like it was written by a retardedmonkey (as the author claimed VP8's assembly code was). He rips on VP8for not including B-frames, even though he acknowledges that there wasno way to do so without violating numerous patents. And where VP8 makessubtle changes to intra prediction to (hopefully) avoid patents, hestill blindly asserts that it is a "patent time-bomb".

Given that they are just about to get sued...it seems his assumptions were correct.

As for the legal analysis, it sounds like it was written by a retardedmonkey (as the author claimed VP8's assembly code was). He rips on VP8for not including B-frames, even though he acknowledges that there wasno way to do so without violating numerous patents. And where VP8 makessubtle changes to intra prediction to (hopefully) avoid patents, hestill blindly asserts that it is a "patent time-bomb".

Given that they are just about to get sued...it seems his assumptions were correct.

It's interesting that Ars chose to mention Apple's suit against HTC without mentioning that Android also infringes on Microsoft patents which HTC licensed (presumably to avoid a lawsuit). Just sayin'

With regard to H.264 it already is open: there are open source implementations as is pointed out in the article (ffmpeg, x264). The spec is available to anyone who wishes to work on their own implementation. It's been adopted across the entire ecosystem of video technology from satellite broadcasting to cell phones and everything in between (the first time that's been true). Most importantly (to me) it's technically superior.

The patent situation is entirely orthogonal to the openness question. Google could buy a dozen On2s and open a dozen VP8s and unless one of them uses some completely different technique from H.264 they're all going to infringe.

I wish we could all just settle on a single codec instead of fragmenting the market. I recognize that this poses a slight problem for a few projects, namely Mozilla, but they have a clear path that they simply choose to ignore: use the H.264 implementation provided by the host OS. On Linux that will mean getting the codec from a project that resides in a country that doesn't have a software patent problem.

The cynical side of me suspects that this is exactly what Google wants and that WebM is just a stick to wave at MPEG-LA. Which is fine and I hope it works but hopefully not too much energy is wasted on what turns out to be a PR move.

As for the legal analysis, it sounds like it was written by a retardedmonkey (as the author claimed VP8's assembly code was). He rips on VP8for not including B-frames, even though he acknowledges that there wasno way to do so without violating numerous patents. And where VP8 makessubtle changes to intra prediction to (hopefully) avoid patents, hestill blindly asserts that it is a "patent time-bomb".

Given that they are just about to get sued...it seems his assumptions were correct.

That doesn't mean anything... it's possible that certain patent holders are reading the controversy (like this article) and just blindly suing in the hopes of the lawyers digging up something that can hold up in court.... until some documents come out detailing what anyone's suing for, it's just schadenfraude.

Why? Do tell why all those companies who made the tech behind MPEG deserve to have their worked copied without compensation.

I'm still waiting for Microsoft and Apple to put out the call for royalties on H.264 to be entirely abandoned and create a GNU license for it. They're two companies who should have an interest in advancing web standards over making a little cash (especially Apple, I imagine they're likely to pay more in licensing fees than they get back in their share of the royalties), yet we rarely hear a peep out of either company that they stand to make money off it.

Its funny that these articles don't mention Jason's earlier post where he gripes about the whole patent thing with H.265 (the next one)...

Quote:

This has led me to suspect that there is a great deal of cheating going on in the H.265 proposals. The goal of the proposals, of course, is to pick the best feature set for the next generation video compression standard. But there is an extra motivation: organizations whose features get accepted get patents on the resulting standard, and thus income. With such large sums of money in the picture, dishonesty becomes all the more profitable.

The Main profile is from the initial H.264 specification, which was finalized in mid 2003. The High profile was added in 2004. Which makes Garrett-Glaser's implication that VP8 is a poor copy of H.264 High profile quite specious...

> Garrett-Glaser said. "This doesn't mean that it's sure to be covered by patents, but until Google can give us evidence as to why it isn't, I would be cautious."

It is impossible for Google to 'give' anybody evidence as to it not being covered by patents, other than for them to preemptively sue the holder of EVERY patent (and patent-pending) in court, and have a judge declare it to non-infringing. Then repeat in every juridiction that permits patents on software, and perhaps even otherwise (as it could be implemented in hardware as well).

Once all these lawsuits have come to completion, then Google would have evidence that they aren't covered by patents.

Of course, EVERYTHING ELSE anybody produces/creates is under the same threat, that somebody else may happen to decide that what you produced is covered in whole or in part by something they think is unique but is actually remarkably obvious to people in the industry.

"It's possible that patent holders will sit back in the shadows and wait for the format to gain significant traction before they start enforcing their IP. If they wait until there is a large audience of heavily dependent users, they can get more licensing revenue from patent royalties."

This, if anything, needs to change. Too often is this the case where the only way you know you are infringing is by success.

This is an excellent question. What purpose does not aggressively enforcing your patent do for... well, anyone, really? Outside of patent trolls, there's no useful function to this loophole and it seriously needs to be closed...

People suing at this juncture one can at least respect for defending their intellectual property, even if we, as proponents of an open internet, disagree with the principle. Submarine patent trolling, on the other hand, is a bane to EVERYONE. Obviously someone influential doesn't want this hole closed or it would have been already. Anyone care to enlighten us as to why?

Even with a 'real property' analogy, this can be debunked. In many areas (not all, but some), one can successfully acquire land and property if one invests the resources and time to develop it due to the actual owner's neglect or lack of said resources. If pressed, it can be taken to court and successfully argued if you have a good enough lawyer and argument... why can't this happen with patents? If someone can claim significant development of a particular resource, where no one else is developing it, they get to keep the patent? Well, not exactly this, as I see obvious problems with it as stated, but something like it to prevent people from sitting on their laurels while a patsy does all the work so that they can then release the lawyers to get a free paycheck... that is so not free market.

> Garrett-Glaser said. "This doesn't mean that it's sure to be covered by patents, but until Google can give us evidence as to why it isn't, I would be cautious."

It is impossible for Google to 'give' anybody evidence as to it not being covered by patents, other than for them to preemptively sue the holder of EVERY patent (and patent-pending) in court, and have a judge declare it to non-infringing. Then repeat in every juridiction that permits patents on software, and perhaps even otherwise (as it could be implemented in hardware as well).

Once all these lawsuits have come to completion, then Google would have evidence that they aren't covered by patents.

Of course, EVERYTHING ELSE anybody produces/creates is under the same threat, that somebody else may happen to decide that what you produced is covered in whole or in part by something they think is unique but is actually remarkably obvious to people in the industry.

"It's possible that patent holders will sit back in the shadows and wait for the format to gain significant traction before they start enforcing their IP. If they wait until there is a large audience of heavily dependent users, they can get more licensing revenue from patent royalties."

This, if anything, needs to change. Too often is this the case where the only way you know you are infringing is by success.

This is an excellent question. What purpose does not aggressively enforcing your patent do for... well, anyone, really?

There are some valid cases where you might not want to enforce a patent right away, but need to reserve the ability to do so later under certain circumstances. Defensive patents are a very important example, like if you just want to hold on to a patent and only use it to establish a cross-licensing agreement when somebody else tries to sue you for infringing one of their patents.

Apple is part of the MPEG patent pool, but they didn't want to be. They were forced into it (on threat of lawsuits). Then they ended up in bitter fights with the group over licensing terms.

Microsoft had similar problems when it was sued over VC-1 and it became part of the consortium.

So the question is whether Apple and Microsoft consider Google a bigger problem than they consider MPEG-LA.

Microsoft and Apple joined together long ago to come up with TrueType. Before that if you wanted scalable fonts on your PC you had to install resource hogging Adobe software on your machine. Throwing their support behind an open alternative to MPEG would be a similar move.

Don't complain about MPEG-LA, the existence of that license association is due to software patents, the real source of the problem. The patent systems purpose has been contorted to help big companies to grow rather then help innovators, same goes for copyright. At this point I'm not sure we can really reform the system, and even if it were possible the big companies are the ones with the money and we know it's hard to pull congress away from the money machine.

Without patent pools, that process would likely be prohibitively costly and impractical.

Good, i'd prefer it that way, maybe then these companies would do something to stop this ridiculous crap rather than just banding together to pay each other.

So you would like to still be using a VCR?

False choice

Why? If no one can produce a new product because of the patent negotiations required, you will be stuck with old tech. You may come up with a fantastic video encoding scheme but I have to negotiate with 14 different companies in order to produce your tech, then why should I? And if no one can produce a product with your wonderful encoding scheme then why should you go through the cost of developing it?

With regard to H.264 it already is open: there are open source implementations as is pointed out in the article (ffmpeg, x264). The spec is available to anyone who wishes to work on their own implementation. It's been adopted across the entire ecosystem of video technology from satellite broadcasting to cell phones and everything in between (the first time that's been true). Most importantly (to me) it's technically superior.

If it's already open, why is Google bothering with WebM? Can someone explain the interplay between H264 and x264?

... as the techniques behind VP8 were INDEPENDENTLY DEVELOPED, and should therefore not be liable for suing in a fair world.

Independent development doesn't mean squat.

And this is why the patent system is far, far worse than the copyright system. Stopping people from directly copying your work is one thing. Prohibiting people from marketing a valid alternative is another entirely.

With regard to H.264 it already is open: there are open source implementations as is pointed out in the article (ffmpeg, x264). The spec is available to anyone who wishes to work on their own implementation. It's been adopted across the entire ecosystem of video technology from satellite broadcasting to cell phones and everything in between (the first time that's been true). Most importantly (to me) it's technically superior.

If it's already open, why is Google bothering with WebM? Can someone explain the interplay between H264 and x264?

x264 is an open-source implementation of the published h.264 specification. Using x264 doesn't absolve the user of need to obtain a license for the use of the patents covered by h.264.

Without patent pools, that process would likely be prohibitively costly and impractical.

Good, i'd prefer it that way, maybe then these companies would do something to stop this ridiculous crap rather than just banding together to pay each other.

So you would like to still be using a VCR?

False choice

Why? If no one can produce a new product because of the patent negotiations required, you will be stuck with old tech. You may come up with a fantastic video encoding scheme but I have to negotiate with 14 different companies in order to produce your tech, then why should I? And if no one can produce a product with your wonderful encoding scheme then why should you go through the cost of developing it?

You already have to negotiate with at least someone, usually someones, with patents already in the open. Then you have to worry about the 14 companies with overly broad patents lying in wait for you. And even if you do come up with a fantastic new encoding scheme, completely independently and without clear historical precedent, you *still* have to worry about the trolls. If you're a startup, the legal fees alone can be business-breaking.

These questions would only make sense in a world without open source development (and profit from it), countries with limited or no software patents, and companies that patent only defensively. It remains a false choice.

With regard to H.264 it already is open: there are open source implementations as is pointed out in the article (ffmpeg, x264). The spec is available to anyone who wishes to work on their own implementation. It's been adopted across the entire ecosystem of video technology from satellite broadcasting to cell phones and everything in between (the first time that's been true). Most importantly (to me) it's technically superior.

If it's already open, why is Google bothering with WebM? Can someone explain the interplay between H264 and x264?

x264 is an open source implementation of the H.264 video codec. Open source generally means that from a copyright perspective, you can distribute a work freely as long as you abide to certain terms. However, just because something is open source doesn't mean that it isn't covered by patents held by another party. Since there are a large number of patents covering the H.264 codec, you have to pay patent royalties to MPEG-LA to distribute and/or use x264 in jurisdictions in which software patents are enforceable.

So H.264 is open in the sense that the specs are freely available and there is an open source implementation, but not in the sense that it can be freely distributed without running afoul of patent laws. If Google's claims are correct, WebM does not have that problem (there are patents covering its implementation, but if Google is to be believed Google owns those patents and is licensing them freely to those distributing/using WebM implementations). Of course, there's no guarantee that Google's claims are in fact correct.

There's a small chance that the result of the Bilski case will make all the software patents go away (in the US anyway), but it seems somewhat unlikely they'll be banished completely. We should find out sometime in June.

... On Linux that will mean getting the codec from a project that resides in a country that doesn't have a software patent problem. ...

This is the key right here. If the solution is to pull a Pirate Bay and just offshore our way around patent infringement, we've picked the wrong patent system or the wrong patent holder. Probably both.

Doesn't On2 have its own patent(s) ? I mean it has been in the codec field since the 90s; so H.264 must be infringing on some of its patents ? Or maybe I'm missing something :|

The MPEG LA is supposed to be a front for, in theory, all the patents involved with H.264. Presumably, any patents On2 has to H.264 have already been determined and are licensable through MPEG LA with On2 being distributed the relevant revenue.