In the WebRTC standardisation post I mentioned that one of the controversial discussions in the IETF context was the mandatory to implement (MTI) video codec battle for WebRTC. While there are some technical arguments on the topic (i.e this VP8 vs H.264 – subjective evaluation and this performance comparisons discussion), there is no dispute both are high quality and efficient video codecs. The issue here is all about IPR and licensing as described in this interesting and ongoing discussion: “VP8 vs H.264 – the core issue“.

Why a mandatory WebRTC video codec matters

Some of you might be wondering why the IETF is having such difficulties reaching consensus on this topic. To answer this, let’s start by describing the importance/relevance of selecting a mandatory to implement (MTI) video codec to begin with. The short answer is Interoperability. As described in the specifications, the goal of having a mandatory to implement codec is to prevent negotiation failure, not to preempt or prevent negotiation. As discussed in previous posts, codecs in WebRTC are negotiated via de SDP Offer/Answer mechanism, and the lack of a common codec will result in negotiation and session setup failure. Having a MTI codec does not mean this is the only codec that can be negotiated, but it does mean that as long as the two parties conform to the specification the MTI can be used always as last-resort if no other options are available. As one can imagine, the absence of a MTI video codec might easily lead to fragmented deployments with some services using one codec and some browsers using others and no way for them to communicate. So it seems that guaranteeing video interoperability is something good. However, should we force implementors to pay licensing fees to accomplish that? That is the main question here.

As this debate has raged on, the lack of decision in this area has been a blocking issue in field trials and proof of concepts (POC). I’m involved in a number of projects where customers are willing to extend (and interwork) ‘legacy’ videoconferencing services over the WebRTC domain. Most of those ‘legacy’ platforms are based on H.264 codec (just few are ‘upgrading’ to VP8) while Chrome and Firefox only implement VP8 (do you remember this – Google Dropping H.264 Codec from Chrome Browser?). Few of the WebRTC-to-SIP interworking gateways in the market include some video transcoding capabilities. Those that do are mostly based on software with varying degrees of scalability. In most cases video transcoding is just a ‘roadmap item’ as it seems some vendors are waiting for this to settle down before deciding whether to implement it or not. That means for the time being we need to use VP8-capable softphones on the SIP/IMS side if we want to test/demonstrate interworked video sessions. A couple of weeks ago I participated at the Iberian Telecom Summit in Madrid, and one of the main topics of discussion was whether customers were willing to pay for video transcoding in a WebRTC context. The answer was not clear at all.

During the last IETF in Berlin this topic was not on the agenda – if you remember, main discussion there was DTLS-SRTP vs SDES. When I started preparing for the next IETF meeting several weeks ago (which is next week in Vancouver), I wondered whether the IETF would finally address this topic or would keep postponing it. So I decided to send this email to the IETF RTCWEB WG and the answer I got was the following plan, which basically means we’ll try to pick a MTI video codec while in Vancouver. Of course there were voices against this plan with some arguing video codec should not be an IETF issue at all but should be covered by the W3C instead. So far those arguments have led to nowhere. However, I wouldn’t be surprised to hear the same type of arguments at the beginning of the upcoming session. The agenda for the meeting can be found here and, as it can be seen, the video codec discussion will take place on Thursday. If you’re willing to follow the discussion remotely you can use any of the tools listed here.

The combatants

As indicated in the plan, the chairs asked for proposals by Oct 6 2013, there are two proposals on the table:

one advocating for H.264 as MTI codec for WebRTC and

another advocating for VP8 as MTI codec for WebRTC.

The VP8 proposal is only backed by Google, the H.264 proposal is backed by Ericsson, Nokia (or shall I say Microsoft?), BlackBerry, Qualcomm, Orange, Cisco, Microsoft and Apple — note that codecs is the only WebRTC-related subject where Apple has participated and expressed their opinion. Note also that Mozilla is not co-authoring any of those drafts. In the past they did comment on the topic via the mailing list but recently it seems to me they’ve been silent on the topic – if they didn’t, then my apologies as I missed it. Overall, while it seems to me they would be very happy to see VP8 adopted as MTI Video Codec for WebRTC, it is not totally clear to me how unhappy they would be if H.264 was selected instead.

This basically means that H.264 support is only for decoding (relying on underlying OS support) and is not available for WebRTC.In any case, I am wondering whether at some point Mozilla would consider taking advantage of this for WebRTC too. This topic is especially interesting for Firefox OS based devices, where Mozilla is collaborating with Qualcomm, one of the H.264 proponents.

The only ‘browser’ I know of that supports H.264 for pseudo-WebRTC is Ericsson’s Bowser (I did not forget the ‘r’). I am not sure if this is being actively maintained any longer as I’ve seen several emails on the mailing list complaining about it:

1

Looks like it was more ofapush totrytogetH.264supported.Idon'tthink any work has been done since last year.

In any case, none of the customers I’ve seen playing with WebRTC considered Bowser more than a nice experiment. For mobile environments they are currently testing mobile versions for Chrome and Firefox on top of Android. Here’s an example from earlier this year.

The<ahref="http://html5labs.interoperabilitybridges.com/prototypes/object-rtc/object-rtc/info">prototype</a>demonstrates how tosupport<strong>H.264/AVC</strong>video without SDP,by building the appropriate JavaScript code andwithout introducing any changes inthe relevant parts of the ORTC specification.

What the laywers have to say

Going back to the core issue of the matter, the above mentioned IETF drafts include considerations on VP8 and H.264 IPR/licensing.

o<ahref="https://datatracker.ietf.org/ipr/1571/">https://datatracker.ietf.org/ipr/1571/</a> - by Google, declaring that the technology is royalty-free.

o<ahref="https://datatracker.ietf.org/ipr/2035/">https://datatracker.ietf.org/ipr/2035/</a> - <strong>by Nokia, which does not declare a royalty-free license</strong>.

The licensing terms forGoogle'sIPR are available at<ahref="http://www.webmproject.org/license/additional/">http://www.webmproject.org/license/additional/</a>.

The Nokia IPR mentioned above includes IPR that has been asserted inongoing litigation inGermany(Nokiav.HTC,District Court inMannheim,

Germany.7O201/12);on one of the patents,<strong>the court has ruled that the phones inquestion(which support VP8)are notinfringing</strong>.

Asmentioned in<ahref="http://blog.webmproject.org/2013/08/good-news-from-germany">http://blog.webmproject.org/2013/08/good-news-from-germany</a>.html?m=0; the case is still ongoing.

The following companies have asserted that<strong>any IPR relevant toVP8 they might have isavailable forlicensing by Google underaroyalty free license</strong>;the licensing terms are available at<ahref="http://www.webm-ccl.org/vp8/agreement/">http://www.webm-ccl.org/vp8/agreement/</a>, as well as details on the licensors:

to<strong>endusers will never be charged royalties</strong>[<atitle="&quot;MPEG LAs AVC License Will Not Charge Royalties for Internet Video that is Free to End Users through Life of License&quot;"href="http://tools.ietf.org/html/draft-burman-rtcweb-h264-proposal-02#ref-MPEGLA">MPEGLA</a>].Real-time generated content,the content most applicable toWebRTC,was free

already from the establishment of the MPEG-LA license[<ahref="http://tools.ietf.org/html/draft-burman-rtcweb-h264-proposal-02#ref-MPEGLA-License">MPEGLA-License</a>].<strong>License fees forproducts that decode andencodeH.264video remain though</strong>.

tofollow the ISO/IEC/ITU common patent policy[<ahref="http://tools.ietf.org/html/draft-burman-rtcweb-h264-proposal-02#ref-IsoIecItuPolicy">IsoIecItuPolicy</a>],but IPR statements cannot be expected there forstill some time.There isno guarantee that IPR statements inMPEG will be royalty free(option1),but may just aswell be"Fair, Reasonable And Non-Discriminatory"(FRAND,option2).

How will this end?

What will be the outcome of the discussion next week in Vancouver? Who knows, but I’m sure we’ll see some room flooding with folks that never attended before an RTCWEB WG (or even IETF) meeting in order to try to influence the decision. The following are potential options:

VP8 is selected as MTI Video Codec for WebRTC — according to Google it seems pretty much every SoC vendor is now shipping HW VP8 acceleration but, would Apple and Microsoft (incl. IE, Lync, Skype) implement VP8 just because the IETF mandates that codec? Would that be yet another reason for them to not join the standard WebRTC bandwagon?

Both H.264 and VP8 are selected as MTI Video Codecs for WebRTC — not sure this is realistic from the implementation perspective. While I could speculate H.264 proponents might potentially live with this decision, I don’t think the same would be true for the VP8 side. Ideally this would allow browser-to-browser and browser-to-something-else communication without requiring video transcoding.

An older codec whose IPR has expired is selected as MTI Video Codec for WebRTC — so far no one has formally proposed this option in a draft but it would follow a “better-than-nothing” approach. This would basically mean having a last-resort that then let clients upgrade to VP8 or H264 as they see fit. This codec would likely not be as technically as good as VP8 or H.264. It is also interesting to note that codecs could potentially be implemented as a plug-in.

No MTI Video Codec for WebRTC is selected — again, no one has formally proposed this option in a draft, but it would basically mean that Video communication would not be a guaranteed feature in WebRTC — in other words, and as someone put it on the mailing list, in the event that a common Video Codec cannot be negotiated during video session setup, the session should fall-back to voice-only negotiation where there is a MTI (G.711 and Opus). To be frank, I wouldn’t be very happy with this but I tend to think this is the direction we’re heading towards.

I’ll post an update next week after the meeting. Have you already subscribed to our mailing-list, if not please click here as it just takes few seconds to join it. You can also send me an email to victor.pascual.avila@gmail.com or follow me on Twitter at @victorpascual.

Want to keep up on our latest posts? Please click here to subscribe to our mailing list if you have not already. We only email post updates. You can also follow us on twitter at @webrtcHacks for blog updates.

How the 100k units per year should “be adequate for many WebRTC innovators or start-ups to try out new implementations on a large set of users before incurring any patent royalty costs” is a mystery to me. What happens after you try it out? You cannot afford it and you cast it aside. I’m part of a startup and I’m well aware of the fact we wouldn’t absolutely be able to afford the licensing costs of H.264 for the applications we’re building on top of WebRTC (conferencing, streaming, etc.).

The choice of H.264 would only result in a full stop to all of the innovativations and fresh ideas people have brought to WebRTC so far, because the licensing burden would not only apply to us, but to tons of other implementors that are doing great stuff right now.

Most of the H.264 proponents have contributed zero or close to zero when it comes to WebRTC: they may be doing something on the ML, but I’ve seen nothing relevant when it comes to implementations, nice scenarios and so on (and no, I don’t consider Bowser relevant, and I don’t even consider ORTC at all: as if we needed another Silverlight vs. Flash or ASP vs PHP clone). The only stuff I’ve seen done was great browser implementations by Google and Mozilla, and excellent applications by WebRTC enthusiasts that at times pushed the concept beyond what I thought possible. When you don’t write a single line of code, just stay on the side watching, and then eventually try to impose your codec, I can only see one thing: you’re scared you won’t be able to compete based on merits, and so you’re bullying to impose your coded, to make money out of other people work or stop those who can’t afford to pay you, no matter how great their efforts were. This is defending the status quo, something that unfortunayely is not new in the IETF where we all know how strong large companies are: and something I will never accept or tolerate, especially when a technology that could affect, in a positive way, the future of multimedia technology is involved.

If it can’t be VP8, let it be something else (H.263 or heck, even H.261 would work at this point) but I will never accept H.264. Legacy applications are called like that for a reason: WebRTC is moving towards the future, and a rock tied to its foot to keep it from evolving is the last thing it needs.

The quote from the Skype (Microsoft) employee is interesting, but so was the response to that message on the mailing list which pointed out that if Skype (Microsoft) wants that position to be considered they really should submit an IPR declaration to the IETF.

I have a concern about the table showing Google as the only VP8 backer. It is correct that Google is the only big organisation backing VP8. However, I am sure there are many, many, small start-ups (who don’t have the time or money to engage with the IETF) that support VP8 simply because there is no royalty-free option for H.264 at this time (I agree with the sentiments of the first commenter here – the 100k unit thing is just a fig-leaf). I believe that the most innovate use of WebRTC will be from these smaller companies and I do worry that the need to use a royalty-bearing codec will stifle that.

To be blunt, the big organisations that are in favour of using H.264 can afford to switch to VP8 more than smaller organisations can afford to worry about H.264 licensing. Even the administration required to count your H.264 usage (required even if the first 100k units are free) may be too onerous for one/two man organisations with downloadable software products (what do you count, downloads, license keys, API keys – how do you manage all this without cost?).

VP8 might not be royalty free in the future, but that still makes it a far better choice than H.264 which is not royalty free now.

1) I don’t know why any non-browser vendor opinion matters (aka. Cisco, Ericsson, Orange….). So even if that table shows a lot of companies in the right side, Google and Mozilla should be 90% of the decision and Opera, Safari and MS rest 10%.

2) The only future proof approach in my opinion is VP8 as it ensures a simple transition to VP9.

I think you are right – it is about to get really ugly! Thank you, Victor, for your comprehensive overview and good fortunes in Vancouver. I had not quite appreciated for imbalance of those at the table for H.264 until I stared at your table, which will make it a tough fight for a royalty-free VP8/VP9 in WebRTC. Lorenzo Miniero above is right that all the small start-ups and innovators who want licensing-unencumbered video codecs are not represented except through Google. You also point to the considerable FUD floating about – “who knows is VP8/VP9 is unencumbered, we may still be able to get you!”. Google has now spent over $300M trying to “free” all the codecs and code and it is disingenuous for companies to keep turning up at IETF saying “well Google hasn’t negotiated/paid/fought with us yet” within a standards body supposedly committed to “a long tradition of preferring non-encumbered IPR whenever possible, and especially to avoid IPR where using the technology requires making agreements with and payments to third parties as part of the cost of doing business”. Fare forward.

I’d like to make an announcement material to the conversations around MTI video codecs in rtcweb.

Cisco is announcing today that we will take our H.264 implementation, and open source it under BSD license terms. Development and maintenance will be overseen by a board from industry and the open source community. Furthermore, we will provide a binary form suitable for inclusion in applications across a number of different operating systems (Windows, MacOS, Linux x86, Linux ARM and Android ARM), and make this binary module available for download from the Internet. We will not pass on our MPEG-LA licensing costs for this module, and based on the current licensing environment, this will effectively make H.264 free for use on supported platforms.

We believe that this contribution to the community can help address the concerns many have raised around selection of H.264 as MTI. I firmly believe that with H.264 we can achieve maximal interoperability and now, do it with open source and for free (well, at least for others – its not free for Cisco J)

More information on the open source project can be found at http://www.openh264.org, which is sparse now but more coming soon.

All the people working on getting the openh264.org and announcements out over the past few weeks were reading this article but sort of afraid to post anything on given what was coming. Great article – In my mind the big thing that has changed today with respect to this article is that Firefox is going to have both VP8 and H.264.

We don’t think choosing an older codec really works – if you look at the video, it is so awful, no one wants it. Thus we think we have to break the log jam at IETF. H.264 as MTI does that and we hope that browsers support VP8, H264, VP9, H265, Daala and any good codec. If the handful of major browsers have wide codec support, and a common fallback of H.264, many applications can be build that use the codec they want and things will work well.

Give users the choice, but make sure there is a common fallback. And my view is microsoft and apple were never going to do VP8 as the MTI.

I think Cullen makes a key point here – if Microsoft and Apple are ever going to eventually arrive on board the WebRTC train the path of them having to adopt Google-driven VP8/VP9 as the MTI in the standard is probably not the most plausible path. So this is a very interesting response by Cisco to the video codec dilemma. Plus, in some sense, Cisco is putting a challenge on the table here to get Apple to do the same thing – make H.264 easily available in iOS to other apps without further fees.

As Shakespeare would have said, from my point of view it’s “Much Ado About Nothing”.

I don’t see any real value in this Cisco opening. First of all, as per the FAQ you cannot ship the module when you’re installing your application that depends on it: you have to download it on the fly, possibly getting the free license somehow in the process. Far from ideal, especially in case the servers are not reachable for any reason. Besides, as far as I’ve understood the “free” only applies to Cisco patents: the FAQ again says that this does not cover other potent holders tha may assert claims, which is EXACTLY the same situation VP8 is allegedly in now.

I find it unbelievably rude to credit Wikipedia for the painting illustrating your article, where in fact the work of Ilya Efimovich Efimovich Repin, courtesy of the Pushkin Museum. As if I were to credit Saint Gobain for the Chateau Margaux that I drank last night :o).
Other than that, great article !