Using Native WebRTC simulcast support in Chrome (or how to be as good as Hangouts) [WiP]

Some weeks ago Philipp Hancke presented an awesome analysis of how Google Hangouts uses WebRTC in Chrome. In that blog post he explained that based on the SDP contents Google is using simulcast but didn't entered in the details of how to activate it. So I had a lot of curiosity and thought that it could be great if people (beyond Google) could use this feature so I tried to replicate their behavior.

Step one: Add simulcast ssrcs and "SIM" group to the offer SDP

The first thing I tried is to implement some SDP mangling to make my SDP look like the Hangouts SDP. That means adding 3 substreams grouped by simulcast semantics. This is the code of my quick and dirty implementation:

https://gist.github.com/ggarber/a19b4c33510028b9c657

Result: no simulcast :(Step two?: Add a renegotiation

I
saw a a renegotiation in the initial connection from Google Hangouts
(when there are no other participants) and I thought this
was needed to enable simulcast and implemented it but this is not needed to have simulcast.

Step three: Add the x-conference flag to the video track in the answer SDP

The third thing I tried was to set those misterious conference flags in the SDP offer:

You can see this lines in Chrome Canary logs:VP8 number of temporal layers: 3Simulcast substream 0: 960x540@450-900kbps with temporal layersSimulcast substream 1: 1920x1080@600-2500kbps with temporal layersSimulcast substream 2: 3840x2160@600-2500kbps with temporal layers

Bonus track: Apparently from those logs we not only enabled simulcast but also VP8 temporal scalability. I have a snippet to parse the RTP packets and check it but didn't have time yet to confirm it.

Step four: Decide what substreams you are interested in

Based on the RTCP Receiver Reports and PLIs received, Chrome will decide which streams to send from the list of streams you configured in the SDP (in Step one).

PENDING: The logic behind these decisions is not clear yet

Apparently you can optimize also the CPU usage by setting the googSkipUnusedFrames constraint that Google Hangouts uses and that way the video substreams that are not going to be sent won't be encoded either. (To be tested)

Step five: Increase the bandwidth

Chrome changes the number and quality of the different substreams based on the bandwidth estimations. The configuration (resolution, temporal layers) for a given original resolution and available bandwidth can not be changed as far as I can tell.

To increase the number of substreams and quality you have to use the REMBs or maybe periodic RR.

Desperation note: At some point I was so frustrated not being able to get it working that I even tried to run my test page simulating I was behind the google.com domain to see if Chrome was behaving differently.

Get link

Facebook

Twitter

Pinterest

Email

Other Apps

Comments

If you are already in or considering going into the field of early childhood education teaching, make sure you read this article first. Your expectations regarding salary and working conditions may be unrealistic, so it's good to be prepared before having your dreams dashed.http://www.careerpanth.xyz/

Post a Comment

Popular posts from this blog

Bandwidth estimation is probably the most critical component in the video engine of WebRTC. The bandwidth estimation (BWE) module is responsible for deciding how much video* traffic you can send without congesting the network to prevent degradation of the video quality.

In the past bandwidth estimation algorithms used to be very rudimentary and based mostly on packet loss. Basically we used to start increasing slowly the video bitrate until we detected packets being lost. To detect the packet loss you use the standard RTCP feedback mechanisms where the receiver side reports packet loss periodically using RTCP Receiver Report (RR) messages.

Modern bandwidth estimation algorithms are more advanced and try to detect congestion before it is bad enough to make routers discard packets. These algorithms predicts congestion analyzing the delay between packets. The idea is that when you start having some congestion, the buffers in the routers will start filling and the delay will be more va…

There are cases when we would like to limit the maximum bitrate being transmitted by WebRTC to avoid wasting resources in the user endpoints or save money reducing the bandwidth usage in our servers. This is because the maximum bitrate by default in Chrome is around 2Mbps and for many use cases a much lower bitrate provides still pretty good quality. BTW using a lower bitrate can also help with stability of quality in multiparty scenarios by reducing the amount of competition among different streams.

There is no simple API to configure the maximum bitrate in WebRTC (although there is one in ORTC) but there are 3 ways to do this by mangling the SDP.

1. Use the standard b=AS:BITRATE (Chrome) or b=TIAS:BITRATE (Firefox) attributes in the SDP for the audio or video channel[1]
2. Use codec specific attributes (this work at least for opus audio codec with maxaveragebitrate property) [2]
3. Use the proprietary x-max-bitrate attribute in the video channel of the SDP answer. For example w…

It is a common request from WebRTC developers and customers to know how they can differentiate WebRTC traffic from other type in their networks. Usually the goal is to be able to prioritize RTC traffic over other types of less important traffic.

By prioritizing the WebRTC traffic in the edge routers it is possible to improve the quality in some network scenarios. The most common cases where this marking may help are:

Congested broadband uplink where the router can discard other type of traffic instead of WebRTC traffic when queues get full.Congested local wireless network
One obvious way to do this is forcing all the traffic to be relayed through a TURN or SFU server and se the priority based on IP addresses. That's easy and always work, but the problem is that it is very difficult to maintain when your infrastructure changes often (scaling elastically, new datacenters...).

Other way to accomplish this is to use Differentiated Services Code Point (DSCP) marking. With DSCP you c…