YouTube Quality at Lower Resolutions was Awful in my Tests

I’m preparing for my per-title presentation at Streaming Media East next Tuesday (presentation description below) and found some anomalous results for lower-resolution files produced by YouTube. I’m including YouTube in the analysis because I’ve always respected their work and for perspective vis a vis other per-title technologies that I’ll analyze (including Elemental MediaConvert QVBR, Capella Systems SABL, Epic Labs Lightflow, QEncode, and hopefully Bitmovin).

To test all technologies, I uploaded the same set of 21 test clips and then downloaded the encoding ladder produced by each technology, YouTube included. Then I analyzed the clips in the Hybrik Cloud platform, producing a range of metrics like VMAF, SSIM, and PSNR. When reviewing the YouTube scores, I was surprised by the poor quality of lower-resolution clips as measured by Hybrik so I wanted to confirm them with the Moscow State University tool on my local workstation. In this video, I do that and show that the low scores measured in the YouTube clips correspond with noticeably poorer frame quality than the baseline clips encoded using 200% constrained VBR.

Not sure why this would be; if anyone has a clue, please let me know. In all previous analysis, YouTube quality has always been top notch and these results are very surprising.

If you’re in Manhattan next week, stop by and say hello at Streaming Media East.

Here’s the video best watched in full screen.

Here are the details of the per-title presentation.

T103. A Survey Of Per-Title Encoding Technologies

Tuesday, May 7: 1:45 p.m. – 2:30 p.m.

Per-title encoding techniques customize the encoding ladder to match the encoding complexity of the source, saving bandwidth on easy-to-compress videos and ensuring the quality of more complex footage. Codec specialist Jan Ozer compares the efficiency, implementation issues, and costs of multiple commercially available and open-source alternatives for live and VOD per-title encoding. Learn what per-title encoding is, how the various options work, and which is the best option for you.

Author: Jan Ozer

Ozer is a leading expert on H.264, H.265, and VP9 encoding for live and on-demand production. In his consulting practice, Ozer helps streaming publishers produce highly optimized and deliverable streams and to choose encoders, transcoders, and workflows that optimize cost, efficiency and flexibility. Ozer is the owner of the Streaming Learning Center and is a contributing editor to Streaming Media Magazine.

Hi Jan – taking a stab at explaining your results. When you upload a video to YouTube, it gets transcoded using both VP9 and H.264/AVC. From what I’ve seen, the AVC encoding is done first as its faster and the video is prepared for publishing. Subsequently, the video is encoded in VP9.

From my visual tests, the VQ of the AVC encodes is generally poorer than the VP9 encodes. And I suspect, this is what you might have seen in your tests. Could you inspect the streams and check if it is VP9 encoded of AVC?

I tested a popular soccer video (https://www.youtube.com/watch?v=2P9t-GeznRI) and played it back in two Chrome windows – one window playing the VP9 version and the other playing the AVC version by using the plugin I’ve linked above (to confirm, right-click on the video and choose “Stats for nerds”) and the visual quality difference is apparent at 720p itself. VP9 has sharper grass content and AVC is washed out.

As we go lower down the bitrate ladder, the different is even more apparent. When I streamed this video at 480p, YouTube streamed it in He.264/AVC and AV1. The difference is stark!

In conclusion, I think it comes down to the codec YouTube decides to use when you playback/download the video.

Hope this helps in your analysis! I don’t have a way to link the screenshots here – if you’d like them, please let me know.

Krishna: That looks to be it. I saw the AV1 stream in Chrome, but when I downloaded in the Wondershare Video Converter (which downloaded and didn’t transcode), I got the H.264 stream which didn’t look all that great. I’d love to know the specs on that AV1 file if anyone can download it. The H.264 file was 720p@ 1.123 kbps; which is pretty aggressive encoding.

Not sure what to take from this – I’m guessing YouTube doesn’t create AV1 videos for the vast bulk of their encodes (they didn’t with VP9; it was something like 1%) and the soccer video has over 6.5 M views.