FFmpeg Quality Comparison

Any­way I used to use Medi­a­Coder to con­vert to flash video, but when it gave me errors, and refused to tell me the specifics of those errors, I took it old school to the com­mand prompt with FFm­peg (which Medi­a­Coder uses any­way). This gives you a lot of use­ful info about the source file you’re encod­ing, such as audio sam­pling rate, frame rate, etc.

Want­ing to find a bal­ance between pic­ture qual­ity and stream­a­bil­ity, I began encod­ing a short length of AVI video at dif­fer­ent com­pres­sion lev­els. FFm­peg calls this “qscale” (a way of rep­re­sent­ing vari­able bitrate qual­i­ties, much like LAME’s –V para­me­ter), and the lower the qscale value, the bet­ter the qual­ity. The avail­able qscale val­ues range from 1 (high­est qual­ity) to 31 (low­est qual­ity). Going worse than a 13 qscale pro­duces unac­cept­ably poor qual­ity, so that’s as low as I went for the pur­poses of this test.

I encoded 3:14 min­utes of an AVI, resiz­ing it to 500×374 pix­els, and encod­ing the audio at 96kbps and 44.1KHz, which sounds fine, and is a neg­li­gi­ble part of the ulti­mate file size, so going lower wouldn’t be very ben­e­fi­cial. Plus I find that good audio can cre­ate the illu­sion that the whole thing is of higher qual­ity. Poor audio just makes it sound like “web video.”

Here are the results, cour­tesy of Google Spreadsheets:

The file­size, of course, goes down as qual­ity goes down. And the loss in file­size also decreases, not just in amount, but in per­cent­age as well, as indi­cated by the red line. For instance, the value of the red line at qscale 3 is 33.97%, which means that in going from qscale 2 to qscale 3, 33.97% of the file­size is shaved off.

How­ever, because these losses are not per­fectly expo­nen­tial, I knew that there had to be qscale val­ues that were more “effi­cient,” in a sense, than oth­ers — val­ues that, despite being high, and caus­ing a lower change in file­size than the pre­vi­ous step in qscale, still caused a com­pa­ra­bly large change in file­size. For instance, still look­ing at the red line, you’ll notice that going from 2 to 3, as I said, shaves off 33.97% of the file­size, while going from 3 to 4 only shaves off 23.93% of the file­size; and that is a 29.56% decrease in change-in-filesize, which is a rel­a­tively large cost. We want the change-in-filesize to remain as high as pos­si­ble for as long as possible.

Now, if you fol­low the red line from 4 to 5, you’ll see that that’s a 20.32% loss in file­size, which is pretty close to our pre­vi­ous 23.93% loss in file­size in going from 3 to 4. In fact, we’ve only lost 15.09% of change-in-filesize from the pre­vi­ous step. So these are the val­ues we really want to exam­ine: change in change-in-filesize, rep­re­sented by the orange line.

This is nowhere close to expo­nen­tial, nor does it fol­low any pre­dictable decline. It darts around, seem­ingly at ran­dom. And we want to catch it at its low­est val­ues, at points that rep­re­sent changes in qscale that were nearly as effi­cient as the pre­vi­ous change in qscale. So the most desir­able qscale val­ues become, quite obvi­ously, 5, 9, and 11.

What this means is that if qual­ity is your pri­mary con­cern (and you’re not crazy enough to encode at qscale 1), go with 5. qscale 5 turns 3:14 min­utes of video into 30.62MB, which requires a down­load rate of 157.84KB/s to stream smoothly. qscale 11 will give you about half the file­size, and require a down­load rate of 77.37KB/s. But, because that’s the level at which pic­ture qual­ity really begins to suf­fer, and because most peo­ple don’t really mind buffer­ing for a few sec­onds ini­tially, I’m prob­a­bly going to stick with qscale 9, whose videos take up 91.58 kilo­bytes per sec­ond, and which is by far the most effi­cient qscale any­way, with only a 4.92% change in change-in-filesize.

One caveat: This whole exam­i­na­tion pre­sup­poses (as far as I can tell) that if it were pos­si­ble to mea­sure and chart the changes in the actual per­ceived visual qual­ity of videos encoded at these qscale val­ues, the curve would be per­fectly geo­met­ric or expo­nen­tial, with no aber­ra­tions sim­i­lar to those above, and with all extrap­o­lated delta curves show­ing no aber­ra­tions either. Given that, it might be eas­ier to believe that every step you take through the qscale is of equal rel­a­tive cost, and that there are no “objec­tively prefer­able” qscale val­ues. But that is a lot more boring.

Thanks for this! Saved me a bunch of time doing it all on my own. I still want to inves­ti­gate the effects of chang­ing the pic­ture size when using qscale to try and get the best qual­ity when encod­ing dvd res­o­lu­tion videos. Will post here if I get around to it!

I would like to see this simul­ta­ne­ous com­par­i­son includ­ing PSNR (peak sig­nal to noise ratio) and SSIM (struc­tural sim­i­lar­ity index) on the same run of video. This would give you the qual­ity com­par­i­son data, and then can eas­ily cre­ate the change-in-PSNR and change in change-in-PSNR num­bers. (and SSIM)

@Daryl: sounds like a great call-to-action. I’d be curi­ous to see the results, too.
Jay: thanks for an excel­lent post. Is it pos­si­ble these val­ues have changed since you orig­i­nally ran this, if ffm­peg has been upgraded since then?

I’ve down­loaded “Lupe (4k resolution).mp” from youtube. I’m using a script I’m transcod­ing from h264 to mpeg but vary­ing the qscale from 1 to 31. The script is only at –qscale 10 now, but I got a bizarre step in the result­ing file­size at –qscale 7. ie:

I think your rea­son­ing for your qscale choice is faulty. Take for exam­ple the tran­si­tion from q=4 to q=5. For a (com­par­a­tively) small decrease in file size, you’re sac­ri­fic­ing a (how­ever big) amout of pic­ture qual­ity. Think of it this way: If the orange graph was “near 0% at the tran­si­tion from q=4 to q=5, what would you pick, the bet­ter qual­ity q=4 or the worse one, q=5?

So bas­cially, what I’m try­ing to say it that your qscale pick is “one-off”, and this choice is “for the worse”. The cor­rect choice in your case would be to pick 4 or 8 or 10.

If your “change in change-in-filesize” or 2nd-order deriv­a­tive, what­ever one prefers to call it, is approach­ing 0%, then what you get when tran­si­tion­ing from qual­ity q (bet­ter) to (q+1) (worse) will net you near 0% of saved file size.