Any sufficiently advanced technology is in distinguishable from magic.

Google says it cuts JPEG size by 35 percent with new Guetzli algorithm, but our tests don’t back that up

Update: Two of the slides in the bees.png file were swapped relative to what the text said was being compared. This error has been corrected.

Google has announced a new and improved JPEG implementation that it claims can shrink file sizes by as much as 35% with better image quality than you’d see in an equivalent standard JPEG file. That kind of improvement would be significantly useful, if true — repeated studies have shown a distinct correlation between website load times and how likely a user is to remain engaged. Speed matters on the Internet, and Google has been focused on improving web performance for years.

A joint blog post by Robert Obryk and Jyrki Alakuijala describes the new compression method as similar to Google’s Zopfli, which can be used to create smaller PNG images and gzip files. Here’s how they describe the new Guetzli (it means “cookie” in Swiss German).

The visual quality of JPEG images is directly correlated to its multi-stage compression process: color space transform, discrete cosine transform, and quantization. Guetzli specifically targets the quantization stage in which the more visual quality loss is introduced, the smaller the resulting file. Guetzli strikes a balance between minimal loss and file size by employing a search algorithm that tries to overcome the difference between the psychovisual modeling of JPEG’s format, and Guetzli’s psychovisual model, which approximates color perception and visual masking in a more thorough and detailed way than what is achievable by simpler color transforms and the discrete cosine transform. However, while Guetzli creates smaller image file sizes, the tradeoff is that these search algorithms take significantly longer to create compressed images than currently available methods.

Google included some of its own image assets to demonstrate the quality difference between Guetzli, a standard PNG, and a conventional JPEG.

16×16 pixel synthetic example of a phone line hanging against a blue sky — traditionally a case where JPEG compression algorithms suffer from artifacts. Uncompressed original is on the left. Guetzli (on the right) shows less ringing artefacts than libjpeg (middle) and has a smaller file size.

The Guetzli image is clearly better than the conventional JPEG file created with the libjpeg library, but we wanted to create some comparisons of our own. Luckily there’s a Guetzli binary image available to download. It even includes a sample file if you want to compare image quality yourself. We were curious to see how the program’s output would stand up in testing, so we’ve done some testing of our own. All of our image editing and standard JPEG creation was done in version 4.0.13 of Paint.net.

Test Setup

The slideshows below contains multiple types of images, so we’re going to walk through them in advance. The first three slides consist of the bees.png file Google provided in Github, followed by the same file saved as a normal JPEG, followed by Google’s Guetzli implementation. The easiest way to see the differences between them is to open each slide in a new window, enlarge the image to 500-600%, and then tab between them. Check the flower in the lower right-hand-corner of the image and you’ll see that Guetzli does a better job of maintaining proper color compared to a regular JPEG file.

Our baseline bees.png image

If you zoom in on the top left bee (800% ought to do it) you’ll see a similar effect. Guetzli and the original PNG clearly show the flower’s anther — that’s the brown bit at the tip that actually contains the pollen. In the standard JPEG file, that information has been lost. The next two slides are differential comparisons. First we compare the PNG file against the normal JPEG encode, followed by a comparison of the PNG file against Guetzli. The more spots there are in the image, the greater the difference between the original PNG and the JPEG.

The original, Google-provided bees.png file. We recommend clicking on the slide to open it in a new browser window and substantially increasing your magnification rate.

So far, so good. The Guetzli compression clearly loses less data. But how about something a little more challenging? The next slideshow is an extreme close-up (500%) on a 4K PNG image capture from Skyrim’s special edition. You’ll want to open each image in a separate window and flip back and forth to compare them. Look at the edges of the foliage on the left-hand side to see the variations between the three files. The original image is shown below.

Our base image in standard JPG (uploading a 4K PNG of this sucker would’ve left you fine people with a 32MB file to download.

We perform the same differential comparison as we did in the first slideshow, but this time the results are rather different.

An extreme close-up of a scene from Skyrim’s enhanced edition. This is the original PNG file.

In this case, which image looks best is arguably a matter of personal preference, and it depends on where you look. Generally speaking, Guetzli does a better job of replicating color, but look at the diagonal rock in the lower left hand corner. The standard JPEG algorithm handles that edge more closely to the PNG file, while Guetzli blurs it. Guetzli seems to struggle with accuracy in a few areas in our 4K image, producing an end result that’s actually more inaccurate than the standard JPEG. This isn’t automatically a problem since the application is still in development, but it does point to a need for further improvement.

Finally, there’s the question of file size. Google claims to achieve a 35% compression improvement over JPEG, but our own tests showed exactly the opposite. Both versions of Bees.jpg were 38KB, compared to a 174KB PNG, and the 4K Skyrim scene was 5.56MB when encoded with Guetzli and 5.10MB when encoded as a standard JPEG.

When we encoded the bees.png file again but at a lower quality rate (85 versus Guetzli’s 95 default quality setting), we finally did see a difference in file size. The Guetzli encode was 20KB, while the standard JPEG file was 23KB. That’s still only a 13% improvement, but if you tend to rely on lower-quality JPEG files Guetzli can theoretically reduce their size while still offering better quality than a traditional JPEG.

To be clear, we aren’t accusing Google of misrepresenting anything about Guetzli. It’s possible that Paint.net isn’t the right tool to use for saving standard JPEG files if you want to see the compression differences Google claims Guetzli can make. The program is still in the early stages of development and there may be something specific about our Skyrim image that the application had trouble with. We stand by our results, particularly since we encoded our JPEG files at the Guetzli default quality level of 95, but there are a number of factors that may explain why we didn’t see the improvements Google says exist.