Get Compressin’!

The best way to counter negative effects of loading image assets is image compression: using tools such as Kornel Lesiński‘s ImageOptim, which utilizes great libraries like mozjpeg and pngquant, we can reduce image byte size without sacrificing visual quality. And thanks to libraries such as DSSIM we can ensure good visual quality testing different compression levels.

Enter Multiplexing

Part of the solution comes from a popular area of the performance conundrum: HTTP2. One of its main benefits is “Multiplexing”: the ability to handle multiple requests and responses at the same time, all using the same TCP connection.

With Multiplexing, website assets load faster. Depending on site architecture, you can also prioritize resources inside a multiplexed connection: flagging assets such as critical CSS with high priority in HTTP2 will make them load sooner. On top of this, pushing not yet requested but crucial assets via HTTP2 Server Push can create a super fast perceived performance, when applied correctly. More on this later.

Multiplexing has a curious side-effect when it comes to image loading: certain kinds of images load significantly faster in terms of perceived performance because initial image information can be downloaded in parallel via HTTP2 Multiplexing. Progressive JPEGs and interlaced PNGs benefit from this.

Progressive All The Things

Progressive and interlaced images are like layer cake: they contain information not in a single stream that renders images from top left to bottom right, but as a stack of layers, each improving on information already shipped in earlier layers. Each individual layer is more lightweight in terms of byte size than the final image.

Since browsers loading website assets via HTTP2 Multiplexing will initiate almost all image downloads simultaneously, the initial, lightweight layers of progressive and interlaced images start rendering much more quickly than sequentially encoded images. Sequentially encoded images render in a windowblind manner: line by line until all image information has been shipped.

That perceived performance and the Speed Index benefit from delivering progressive JPEG scan layers via HTTP2 Multiplexing was already observed by Google’s John Mellor in 2012. He was experimenting with the SPDY protocol, a precursor to HTTP2. Today, we can improve on this discovery to make progressive images appear even faster:

Take The Power Back! Or: The Scans File

Progressively encoded JPEGs contain ten scan layers by default. That means ten iterative layers of image information build on each other to deliver the final visual quality of the image. The first visible scan layer of a progressive JPEG is always highly pixelated and often black & white because it saves on color channel information. If you want to check out how each scan layer looks, use Frédéric Kayser’s “jsk” tool to split a progressive JPEG into its individual scan layers.

Each line in the scans file defines a new scan layer. They contain multiple parts of information on color channel, matrix index and lossiness.

The three channels are brightness (‘Y’), blue (‘Cb’) and red (‘Cr’), which respectively have the numbers 0,1 & 2 in the scans file. The matrix index in the scan file goes from 0 to 63, covering a 64-pixel block. (JPEG encoding has a native 8×8 block setting.)

Getting Creative

Our goal is to show meaningful image contents sooner while enabling browsers to lay out the site speedily. Our initial scan layer should therefore be lean but meaningful, followed by an as steep increase in perceived visual quality as possible.

The custom scans file displayed here ensures a first scan layer with appropriate colors. At the second scan layer, we already have a highly acceptable preview. Scan layers three and four deliver the necessary color information: the red channel before blue channel since it is likely that red color information is more important to improve visuals, e.g. when showing faces. After the fourth scan layer, the image looks complete and the final fifth scan layer improves only fine high frequency details. These improvements are reflected in an ~6% better Speed Index, reflecting the perceived performance.

Push! Push! Push!

HTTP2 offers another tool we may use for even faster delivery of image contents: Server Push. In supporting HTTP2-enabled web servers, it is possible to flag individual scan layers of progressive JPEGs with high priority and making the server push those scan layers into the client browsers’ Push cache even before the request for the respective image is initiated. Browsers then can lay out the page and render initial scan layers with the performance of a warmed cache, making users perceive the site’s images as rendering exceptionally fast.

This technique should only be used strategically: find out which images are crucial for creating user engagement on your page, e.g. an emotion-evoking hero image or a product overview image, and only apply Server Push to the initial scan layers of those JPEG images. This will enable you to increase user engagement and thus successful conversions without harming overall site asset downloads.