Responsive images have been around long enough for most of us to have taken them for a spin, or at least to have learned from the experiences of those who have. Beyond doubt, the responsive images1 specification is a great win for the web. However, quite a few reports from the front lines suggest that responsive images can become pretty ugly.

The good news is that there is a fix! No, not throwing JavaScript at the challenge, but by asking the web server for a helping hand. Enter Client Hints, an initiative spearheaded by Google that is already available in browsers (Chrome and Opera) and that is super-simple to use. Let’s see how Client Hints can reduce both image size and verbosity of the responsive images markup.

This article will not demonstrate the challenges of responsive images. Several other sources have already2done3that4. Instead, we’ll focus on how to address the issues, with a little help from the web server and a new way for the browser, or client, to request images with specific properties. Even if this is called “Client Hints”, it is pretty specific. Let’s dive in!

Client Hints is a new feature already available with Chrome 466 and Opera 33. More browser vendors are following. It is an initiative by Google7 (spearheaded by Ilya Grigorik), and, as its progress indicates, it is likely to be “a thing”. The initiative was also recently adopted by the HTTP Working Group8.

You can think of Client Hints as the missing link between the browser and the server when it comes to layout information. Instead of specifying every possible image breakpoint, pixel density and format in your responsive images markup, Client Hints append the current setting to the HTTP request, allowing the web server to pick the perfect fit — also known as content negotiation9.

The current way of dealing with responsive images is typically to specify different image sources based on the receiving device’s pixel density, preferred image format and viewport size. If you go through the process of picking breakpoints and formats, your markup might end up something like this:

10Even relatively simple use cases can become quite complex using the responsive images syntax. The code shown here is from A List Apart11. (Source: Brad Frost12)

I think many of us agree with Brad on this.

To be fair, the example above include different crops of the image too. Which is something Client Hints alone can’t help you with. Still, crafting markup like this is not something a developer or designer should be spending their time on13. We need an automated process. Even if the server can generate dynamic markup automatically, client hints decouples the markup from the image which makes it easier to perform operations on the image without having to worry too much about the markup.

Imagine if the web server knew the pixel density, the viewport size and also the actual size and format of an image. This is what Client Hints does! Browsers that support Client Hints add a few HTTP headers to requests. The latest draft14 mentions these hints:

DPR
This stands for “device pixel ratio,” the ratio of physical pixels on the screen to CSS pixels.

Viewport-Width
This is the width of the viewport in CSS pixels. (CSS pixels means the units used in CSS to describe a layout. A CSS width of 100 pixels would be 200 device pixels (DP) if the device pixel ratio (DPR) was 2.)

Width
This is the actual width of an image in real physical pixels (similar to the w descriptor made famous by responsive images).

Downlink
This is the client’s maximum download speed.

Save-Data
This boolean indicates whether extra measures should be taken to reduce the payload.

Downlink and Save-Data are not available in Chrome just yet, but you can imagine their purpose. Let’s focus on the currently available hints. First, we have to tell the browser to send the hints.

15With a <meta> tag added, Client Hints are enabled and additional HTTP headers are sent with the request. (View large version16)

You’ll have to opt in to enable Client Hints. The reason for this is that additional data shouldn’t be added to a request unless it is used for something. If it’s not used, it would work against the very purpose by adding data to the payload. Web servers can also advertise their support by adding an HTTP header to the HTML response which lists the hints accepted by the server:

Accept-CH: DPR, Width, Viewport-Width

If adding an HTTP header is not an option, you can also add this meta tag inside the <head> element in your markup:

<meta http-equiv="Accept-CH" content="DPR,Width,Viewport-Width">

That’s all we need. The browser will now append the DPR, Width and Viewport-Width headers to all subsequent requests generated from the HTML, including CSS, JavaScript and so on (with exception of Width, which, in practice, applies only to images).

Aside from images, Client Hints might be useful if the CSS file contains breakpoints based on the viewport size or device pixel ratio. Knowing the viewport size before the CSS is returned to the browser, the server can strip unqualified blocks from the CSS file before sending the response. That is another story. For now, let’s look at an example with images.

Imagine we have a page with the image tag below. Let’s display the flower.jpg image at a width of 200 CSS pixels.

<img src="flower.jpg" width="200">

With Client Hints enabled, the browser will send the following request to the server:

17With just the meta tag added, the browser appends DPR and Viewport-Width to the request. (View large version18)

The browser “hints” to the server that the requesting device has a pixel ratio of 2. As a bonus, we get the size of the viewport, too, since this is known by the browser at the time when the request is being made. Because the DPR is 2 and our page’s design need this image to be 200 pixels wide, we would like to serve an image that is 400 actual pixels wide (200 pixels × 2).

But we’re missing information about the intended display size, even if the img tag says width="200". The specification explains19 that the sizes attribute has to be set in order for the Width header to be sent. In the near future, the width attribute on the image element is likely to be included in the algorithm too, but for now, we’ll have to stick with sizes. The sizes attribute describes the layout and display size of the image. In the beginning, it might be less intimidating to think of it like the good old width attribute or like a CSS property:

<img src="flower.jpg" sizes="200px">

Using pixels might make it easier to “retrofit” existing markup, but using relative units20 such as vw is recommended to make the page more “responsive”:

<img src="flower.jpg" sizes="25vw">

Now, the request for flowers.jpg will look like this:

21With sizes attribute added in addition to the meta tag, Width is added to the request. (View large version22)

The browser calculates the intended display size of the image based on the current size of the viewport and the device pixel ratio of the device in time for the preloader to fetch it. In the example above, the viewport (Viewport-Width) is 774 pixels wide . The <img> tag specifies that the image should cover 25% of the viewport size: 193.5 CSS pixels.

Because it’s a high density display, with a pixel ratio (DPR) of two, we multiply the CSS pixels by the pixel ratio and arrive at 387 actual pixels (Width). You might recognize this from how the selection process work with “regular” markup for responsive images with multiple image sources listed. The difference here is that the information is appended to the HTTP request, rather than an image source being picked from the srcset attribute.

What just happened? Basically, we boiled our verbose responsive images tag down to something that looks very familiar, but with the same responsive functionality. With the pixel ratio and width information, the server can now pick, or generate, the appropriate size of the requested image.

The Width header tells the server the exact width of the image (relative to the viewport) that the browser need to fit in the layout.

Actually, we don’t need srcset at all. The sizes attribute is our new hero! Based on the relative value of sizes, the browser translates this into the actual size in physical pixels. Also, remember that media conditions can be applied if you want the image to have different sizes as the width of the viewport changes:

<img src="flower.jpg" sizes="(min-width: 30em) 100vw, 50vw">

The Width header will, of course, reflect this, too. For a deeper dive, Jason Grigsby has written a great introduction to sizesover on Cloud Four23

But what about type? Responsive images allows you to define different formats, or mime types, of an image by using the type attribute: type="image/webp"

Client Hints does not cover this, but its older brother, the Accept header, may contain valuable information.

Accept: image/webp,image/*,*/*;q=0.8

This example is from Chrome, which is nice enough to tell us that WebP is preferred. Other browsers might just say */* which basically means, “Send me anything.” In those cases, you could apply your own rules, or, even better, the server could implement more advanced device intelligence solutions to decide the best image format to return to the client.

We could say that, with Client Hints, we’re moving the responsibility of selecting an image source from the browser to the server. This means, of course, that we need to implement some logic to act on the Client Hints server-side.

The bonus of involving the server is that, rather than just selecting the best match from a list of pre-generated image files like the browser does, the server can generate the perfect fit on the fly! On a small scale, this is quite achievable because we have all the information we need in the HTTP header.

However, if this task is a bit daunting and if performance is a priority, a few web services — image proxies, if you will — already support Client Hints. One of them is the free ImageEngine24. Using ImageEngine, for example, we first have to “prefix” our images with the URL of the service.

If your image src’s URL is http://example.com/image.jpg, then we’d have to change the src to http://[key].lite.imgeng.in/http://example.com/image.jpg. [key] is the personal token you get when you sign up. As long as the meta tag is present and that sizes attribute is present in the image tags, we’re good to go. Looking at the response with cURL, we can see how the server responds:

The request has a DPR of 2, a Width of 150 pixels and a Viewport-Width of 800. The server then responds with the Content-DPR header, whose purpose is to confirm to the browser25 what the pixel ratio is of the returned image, so that the browser can fit it on the page correctly.

In the example above, the Content-DPR will always be the same as the DPR request header, because ImageEngine is scaling the inputted image to the exact value of Width. As a bonus, even if Width is not set, ImageEngine will fall back to Viewport-Width and then to screen size data from WURFL, a database of devices.

If you implement the server yourself, and you chose to mimic browser behavior by selecting the closest match from a set of pre-generated image sources, then the Content-DPR header might be a different number than the DPR hint from the client. The browser will make use of Content-DPR to scale the image to its display dimensions.

Also worth noting is the Vary header. The purpose of this header is to tell the client (the browser or proxy) that the response from this URI will vary according to the value of the Width header. This enables web proxies and content delivery networks to cache the image better — at least compared to caching based on the User-Agent.

Before you run off and start implementing support for Client Hints, you should know that not all browsers support them. The last <img> tag above will likely mess up a page viewed in a browser that does not support Client Hints. So, what are our options?

Maybe a JavaScript polyfill26? In this case, we’d have to rely on cookies, not separate HTTP headers. Cookies would create problems for content delivery networks and cache proxies because the cookie value must be included in the cache key, which would likely lead to cache pollution. Further, and more importantly, the browser’s preloader27 would not have any knowledge of the cookie’s value.

Until support for Client Hints has reached critical mass, the safest approach is to combine hints with explicit – but relative – widths and heights to make sure the layout doesn’t break. If the browser doesn’t send Client Hints, but the image server expects it, you need the layout to handle an oversized image if that is the default behavior of the image server. Moreover, to reduce the risk of serving too big an image to a non-supporting browser, an image optimization service is recommended, as described above. ImageEngine is useful because it handles mobile devices fairly well (although I’m biased here!). If a mobile device does not support Client Hints, then ImageEngine will never serve an image wider than that particular device’s screen.

Aside from automation, the motivation for implementing Client Hints is, of course, image performance. It’s difficult to compose a fair test case, but to give an idea of how a Client Hints-based image request performs differently than a “regular” responsive images request, I’ve put together a little demo28 with the two scenarios. Below is a table showing the bytes transferred and the actual size of the image put on the wire at different viewports. The selected image breakpoints and viewport sizes in the demo are arbitrary.

Responsive images w/srcset

Client Hints with server scaling

Viewport width

KBytes (w/srcset)

Actual width (pixels, w/srcset)

KBytes (w/client hints)

Actual width (pixels, w/client hints)

320

39.6

480

16.1

288

480

39.6

480

28.6

432

800

81.7

768

63

720

1100

138

1024

113

990

1400

138

1024

186

1260

While the preselected image breakpoints span portions of the viewport’s size, Client Hints, along with an image server capable of resizing images, addresses the continuum of viewport sizes. With Client Hints, we have surgical precision. On average, the Client Hints approach serves 19% less data. If we exclude the 1400-pixel viewport, to which the responsive images approach serves too small an image (1024 pixels), then the data served is 32% less with Client Hints, which is substantial.

The data sample in the chart above is too small to draw any conclusions, but it illustrates the purpose of Client Hints well. No surprises there. Worth noting, though, is the use of DNS prefetching32 for the external host name try.imgeng.in. Referring to the table above, the time it takes to download the bytes is as expected. Client Hints works as advertised.

While responsive images (<img>) element with srcset and sizes) implemented in markup only allows the browser to pick the closest match from a list of image resources, Client Hints enable the web server to serve an image tailored to the browser’s needs, which in most cases mean less data and better image quality. It does require some programming if you’re implementing server-side support by yourself; luckily, some content delivery networks and web services already support it.

The future of Client Hints looks promising. Adding hints about the user’s connection and instructions to reduce data traffic are on the way33. With this information, the server can increase image compression, avoid serving high-density images or reduce the byte size of images in other ways.

As developers, we have no reason not to start exploring Client Hints right now. The key benefits are less verbose and more maintainable responsive image tags, fewer image bytes transferred and, ultimately, happier end users. Only a few steps are needed from you:

add the meta tag to the head element,

put the sizes attribute in your image tags.

Then make, or pick, your image server. I’ve mentioned the free ImageEngine34 optimization service above, which supports Client Hints. The best way to find other services that support Client Hints is to stay tuned and search on Google35 (duh!), because this is a new thing and more providers are announcing support as we speak.

Jon Arne is a mobile guy. He has been an active member and thought leader in the global mobile community since the late nineties.
Jon Arne is a developer by trade, and truly believe that the key to success is a happy end user, and that innovative use of technology will help achieve that truly optimal user experience. Coding now is just done for recreation as he is now heading the Innovation team at ScientiaMobile.
Jon Arne is also an occasional writer, blogger and speaker. Tweets as @jonarnes.

Marc

This seems like a terrible idea to me, for a very simple reason: viewport size can change after the server request has happened. “Page orientation” on mobile devices is the clearer example to this. If, like you suggest in the article, you use client hints to discard some CSS based on viewport size, you have a problem when changing page orientantion or simply resizing the window in desktop browsers.

The solution would be the request the page again everytime the viewport changes sizes. Very bad idea, sorry.

Kelly Sutton

It’s important to remember that Client Hints are exactly that: hints. They expect the browser to reason a bit about current conditions to decide whether or not to issue another image request.

Viewport size changes are important to consider, but a minority use case. Client Hints allows for delivery of optimized assets for the 90%+ cases where the viewport will not change, and defers to the browser to handle viewport size changes.

2

4

Seriously

Did you make the article so long so that no one would read it? Worked for me! That’s okay I still know of the issue in details from other sources and it’s really clear this article is incredibly disingenuous.

You like long shit? Here ya go:
(Google fanbois should just stop reading here and personally insult me in the comments or act like a prozac bot and say something nice about Google. Either way, reading on will just either rock your world or it’ll be incomprehensible to you)

First let get something straight:
It’s very disingenuous to say “Even relatively simple use cases can become quite complex using the responsive images syntax”. Listing a couple alternate sources per image really isn’t that “complex” or even hard. Also there are other ways to do it that are even easier.

Also disingenuous:
“This enables web proxies and content delivery networks to cache the image better . . .” Do you really think they aren’t going to cache ever version of each image? And if they did, that would be “better”?
More:
“With Client Hints, we have surgical precision.” Now we have vague “hints” (based on “width” not “device-width” so size could change at any time from 380 wide to 4k wide.) with the other methods we have precision AND flexibility. And that’s what proper responsive web design is all about.
Even more:
“we have no reason not to start exploring Client Hints right now.”
I can think of plenty foremost is already mentioned in your article. It’s only available in Google Chrome (And still only partially supported by them). Its best for us all not to waste our time on such things, ever. And since we’ll have to use our old methods on top of this, I’m sure that whomever is paying us would appreciate it if we skipped it.

Now let’s talk about what you novel-disguised-as-an-article didn’t say:
1- Google has a vested interest in breaking the internet
2-This proposed “solution” works toward breaking the internet by making headers larger and more complex.
3- Do we really want to have to teach rookie web designers about http headers?
4-We want browser manufacturers touching headers as little as possible, not as much as possible.

fwolf

Nah, not really. My own portfolio site is basically a static HTML page with lots of CSS and a bit of JS. The only dynamic parts are a lil bit of routing, the form mailer and the image generation. So after having worked on the new version of the site, optimized the hell out of it, I went on to the Responsive Image issue .. but: When I found out I’d have to add those horrible attribute mess to my img tags, I was like: Are you .. freaking kidding me? Thats quite … insane?!?

So the mentioning of Client Hints came in as a big saviour, (kinda like) the metaphorical White Knight, saving the day. Knowing there is the option to use a meta-tag makes it all even better. For the non-supporting (or is it “non-supported”?) browsers I just gonna continue to use my own CSS-JS-mishmash polyfill-ish helpers. Though a bit more refined than what’s currently “in action”.

Now, this works for a hand-crafted, 90% static (single-page) website. No more srcset “wading through that mess” nightmares. With a CMS or refined toolkit at hand, the situation might be different, ie. less complicated, because of the simple automated generation vs. hand-crafted code (eg. with WordPress, as mentioned in Smashing Magazine about a month ago).

cu, w0lf.

-1

12

Ivo Petkov

There might also be problems with CDNs. Still, more information on the server is… more information.

Actually, I’ve wanted such information long time ago, but since `srcset` is here there are much better solutions. And because there is not a single great solution now, I’ve built a library that can be useful for you too – https://github.com/ivopetkov/responsively-lazy

Kelly Sutton

CDNs do present a bit of an issue because of the requirement to vary on Width. An incorrect implementation at the CDN layer leads to an explosion in cache key space and lowering hit rates, thus negating one of the major benefits of using a CDN. Thus, you need a CDN with sophisticated edge logic to handle cache spaces in an intelligent way. imgix is the only service that currently does this.

Steve Kamerman

For clarification, the service mentioned in the article, ImageEngine, supports image resizing on the CDN edge via client-hints and server-side mobile device detection for devices that don’t yet have support for client hints.

Disclaimer: I work for ScientiaMobile, the company that owns ImageEngine

Andrea Ganduglia

Artur

Client-hints can’t do real media queries, so comparison to the full <picture> syntax is unfair. They’re closer to just a static version of srcset, which has a quite terse syntax already.

This feature seems to be aimed at a narrow case of CDNs that can dynamically modify served images, can find higher-resolution versions of these images, can modify <head> of the page, but can’t change the <img> tags.

I’ve implemented Client-Hints in our company’s image server, but it turns out nobody is using it.

Without the meta opt-in and high-DPI originals it’s not possible to automatically improve old pages, and new pages use srcset or require real art direction that doesn’t break when user resizes the window or rotates the device without clearing the cache.

Sam

There are some concerns about shifting the decision logic to the server based on HTTP Header values.

1. It will play havoc with CDNs, since they cache output without any processing logic. By vary-ing each combination of client hint headers, would effectively render CDNs useless.

2. Inability to show new images on resize.

3. Client-side apps still need the ability to determine the image they want to render, regardless if a server will do negotiation based on HTTP headers, so the client side html markup we already have would never go away, so seems pointless to confuse the situation with duplicate behaviour via http headers.

P.S. What’s wrong with all the extra markup for images? Yes it’s not as pretty as a simple img tag, but what’s the fear of adding a few additional elements vs achieving a lot of additional functionality. It’s not like devices and modern bandwidth can’t handle it?

Sam,
Thank you for commenting. WRT to CDNs it is clear that the number of cached resources will increase. How much it will increase remains to be seen, but I doubt it will lead to cache pollution. IMHO it is better to put as much of the responsibility on the caches as we can. Client Hints’ interaction with caches also specify the use of the Key header to improve cache efficiency.

It is the responsibility of the web developer to choose the best method for displaying images. If the change-image-when-resizes use case is key, then Client Hints alone won’t do the trick. Further, it is not like Client Hints will replace the client side responsive images we’ve just learned to use. The two are very complementary and not mutually exclusive.

4

22

Nigel Wade

I don’t think this is really ‘Responsive’. Isn’t this more ‘Adaptive’?

I was hoping that this would be something that would be sent with the initial page request. Having set the headers up on the server to accept ‘Accept-CH’ I only get the headers for subsequent requests for images, JS, CSS etc., which is a shame.

Unfortunately the terms we use have been a bit diluted. I agree that this is more “adaptive” (or adaptive delivery or dynamic delivery, like Google calls it) because the server is adapting the content. On the other hand it is closely related to responsive images markup and the way that is implemented in the browsers.

Regarding the Accept-CH, this should work if you set this header in the response of the initial html. At least with Chrome. Alternatively, try a meta tag.

Today, too many websites are still inaccessible. In our new book Inclusive Design Patterns, we explore how to craft flexible front-end design patterns and make future-proof and accessible interfaces without extra effort. Hardcover, 312 pages. Get the book →

Meet the new Sketch Handbook, our brand new Smashing book that will help you master all the tricky, advanced facets of Sketch. Filled with practical examples and tutorials in 12 chapters, the book will help you become more proficient in your work. Get the book.

Meet SmashingConf San Francisco 2017, featuring front-end ingredients, UX recipes and nothing but practical beats from the hidden corners of the web. Only practical, real-life techniques and recipes you can learn from. Get your ticket now!