In an early post, I offered up a sample picture element implementation in JavaScript (a polyfill) that we could use to play with and plan how the element should behave when/if implemented. The script did “work” in that it used the proposed picture element markup to deliver the appropriate image asset based on source elements with media queries, but it came with the common drawback that the fallback image request could not be prevented, incurring unnecessary overhead in larger-screen environments. Not ideal for production use…

However, with a small addition to the boilerplate picture markup and a tweak to the JavaScript, we can feasibly kiss that overhead issue goodbye without sacrificing accessibility in non-JS environments, giving us all the benefits of the picture element right now, provided we’re okay with the addition of a noscript wrapper (*) around our fallback image. Those newiPad images already begin to sound a lot less scary…

So, is it responsible to suggest that developers might use something like this today, given that it provides a workable solution to this very real problem, even if the markup is not yet standard or supported natively anywhere? I’m certainly attracted to the idea, given how hard it is to do this through other means. Perhaps it’d even encourage browser implementations more if the pattern gained traction like this, I’m not sure.

A native implementation would still of course allow image sources to be fetched immediately when the document is parsed, but as a fallback polyfill for today, I think I’m comfortable with the advantages that this slightly ugly workaround brings.

Feedback welcome!

Thanks

Scott Jehl

* The noscript idea here isn’t entirely new… Head London cleverly proposed it a while back. For some reason, it feels a little more purposeful to me when the noscript is used as fallback content in a picture element.

80 Responses to Polyfilling picture without the overhead

Scott, this looks pretty awesome. Two thoughts that come to my mind on this update.

First, I’m curious about the SEO impact of wrapping an img tag with a no-script tag. Does this cause any negative effect on imgs/alt tags? I have no idea if it would/would not.

Second, Brian McAllister mentioned this about the previous version of the polyfill:

“Can I play the devil’s advocate and suggest a “reassess” attribute on the picture element.

This would give developers an opt-in to the “reload on resize” question on an element-by-element basis.”

I think that’s a good idea. I’d be fine with an attribute on the picture element, something like reload=”true|false” or some better wording where you could specify if images in the picture element should be reassessed on resize as opposed to reassessing all of them. Just wanted to throw that out there as an optional setting to add to the polyfill.

@Anselm, I would scratch the idea about the reload attribute on the native picture element. This should be the native behavior of the picture element anyways. My thinking behind this was to prevent additional HTTP calls after all of the media queries on the picture element have been hit.

Originally in Scott’s demo, if you opened the browser on a desktop and scaled it down it, there would be HTTP calls to get the large image, the medium image, and the small image. Then as you scaled back up, it would send additional HTTP calls to get the medium image again, and the large image again, once you went past those breakpoints again.

It looks like Scott’s updated the demo to prevent these additionally HTTP calls so each image is only called once.This is exactly how I would expect the native implementation of the picture element to work.

In regards to using this polyfill in an actual project, I’d say once one browser implements this naively, I’d would love to see Google Chrome do this, I’d feel comfortable using this. It really is a brilliant solution.

This looks really interesting. I think implementing this now wouldn’t be a bad idea, either. As you said, in might even speed up the element becoming standard. The first thing that came to mind for me was ; its survival was definitely determined by developers continuing to use even once it was taken out of the spec, and this seems like another situation where just using it would help a lot.
And it’s great that the script checks to see if the picture element is already supported. So this seems really future-friendly, too. Great work!

This is great. The new* iPad is going to force a a way to handle this, and working on it is good.

If I might chime in on standards approaches, content negotiation, much ignored, was to some extent made to encompass this sort of thing. There is a lot of room to get this right in the http specs** without Rube Goldberg machines, without inventing new things (other than wanting to, or needing to).

*sigh.

**Yeah, sigh, again, and mumble speedy, of which I have no idea yet. need to get on that, anyone have a spare 28 hours?

Curious as to what happens to that extra markup. Definitely makes sense and works, but having a hard time thinking about the maintenance aspects with this for someone that has a large scale need (such as Flickr).

This is pretty cool, and I love that it works (you genius man Mr Jehl!) – but I think it’s really quite dangerous to promote it’s use right now.

Polyfilling for a defined and standardised solution is one thing, but polyfilling for a moving target we know has some (in my mind very serious) unanswered questions is dangerous and misleading. It’s fine in the hands of people who understand the nuances of the issues, but in the hands of people not familiar with the surrounding caveats and such… I’m not comfortable with that.

I don’t want to encourage this exact solution when it’s entirely possible that we’ll discover it’s a less-than idea solution in the future and want to back-pedal. It’ll be very hard to do that by then.

Additionally, if people take this up then we are encouraging the use of completely non-standard mark-up in the wild, and that too is something I have philosophical troubles with.

This is pretty cool, and I love that it works (you genius man Mr Jehl!) – but I think it’s really quite dangerous to promote it’s use right now.

I don’t think Scott’s necessarily doing so, Matt. But speaking for myself, I see this polyfill as a great way to jumpstart the discussion about what this element ultimately needs to look like, how it functions, and how we can reasonably deliver fallback content to non-picture-aware UAs.

I see a big problem of the polyfill that the whole syntax is not ready yet.
We don’t have done a real decision on how the syntax works best currently and many people might already use this polyfill.
I hope this isn’t going the same way prefixes did: People use it/them without knowing about it and if this is valid syntax.

I won’t imagine to have pressure adapting our syntax to this syntax just because everyone already uses this syntax.
Maybe I’m too scared here but I wanted to mention that.

Nevertheless: Thanks for the great polyfill. It’s an interesting and very good concept.

I saw Head’s noscript idea a while back and liked it, it seems the better way to solve the immediate problem and start delivering responsive images, but, as others have said, it doesn’t put us in a good position to go forward from (not dismissing what you’ve done here Scott – it’s great work and getting a working polyfill helps determine what syntax will and won’t work)
Just as a complete aside, I know that we’re currently following the video pattern, but source src still feels awkward to me.

I’m still working on solutions for IE7/8/9, and frankly getting no where.

In IE7/8 you can select picture elements, and you can select source elements, but you can’t select source inside a picture element (as in document.getElementsByTagName( "picture" )[0].getElementsByTagName( "source" )). I’ve experimented with selecting img, and then the parent, which could work, but, this would mean you couldn’t have picture as siblings, which for me rules this out as a solution. Another option would be to regex it to pieces, but this isn’t cool.

In IE9 you can select picture elements, but you can’t select source elements, because the source elements are ripped out of the html, and the DOM. I haven’t been able to access the source elements in anyway. Windows Phone’s default web browser rendering engine that is based on Internet Explorer 9, so I assume the phone has the same behavior.

Has anyone else been looking at possible solutions for IE? Has anyone ideas for other approaches to it?

Ethan – fyi, your comment came through with Matt’s text as if it was yours I think 😉

For those reporting that there’s a problem in this browser or that, I’m sure that can be tidied up in no time. It’s a very simple script, but there’s perhaps an unchecked property in use that’s throwing off Android. This also would need a modified HTML5 shim for IE support.. maybe just a document.createElement( “picture” ) is all, but I’m not sure yet.

Either way, the main benefit is that the img fetch is now prevented, and that was the hardest part to fix.

As for advocating this for use now or not, we’ll check in with some browser dev-rel folks to get their take. Once one UA implements it, it’ll feel a lot more comfortable using it. For now, it’s use at your own risk. 🙂

I’ll have a look at document.createElement( “picture” ) – this might help solve the IE7/8 issues.

The main problem though is with IE9 removing source from inside the picture element, and this occurs as the page is rendered, before any javascript can be applied to the rendered elements, which makes me think it can’t be fixed with JS. Any ideas?

I do like the picture/source syntax, and would like to be proved wrong!

I’ve wrapped the picture element in a video element, if it is IE9, with IE conditional comments. This stops IE9 removing the source elements. Then use JS to move the picture element from inside the video element to outside it, and the source elements are preserved. Then Scott’s Script runs perfectly.

Needless to say this solution it less than ideal, and I’m still on the hunt for better solutions (any suggestions?). But if anyone wants to look over it, it’s on GitHub.

Another thought… if you’re in need of this fix today, but don’t want to risk using the syntax, it’d be very simple to reproduce the entire approach using divs with data- attributes by modifying the JS a bit. It’ll mean your markup will be locked into using the polyfill JS forever, but it’ll also avoid the risk that the picture tag is implemented differently than this presumes.

That said, if large numbers of developers began using this markup pattern, perhaps browser developers and spec authors that decide to implement picture one day may be less likely to break that use case and implement picture as this community has proposed. The proposed markup mimics the video tag too, so there’s precedence in something already functional. It’s also been in discussion for quite some time now – probably as long as some other HTML5 elements took to reach implementation.

I think the concept of web developers having an influence like this in the direction of their language is pretty interesting, especially in cases like this where that community tends to be more in agreement that a particular feature is needed than the implementors or spec authors might be (with no disrespect to those groups). There’s not a lot of precedent for this, I don’t think, but if both browser developers and spec authors can invent markup solutions that don’t already exist, why couldn’t a community of web devs rally to do the same?

Hey all,
It is my first post here but have been paying close attention to this responsive images party for quite some time now.
I definitely don’t want to hijack this thread either, so apologies in advance if it seems so.

First of all let me say to all those doing the work and pushing things forward, kudos indeed are in order and many thanks for all of the thinking involved.

Now for the ‘but’… unfortunately.

One statement that scared the jimmy’s out of me was “Once one UA implements it, it’ll feel a lot more comfortable using it. For now, it’s use at your own risk.”
From Matt Wilcox:
“I don’t want to encourage this exact solution when it’s entirely possible that we’ll discover it’s a less-than idea solution in the future and want to back-pedal. It’ll be very hard to do that by then.”
“Additionally, if people take this up then we are encouraging the use of completely non-standard mark-up in the wild, and that too is something I have philosophical troubles with.”

For me I’ve gotta say…. Easy now, relax on the UA implementations it’s much too soon for that!
We need to really slow down on things, because I fear we’re going to be in the unfortunate situations Matt speaks of. It’s more possible than not possible.

With this proposed new picture element markup, I have a really hard time asking for a few things:
1. Extra time from the Content Producers so they can re-size, cut and crop 3 additional images / per image, on top of the 9 they already cut. Up to 36 total? Are you serious? For 1 image? Holy shit.
– I showed the proposed new code to a Director of Content at a big Media company and she had a hissy fit… almost fell out of her chair.
– Her Content Producers and Bloggers do some basic html coding sometimes, as well as write a lot of blog posts, with embedded images, how would they embed a responsive image into a post with a custom built cms? Uh oh…
2. Time from the back-end devs to update code behind / user controls / scripts / configs / cms etc. to be able to utilize the new markup.
3. Time to go back to every instance of every image on all the sites to update img to picture, and with a noscript? Wow.
4. Additional time built into every and all projects going forward just to deal with images, plus additional time allotted on top of this just for the maintenance of the images before they even hit the website.
5. An unknown amount of new time for ongoing new maintenance of image directories – of which a new naming convention will need to be created, docs, scripts, ps actions created. Another holy shit.

Personally I can’t in good faith go to my VP and ask for an additional “X” number of dollars next year be put into the budget to deal with labour surrounding images. It’s bananas. What’s the number? Half million? Who knows, could be.

Some stuff that I do know…
I know this responsive images party is a huge issue.
I know many people are racking their brains trying to figure it out.
I know Browser makers should be shouldering a little bit more of the weight of this problem than us coming up with the picture element.
– UA devs building into browsers a dynamic image re-sizer?
I know we need to get more than just FE devs opinions on the new markup – Back-end, Content, IT??
I know we should be soliciting server makers for ideas — IIS responsive images module? Why isn’t microsoft in on this party?
– Would this not solve the large issue of people not having access to servers?
I know polyfils are good for testing in cases like this, but…
I know with the shear amount of polyfils, compatibility scripts etc. that something is hugely wrong in the dev world right now.
I know the dev world, specifically the front-ends, need to slow down and let the browsers catch up — which is directly related to this topic, and the rest of the polyfil nation.
I know the guys at Apple recommendation for css4 background images is highly intriguing and is worth a serious investigation into porting the concept to html markup.
– going so far as a demo with polyfil, the same as picture.
I know if we go with a pure html markup with no help from UA’s or servers, I will most likely jump out of a window.
I know it’s logical to follow a defined spec with regards to ‘video’ when proposing a ‘picture’, but the practicality and residual affects far outweigh us going that route.
I know I’ve been thinking about it as long as everyone else, with tonnes of notes, but somehow always comes back to having a server and/or UA do part of the work.
I know we’re making things a whole lot more difficult than they need to be. KISS – Keep it simple stupid applies greatly here.

That’ll take some time to sift through, but my initial thoughts are that yes, this is a difficult problem to solve, but it is real a problem that users are dealing with today. This is one way we can address it now, and in my opinion, it may be the most flexible and future-ready (media queries self-maintain in ways device detection simply cannot). If the markup makes you uncomfortable, that’s understandable… using divs and data attributes to achieve the same structure and fallback seems a perfectly viable approach for today to me.

No matter the approach you take, whether your using device detection, automating image scaling, or manually pushing out a few sizes, delivering responsible images takes some additional work on your part (or on producers’ part perhaps). However, it’s fairly common to find CMS systems – WordPress, etc – that’ll automate image sizes for you upon upload, and large sites and newspapers have long had workflows in place for generating image sizes. I don’t think this approach implies you’d have to use multitudes of sizes either – 2-to-4 sources for a large image asset would probably deliver a great experience – that’s what we ended up doing for BostonGlobe.com and it seems to scale pretty well across devices.

Also, I don’t imagine picture would be necessary for every image element on a page… you might only use it for images that are large on large-screen devices, and small on smaller devices, like a large article feature image.

You can always choose not to optimize, but I tend to think this is one of those situations where we’re able to do some of the hard work to deliver a much better experience to our users.

If anyone is interested, I’d be happy to make a branch of Picturefill that is designed to work with div[data-picture] markup pattern.

Maintenance is definitely the most important issue if not among the top 5 things we should be keeping in mind as we go along. Of course Not every element will require the picture element like scott says, but it will certainly take a bit more effort on our part to deliver the proper experience. If you’re an android developer for example, you’re expected to use a few sizes for density. A device such as Android has hundreds of device types with several different screen sizes, ranging from small phones to large TV sets.

I copied this snippet from the Android developer docs and I think it is super important to keep in mind for this particular situation.

Android Documentation http://developer.android.com/training/multiscreen/screendensities.html
Defining layout dimensions with pixels is a problem because different screens have different pixel densities, so the same number of pixels may correspond to different physical sizes on different devices. Therefore, when specifying dimensions, always use either dp or sp units. A dp is a density-independent pixel that corresponds to the physical size of a pixel at 160 dpi. An sp is the same base unit, but is scaled by the user’s preferred text size (it’s a scale-independent pixel), so you should use this measurement unit when defining text size (but never for layout sizes).

SVG in my opinion combined with the ability to work with the img element can really help to solve all these maintenance and workload issues permanently. This would allow us to link to one image file and be done with it without the extra this and that mumbo jumbo. Future friendly and way simple.

I just makes no sense to convert a real photo into a ScalableVectorGraphic. Because we would have a 1gb file and less colors than in JPEG.

It works for graphics but not for photos. So we are in need of a file-format which can serve diff. outputs for a photo. This is what I always proposed. But what format? I wrote to ppl of WebP and proposed but never got an answer. :S

But also we need a different approach doing this in our HTML markup. Because then we can serve different content.

Hey Daniel,
The following branch has a div-based picture implementation you could certainly use. It works the same and doesn’t risk conflict with potential future picture implementations. That said, the markup’s a little clunky to those with an eye for semantics. Harmless to the user, but clunky.

The solution above sounds like a nice complement or alternative to picture, but I think the likelihood of it working on a device or browser that exists today is low, and browsers stick around a long, long time – especially on devices (see the hundreds of Androids coming out running 2.2 and 2.3 right now). We don’t have any tools to generate an image like this today either, and Photoshop has yet to land support even for png8 alpha opacity, something we’ve needed for over 10 years! The drawbacks you mention are drawbacks we already deal with in CSS and in other HTML elements like video. Perhaps one will decide to add more source elements to their markup one day as devices come out with features we never thought to query, but if it’s resolution increases you’re worried about, that’s fairly easy to set up for the future right now with media queries. iPad’s retina screen, for example, is something we could have targeted agnostically with CSS media queries long before the device even existed – indeed, you can target it easily with picture/Picturefill. Forward-looking developers can certainly prep images and queries for resolutions that don’t exist today – it’s just rarely seen.

The picture element combined with a JS polyfill offer an approach that work in non-native implementations right now, don’t require UA workarounds, don’t add unnecessary overhead, work with existing tooling (Photoshop, etc), and will kill itself off as native implementations appear. It’s not perfect, but I think the case for “good enough” is pretty strong here.

I agree Scott, but it seems like we have file formats that support progressive loading (JPEG, PNG, etc.) and file formats that support multiple images (ICO). Is my head too far in the clouds if I say I’d love to see those capabilities combined into a format that can have the appropriate image segment downloaded for a given context?

FWIW, I agree picture is probably good enough for now, but it seems like a single package format would be the best option all-around. It may not be ready for prime time for 2-10 years, but it seems like a worthy goal.

I like the idea to give WebP a chance to do this. But I don’t know who is behind that format. Maybe you know and you can ping them?
But as I said it is not made for diff. content then. Why for different content? See this: http://css-tricks.com/lark-queries/ – I have similar ideas to do for images. BTW: Could we extend our syntax to serve text reponsively, too? Maybe we need a element instead of ?

I’d love to see both tools available to us one day, but we do need something today, and I think picture offers a “now” solution that doesn’t feel rushed or shortsighted.

I can hardly envision the problems we’ll be facing a year from now, but I’m guessing we could be using picture + media queries for polling all sorts of things beyond device resolution. In that way, a technique that utilizes a feature that is constantly evolving in browsers (media queries) seems like a pretty safe bet. I’m not sure how header data and such could give content creators the control they need in generating a multi-image format, but it does seem interesting and useful in different ways, at least.

“It may not be ready for prime time for 2-10 years, but it seems like a worthy goal.”

You might have missed but I always said this. BUT this is a whole other idea not in conflict with ours. Your idea is to have a responsive file-format. This isn’t bad, it’s best for same content (images). And there are image formats supporting such stuff but not in that purpose etc. As Scott said this would take years to achieve. So we need a interim solution and we need a solution for serving different content responsively.

I’ve just been working with this script and hit issues with older devices – Android 2.2.1 and iOS 4.2.1 rendered no image at all. It looks like a similar issue to IE in that the source elements were either stripped out or couldn’t be accessed. The script found the picture element, but not the sibling source. I switched to the branched version that used divs and data attributes, and it worked in both cases.
I like the picture pattern as it follows an existing precedent, but it’s going to be a huge problem if it’s not backwards compatible (even with a polyfill).
This issue alone to my mind means we’d need to look elsewhere.

I’d agree that backwards compatibility (with a polyfill) is a must for any solution!

But I think the latest version of Scott’s polyfill has fixed this problem, but the source elements need to be duplicated in comments. I don’t think this solution is ideal (and am still looking for an alternate solution), but it is preferable to changing the syntax.

Have you used the latest version in the repo? I did a pretty big refactor and the device support currently tests out very well now, at least on what I’ve been able to test (which does include the devices and browsers you mentioned).

Check out the Readme as it has updated information on support, and let me know if you still run into issues – the bug tracker is ideal for that.

My bad – I was using the previous source html that didn’t have the comments in.
It’s working in those older devices – even working in the oldest BlackBerry we have.
It’s not ideal to have to fallback to comments, but if it gets us up and running then I’m happy with that for now.
Great work Scott, thanks.

While we experiment with this, do you think it makes sense to use x-picture instead of picture? It seems to follow the “experimental markup” setup a bit better and will be a little more explicit for users who may think this is ready for prime time.

Hey thanks, Aaron – sounds interesting. Do you think that the “div” branch should head in that direction for now, instead at least? Any non-picture markup is fine by me in that regard. Care to send a pull request with what you’re thinking? https://github.com/scottjehl/picturefill/tree/div-markup

As for the main branch, I’m not sure… I guess I could be swayed, but I do like having a 1-1 implementation of our proposed markup, both for planning and future-polyfill usage.

While there may be advantages in not using the proposed tag name, I think the benefits likely outweigh the problems. But then, I think this is probably the best solution I’ve seen for the responsive side of image handling.

It still leaves open the difficulties of trying to respond in a bandwidth appropriate manor, but perhaps that should be on the browser makers to give us a media query to detect connection type? -webkit-bandwidth-max/min perhaps.

I didn’t realize how far along talks were about an actual spec to solve this, and hadn’t talked with anyone yet, so this was basically just a stab in the dark.

I used data attributes because I was expecting it to just be a hold-over solution until there was something formally introduced and adopted, and data- prefixed attributes would still allow people’s html to validate (though I guess I would have been leaning towards using a div instead of for that to mean anything, eh?)

Width and height could/would be handled either in the source or through the css (which is where I was figuring it would belong).

I’m not sure if there’s anything more I could contribute to Scott’s approach as it looks much more thought out than mine, but I think mine was on the same evolutionary path, and wanted to sort of chime in that I agreed it is a good approach overall and allows for some very interesting things to happen.

I think introducing an element like this is a pretty clean and efficient solution. The same can be accomplished with divs, media queries in css, and background-image, however I prefer to avoid divs with background images when it’s possible to use a real element. The only issue would be that if this were introduced as a new HTML tag, there would be backwards-compatatbility issues. As a javascript-managed tag it works fine with browsers that don’t already support the tag, but I’m not sure what a good solution would be if it were a standard tag.

This is a great solution for the time being. I have seen several complaints about “I have to crop out HOW many images?!!?” To that I would say, that isn’t the problem this is solving.

This addresses the issue of load times on smaller devices extremley well and, while it might not be super SEO or future (due to the no script) friendly, it’s a heck of a lot better than anything else out there right now.

If you are running a large blog or article based business, chances are you’re running a cms with an image uploader. It shouldn’t a huge stretch for your backend guys to modify the CMS you are using to save out a whole mess o’ images with different sizes and different names and then spit out this code into some simple copy paste or default setting.

And i’m really interested in implementing this solution on my future projects, specially when it comes to responsive and image load issue.

However, Im concerned that there is no new news or updates for this since july of last year. I’m wondering if this still works on all major browsers.. IE8+, chrome, firefox, safari and andriod default internet browser?

I hope you can enlightened me with the supported browsers and if there’s any known issue too.

A lot of progress has been made since this post. Browsers have begun to implement native solutions for the picture element and srcset attribute. More info on the latest here:http://scottjehl.github.io/picturefill/