So Many Colors

September 12, 2008

This summer, I’ve been working on getting Mozilla’s color management backend ready for the prime time. We’re finally turning it on in tonight’s nightly builds, so I thought I’d give a bit of background on the history of color management in Mozilla and on color management in general. You should all go download the latest nightly when it comes out and give it a try.

In a nutshell, color management involves using information about a computer’s display, and in some cases information about certain images, to render pages more accurately. Contrary to popular belief, an RGB value (such as #ffffff) alone does not represent a color. Rather, the values for the red, green, and blue channels can be thought of as coordinates in a color space that contains the actual colors. The color that an RGB tuple represents thus depends on how the color space is defined (see Wikipedia for more information).

This type information is generally stored in a file called an ICC color profile. Computer displays often ship with ICC profiles that specify exactly how various RGB values will look on the screen. Additionally, JPG and PNG images are often tagged with ICC profiles to describe the color space of the pixels. In theory, every image should come with a color profile specifying its color space. However, many images do not contain embedded profiles, nor would it make sense for a CSS stylesheet to embed a color profile for CSS colors. As a result, the W3C determined that untagged web content should be assumed to be in a standard color space called sRGB. So color management in Mozilla is really a two-part process: we map incoming RGB values out of their respective color spaces, and into the color space of your display (which we obtain by querying the operating system).

Colors are more complicated than you might think.

About a year ago, Mike Beltzner unveiled Mozilla’s color management support in a blog post. Unfortunately, there were some major issues that kept the color management mode set squarely to ‘off’ for the release of Firefox 3.0.

First of all, there were quite a few rendering bugs filed for our color management implementation. I was able to confirm a good number of them, and discover a few more myself, and I’ve landed the necessary fixes. There were also quite a few issues with our Reftest rendering test suite that needed fixing so that the tests would pass with color management on. These issues have also been fixed. However, several people reported bizarre rendering behavior (whites showing up yellowish, blues showing up purple) that turned out to be the result of bogus display profiles on the reporter’s system (many thanks to Chris Murphy for lending his expertise on this issue). This is scary, because it raises the possibility that any number of people might see inexplicably weird colors in their browser once we turn on color management. As such, we’re hoping to get as many testers as possible in the nightlies and betas to see how widespread the problem is.

Another concern we had over color management had to do with plugins. Since plugins like Flash and Java do their own drawing, we don’t have any control over whether they use color management. While an extension to the plugin API to allow this is in the works, there’s currently no way for the browser and plugins to coordinate on color management. If the browser color corrects a color while the plugin doesn’t, they no longer look exactly the same side by side. At this point, it’s unclear exactly how much of a problem this is, but we’d appreciate feedback.

However, the biggest reason color management was disabled for Firefox 3.0 was that it regressed performance by about 20-30% (according to our Talos tests) when switched on. This was deemed unacceptable for a default setting. As such, I spent a great deal of time profiling code and then devising, implementing, and testing dozens of strategies to squeeze the performance overhead as close to zero as possible.

This turned out to be really hard. Color management is a per-pixel transform, meaning that we need to call a function for every pixel of an image (CSS colors are easier since repeated pixels are known to the rendering code). Such an embarrassingly parallel problem would be a perfect candidate for GPU acceleration, but we’re just beginning to draw up plans for OpenGL and Direct3D backends to Cairo, so it will be quite a while before we can just write a custom shader and make the entire problem disappear. As a result, I dug in my heels and worked on making the transformation of one pixel really, really fast. After some rearchitecting, various precaching strategies, and a whole lot of hand-tuned SSE2 assembly, the performance overhead of full-page color management is finally at about 4-5% on all platforms.

For the past few months, this seemed to be enough. Beltzner was receptive to the idea of turning it on, and nobody in either of the development meetings raised any gripes on the issue. Robert O’Callahan and David Baron had voiced some concerns about standards compliance and interoperability, but both were willing to turn it on for Firefox 3.1 Beta 1 in order to get feedback from the community. However, as Vlad and I were landing the final patch to turn color management on, we were beset by a flurry of naysayers who didn’t feel like the performance hit was worth it.

As a contingency, I had previously implemented a third mode for color management, one that color-corrects tagged images only, that has a performance hit of about 1%. Safari has been color-correcting tagged images for a while now, and this is certainly the dominant use case for color management. Given the pushback on full color management, it looks like we’ll have to settle for a default of “tagged only” for now. The capability for full correction is available in Firefox, and hopefully we’ll have an easy way to switch modes by the time 3.1 rolls out the door. For now, you can go to about:config and set gfx.color_management.mode to ‘0’ for no color management, ‘1’ for full color management, and ‘2’ (the default) for tagged-only correction (for technical reasons, you’ll need to restart your browser for this setting to take effect). Deb Richardson put together a great blog post where she highlighted some of the differences between color-managing and not color-managing tagged images. I’d encourage you to check it out.

Color Management Off, Color Management On (Courtesy of Deb)

Personal disappointment aside, I think it would be a good idea to open up discussion on the subject of full color management on the web. In a world where every browser supported color management and nobody had bogus display profiles (idealistic, I know), a web designer could pick a color scheme for his or her web page and rest assured that it would actually look the same for each viewer. Whites would really be white, and that soothing hue of green would look just as good on your mother’s laptop as it does on your desktop. I’d encourage people to try browsing the web with gfx.color_management.mode=1. Is the performance impact noticeable? Do you run across any web pages that don’t look right? How many color-matched plugins do you see on a daily basis? Do you care about color correction? As always, Firefox is your browser, and we want to know what you want. So tell us.

I’ve been using CMS ON for as long as it has been available (Gecko 1.9.0 nightlies), and never ran into performance problems (except *maybe* one, bug 408029, although results are the same with it OFF). At least nothing that I can attribute to Colour Management.
The main issue is Flash (colour mismatch), but I’m not a Flash guy, and FlashBlock does a good job…
That said, I can understand the decision to turn CMS only partly ON; and at least you’ll have parity with Safari. I’ve seen enough bad profiles, in BugZilla, but also when I was full time working with Photoshop.

Hopefully, LCMS can be further fine-tuned to enable full colour management by default (and no, I won’t change my settings – fully ON). It does give good results here, across 3 Macs.

It’s a shame it’s only for tagged images at the moment, one of the complaints that gets thrown at Safari is that it introduces rendering problems by correcting only images (similar to the issue with plugins)

I’ve also had colour correction turned on since the FX3 beta’s, the only problem I had was with a terrible profile that came with a new monitor I bought (It made white a dark yellow colour), on my Mac though, it’s been beautiful.

Oh, regarding performance, I’ve got a second gen or so eMac, one which was out of date before Mr. Jobs even announced the Intel switch, and I haven’t noticed any significant speed issues (It’s still faster than FX2 for me)

Wow, the ICC profile Windows Update gave me for my Acer P243w is broken beyond belief. White is yellow, and grey is pink! The update was an optional one, so at least it wouldn’t be installed automatically, but I imagine most people would think like me “Oh, there’s a colour profile available for my display? Awesome!” and not think twice about updating. I also haven’t noticed any problems in other programs (i.e. the GIMP).

In Windows Vista, there are five settings for “ICC Rendering Intent to WCS Gamut Mapping”; would messing with these make any difference? Alternatively, which ‘default’ colour profile should I try rather than this one? Oh, if only I had a calibrator..

@Laurence – Just to clarify, the display_profile pref is a string that allows you to specify the file path of an ICC profile. If it’s blank or invalid, mozilla will query your system instead.

@Emanuel – Yikes! It’s odd that you don’t notice any problems with the GIMP, since it uses LCMS for color management also. Do you have it switched on for the GIMP? On the issues of rendering intent, you really want some sort of relative/perceptual mapping, which I’ve made to be the default in mozilla (see gfx.color_management.rendering_intent). Mozilla does its own color management, so the OS settings shouldn’t effect anything. Also, I know nothing about windows display profile calibration. You can try googling for a better profile for your display, or you can experiment with different tools and let us know how it goes? I’m quite curious.

I got the latest nightly and right off noticed the current tab turned an ugly yellow (must have had the color tag in the image). Then I tried putting the about:config value to 1 – everything turned bland yellow. Switching it to 0 made everything look normal again.

I think my color profile is bad :(. I have a Samsung SyncMaster 245T running on Windows Vista.

@bholley,
I suppose I probably don’t have it turned on for the GIMP; I figured since it’s a photo editor it would be on by default, but it didn’t look any different than any other Windows application. I switched back to the sRGB profile and everything is fine, so it was definitely the profile. Would it help you at all if I sent you the offending profile? As for calibration, all I know is you can buy these devices for like $350 that will generate a colour profile based on the actual output of your monitor. I might look into ‘software only’ calibration a bit more later – I’ve already done some through my video card settings, but it’s generating an ICC profile that interests us here.

@Emanuel – As a matter of fact, we’re trying to collect as much information as possible about bogus profiles, so sending any display name/profile file pair that generates bad results would be very helpful. Unfortunately, I’m about to head out of town and will be without internet access for the next week. Can you email Vlad (vladimir@mozilla.com) and ask him how he wants to organize the collection of the data? Thanks for all your help.

When this talk about color profiling began, I understood why some (of my own) websites looked wrong in Safari. Now, I cannot remember how it was – did Safari apply color profiles to all images or did The Gimp secretly embed a color profile behind my back? – but my conclusion was that color profiles can ruin a carefully stitched web page layout. It may enhance a list of photographs immensely, but if you use a PNG to fade a background image into a solid color, you really do want the opaque part of the PNG to match the solid color exactly. I’m all for the infinite progress of vivid colors, but if automatic color correction implies that images cannot be reliably used together with solid CSS colors, I strongly encourage the continued management of *tagged* images only. Then, at least it will be my own problem to make sure that images are not tagged. Also, if one non-alpha image is used used to overlay another, it may be of vital importance that the colors look exactly the same. I’m superstitious enough to fear that automatic color correction may optimize each image individually – say, maximize the contrast – with disastrous consequences for patchwork-like designs. I’ll try the new build max pronto and report my fairings with the full-on config – just wanted to remind you guys about this particular use of images.

I think Safari applies colour correction to tagged images, and treats everything else as if it’s in the monitor profile, Which is what setting 2 does (correct me if I’m wrong, about Safari and Mozilla)

Setting 1 corrects tagged images, and assumes everything else is in sRGB, which it then corrects to the monitor profile, Ideally everything will match (You won’t have the mismatch between CSS colours and images that you can have in Safari)

On the ICC site is a simple page that can be used to verify functional color management for tagged images: http://www.color.org/version4ready.xalter
(Both Apple and Microsoft have used it in their testing)

ICC users were recently invited to try out a tool for making ICC profiles for your display. Ken says Firefox users are welcome to try it, too. Below is the invite he sent to ICC users.

Lars

“Adobe is making available a prerelease version of a tool called Simple Monitor Profiler. The tool is a small Windows application that creates a monitor profile from the chromaticity and gamma values in a monitor’s EDID data, if the data is available from the display. In addition, the tool will ask if you would like to voluntarily send back to Adobe the color data used to create the monitor profile.

We are studying the availability and suitability of EDID color information for auto-generating monitor profiles for those Window systems that don’t already have a monitor profile. Mac OS X auto-generates monitor profiles from EDID, and we are interested if this would be beneficial for Windows as well, to improve the color managed pipeline.

If you are interested in testing the Simple Monitor Profiler tool, please send me the following information: First Name, Last Name, Email address. We will then send you the instructions for accessing and downloading the tool.

The greatest problem with profiles is that you can almost never use the factory ICC profiles for anything. Each monitor is individual and the profile only ever is 100% meaningful if the lighting of the room is correct. This does not mean a 90% parity isnt better than 0. Just that #fff may look white on ALL monitors, or at least as white as that monitor can make.

So everyone would need to measure their monitors individual space. Worse every graphics artist out there would need to learn about what happens when colors don’t reproduce. And yes then theres the damn alpha blending between the damn things. (alpha by the way is also a bit hard so do you guys for example in general know if the formats you use have straight or premultiplied alpha? Would you care. Well in pngs at least its defined, but not all implementations know it)

Quite frankly im seeing people have enough problems with profiles in print which has at least in theory been using this system for 10 years now. I have to this day wirtten 2 guides for color correct workflows but see its just too much for too little practical benefit.

So for those of you who dont know how to handle icc profiles whatdo you do if your monitor IS yellow but i want it to be white? Hell you spend nest 3 days learning something VERY tech and testing yoru ass off. Not many a budding web artist has the personal ethics to do this.

I’m 99% sure that the issue isn’t a Firefox bug per se, but rather that our color management engine uses some rather marginal data chunks in the PNG spec to construct an ICC profile if it doesn’t find a full profile embedded. I’m reasonably sure Safari ignores these chunks, hence the discrepancy.

Feel free to CC yourself on the bug and give us your input if you’re interested in having some input as to where we go with this.

I’m disappointed to hear that mode 2 will be the default for the next release because I think color-management for un-tagged images is very important: Most images on the web are un-tagged (and created on sRGB small gamut displays). Now when you have a wide-gamut display as I do you’ll get very saturated colors in all those images which just looks bad. Interpreting un-tagged images as sRGB would be a wise choice whereas displaying them with the monitor profile does not make sense to me at all because the color you get will be off in most cases. What’s the rationale behind this decision apart from the performance advantage?

I think Shaver’s rationale was mostly that 90% of the people who care about color management care about it insofar as it makes dramatic differences in the rendering of tagged content. After that, more people care about the snappiness of their browser than care about precise color rendering. *shrug*

For the people who do care though, full color management is just an about:config pref away. I see no reason why it will stop being supported in the foreseeable future, and will do get it running even faster.

[…] and Firefox added color profile support with version 3.0 if users manually enable it. With version 3.1, Firefox applies color profiles for images that have been tagged with one. As a result, images on my high-gamut monitor at home look fine in Firefox, but in Chrome they’re […]

[…] and Firefox added color profile support with version 3.0 if users manually enable it. With version 3.1, Firefox applies color profiles for images that have been tagged with one. As a result, images on my high-gamut monitor at home look fine in Firefox, but in Chrome […]

I tried enabling color_management and found that I too had a broken icc profile. I have two philips 200WB screen, and got the color profile from philips own site (bundled with the drivers). Once enabled grays become yellow, and blues gray. I enabled the sRGB profile instead, and things seem to work well.

It’s excellent to hear that these rendering issues are being addressed. I’m running 3.0.5 and the blue-purple problem is so bad on my Macbook screen (probably due to its limited colour production capabilities) that I’ve had to disable CM for now.. I knew it wasn’t an OS problem because Safari has no problem rendering blues properly.

Here’s a strange color management issue I’ve run into using FF: on my 13″ MacBook (onboard Intel graphics), with FF color management turned on, certain color will render very differently on my external display than on the built-in LCD. I’ve calibrated both using a colorimeter, several times, always with the same result. On the MB screen, colors remain true, on the external display blues will turn purple, greens and yellows will become garish. Tagged images seem to display correctly, but the Gmail page, for instance, looks all wrong. Using Safari, which supports color managent by default, colors render the same on both displays.

I’ve had the logic board replaced, thinking that the graphics card was at fault, but I’m starting to think it has to do with FireFox’s color management. When I turn color management back off, everything displays the same on both displays.

I’m a motion graphics artist and very particular about color. I love firefox and use it over safari but I just noticed today that the quicktime movies I send to clients are not displaying the right colors in firefox. I checked in safari and the colors were exact. I tried Sean Hayes’ Color Management add on and the colors did not change at all. I updated to the newest version of firefox (3.0.5?) and still no luck. I even tried the hack (about:config) and still no change. What am I doing wrong. I’m going to have to ask clients to use safari when viewing the animations and really don’t want to do that.

Just some info about the Quicktimes is that they are rendered with H.264 codec. And I just checked the Sorenson 3 codec, same issue.

Also, Im on a G5 mac non itel, OSX 10.4.11 and use a custom color profile I calibrated especially for my monitor.

So something is definitely up with quicktime and firefox. Let me know if you can help me out. Sean Hayes said I should contact you but if there is someone else I should contact instead please let me know.

@Luis – First of all, color management isn’t on by default in all 3.0.x builds. To get all the new color management stuff I’ve been doing you need to do 3.1* or higher.

However, the real issue here is that plugins do their own drawing, so firefox can’t do color correction on them. There’s been talk of a universal API to tell plugins to do color management, but so far nothing has come of it. If quicktime movies are color management in safari, it probably means that apple is doing something special. I’ll look into whether there’s anything we can do to support it.

Pretty much every digital camera on the market today generates image files that have the colorspace indicated with a simple note in the Exif data. The Exif/DCF standards allow both sRGB and AdobeRGB to be specified this way (as described here).

Does the new color-management stuff respect those metadata tags? I would hope so, because the vast majority of photographic images available on the web have their colorspace marked that way rather than provided via embedded profile.

@Jeffrey – I don’t think we currently do anything with the EXIF tags. Untagged images are by default assumed to be in sRGB, so we don’t lose anything if that’s the case. Do you know how widespread the use of the AdobeRGB tag is?

Uncommon in point-n-shoot cameras, widespread in SLRs, although they generally (always?) default to sRGB. The problem is when the advancing amateur hears “AdobeRGB is better than sRGB” and changes the setting without really understanding the ramifications.

I had the impression that sRGB would be assumed for images only when gfx.color_management.mode was set to 1, whereas the default was 2. I was hoping that the Exif/DCF tags could also be honored in the default mode, because they are exactly as informative as an embedded profile, but much more prevalent (again, it’s how images files directly out of the camera are done).

I understand that mode 1 is a huge jump (color managing CSS, etc), but the social concerns associated with that don’t apply to images with a colorspace noted in the metadata, since you’ll generally find those only on photographs. A web designer making a company-logo masthead will certainly not add those metadata notes, and so even if you include support for the metadata notes in color-management mode 2, “web page graphics” won’t be affected and their unmanaged colors will still match the unmanaged CSS.

BTW, when Safari came out, I had discussions with Apple about this. Their reason for not implementing full color management (comperable to your color-management mode 1) was that they had no way to manage Flash colors, and so were not willing to bite that bullet.

I have installed firefox 3.0.10 and also the latest beta I am also experiencing the blue to purple problems with color management enabled (I also tried all the variations) and on a calibrated monitor.
This is issue is not present inside Safari! As far as I could see the problem is mostly apparent on non-tagged images, which should be rendered to the monitor profile or the sRGB profile. This is not happening in our case, since I was able to test it with other color managed applications and see the same I see with safari (correctly rendered color!) something not happening within firefox though!

Could you please clarify what are the image formats in Firefox that support color management through embedded profiles? PNG, JPEG, TIFF, GIF?
Another question – in scenario when there is no color profile associated with monitor on OS level, does Firefox automatically assumes sRgb for monitor?

@Gennady – Firefox does color correction on jpegs and pngs, and assumes sRGB if nothing comes back from the OS. This is at least this was the case with lcms. Firefox now uses qcms (see my most recent post), so things may have changed (though I highly doubt it). confirm with jeff for the full scoop though.

So I’ve installed the latest stable release of FireFox today, 3.5, and I noticed JPG images on some new site’s I’ve been working on look overly saturated. I was completely baffled at first, but came to the point that this just suddenly occurred after I installed the new Firefox. So I began my hunt on color correction and stumbled up on this site and MDC’s article about the new ICC Profile feature.

By default , ICC Profile are checked whenever I export JPG images from Photoshop. So now I have the problem of many websites we designed displaying overly saturated or too dark images.

So now it seems I need to find every image I ever exported with ICC profles on. and now turn them off to prevent this from happening. Because user will mostly likely upgrade their Firefox and run into this display problem. New unnecessary work for me.

@Edward – I know it sucks when this sort of thing just crops up, but the idea behind it is to make images show up more like you intended, not less. If the images look over-saturated, then you probably need to fix the color profile on your system.

As a developer, I am extremely unhappy with this situation. I mean I can’t exactly tell all my clients and site visitors to go into the backend about:config and then set gfx.color_management.mode=0. So instead, I’m having to remove the tags off of every photo I ever put up on the web??? Frustrating indeed.

Please check this link http://www.wfd2011.com/events/film.html both in Mozilla Firefox 3.5.9 and IE 8. The header picture (the picture of film strips) is darker in the Firefox browser and it is matched in IE 8.

It doesn’t make sense at all because I can’t explain every user to fix the color management tags in FF 3.5 and up. They will think I designed that way which is not fair. Is there a way to correct the picture in FF without having to downgrade or to fix the FF browser bug?