Lessons from the dress: The fundamental ambiguity of visual perception

The brain lives in a bony shell. The completely light-tight nature of the skull renders this home a place of complete darkness. So the brain relies on the eyes to supply an image of the outside world, but there are many processing steps between the translation of light energy into electrical impulses that happens in the eye and the neural activity that corresponds to a conscious percept of the outside world. In other words, the brain is playing a game of telephone and – contrary to popular belief – our perception corresponds to the brains best guess of what is going on in the outside world, not necessarily to the way things actually are. This has been recognized for at least 150 years, since the time of Hermann von Helmholtz. As there are many parts of the brain that contribute to any given perception, it should not be surprising that different people can reconstruct the outside world in different ways. This is true for many perceptual qualities, including form

Rubin’s vase: A classical example of figure/ground segmentation. The image is fundamentally ambiguous. People perceive a vase or faces, but not both at the same time.

and motion. While this guessing game is going on all the time, it is possible to generate impoverished stimulus displays that are consistent with different mutually exclusive interpretations, so in practice the brain will not commit to one interpretation, but switch back and forth. These are known as ambiguous or bistable stimuli, and they illustrate the point that the brain is ultimately only guessing when perceiving the world. It usually just has more information to go by and disambiguate the interpretation.

A bistable motion stimulus. Do you see the dots moving from left to right or up and down?

This is also true for color vision. The fundamental challenge in the perception of color is to identify an object despite changing illumination conditions. The mixture of wavelengths that reaches our eye will be interpreted by the brain as color, but which part is due to the reflectance of the object and which part is due to the illumination?

This is a inherently ambiguous situation, so the brain has to make a decision whether to take the appearance of an object at face value or whether it should try to discount the part of the information that stems from the illumination. As the organism is not primarily interested in the correct representation of hues, but rather the identification of objects in light of dramatically varying conditions (e.g. a predominance of long wavelengths in the early morning and late afternoon vs. more short wavelengths at noon), it is commonly accepted that the brain strives for “color constancy” and is doing a pretty good job at that. But in this tradeoff towards discounting, something has to give, and that is that we are bad at estimating the apparent absolute hue of objects. For instance, a white surface illuminated by red light will objectively look reddish. The same white surface illuminated by blue light will objectively look blueish. In order to recognize both as the same white surface, the subjective percept needs to discount the color of the light source.

So it should not be surprising that inference of hue can be dramatically influenced by context. The same shade of grey can look almost black on a bright background but almost white on a dark one.

Lightness illusions are common. The shade of grey above and below the horizon are the same. You can see this by covering the strip in the middle

Note that this is not a bug. It is a necessary tradeoff in the quest to achieve a stable appearance of the same object, regardless of context or illumination.

White/gold vs. Black/blue: Some people perceive the image on the left, others the one on the right. Others switch back and forth.

As noted before, this kind of divergence of interpretation might be rather common with complex stimuli. The importance of the “dress” stimulus is the extent to which intersubjective interpretation differs in the color domain. To my knowledge, this is by far the most extreme such stimulus. Of course one has to allow for the fact that not everyone’s monitor will be calibrated in the same way and viewing angles might differ, but this doesn’t account for the different subjective experience of people viewing the exact same image on the same monitor from the same position. And of course the reason why the “true colors” of the dress are in dispute in the first place is the color constancy phenomenon we alluded to above. This was likely a black/blue dress that was photographed with poor white balance, giving it an ambiguous appearance. But that doesn’t change the fact that some people sincerely perceive it as white/gold.

That the interpretation of the color values itself depends on context can readily be seen if the context is taken away. In the image below, some stripes were extracted from the original image without altering it in any other way. The “white/blue” stripe can now be identified as light blue and the “gold/black” stripe as brown.

Stripes, out of context: One now unambiguously looks like blue, the other like brown.

But why the difference in interpretation? That is where things get interesting. If the ambiguity derives from color constancy (and it looks like it does), the most plausible explanation is that people differ in their interpretation of what the illumination source is. Those who interpret the dress as illuminated by a blue light will discount for this and see it as white/gold whereas those who interpret the illumination as reddish will tend to see it as black/blue. Interestingly, the image itself does allow for both interpretations, the illumination looks blueish in the top of the image, but yellowish/reddish in the bottom. On a more fundamental level, a blue/black dress illuminated by a white light source might be indistinguishable from a white/gold one, but with a blueish shadow falling onto it.

But if this is the case, one should be able to consciously override this interpretation once it is pointed out, but – for most people – this does not seem to be the case, in contrast to most other such ambiguous duckrabbit displays. People are able to willfully control what they see.

Inherently ambiguous stimuli. Interpretations switch, they don’t blend. But people can consciously override interpretations once they are pointed out.

This raises several intriguing possibilities. For instance, it has been recognized for quite a while that the human “retinal mosaic” – the distribution of short-, medium- and long-wavelength cones in humans is radically different between observers, but this seems to only have a minute impact on the actual perception of color. Perhaps it is the case that differences in retinal mosaic can account for difference in the perception of this kind of “dress” stimulus. Moreover, there is another type of context to consider and that is temporal context. We don’t just perceive visual stimuli naively, but we perceive them in the context of what we have encountered before – not all stimuli are equally likely. This is known as a “prior”. It is quite conceivable that some people (e.g. larks, owls) have a different prior as to what kind of illumination conditions they encounter more frequently. Or a complex interaction between the two.

While we must confess that we currently do not know why some people consistently see the dress one way, others consistently in another way and some switch, it is remarkable that the switching happens on very long timescales. Usually, switching is fast, for instance in the Rubin’s vase stimulus above. This could be particular to color vision. There are no shortcuts other than to do research as to the underlying reason that accounts for this striking difference in subjective perception.

Meanwhile, one lesson that we can take from all of this is that it is wise to assume a position of epistemic humility. Just because we see something in a certain way doesn’t mean that everyone else will see it in the same way. Moreover, it doesn’t mean that our percept necessarily corresponds to anything in the real world. A situation like this calls for the hedging of one’s bets, and that means to keep an open mind. Something to remember next time you disagree with someone.

Who is right? Does the question even make sense? Or is it important to see the bigger picture?

I scrolled back to the original picture and I immediately perceived it as darker. The cue from the darker picture apparently altered my interpretation very quickly. I wonder if that shift would be context-specific, i.e. just that picture of that dress, or if the “white-balance” in my visual world was generally altered?

I’m sure you’re inundated right now, but I wanted to mention one group of people that no one’s talking about: those who see the image as photoshop “sees” it. I.e., those who see baby blue and burnt orange or brown or olive and don’t correct for a blue or a yellow background. I’ve been informally polling friends in facebook and at work, and those who see it that way have, thus far, all been people who work in the arts (a painter, a photographer, an art teacher, and two photo editors) or children. The BBC video has one example of someone who sees it that way. This phenomenon seems potentially more interesting than, or at least as interesting as, the norm. I’m not in that group, personally. For me, the color switched after I stared at the blue-corrected image for a few minutes, and that switching experience has made me a bit obsessed with the whole thing (I’ve been playing around with other ways to make the color switch).

This is SO fascinating! In the hopes that more data will help vision and cognitive scientists figure out exactly what is going on, here is my family’s experience with “the dress photo:” When my 13-year-old son first showed this to me on his iPhone, I thought we were just going to argue about whether the dress was purple or blue, so I was surprised when (a) he said he saw white and gold and (b) when he ‘inverted’ the colors on his iPhone screen, it turned white and gold (in reverse for lace vs fabric) to me but was always white/gold or gold/white to him regardless of original vs inverted color on the screen. We tried dimming the lights in the room, dimming the iPhone brightness, etc. but nothing changed for either of us then.

More data: Our family is split down the middle: looking at the same photo on the same device at the same time, half the family sees blue/black (N=2) and the other half sees white/gold (N=2). The split in color perception is not correlated with gender or age (i.e., male/female and adult/child). Everyone has tested recently with normal vision, including normal color vision.

More data: I can’t switch between the colors looking at the entire image at once (it’s always blue/black) but I can switch in continuum when I look at a large version of the photo on a laptop screen when only a portion of the image is visible at once. Viewing the top part of the image only (with the bright light source in the top right corner of the image), the dress looks white/gold. As I scroll down, the colors change rather continuously to light blue/brown then to dark blue and black. Scrolling up and down so that the photo was entirely or only partly visible, this color change in my perception is 100% repeatable. There is nothing subtle about it nor am I making any (conscious) effort to try to make my ‘mind flip,’ as in the mammal/bird or Ruben’s vase bistable examples. It’s whether the bright light in the upper corner of the photo is visible or not (with respect to the rest of the dress) for me.