This illusion is pretty fascinating. It might actually point the way to a new discovery.

The color constancy effect has been known for a long time, and there are many illusions around that show (to the same viewer) how a colored surface can appear to be one hue or another depending on the circumstances.

There are two things that are surprising about this particular image:

Unlike many dual-interpretation illusions, this one is really stable. People see it one way or another, and there is very little that can be done to change their perception.

How people see it is not very affected by viewing circumstances. Two people can look at the same picture in the same light and see something completely different.

This raises two interesting questions: What is it about this particular image? And what does this say about possible differences in people's visual systems?

Why this image?

There are a few qualities about this particular image that may explain why it is triggering a unique illusion:

The visual system processes color at the earliest stages in two parallel neural pathways. One is a blue-yellow information channel and the other is red-green (as in red-green colorblindness). This illusion has something going on in the blue-yellow channel, it would appear. The blue-yellow channel is the oldest one, in case that matters.

The "colors" that are the subject of debate are some of the most ambiguous colors that the brain processes. White, black, and gold are all unusually problematic for the brain for different reasons that all have to do with context. More on that later.

Unlike a lot of illusions that show colors in the abstract, e.g. colored tiles over other colored tiles, this one shows a concrete, tangible, familiar object: a dress with texture and highlights in a "just photographed" context. This makes it feel real and incontrovertible. And because it seems like a concrete object, it is hard to "unsee" the version that struck you first once you have seen it (if that is what is going on).

What is special about white, black, and gold?

White, black, and gold are all particularly ambiguous as colors go.

A white shirt under red, blue, or green light still looks white, even though the light coming off of it is a "pure color".

Black is never really black, because if it was, you wouldn't see it. True black looks like a hole or featureless void. Black fabric still has texture and shades and a surface appearance even though it is "black." The non-black component of black is attributed by the visual system as a reflection of ambient light in the environment -- a point that is key to this particular illusion. For more information on black, see What is black?

Gold is particularly problematic for two reasons. First off, it's not a real color. There is no "gold" in the rainbow. Gold is an illusion created when yellow light is overlaid with a "metallic" appearance. Metallic, in turn, is a visual effect caused by "specular reflection", which is evidenced by sharp transitions of superimposed white light (the "shininess" of metal). The second problem with gold is the "yellow" part of it. Yellow can be composed of red & green light (with no yellow light mixed in) and changes color depending on luminosity and context. The brown of brown shoes is the same hue as the yellow of the sun, just darker relative to the surrounding.

So given the involvement of three of the more ambiguous colors, it is not surprising that they could form the basis of a striking illusion.

Pascal proposes two possibilities: that it could be due to differences in the color receptor make-up of different people's retinas (which are known to vary from person to person), and that it could be a temporal adaptation effect (once you have seen it one way, you can't unsee it).

I'll toss out a third possibility: people's internal color processing systems could actually be different, possibly shaped by experiences of understanding the visual world from early childhood.

Why do some people see the world in "black & white" and others in "shades of gray"? Why do people brought up reading from right-to-left (e.g. Hebrew, Arabic) give attention to the right side first? Why do people have favorite colors that are different?

This dress illusion may finally help us answer a centuries-old question from philosophy: do people perceive colors the same way?