I'm working on a chart that displays and names a couple hundred color swatches. I have all of their color data in a table, and I want to arrange them by hue and lightness, so I applied the Rec.709 coefficients to my data. This arranges my colors into a natural gradient--except for the reds, which have some glaring missplacement. Any idea why .709 coefficients do this, or what coefficients I should use instead?

Okay, so it'll take me a while to digest all of that, but thanks in advance just for knowing what I'm talking about! I've asked this question to a half dozen people and only made it to a shoulder shrug.

Changing the gammas absolutely changes the order of the values. The distortion I was originally experiencing is due to the fact that the linear coefficients give an intense value (say 255,0,0) a lower Y than a moderate shade of the same hue (say 175,30,30). I don't fully understand the science behind gamma correction, but mathematically, if we want to boost the values of intense hues relative to others, the exponent must be greater than one; using 1/2.2 makes the problem worse.

For example, I have three values in my table: red (229, 0, 0); cardinal (178.6, 28.8, 52.8); and oxblood (122.1, 32.1, 41.5). Red is clearly the lightest, oxblood is clearly the darkest, and desaturating programs across different software and monitors agree.

Ranked light to dark by the Average method, we get Cardinal>Red>Oxblood

With the linear .709 coefficients, we get Cardinal>Oxblood>Red

With a gamma of 1/2.2, we get Cardinal>Oxblood>Red

With a gamma of 2.2, we get Red>Cardinal>Oxblood.

The last formula works emperically for my data, but I suspect it should actually be a little smaller because my resulting data seems skewed to the extremes.

My application is really pretty trivial--I'm just making a table of color names. But now I'm caught up in the intellectual exercise. You can take a look at my old draft (using linear .907 coefficients) here.

and I am not supposed to use the inverse of the gamma (1/2.2) after all?

This seems to put my names in the right order, but it skews my data very dark (about one third of my data points wind up with less than 20% lightness).

Is it safe to assume that my RGB values are gamma-encoded? My values are based on component averages of responses to digital color swatches, so they haven't been converted into RGB from somewhere else.

At this point I'm tempted to just desaturate the image in PhotoShop and manually eyedrop each color into my data. None of the formulas we've discussed in theory match the function in practice-- I don't mean to be a pain about it, I'm just surprised that it's such a mystery.