Eigenfaces for art

By Mason Simon

If you discard color, a 100×100 pixel image has 10,000 pieces of information–one for each pixel. Consider each image a data point. To plot them, you’d need graph paper with 10,000 axes, because each pixel’s value tells you the position of the image (data point) along the corresponding dimension. And you thought images were 2-D.

The reason for putting images in this 10,000 dimensional space is to find how far one point is from another; in terms of images: to find how similar one image is to another. This can be useful for face recognition. But 10,000 dimensions are too much (google “the curse of dimensionality”). Intuitively, we don’t need that many, because to describe how one person’s face differs from another, we say things like “his nose is bigger than hers”, “her mouth is wider than his”, … and these high-level differences aren’t captured by any single pixel.

Now forget about faces for a moment, and think about colors. All the colors your computer screen can display are formed by mixing red, green, and blue. So plot each color you’re interested in using 3 axes–one for red, green, and blue–and look at how close they are to each other in 3-D.

The critical step with colors was to recognize that each is a mixture of 3 components. Faces (and other images) can be analyzed this way too. Now the components aren’t obvious like they were for colors–you have to choose them yourself, and you’ll probably want more than 3. Principal Components Analysis (PCA) is the best way to choose them because it picks the components that express the most difference between faces–so just knowing that face A is 40% composed of component C1 (the best) while face B is 70% composed of C1 should tell you that A and B are very different. In the same way that red, green, and blue are themselves colors, each component is itself an image, like this one:

David Billington taught me that good engineering often produces good aesthetics, and that happened in this case. My next idea for eigenfaces as art is to incorporate color, which will take them out of the engineering realm (because color varies too much due to lighting to be useful for recognition, I’ve heard), but I’m curious to see what colorful eigenfaces look like.