Heather Dewey-Hagborg ~~ www.deweyhagborg.com

Stranger Visions: Software

July 9, 2012

I am still very much in the middle of my wetlab work for Stranger Visions but the software side – the part that takes DNA information and produces a 3d model of a face – is mostly complete. I thought I would talk a little bit about the software and the concept of inductive bias in this next installment on Stranger Visions. This is a slightly technical post but there is a kind of ethical question at the end…

I have worked with facial recognition algorithms in the past and one technique I had read about was the use of a morphable 3d facial model to attempt to recognize faces at weird angles. As I have previously turned systems like this, intended for surveillance, into systems for creativity, it naturally occurred to me that the same system could be used to generate faces. And if you can determine what correlates certain types of faces you can then generate faces with those characteristics using principal components analysis.

It turns out the research group behind the Basel Morphable Model realized this as well and after much digging I figured out how to use their matlab model as a starting point for many different types of parameters. They had already found the primary axes for gender, age and weight so the main parameter missing was ethnicity.

The issue here of course is that all these models are heavily skewed towards people of european descent so I knew from the beginning that I would have to deal with significant inductive bias. For those who have never read AI textbooks, inductive bias describes the set of assumptions a programmer makes in the design of a machine learning system. Or as Luger puts it in Artificial Intelligence “Inductive bias refers to any criteria a learner uses to constrain the concept space or select concepts within that space”. It is actually a pretty fascinating subject in its own right that dates back to the origins of inductive logic with Aristotle. The basic idea is that by looking at a certain number of examples of something we can form generalization about all things of this type. Now we all know what happens when we make assumptions based on limited experiences, so the flaws of inductive reasoning are pretty obvious. Then again, all reasoning rests on making some initial assumptions (otherwise we have nothing to deduce from!) so at the end of the day some amount of induction is unavoidable. For more on induction, deduction and their interdependence see Hume’s An Enquiry Concerning Human Understanding.

In programming inductive bias can be anything as simple as how many clusters to look for in a data set, to much more difficult issues like how to get a representative sample of images of a population for a face recognition system. (And just what is “representative” anyway?)

The Basel model I started with was trained on an even split of men and women but the majority of people in the training set were of european descent. This means that even though the model can be fitted to parameters representing other ethnicities it always has a slight in-built bias toward caucasian faces. With this in mind I then trained a new version of this program by fitting images from the FERET face recognition dataset (which is more representative of ethnicity composition in the states but still not perfect). I trained the system based on my own subjective interpretations and thus, like much of an artist’s work, the faces it produces are as much a reflection of me and my view of the world as they are any kind of objective scientific model. Then again, most academic researchers follow the same technique, utilizing the judgements of 5 men in an office instead of 1 woman in a studio.

I did all this work in matlab which I am by no means an expert in, and so I switched to Python for the rest of the code – mainly reading in various data files and turning that info into parameters for the matlab morphable model. The Biopython library proved valuable here!

And now I have a bit of an internal debate as to whether I should open source all this work or not. My impulse is to post the code online and let people do what they wish with it. But then another part of me wonders who will want to use it and what for? The point of the project is criticality and dialogue, not proving that this is technically possible. I never had the intention of making a “genetic portrait toolkit”. Do I really want to make this easier? Thoughts?

Like this:

Related

I hope you make this public or obtainable through some means. I cannot tell you how much I would love to use this software. I recently had my genetic profile looked at by 23andMe and would love to extend this to your software.

I find what you’re doing fascinating. I think its an interesting cut at capturing generic portraits, however, until we have a good means to determine genetic and epigenetic expression, and all the other life impacts reflected in any given sample, there is no good way to get any closer than a recreated face similar to its owner. I don’t imagine this will a tool that’s easy to misuse for some time. Of course, technology blasts ever faster forward at an ever accelerating pace and perhaps the day of being able to reconstitute a face, and many other tendencies is not far away. Great job.

Heather i think what you did is amazing and i could thing in thousands of apps for the benefit of the humanity and that will be your legacy to the world. You will help many people that is buried xxx for somebody to recognize them, give peace to parents searching for their sons. I would like to buy my portrait for mere curiosity. You can put your algorithm or app to a foundation to raise money for human causes. Just to give a couple of ideas. I can send you samples from Latin America for your project.

As a programmer, I am curious about the code and algorithms used so I hope you post it. Since you have already provided the critical information that something like this is possible, – anyone that is sufficiently motivated should be able to duplicate the work given enough time and energy.

As an artist I am curious about the whole process as well. Could you say what was involved in the actual physical mask creation once the STL file was done? What printer did you choose and how did you do the coloring?

Very great post. I simply stumbled upon your blog and wished to mention that I’ve really loved browsing your blog posts. In any case I will be subscribing for your feed and I hope you write once more soon!