Commercial AI is great at recognising the gender of white men, but not so good at doing the same job for black women.
That's the conclusion of a new study, "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification", that compared gender classifiers developed by Microsoft, IBM, and Chinese startup …

Re: Is spreading

Re: Is spreading

Surely it's absolutely not racist since the one thing it can't do is discriminate?

It is not the computer's fault to start off with. Basic photography 101. With the same exposure levels for a colour photograph you will get less contrast and less feature differentiation for darker skin colours.

If you want to make the computer job easy, change the spectrum band in which you take pictures. I suspect that you can get significantly lower error rate going into near IR. This is already being done for number plates by the way.

Re: Is spreading

Well, this is my first thought, poorer contrast. But it is an assumption. Even with a balanced training set tuning of parameters is often already done first on a separate set, so it could be at that stage too (and coming back the the contrast/IR issue, evaluation against other methods is often on smaller sets before being applied to larger ones, so the need for an appropriate imaging method can get missed).

Re: Is spreading

Re: Is spreading

Women tend to have more varying hairstyles, make up, glasses (if they wear them) than men, across cultures. But I agree moving the cameras into a part of the spectrum where melanin is not seen would probably help.

Re: Is spreading

the one thing it can't do is discriminate

Given the increasing fluidity of gender identity, I can't really see why attempting to form two tidy distinct subsets is even considered a worthwhile goal - unless they're trying to maximise the revenue from Valentine cards.

Re: Is spreading

Surely it's absolutely not racist since the one thing it can't do is discriminate?

That's a bit like saying that there's no problem with racial minorities being shot by police, because guns can't see skin color.

Computer software can't help but reflect the biases present in the data sets it's trained with (AI), or validated against (manually coded algorithms.) One of the dangers here is that computers will become a way to codify bias in a socially acceptable, plausibly deniable way. "It's not me, it's the computer."

Re: Is spreading

Re: Is spreading

"We now have racist computer systems. When will it end!"

On the bright side, we are, even here in the 21st century, still seeing regular news stories of black people being stopped and searched or assumed to be up to no good by white police officers. If the facial recognition is so much more poor with darker skins, then mainly white "persons of interest" will get flagged up by facial recog.

from three African countries (Rwanda, Senegal, South Africa) and three European countries (Iceland, Finland, Sweden) selected for gender parity in the national parliaments"

The authors are so blinded by their desire to find a bias on gender and skin colour that they've picked a test dataset that is massively skewed on ethnicity, age, health and social class.

They also tie themselves in knots trying to reconcile biological gender with gender identity, so their benchmark gender classification is suspect in the first place (it's based on Mr, Mrs and the "appearance of the photo").

The word "Intersectional" is the giveaway. A study declaring itself Marxist/Leninist would have identified that the discrepancy was class based and constituted oppression of the proletariat using exactly the same data and for exactly the same reasons (confirmation bias).

That doesn't mean they are wrong about the differential accuracy of course. It just means they have pointlessly poisoned the well regarding the integrity of the study.

Agreed. To be representative you really need to sample everywhere. Otherwise there are always going to be big gaps in its applicability domain. In this case Scandinavia is over represented, so it's going to be over trained on Vikings, while they've got just three really distant countries representing all of Africa.

I'm not sure if serious.

Many times it has been stated, there is more diversity and difference inside groups than there is between groups. So while we notice the differences of vast distant populations, we ignore through confirmation bias or social normality, the massive differences locally.

Re: I'm not sure if serious.

Many times it has been stated, there is more diversity and difference inside groups than there is between groups.

That may be true for genotypes (it's actually an oversimplification), but it doesn't hold for phenotypes, particularly not when you're only considering a tiny subset of phenotypical data which the human brain has specifically evolved to evaluate (faces).

Genetic diversity

Isn't 85% of all genetic diversity in humans found in Sub Sahara Africa (phenotype as well)? Remembering hearing that. Too lazy to get real reference but this wikipedia snippet will do for now.

"Sub-Saharan Africa has the most human genetic diversity and the same has been shown to hold true for phenotypic diversity.[36] Phenotype is connected to genotype through gene expression. Genetic diversity decreases smoothly with migratory distance from that region, which many scientists believe to be the origin of modern humans, and that decrease is mirrored by a decrease in phenotypic variation. Skull measurements are an example of a physical attribute whose within-population variation decreases with distance from Africa."

Re: Genetic diversity

Isn't 85% of all genetic diversity in humans found in Sub Sahara Africa (phenotype as well)?

Most of the genome has no external phenotypical expression and whole-genome genetic distance does not necessarily correlate with phenotype as the variation in dog breeds and the quasi-canine appearance of hyaenas illustrates. Conversely, sub-saharan Africans and aboriginal Australasians have the greatest genetic distance but are often considered to be phenotypically similar.

Sub-saharan Africans do not have any Neanderthal or Denisovan genetic contribution (3-5% in everyone else) besides some rare instances of genetic backflow from the Levant. This impacts externally observable phenotype, particularly in terms of eye and probably hair colouration (plus the immune system, hair texture, respiratory metabolism and a number of other areas).

Human perceptions of phenotype are evolved to assess ingroup membership rather than genetic distance per se. That means the environmental (in human terms, cultural) aspect of phenotype is often of greater importance. For a human, how an individual dresses and behaves (e.g. a military uniform, prayer rituals) is part of the phenotype just as the exact design of a nest is for a bird.

If linguistic patterns dating back a long long long time - several thousand years at a minimum - are a clue, Finns and Swedes are very different groups... one would expect the Finns are more closely related to Siberians than Scandinavians.

"I wouldn't call it representative if there are no or very few people from the Indian subcontinent or east Asia, not to mention South America."

Beat me to it. Between them China and India cover over a third of the world's population. It looks more as though they deliberately took the extremes of some of the whitest and blackest populations they could find. Which is fair enough for a certain kind of testing, but can hardly be considered representative and is likely to paint things in a particularly bad light.

Skin tone is not binary with the meta groups

So was the correlation purely on darker skin tone as such, or was it actually caused by a correlation between darker skin tone and different facial characteristics that react less well to the identifying parameters used by the algo?

Re: Skin tone is not binary with the meta groups

Computers are less good at recognising that black people actually have faces. That is due to the way cameras have been optimised to pick up lighter skin tones.

Also, secondary sexual characteristics interact with racial differences to make it difficult to gender people across different races.

One example is shape of forehead. If you look at white people, women tend to have a more vertical forehead and a low hairline, men tend to have a more diagonally sloped forehead, a more pronounced ridge above their eyes, and a higher hairline.

These differences also appear in black people, however black people tend have a more diagonally sloped forehead and higher hairline than white people, so you might find more black women with a similar shaped forehead to white men.

Also, black people are far more genetically diverse than other groups, so you can't make the same generalisations about them. There could potentially be more difference in DNA between one black person and another black person, than there is between one of those black people and a white person.

I love this misconception. Attention Black Women! Do YOU have an issue recognising yourself? Are you struggling with the mirror in the morning, or perpetually forgetting your own name? We have the solution: a Ph.D course in developing facial recognition software. Because no-one else will do it for you.

So facial recog is not reliable for non-white skin

If I recall correctly, there's an issue in the USA between the police and black people. This is certainly not going to help.

It is a curious result, though. One would think that the color of the people who wrote the facial recog code doesn't matter, a process was thought of, agreed upon and implemented and there isn't any reason why code examining a face should have a harder time detecting skin tone variations in dark pixels than in light pixels. I think that this might be a sign that the cameras taking the pics are having a bit of trouble properly capturing dark tones. If the data in is insufficient, the data out will be flawed.

Maybe the coding teams can find a way around that, but it looks like it's going to be difficult.

Re: So facial recog is not reliable for non-white skin

If I recall correctly, there's an issue in the USA between the police and black people. This is certainly not going to help.

A made-up issue, really. The truth is that the police and the "justice" system in the US are just brutal in general, and it's black people that happen to have recognized it because of the greater level of contact with the system due to higher rates of crime (and the root of that probably goes back as far as the reconstruction era attempts to keep the freed slaves enslaved in fact while being "free").

If you look at police shootings per unit of police contact, whites get killed slightly more often than blacks, and you can bet the whites from those socioeconomic strata don't have any illusions about the police being nice to them. It's just a needlessly brutal system, with the highest rate of incarceration of any western liberal democracy. Got to keep those private prisons filled and profitable, you know. Some things don't lend themselves to privatization, no matter how much I generally prefer the government get out of things. This is one of them, because there wouldn't be any prisoners in those prisons without the government to send them there in the first place, so it's not really private in any real sense... just an unholy mix of public and private that should not be.

Re: So facial recog is not reliable for non-white skin

Re: So facial recog is not reliable for non-white skin

It's just a needlessly brutal system, with the highest rate of incarceration of any western liberal democracy.

That's true, but one of the reasons the US has such high rates of imprisonment is that it has the prison places to keep people in for more of their sentence. Here in the UK, the lack of prison spaces means a "life" sentence usually equates to fourteen years, but most violent criminals are routinely released halfway through their sentence anyway (and many get shorter sentences by claiming they didn't mean to kill, so that's manslaughter and shorter sentence). So murder somebody, and in some instances murderers are released after six or seven years. In one recent UK case, some evil, criminal scumbag who'd been put away twice before for the manslaughter of TWO previous partners was released early and killed a third partner.

Personally I'd rather we built a few more UK prisons, although I accept they are expensive and don't reform prisoners.

Re: So facial recog is not reliable for non-white skin

"So murder somebody, and in some instances murderers are released after six or seven years. In one recent UK case, some evil, criminal scumbag who'd been put away twice before for the manslaughter of TWO previous partners was released early and killed a third partner."

Is that some kind of proof that higher incarceration makes the world a better place? Or some kind of random factoid?

Re: So facial recog is not reliable for non-white skin

"A made-up issue, really. .... If you look at police shootings per unit of police contact, whites get killed slightly more often than blacks"

No, not really a made-up issue. Stats like the one you mention are tricky because the denominator of your ratio, "unit of police contact", is not at all uniform across races in the US. Blacks get subjected far more than whites to unjustified police contact (for example, "driving while black" really is a thing, https://www.washingtonpost.com/news/wonk/wp/2014/09/09/you-really-can-get-pulled-over-for-driving-while-black-federal-statistics-show/). As a result, a much higher fraction of police contacts with black people are with non-violent, law-abiding citizens so naturally fewer of them end in shootings. If the measure were "shootings per unit of justified police contact" then it would be higher for black people.

But even measuring "justified police contact" is tricky because "justification" is complex, and racially tinged. Earlier, the police's word would be unconditionally accepted as "justification", eg "he tried to grab my gun". Now, with a number of truly disturbing cases of police lying, planting of evidence etc, that "justification" is no longer a straightforward matter.

Re: So facial recog is not reliable for non-white skin

Blacks get subjected far more than whites to unjustified police contact (for example, "driving while black" really is a thing...

I remember being somewhat startled to find out that every black person I knew had a story about being pulled over and harassed by the cops for no reason other than being black and driving a car. And these were well-off professionals.

You can normally find a fact to prove your case because any bias can also be within the studies objective.

All test images should have been picked at random, without knowing the colour or persons origin from the start. Results should have been listed for fat people, thin people, ill people, people with make up and not etc etc.

The results may have shown a inaccuracies with fat white people as well, perhaps also thin and people with a cold may have also have been inaccurate.

Perhaps the software isn't racist but just generally not good. Is Face recognition better or worse for detecting fat old white men with stubble on their face?

I'm not being racist...

...but they all look the same to me.

I have a slight form of Face Blindness. I have to study someone's overall features (clothes, a scar, hair type, mannerisms, size of ears, voice etc) to help me recognise people. I need to get a handle on someone and I suspect many other people have similar problems, whether they know it or not.

I have more problems with a woman's face partly I feel because so many have make up on which smooths out features. Chubby faced people have skin pulled a little tighter which removes wrinkles. I'm white and black people tend to look younger to me, their faces often look smoother and more round and I've thought this may be to do with darker skin not showing shadows so much on the face.

Is the lack of contrast from light to dark on their faces a valid reason I wonder? Do they have less wrinkles? I think I can think of a few other reasons computers would struggle rather than racism.

Another may simply be that most of the programmers working on these products are white men who when teaching a computer what to look for are using their own "white man's" algorithm in their brains to recognise friend or foe. The area's I have lived in, rural England, simply haven't had many black people in for me to identify them as well as white...perhaps computer geeks who have spent most of their lives in darkened rooms surrounded by other white people and computers in just don't know what the black peoples little differences are.

No, I'm not sold on the racism card within face id.

You strip the hair and makeup off of anyone and I wonder how accurate a human would be at spotting gender. I'm willing to bet your own accuracy would change between different races and colours.

Also, go find a group of black programmers who have made a face ID software and test their results.

Re: I'm not being racist...

Re: I'm not being racist...

"Prosopagnosia. I've met several people with that in my previous work." -- TRT

Me too, but mine is so severe that I'm not sure how many times I've met them.

On one occasion (apologies if you've heard me mention it before) I didn't even recognize my Dad when he paid a surprise visit. I also mistook (ditto) Zoe Wannamaker for a mature student of mine because her face was so familiar.

Re: I'm not being racist...

Re: I'm not being racist...

Prosopagnosia: Mine isn't so severe however I have to really concentrate on matching names with faces and give me photos of people that I don't know very well and I struggle connecting them at all, particularly if there are time differences in the photos or the location is out of context.

It's quite annoying/upsetting at times when you are out with people who don't have this issue and when you both meet somebody that you both know but haven't seen for a few years and they recognise them instantly and I'm there looking blank and clueless. It's not that I don't "care" about the missing individual at all, it's just that I genuinely don't recognise them even if I could recall their name and a lot of things about them.

Re: I'm not being racist...

"You strip the hair and makeup off of anyone and I wonder how accurate a human would be at spotting gender. I'm willing to bet your own accuracy would change between different races and colours."

Experiments show that the ability to recognize individuals varies with experience with identifying individuals of a given race... which is generally greater and learned earlier and more thoroughly with one's own race. It would be likely hat the ability to discern things about people one does not know is similarly affected.

Contrast

Could this be a problem with the light contrast present in different faces?

I've just looked at a series of images of faces and the white faces had a greater contrast range. If I squinted at the images, the white woman with the ridiculous Groucho eyebrows was still recognisable whilst the other faces merged into blobs.

If facial software it trying to identify points on a face isn't it going to struggle with a face with lower contrast? Blonde eyebrows on a white face and black eyebrows on a black face = low contrast. Of course that doesn't explain why it's accuracy is lower with female faces. Do women do anything to alter the shape or colour of their faces?

Re: Contrast

"Of course that doesn't explain why it's accuracy is lower with female faces. Do women do anything to alter the shape or colour of their faces?"

Suppose you have four people, A, B, C and D. A and B are 'male', C and D are 'female'. A, B and C look very similar, D looks different. Software will guess 'male' for A, B and C, 'female' for D. It gets 100% for male, 50% for female. So if there's greater diversity in the one dataset, it will get a worse mark on it.

Film and photography have a similar issue

This needs to be fixed for sure, its a failure to train and model on a full reference dataset.

This is not just a computer fail issue, when Kodak ruled the color film market in the 60's they calibrated the stock to 'Shirley Cards' which was a perfectly balanced picture of a white woman.

This meant they had issues with dark skin tones but was it even possible to have one stock for all tones? probably not, there is a whole specialism on lighting and filming darker skin tones today with digital movie cameras.

if you search for 'The hardest part of being in a biracial relationship is taking a picture together.' it also highlights difficulties with getting a good picture for light and dark skin tones at the same time, ML will have to deal with this as well.

Just saying its not a slam dunk to fix, there are technicalities to deal with.

Re: HDR!

That's IF the facial recog software can even acknowledge there IS a face, which with a low enough contrast it may fail (too low and the black face is washed out, but too much and a white face is washed out, and there's no easy way to know which way to take, especially if you have both types in the same scene).

Re: HDR!

"That's IF the facial recog software can even acknowledge there IS a face,"

Or consider the photo of myself and another person I uploaded to Facebook last year. Facebooks face recognition draw a rectangle around her face, and another one around my foot. I'll admit' I'm ugly, but my feet are uglier, so that's no excuse.

No. No and No.

Give a camera some infra red or some ultra violet vision etc, and it will probably out perform humans for fingerprinting (though may not for general recognition, with makeup/ageing etc. Though I guess nothing to stop them training it for those too).

Many many people make the error in thinking because a machine fails in the same way a human does, it's subject to always do so. Then forget we put wheels on cars, not legs!

Reminds me of the excellent Better Off Ted series episode https://vimeo.com/29017688 where the entire motion detection system refused to recognize dark skin and all people of colour had to be accompanied by a person of non-colour or something.

Re: Really

A lot of the problem there is you'd really have to be clear on what the intention of the data was. "People who use male pronouns," "people with an M on their driver's license", "people with a Y chromosome," "people with high testosterone levels," and "people with a penis" are sets that do not entirely overlap, but we often act like they do. As a result we tend to ask the wrong questions and get not very consistent results.

It's also worth noting that even given a set of only cisgendered people, humans do not guess gender 100% correctly. I've seen studies that showed faces with more contrast were considered more feminine, which suggests that our mental algorithms are skewed by our "training set," so to speak, having a lot of women wearing makeup in it.

Makes me wish

Sloppiness

While there are sampling problems that limit validity of the data it does point out basic a photography problem. Darker colors tend to show less contrast when photographed unless the photographer makes an real effort to compensate with the lighting and camera setting. My cats are very dark brown and without compensating for their coats facial definition tends to get washed out when I photograph them. I would say the error rate for white females should be a red alert that these packages will probably have an unacceptable error rate for any real identification no matter the race of the individual.

I'm actually not at all surprised. It's always been more difficult for cameras to pick up details from darker surfaces, so of course a machine will have more difficulty picking out facial features on darker skin. You could solve the problem by turning up the light sensitivity, but then lighter skinned faces would get washed out. The real problem is that the way cameras and lenses work just isn't as efficient as the way the human eye works.

Also, as someone who sometimes struggles with face blindness (on good days I barely notice it, on bad days I can't even pick my own sister out of a crowd) I gotta say that the fact that a computer can recognize faces at all amazes me.

Human eyes and lenses are actually auful compared to modern cameras and lenses.

The human brain however is an infinitley better image processor and interpolator than anything we have managed to develop technologically.

contrast is probably an issue as is the data set used, but this can be fixed by running the custoday photo database the police refuse to get rid of through the training profile. this will probably bias the system the other direction, if the custody book from my local Cop Shop is to be believed

One of the tricky things about that is, because a lot of what we "see" is interpolated by our brains, we're kind of fooled into thinking our eyes are much more reliable sources of information than they actually are.

One interesting example is a friend of mine who has migraine headaches that come with blind spots in his vision. He said until a blind spot covers about a third of his central visual field, he can't see it directly; the brain fills in what it thinks should be there, and he ends up looking at objects and not seeing them, or seeing blank pages where there should be text. There's a threshold beyond which the brain can no longer patch things over, and then he sees the blind spot as a shimmery area.

As the joke goes, any engineer who built a camera as bad as the human eye would be fired...although I think we'd cut them a lot of slack if they'd built it out of jelly and meat.

"The real problem is that the way cameras and lenses work just isn't as efficient as the way the human eye works."

Do we have scientific evidence of this or are we glossing over the possibility our eyes are even worse at it but we don't acknowledge it?"

Any serious photographer knows that the human visual system can see things that are difficult or impossible to photograph, while in some circumstances the camera (particularly modern digital sensors) can catch things the human eye cannot.

More often than not it seems to be the camera which is fussier about lighting, subject motion, focus, sensor movement, dynamic range, etc.

Re: It is dark when you leave the room...

Wigs and Heavy Dark Framed Sunglasses Will Beat Watson Most Every Time

The reference points for facial recognition are well known and are easily blocked or distorted. Try making your mouth into a 'rosebud' shape - that is a useful start.

The fallibility of identification technology us amply demonstrated by the HSBC voice analysis that is applied to every telephone call made to the bank.

With a simple tremolo circuit commonly used by musicians and a bandwidth limiting filter, my personal assistant can, and does, emulate my voice and easily persuades the HSBC computer that it is I who is speaking.