Posted
by
msmash
on Thursday July 13, 2017 @01:40PM
from the therein-lies-the-rub dept.

An anonymous reader shares a report: The ACLU has begun to worry that artificial intelligence is discriminatory based on race, gender and age. So it teamed up with computer science researchers to launch a program to promote applications of AI that protect rights and lead to equitable outcomes. MIT Technology Review reports that the initiative is the latest to illustrate general concern that the increasing reliance on algorithms to make decisions in the areas of hiring, criminal justice, and financial services will reinforce racial and gender biases. A computer program used by jurisdictions to help with paroling prisoners that ProPublica found would go easy on white offenders while being unduly harsh to black ones.

Besides, we as their creator are flawed beings so inherently, our creations will be also flawed.

I'm not sure this is a flaw. If the data shows a gender or race bias, the AI will reflect that. Some biases based on gender and race exist, regardless of what the PC version of existence is. You can call it unfair, but not inaccurate.

We read constantly about so-called racism based merely on the fact that one race objectively exhibits a particular trait over other races.

That's called data, not bias.

Ok, let's start with the fundamentals. What exactly is 'race' here? You may think that's obvious, but all people have their own mixture of ancestors, so how are you going to sort everyone objectively into bins? If you can't do that, how are you going to objectively determine the traits of these supposed bins?

Rather than race, think of it as "culture". It's why first and second generation African immgrants vastly exceed 3+ generation African Americans [qz.com] in terms of economic and scholastic success. American black culture is the issue, not prejudice against blacks in general. Biases against blacks are because of the prevalent US black culture creating the dominant image of what a black person is. We have cultural biases, not racial biases... It's not DNA - it's culture.

I'm not so sure about that.Africans and African American are two very distinct races genetically.

So, go take a bunch of people from a culture as genetic stock. Now go ahead and remove any that can't survive a grueling 10 week voyage from the genepool entirely. Next, add selective breeding for about 8 generations as slaveowners try to have the next generation be more efficient laborers.

When you combine all of those, it drastically changes the genetic composure, enough that I would consider them different

"So there is a genetic reason to have bias about hiring people - some people are just "born lazy and ignorant"?"

Not so much lazy and ignorant as a combination of factors. If you look at performance of individuals in western societies, factors representing success correlate pretty well with IQ, to a point. Generally, we see about 80-85% of performance being innate (genetic), while around 15-20% is environmental. We see the same thing in physical performance - no amount of work will make an Olympic athlete out of someone without the body for it.

Black culture is certainly toxic, but it's also a reflection of genetics. They feed back on each other. There has been a ridiculous amount of money spent over decades trying to solve the black-white achievement gap, yet it doesn't work. It can't work.

There are population differences between the black and white population in the US that are compounded by the effects of poverty, malnourishment, and poor education.

Poor education, culture, and poverty feed back on themselves - it takes only a single student to disrupt an educational environment, so if you have a higher percentage of special needs students (or simply disruptive ones), there will be a greater percentage of classes where it's difficult for children to learn. The ability of a school to fund smaller classrooms is a function of its funding, which is often a function of where it's located and its taxbase. Poverty tends to concentrate individuals into areas where mass transit is an option, and so you get a perfect storm of a population that is already dealing with a lower mean IQ coupled with poorer education across the board.

This is also why voluntary busing can help with education, but only to a point. If you bus the non-disruptive students to better schools, they benefit from being removed from their disruptive classmates. If you bus the disruptive classmates as well, you harm the education of wherever they are bussed to.

I went to one of the former schools - black parents with above-average children who wanted their children to receive the best possible education would choose to send their children to my school. They were driven to succeed, and accountable to their families, and it did not adversely affect our education, but it helped theirs significantly.

So, no, it's not that they are born lazy, or ignorant. Those traits may be present as a class as a function of IQ, but like anything else individuals are individuals, who vary greatly. We can draw conclusions about a population, and estimate likelihood based on those conclusions, but you never really know what an individual will do until they are given the chance to do it.

What are they calling "bias"? We read constantly about so-called racism based merely on the fact that one race objectively exhibits a particular trait over other races. That's called data, not bias.

It's a tricky question. Just because something is data, does not mean that it isn't biased: data can be biased-- in fact, 90% of what we do in experimental science is understanding the bias in data and figuring out how to get an unbiased measurement out of a biased data set. Almost all data is biased one way or another.

If, for example, white people caught shoplifting are usually given a warning and let off while black people caught shoplifting are arrested and prosecuted ("shopping while black" [ibtimes.com]), the data will show a higher rate of shoplifting among blacks. You will need to go to the raw data to see the actuality. See: https://www.theguardian.com/la... [theguardian.com]

Of course it can. In fact, it pretty much always will. You can deliberately or accidentally ask leading questions, or survey a non-representative sample set. Then the data is biased in some direction, and if you want the truth then you're going to have to figure out how that inherent bias has affected your data. Or if you don't want the truth, then you figure out how an inherent bias is gong to affect your data, to get your desired goal. Five out six dentists that we asked agree that money is cool.

"Further, the fact that more people of a particular race are prosecuted is not a reflection of bias in the data, rather a bias in the prosecution."

In this case, "persecuted" was more accurate.

Data is Data. It cannot exhibit a bias.

I can only surmise that you're not an experimental scientist. Data has bias all the time.
In physics (my field) the bias usually has no social consequence-- astronomical statistics, for example, are biased toward bright stars (since they're much easier to see than faint ones, and hence overrepresented in the data set). In social "sciences," however, the bias very often does have social consequences. SAT scores from children whose parents spend tens of thousands of dollars on SAT Prep courses, for example-- surprise!-- score better on SAT exams than ones who don't. The data shows a correlation of SAT score with parental income. Is this real? Better correct for the SAT-prep course effect before making a conclusion.

Data is biased. All the time. Be ready for it.

...Plus, being from the Guardian, I am skeptical that they didn't twist the data some to obtain their desired outcome, which ironically touches on the subject of this story.

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.

"What are you doing?", asked Minsky.
"I am training a randomly wired neural net to play Tic-tac-toe", Sussman replied.
"Why is the net wired randomly?", asked Minsky.
"I do not want it to have any preconceptions of how to play", Sussman said.

Minsky then shut his eyes.
"Why do you close your eyes?" Sussman asked his teacher.
"So that the room will be empty."
At that moment, Sussman was enlightened.

The data is incomplete. AI, like humans, makes mistakes like "correlation = causation". The problem is, like some humans, AI doesn't understand this and can't ask for additional information or self-correct.

You're an idiot.

The AI doesn't need to understand anything. Nor does it need to ask for additional information.It absolutely does self-correct. When it encounters data that doesn't match its model it adjusts the model. If the AI is biased to say that a certain sex is more likely to have a certain trait, then if it encounters data that says otherwise the model is adjusted.

This is why AIs have a "training" data set and a "testing" data set. You train it until it's good, then you test it on data it hasn't

The data is incomplete. AI, like humans, makes mistakes like "correlation = causation". The problem is, like some humans, AI doesn't understand this and can't ask for additional information or self-correct.

Very much this. Reading the ProPublica article [propublica.org] (the Axios one in the summary doesn't have anything useful except a couple of links - this being one), it's easy to see that the real complaint is that the sentencing algorithm appears to have problems with accuracy when its predictions are compared to what really happens.

Interestingly, if this article [washingtonpost.com] is correct, race is not one of the inputs into the system in question (Northpointe's Compas system).

Reading the field guide for the system here [northpointeinc.com] I was impressed

Or the bias lies with the notion that everyone should come out to be exactly the same. If you have an AI that doesn't even consider race, gender, age, etc. but still produces results that have an uneven distribution, then it's pretty likely that age, race, gender, or any other characteristics we could care to measure are not meaningless descriptors and are correlated with other factors whether we like to admit it or not.

If an AI program says someone is a bad financial risk without any knowledge of their race, gender, age, etc. then it's because the person is a bad financial risk based on the factors it was given to consider not that the AI is discriminatory. The AI is going to be the least discriminatory thing possible, because it is incapable of having human-styled prejudices unless explicitly programmed to.

Or the data being fed in could be biased. Take for example the idea of repeat criminal offenders. The data may say that in New York City, black men are more likely to be arrested after release than white men. But for years stop and frisk was in place so black men where constantly being stopped and frisked and arrested for minor infractions. So yes, they are more likely to be arrested by that is not the same as more likely to reoffend. They are more likely to be caught because the police stopped them more. So yes, the algorithm fed that data would say black men would reoffend more and it would be true to the data, but not true to the actual facts. Bias can be in the algorithm but it can also be in the data itself.

You have two sets of populations. Say, hypothetically, the exact same percentage of each set carries contraband around, Members of one set are stopped and frisked with no probable cause more often than the other. That set will have a higher rate of arrest for that contraband not because they are more likely to have it, but because they are more likely to be searched.

The data fed into the system has a race bias, so the output necessarily does as well. None of this is a surprise. Other than the indications sometimes that it's the AI programmer's bias, not the data's bias.

Besides, we as their creator are flawed beings so inherently, our creations will be also flawed.

This is the key. And you don't have to spew 8chan-style garbage at an AI to "make it racist." It will pick it up from humans on its own, from training data built with human prejudices. One of the most amazing things about AI is how good it is at copying human biases without having any of the relevant inputs. You may not teach your AI that race is a thing, but it will find from training data that certain factors have some correlation with a certain outcome and it will copy that behavior, and those factors wi

The problem is not that the data set reflects the reality. The problem is not that the AI makes mistakes, but that the particular mistakes the AI makes reflect the bias of the society that programmed it.

We also turned up significant racial disparities, just as Holder feared. In forecasting who would re-offend, the algorithm made mistakes with black and white defendants at roughly the same rate but in very different ways.
The formula was particularly likely to falsely flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants.
White defendants were mislabeled as low risk more often than black defendants.

The problem is not that the data set reflects the reality. The problem is not that the AI makes mistakes, but that the particular mistakes the AI makes reflect the bias of the society that programmed it.

I believe that the newer ways of "Deep Learning" methods of teaching AI will address these concerns

Sounds like just faulty programming on that article you referred to...it said this for the training of their AI:

"orthpointeâ(TM)s core product is a set of scores derived from 137 questions that are eith

You can't have AI that learns on its own and have AI that isn't racially biased unless you artificially code blocks to it reaching certain logical conclusions. Then of course you've just made a dumb AI. The entire point of big data is to ferret out patterns in the noise.

It's easy to provide AI with data. It's hard to make it understand the limitations and biases of that data. For example, the data shows more black people carrying illegal items, but mostly because the police stop and search them more frequently than white people.

For example, the data shows more black people carrying illegal items, but mostly because the police stop and search them more frequently than white people.

... which is itself based on the observation that black people are more likely to carry illegal items.

This is a problem that customs deals with all the time. They discriminate in their searches because it's significantly more effective. In Canada, for example, Americans going to Whistler have their electronics searched because there is a high amount of illegal work. Americans going to Alaska are searched for guns (because they found so many).

They have non-profiling days where all selection is random, and they have mandatory times when everyone gets searched. They do this to validate their discrimination models, and waste a lot of time finding very little.

Evidence-based policing is going to end up racist, because reality is racist.

Are they? His argument is is the reason for blacks getting caught more often is they can checked more often. It's not that they are committing more crimes, it's that they get caught committing more crimes. Very different. It's a self-fulfilling prophecy. You think they commit more crimes, therefore you non-randomly check them more often to see if they are committing crimes and you find some fraction of the time that they are and use this as justification. Maybe the exact same thing would happen if you did t

So true, we were on a road trip when I was a teenager (3 countries blah blah, I read a LOT) and my dad realized if he was wearing his sunglasses at foot and mouth disease checkpoints the car always got stopped and searched (which was a pain because it was packed to the brim). If he took them off before we got to a checkpoint we were waved through without a search.

FYI in foot and mouth disease outbreaks they routinely put up roadblocks in strategic areas and any meat is not allowed through, it's kinda li

The problem in this particular case was something completely different. The program was weighing socio-economic factors like schooling, relation to parents and siblings, financial troubles, all those things that can predict recidivism. And if you had too many of them counting against you, it predicted you as a future criminal. The problem was that many white criminals come from a quite sound background, and most of the factors used to predict the future criminal career were ok with them (good schools, healthy relationships etc.pp.), giving them a good score, better than reality. They were twice as likely than predicted to become repeat offenders. On the other hand, blacks often have many factors counting against them, and thus the program gave them a quite low score, lower than reality. In fact, they were only half as likely to become repeat offenders than predicted by the program.

It was determined, that the program gave too much weight to the sheer number of factors counting against the person instead looking how bad some of the factors were. It would rather give a white guy with repeated offenses against other's sexuality a good score (because for him, only one factor looked bad, all others were ok, like steady income, no drug use etc.pp.) than a black charged with theft, because he might have been a homeless school dropout, with no known siblings or caring parents.

Indeed, I would consider racial bias to be a subset of "faulty programming."

Far from it. A system that lacked the racial bias reflected in reality would by it's very nature be flawed, and racially discriminatory. It would have to be skewed in such a way that it disproportionately benefited specific populations based on their race in the interest of "not being biased".

A simple example to illustrate the point, using something that's not as polarizing as criminality:

Suppose we wanted to estimate cancer risk for individuals. As is often the case in statistics, the goal is to estimate the values of unknown attributes using known attributes.

In this hypothetical scenario, white people have double the cancer risk of black people. We've also decided that for reasons of policy that it's immoral to judge people on the basis of their skin color, whether or not that actually correlates with risk.

If we looked at basketball players (for example), we might see that white people tended to play basketball individually, and focused on activities that could be done by themselves (shooting longer distances), while black individuals tended to grow up in urban environments with busier courts, and that they would focus on shorter shot distances, and skills which would contribute better to 5 on 5 games.

If we train a model using that data, we could easily find ourselves in a situation where the average shot distance ends up correlating with one's risk of cancer, because cancer correlates with race, and race correlates with shot data. This is normal, and expected, because the underlying data itself reflects this reality.

Since blacks have higher criminality rates, and higher recidivism rates, any just risk assessment algorithm is going to end up biased against black individuals. This is true whether their increased crime rates are due to poverty, intelligence, broken families, economic inequality, bad education, increased use of welfare, take your pick.

At the end of the day, the correlation won't tell you why - just that it's there. If the risk is higher for black individuals, and it doesn't assign (on average) a higher risk for black individuals, then the algorithm is a bad algorithm, because it's been weighted in such a way that it will disproportionately favour black individuals. It's social engineering that sends people of other races to prison more often in the interest of political correctness.

Indeed, I would consider racial bias to be a subset of "faulty programming."

Far from it. A system that lacked the racial bias reflected in reality would by it's very nature be flawed, and racially discriminatory.

Stop right there. We're talking about different things.

You are talking about "racial bias reflected in reality", but the article I am referring to is talking about racial bias that is in the output of the AI but is not reflected in reality. The article talks about the comparison of the AI output with actual results that show that the AI overpredicts blacks will commit crimes, and underpredicts that whites will commit crimes. The AI is not "reflecting reality".

the particular mistakes the AI makes reflect the bias of the society that programmed it

Except that this appears to be just speculation: Imagine if (for whatever reason) black American men in a certain situation (income, neighborhood, etc) have a 10% recidivism rate, while white men in the same situation have a 20% recidivism rate. The AI has to give a single number for both groups (since race is deliberately hidden from it), so it guesses (say) a 15% chance of re-offending. So it over-estimates the chances that a black man will re-offend while underestimating the chances for a white man - without any racial bias whatsoever.

Ironically, giving it race as an input would allow it to make more accurate predictions and appear less biased.

There's a chance I've missed something, but barring that, all this demonstrates is that people don't understand statistics and have a strong urge to explain everything as racism.

Actually, I am unaware of any women currently on any NBA rosters. Ignoring the small different in men vs women in the population, about half of random people will have 100% likelihood of not being on an NBA team, and about half have a 99.999% likelihood of not being on an NBA team. Those probabilities may still add up to the same thing, but practically, if I meet a random woman black or white, I still can be absolutely certain she is not on an NBA roster.

Saw a TV ad once for a medical show about a man born without a penis getting a "bionic" one. But the blurb said "Andrew is the only person in Britain born without a penis due to a 1 in 20-million condition". I was forced to infer that women in Britain are born with penises.

That, or that people insist on using gender-neutral pronouns even when doing so leads to silliness. Similarly, sportscasters have a checker history of referring to important "firsts" by "African-Americans" except that they sometimes aren't African-American at all...they may be actual Africans from African countries, or may be dark-skinned people born in Britain or elsewhere in Europe ("European-Africans"?).

What does Formula One driver Lewis Hamilton have in common with former heavyweight champ Lennox Lewis? They're both famous athletes named "Lewis," of course, but they also have the distinction of being two of the most recognizable African-Britons on the planet. What, you've never heard the term African-Briton before? Perhaps you, like certain media outlets we know, need to learn how to use the term "black."

Here's ESPN's correction after Hamilton won last weekend's Canadian Grand Prix:"On a June 11 Mike and Mike in the Morning news update on ESPN2, Formula One driver Lewis Hamilton, the first black person to win an F-1 race, was termed an African American. He is from England."

Here's how the Charlotte Observer expressed regret:"A story in Monday's Sports section misidentified Lewis Hamilton as Formula One's first African American driver. It should have said he is the series' first black driver. Hamilton is British."

Lennox Lewis was also regularly mislabeled, usually by columnists discussing the "African American" dominance of the heavyweight division.

Of course, it's not only athletes who have to deal with this strange combination of political correctness and geographic ignorance from American writers. Brits Naomi Campbell and Thandie Newton have both been referred to as African Americans. (Newton at least has the African part down, as she was born in Zambia.)

Maybe as punishment, the journalists should be forced to listen to a lecture on the differences between African-Americans and black people by Gary Sheffield.

The story of "Harrison Bergeron" by Kurt Vonnegut, which can be found in "Welcome to the Monkey House" [amzn.to], where the government restricts everyone to be average: the beautiful wear masks, the athletic wear weights, and the intelligent have radio implants to make them stupid..

I'm pretty torn on the concept. Logically a computer learning system, should in turn be able to over time figure out the ideal outcomes. IE If any races or genders are more likely to commit certain crimes, it makes sense to let the algorythm factor that in to projections.
But on the other hand, no data set to work with, is free of bias. IE if you are going with arrest reports, there's no way to know whether the people doing the arresting were mostly only watching one particular group etc... and thus a huge

It's not that the AI or algorithm has a bias, but that it's trained or given inputs that have that bias. For example, in the parole system, the software was given inputs that included not just details of the crime and sentence, but subjective ratings by guards who may well be racist. As usual, garbage in leads to garbage out.

But as Wexler’s reporting shows, some of the variables that COMPAS considers (and apparently considers quite strongly) are just as subjective as the process it was designed to replace. Questions like:
Based on the screener’s observations, is this person a suspected or admitted gang member?

And:

The New York State version of COMPAS uses two separate inputs to evaluate prison misconduct. One is the inmate’s official disciplinary record. The other is question 19, which asks the evaluator, “Does this person appear to have notable disciplinary issues?”... An inmate’s disciplinary record can reflect past biases in the prison’s procedures, as when guards single out certain inmates or racial groups for harsh treatment. And question 19 explicitly asks for an evaluator’s opinion. The system can actually end up compounding and obscuring subjectivity.

By definition, you can't claim that system is objective when it calculates a number based on "an evaluator's opinion".

90% of murdered blacks [fbi.gov] were killed by blacks, whilst 83% of murdered whites were killed by whites. And 57% of all murders were commited by blacks. Was it 99%? no - but it wasn't far off from 90%, the real statistic...

Well, when the system uses inputs that explicitly include guards' opinions, and then it's output just happens to show a huge racial disparity that does not correspond to statistical reality, that speculation may just be right.

After political correctness has subjugated humanity, it sets its sights on the machines! I take some small comfort in knowing that it can never actually change reality itself. Even if no one is allowed to notice, the world will continue following the laws of physics.

The AI is only as smart as the data its fed. If the statistics are biased (as in, mathematically, not subjectively), then the AI will be as well. The only way to "fix" this will be to either cook the input, or add political correctness to the algorithms.

I get that the ACLU and others are afraid that this will cause a feedback loop to reinforce stereotypes, but altering the AI is the wrong way to go about it. This is a societal problem that needs to be fixed at the societal level.

This is a societal problem that needs to be fixed at the societal level.

There is no problem.

When black males show less upwards social mobility. When women regularly earn less than men for doing the same jobs...

One way or another there is a societal problem. I can't say if it's whitey holding the black man down, or the black man holding himself back through poor social mores. Either way it's a societal problem.

Remember: So-called, inaccurately named 'AI' cannot actually 'think'; it's just mimicking us -- or at least some of us. It doesn't have a 'bias' of any kind, because that implies congnition, which is a quality it cannot posess. If your 'deep learning machine' or 'algorithm' is spitting out racist/sexist/ageist data at you, blame humans, not the machine. It's only doing what it was programmed to do, it has no 'free will', it has no 'opinions'.

So the real story in their cherry picked example is two fold:-It's wildly inaccurate, and Northpointe's product should be put out to pasture and never used, period.-A system is being used to influence punishment that is not open to auditing because 'proprietary'.

Note that the systems explicitly did not have knowledge of race. So we have two possibilities:-Some criteria that correlates to race is triggering it-The system is perpetuating existing bias in perception and reality. For example:
-"Was one of your parents ever sent to jail or prison?" could easily cause the ghosts of prejudice that caused unjust incarceration to recur today.
-"How often do you get in fights at school?" Again, if one is subjected to racial tension, they may unfairly be a party to fights they didn't ask for.

Yes, I read through the ProPublica article and my takeaway is that the systems are flawed and should be reviewed and either fixed or scapped. If your algorithm is supposed to predict recidivism, and it fails to do so, then it's broken. The fact that it fails to do so in a racially baised way is really icing on the cake.

What is sad about the US in general, and Slashdot specifically, is that the comments here about the actual data and the failures in this correlative model, are basically left alone, while all the racist "See even them super smart computers know nig... sorry... blacks are ebil crooks" shitposts, get to +5 almost immediately.

Slashdot needs a new slogan: Validation of biases. No intelligence found here.

I'm going to argue that in the context of training AIs (neural networks, esp.) on data sets that we may very well be imparting biases on them. If the conclusions present in the data were arrived at by biased means (in this context, I'm suggesting historical prolific racism/sexism), those biases should be present in the behavior of the resulting construct.

That aside, attempting to compensate by overriding the output of the AI with some sort of counter-bias indeed seems like a terrible idea.

Probably making my points here less relevant, I did not see any direct references to neural networking; if these are all just human-programmed algorithms (lacking the abstraction of the neural net stuff), I don't have much else to add.

AI learns from our own biases. Those who claim that reality is biased and not humans tend not to think that many biases are self fulfilling prophecies. Black people are not naturally more violent, but poor people are, for many complex social and psychological reasons. Don't forget that black people started as slaves in North America and that it most often takes many many generations for poor people to get out of poverty, which is getting even harder now with income inequalities. So, are black people more vi

I suppose its just not inflammatory/sensational enough to say: "Some programmers gave an expert system some data to look at and it gave a result."

Instead they want us to pretend there are actual thinking computers that are racist or sexist or something else even more silly, AND lets start changing them to be more politically correct because 'reasons'.

This madness will never end will it? It will just cycle around from obscurity to inflammatory and we have to keep beating it down forever?

If the data being fed in is accurate, I don't see how we can treat that bias as anything other than a rational response.

The real problem isn't that the tool is making an data-driven (even if "biased") assessment regarding the tendencies of a subgroup within the population, but rather that the tendencies of the group are being used to make decisions about how to treat individuals. That is the essence of stereotyping, whether it's done by a human or by a machine. Stereotyping is wrong because it disregards individual choices and personal responsibility; morality aside, it's also a poor guide since the variation within a given

Or, rather, adopt the mindset that an AI is somewhat like a child. A child that grows up in a (racist/sexist/whatever)-ist household is statistically more likely to turn out fairly similar, as is a child whose school curriculum holds such biases. The people implementing/training these things are going to (hopefully subconciously) impart their own biases upon them, or at least the biases present in the training datasets. If you train a parole-bot with all of our (US, but probably most places) historical parole data, of course it's going to be quite racist! I don't know what the 'proper' solution is, but I feel like attempting to manually adjust the AI after the fact is a terrible idea; to me, it makes more sense to manipulate the training data set until you get a reasonable result.

It's just a computer program, isn't it? We could just NOT feed it race and gender information, have it crunch probabilities, and see what kind of determination it comes up with. It should be that easy, shouldn't it?

I work at a company that scores job candidates with an AI system, so I have some experience with this. One thing to keep in mind is that most AI systems these days are deep learning algorithms that depend on a reliable training set. If gender or racial biases exist in the training set (whether justified or not), a good deep learning system will learn these biases and propagate them.
My company makes an active effort to prevent these types of biases from creeping into our system.

So we acknowledge that black offenders are statistically more likely to reoffend than white offenders.

But why is that? I know a lot of people assume that this is “just how black people are.” But the image media paints of “black” is far more socioeconomic than anything else. Do poor blacks commit more crimes than poor whites? What about in the middle class? Upper class? If poor whites and poor blacks have differences in recidivism, is this due to a cultural or genetic difference in how these people handle the stresses and challenges in their lives? And if so does this difference conver advantages in other circumstances?

Something we need to be mindful of is that people often conform to the roles that others assume for them. If you’re black and everyone assumes you’re going to be a criminal, and one day you get an immoral impulse (like ALL humans do), the negative self-image that was handed to you will be a strong influence over how you decide to give in to that impulse or not.

My dad always had this attitude that women were less intelligent than men. He would never admit to that, but there are assumptions he made that had an effect. My sister had dyslexia and she’s female, so there was always this belief that she wasn’t more than “average” intelligence. And once people develop a belief, it is common for them to only notice the things that confirm that belief, while things that contradict it get automatically filtered out. It turns out that she is extremely bright, just not in areas that my father recognized. Long story short, I’m betting that if she had been recognized for her intelligence, she could have channeled that positively. Instead, she turns into a manipulative sociopath.

Other people’s beliefs about you can fuck you up.

The biggest impediment for blacks to get out from under this higher recidivism trend is what people assume to be the cause of the trend. It’s chalked up to something inherent about being “black.” Commonly, when a white male makes mistakes, people are apt to blame it on stress or other external factors, and they’re working hard, and they mean well, and they’re doing the best they can. Only after someone has evidence of nefarious intentions do we change our opinion. If we were to treat everyone else the same way, it would make a world of difference.

Sure, and that's totally fair. The issue comes when, say, 60% of JobsRequiringNavigatingSkills are men and 40% are women, and people say "this is unfair".

To be honest, though, it depends on the job. Men have, typically, much more upper body strength than women, so are more suited to being things like garbage men. Yet nobody's clamoring for equal numbers of women to be garbage *people*.

Yet they are for firefighters, even though firefighting is basically a job where you turn upper body strength into saved lives, simply because they want to be seen as "equal".

People are different and have different things they're good at and bad at. Most HR people are women even though that's a comfortable, high paid, safe job. And I'm okay with that.

Um, wrong.
Blacks aren't more violent. Current popular black culture is violent, which is teaching black youth exposed to it to be violent. Asians aren't "good at math". Most Asian cultures put more of an emphasis on math at an earlier age than western societies. Non Asian students studying overseas from an early age are also "good at math". And children with an Asian ethnicity but born and raised in western cultures are just average at math.

The problem is making policy targeted at individuals based on statistical correlation of a group. We have this individualistic notion in the US at least that every person can forge their own path in life.

That narrative doesn't work when there are systemic barriers put in place pre-emptively due to statistical analysis.

Very few people deny the hard numbers that black people (in the US) commit more crimes. Or that chinese/japanese/korean (in the US, not all "asians") 1st and perhaps 2nd generation people are more academic. I haven't looked up the women and navigation statistics.

The problem comes when you take that general statistic and start making policy that target individuals. E.g. "Looking for a data analyst? Hire that asian-looking guy!"

Even worse when it comes to measures that perpetuate said statistic. E.g. "he's black, so let's assume he's guilty of a crime until proven otherwise".

Blacks are convicted of crimes more often, certainly. Does that mean they're more violent, or that they get caught more? Or that they live in worse situations than whites? Are Asians particularly good at math, or do Asian parents favour certain qualities that lead to more favourable math outcomes? Are they in more stable communities so their kids have a better opportunity to study math? Is it cultural or innate? Are women actually bad at navigating, or is it that we're less likely to take little girls out to go camping and get experience at navigating? Is that your own bias, since I've always heard that women are better at navigating?

We actually have statistics that white people just aren't convicted as often for drug offences despite having similar or higher rates of use and dealing. Based on conviction data, a machine learning system would internalise the bias that blacks are more likely to have an involvement with drugs, despite that not being true. Garbage in, garbage out, right?

(Notice that those articles are from 2009, 2011, 2013 and 2014—this is not new data.)

So generalities are not necessarily based in reality. Indeed, your claim that 'Asians are good at math' is particularly bad since Asia is HUGE and there's no way everyone from that area of the world is good at math. And as a half-Chinese guy that's okay at math but much worse than my white partner, and who knows plenty of Chinese people that have no affinity for math at all, I feel like a lot of these generalities are based on folklore and a few selective tests that aren't really representative of ability.

The USA and Canada are not the bastions of equal opportunity that they purport to be, not for everyone. First Nations people in Canada and black people in the USA are consistently disadvantaged through broad government policy.

So all this to say that getting good, clean data for machine learning systems that remove human bias is incredibly difficult, since most humans are unwilling to admit their biases don't necessarily have a basis in reality, or are the wrong conclusions drawn from incomplete knowledge of data.

Blacks are convicted of crimes more often, certainly. Does that mean they're more violent, or that they get caught more? Or that they live in worse situations than whites?

It means that the first 10 times Johnny White gets caught stealing gum, he gets a warning by the shopkeeper, the next 5 times the shopkeeper calls the cops and he's taken home by the cops, then the 16th time, he's formally warned, having that be the first time there's any formal record of his misdeeds. Tyrone Brown gets charged the first time, and gets 10 years "to make an example of him".

That's why the conviction rate isn't a good statistic, the data shows that the entire system has biases.

It's interesting how you redirected the discussion from "violence" to "drug offenses", which are entirely different things. According to the FBI stats in 2013, there were 2,698 murders committed by blacks, and and 2,755 committed by whites. When you consider that blacks only comprise 12.2% of the population, yet committed nearly as many murders as whites which are 63.7% of the population, there is a significant tendency towards violence. Additionally, 83% of the people murdered by blacks were also black,

Blacks are vastly more violent per capita than Whites, as shown by the DOJ random surveys asking about crimes one has been a victim of in the past year, then asking particulars about who did it. Blacks are vastly over-represented in assaults and robberies in the US, though all felonies are also committed more often by Blacks per capita. Particularly interracial crime is overwhelmingly Black-on-White rather than the reverse, over a 25-to-1 ratio per capita. For rapes it's 95% certain to be a ratio of hundred

In many areas our society has decided on a requirement for equality of outcome. If the applicant pool is 10% black, then your workforce better be about 10% black. Likewise, a criteria of the probation-recommending-AI could be racial equality, where blacks and whites are equally likely to receive probation. This will likely lead to more crimes, but that is something that many people are willing to accept to avoid discrimination.

You cannot filter on inputs, but just avoiding telling the AI the offender's r

That is complete nonsense. That is so far skewed from reality that I do not know where to begin... Are you seriously claiming that a group comprising 6% of the population committing 50% of the murders is less violent because of some sort of mysterious systemic racism? White people are capable of great acts of violence just like any other group but statistically, we're living in pre-immigrant Scandinavia as far as crime rates go if you remove black perpetrated crime from the stats.

In East Menlo Park, the solution was to give everyone enough money to buy a house elsewhere. Then that area next to Facebook's HQ now becomes safe enough for middle class homes to be built as well as various shops like Jack-In-The-Box.