Race in Modern America & Big Data

Modern America is a divided society. Maybe it is a side-effect of our socioeconomic system, which prides itself at thriving through competition, or maybe it is natural. Whatever the cause, America must strive to identify and eliminate those division which have a net negative effect on its people. Of those negative divisions, race is probably the most pernicious. Scholars such as W. Lloyd Warner and Michelle Alexander have argued that the racial divide is so extreme that it almost resembles a caste system. Regardless of exactly how divisive race is, it is clearly still an issue in contemporary America, and it carries incalculable costs. As W.E.B. Du Bois said of racial discrimination, “[It] is morally wrong, politically dangerous, industrially wasteful, and socially silly.” (Katz and Sugrue 1998, 55) All Americans should support actions to reduce the burden of historical racism. Racial division was originally a direct result of laws created within the United States, so it should be possible to use the same system of laws to redress the issue of racial discrimination and division.

Until the racial divide is completely bridged, the United States must take steps to integrate those people who have been marginalized for hundreds of years. However, before policies can be made to ameliorate the situation, it must first be decided how to determine if society is making progress towards the goal. Some type of statistical analysis will be required since any type of manual inspection would invariable be flawed. Psychologists think that people can have subconscious racial biases; they call this implicit prejudice. Without a statistically sound check scientific in place, policy makers could potentially implement programs which unintentionally operate in a biased manner. (Fiske 1998, 375) History has show that there have been many supposedly colorblind initiatives which yielded demonstrably color biased outcomes. The Social Security Act of 1935 and Servicemen’s Readjustment Act of 1944 are two such examples

When Social Security was originally enacted farmworkers and domestics, professions dominated by African American workers, were left out. The explanation for the exclusion was that it would have been too difficult for the government to collect the necessary withholdings. There were proposals to overcome these problems, but they were not embraced by the legislature. The Wisconsin Group, an influential group of Northerners who were involved with drafting the legislation, explicitly rejected unified plans for Social Security and tacitly supported the illusory “separate but equal” policies.

Also, Social Security benefits were determined by contributions, so even when the law was changed to allow farmworkers and domestics to contribute, black workers were far behind white workers of similar socioeconomic background. African Americans who were eligible for Social Security also received less benefits. Systemic wage discrimination meant that they paid less into the system, and years of hard work with little health insurance meant that their life expectancies were shorter than those of comparable white workers.

The Servicemen’s Readjustment Act, or the GI Bill as it is commonly known, also produced unequal results. One of the GI Bill’s main supporters was John Rankin of Mississippi; he was eager to funnel federal funds into his poor state, as long as segregation would not be affected. To this end he made sure that the bill did not place any restrictions on local businesses that would require them to violate “long standing customs” and accept all veterans equally. (Katznelson 2009, 136) White vocational school proprietors in the South were disinclined to admit black students. Since Mississippi would not approve many vocational schools run by blacks, the effect was that blacks were restricted from using the GI Bill to attend vocational schools in Mississippi.

The North was no bastion of equality with regard to the GI Bill either; informal segregation meant that few blacks were admitted into Northern or Western colleges. Ninety-five percent of black veterans used their college benefits in the South’s where there were all black institutions. However, the underfunded black universities were unable to keep up with demand for higher education. Southern black colleges turned away fifty-five percent of veterans, versus a turn a way rate of twenty-eight percent for all universities. (Katznelson 2009, 132) The black veterans who were admitted to colleges went to universities which had inadequate resources and offered few advanced degrees.

The results of the New Deal legislation were skewed so much in favor of whites that Ira Katznelson has labeled the policies collectively as “white affirmative action.” It is unlikely that these biased results would have occurred if equality of outcomes had truly been a concern.

The 1960s civil rights movement in the United states brought about changes aimed at correcting historical wrongs like those of the New Deal programs. After passing laws such as the Civil Rights act of 1964 and the Voting Rights Act of 1965, America appeared to be moving into a post-racial era. These civil rights laws should have helped undo longstanding injustices. However, as Michelle Alexander points out, the civil rights movement itself caused a transformation in criminal laws which eventually offset the explicit civil rights gains. The civil rights protests and riots, which were fueled by the era’s harsh discriminatory practices, quickly became associated with lawlessness. Politicians began promising to bring “law and order” if elected, some even claimed that the lawlessness was a direct result of desegregation. (Alexander 2010, 41) The immediate effect was that police began to crack down on protesters, which due to the nature of the protests occurred in predominately black neighborhoods. The long-term effect has been that people who live in poor, predominately black neighborhoods have been criminalized. Rather than overtly discriminating based upon race, as was the norm in the pre-civil right era, criminal records began to be used.

White Americans tend to deny that institutional racism still exists in the post civil rights era; polling data has shown that racism has steadily declined since the late 1940s. However, some argue that after the civil right movement America entered into an era of colorblind racism. (Bonilla-Silva 2010, 2) Since it is no longer legal to discriminate on the grounds of race, people are unlikely to admit to racial discrimination. Social norms now prohibit blatant racism, and some Americans even argue that “Hitler gave racism a bad name.” (McConahay 1986)

Due to these difficulties, polls, surveys, or other self reporting methods cannot reliably be used to determine the extent of racism, but it must be known if society is actually progressing away from prejudice or if the prejudices are just being repressed. The only way to sense progress is through some sort of measurement, unfortunately the changing nature of racism makes this particularly difficult. Furthermore, the entire population makes countless decisions everyday which could potentially be tainted by bias, so measuring progress on social issues can be a quite difficult task. It seems that the best way to sum those countless individual decisions into manageable data is by examining actual social outcomes. As the Moynihan reported noted, “It is not enough that all individuals start out on even terms, if the members of one group almost invariably end up well to the fore, and those of another far to the rear.” (Moynihan 1965)

Some people may object to measuring social outcomes based upon race; there is a belief that the categorization that comes with measurement perpetuates racial divisions. However, research in psychology suggest that people automatically classify people based upon external characteristics and then form biased opinions based upon those classifications. (Blank 2004, 23) So, the act of recording demographic information likely has little affect upon people’s views of race.

Once a method for measuring progress has been established, attention can then be turned towards making the changes required to achieve equal outcomes. The American society runs on money, so access to money is a crucial component. The preferred way to accumulate money in America is by selling ones labor; equal access to markets in which labor can be sold will be key to equal outcomes. As Vernon Jordan said at the National Urban Leagues 1974 annual conference, “unless there are decent wages for everyone willing and able to work, our nation will continue to be haunted by the specter of a growing number of people for whom there are no useful roles in our society.” (Hamilton and Hamilton 1997, 198)

Employment has long been an issue for African Americans. W.E.B. Du Bois said that employment was the most pressing issue of the day, and unfortunately not enough progress has been made since 1899 when the work was first published. In the late nineteenth century many African Americans fled the oppressive environment of the South. They came north in search of freedom and equality. They found that the North was only marginally better than the South regarding race and employment. When two equally qualified candidates, one white and one black, applied for a job, the white would invariably get the job offer. White business owners claimed that they had to hire whites, lest they offend their white customers. Factory owners claimed that they simply could allow untrained individuals to operate their machines. Blacks did not have access to proper training, and unions were generally not interested in helping them.

Even in the rare cases when African Americans were allowed into industrial settings it was not for altruistic reasons. At the Midvale Steel Works, near Germantown Pennsylvania, one-sixth of the workforce was black. The plant manager was the famous efficiency expert Frederick Winslow Taylor. Taylor hired blacks into positions which would interfere with other minority groups ability to form clans or cliques. (Katz and Sugrue 1998, 103) Even that experiment did not last long, the steel works abandoned the policy in the 1890s.

By the time Du Bois published The Philadelphia Negro, eighty percent of black males were employed as laborers or servants. Black women were restricted to domestic work in the North as well as the South. Although southern black men could take more skilled labor in the South due to the lack of white unions, they were still primarily relegated to farm work. This picture had not changed much when the New Deal policies were being debated in the 1930s.

Neither the National Labor Relations Act of 1935 nor the Fair Labor Standards Act of 1938 assisted African Americans. The South was scantily industrialized when New Deal legislation was originally being debated, so they saw no problem supporting pro-union legislation so long as agricultural and household laborers were excluded. Agricultural work was explicitly agreed upon to be outside the definition of “industry,” which is the only portion of the economy which those laws covered. (Katznelson 2009, 56) In the North union locals prohibited blacks from joining, and the national unions were not willing to “interfere in the internal affairs” of those locals (Hamilton and Hamilton 1997, 13) It was not until the Civil Rights Act of 1964 that discrimination in the workplace was finally outlawed.

Unfortunately policing racial discrimination is extremely difficult. By the end of the 1970s the unemployment rate for black Americans had not dropped below nine percent in a decade. This spurred the national civil rights organizations to lobby for full employment legislation. The fruits of their labor was the Full Employment and Balanced Growth Act of 1978. The act gave private industry five years to get unemployment below three percent in nondiscriminatory ways. If it failed then the government would be required to create a “reservoir of public employment.”

The legislation demonstrates the importance of employment; at near employment there would be less room for discrimination due to the limited free labor pool. As Mary Poole has said, “[Discrimination] could be eradicated only when the drive for efficiency and profit [is] allowed to rule unfettered by racial prejudice.” (Poole 2006, 84)

Even when near full employment is realized, the nation needs to further ensure that the distribution of jobs across the population is equitable. Statistically speaking the percentage of doctors in America which are African American should be directly proportional to the number of African Americans in Society. This is yet another tricky social goal to measure. However, with the aid of modern computing technology it should be within the realm of possibility.

The Internal Revenue Service (IRS) could change its reporting requirements to mandate that employers record detailed demographic information along with the information it already collects. The IRS should sanitize the data by stripping out taxpayer specific information and then publish it to the Bureau of Labor Statistics (BLS). The BLS would then be able to use modern data mining techniques to make very specific measurements of the labor market. The BLS would then be able to measure equality across professions, regions, and even individual firms. If disturbing trends are found in the data, then root cause analyses could be performed and more specific policy recommendations could be made. At some point it could even become possible to tax companies according to their level of equality, if areas prove to have any sort of malignant prejudices. The key is to begin gathering data that can be used to assess the overall level of inequality, and modern technology could make this easier.

Once American society gets to the point at which it can reliably measuring the job market for unequal outcomes, it should then take steps to resolve any identified issues. One tool for resolving disparities is to implement affirmative action policies targeted at the demonstrably disadvantaged.

Contemporary discussions of affirmative action focus on the post civil rights era benefits which were targeted towards African Americans and often veer off into debates about “reverse discrimination,” but once better data is available regarding employment discrimination, then these claims should be easier to silence. It should be possible to realize W.E.B. Du Bois’ and other progressives’ goal of using actual statistics to improve society.

By making policy adjustments in real-time based upon actual employment statistics, policymakers would be able to speed the process moving toward equal outcomes in employment. Equal employment and career outcomes, in aggregate, result from equal opportunities. When the point of equal outcomes is reached, America will also know that it has finally provided equal opportunities. This is within society’s abilities, it just requires the commitment.