Diversity is a double-edged sword, making individuals less likely to be altruistic than they might be in a more homogeneous setting but also inspiring them to scale new intellectual heights and to explore new horizons, argue two UCLA economists in a new book.

"People enjoy being around people they can relate to, and they are uncomfortable with diversity," said Matthew Kahn, a co-author of "Heroes and Cowards: The Social Face of War," which will be published Jan. 21 by Princeton University Press. "But even though people don't like being exposed to people who are different, they benefit from the experience in the long run. They learn the most from those who are different."

While recent research into lower rates of volunteerism and lack of taxpayer support for local projects in diverse communities has reached similar conclusions, the latest findings are based on a surprising set of subjects: 41,000 soldiers who served in the U.S. Civil War between 1861 and 1865.

"Union soldiers, whether in prison camps or in the field, were the most loyal to men who looked like themselves — of the same ethnicity and occupation, from the same state or hometown, or of the same age or related by blood," said co-author Dora L. Costa, a UCLA economics professor. "We believe that by going back so far in time we're getting at an effect that's universal. This reaction to diversity may be hardwired into us."

Curious about the forces that motivate individuals to acts of heroism, Costa and Kahn set out to mine millions of records on Union soldiers that have been gathered since 1991 into a database funded by the National Institutes of Health.

Tracing enlisted men throughout their tour of duty and into retirement, the Union Army Dataset merges official Union records with now-public census data between 1840 and 1910, essentially providing a cradle-to-the-grave look at the soldiers. Details include a soldier's original economic status, degree of literacy and hometown; his military record, including whether he deserted or ended up in prisoner of war camps and how he fared there; and his eventual residence, livelihood, economic status and degree of literacy.

The researchers then subjected these variables to the same sort of statistical analysis as social scientists who conduct research on living subjects in experimental conditions. The results, insist the UCLA team, are just as valid — and possibly even more trustworthy — than glimpses into human behavior gleaned from laboratory experiments.

"Unlike university laboratory researchers, we examined life-or-death choices," Costa said. "We could never orchestrate an experimental exercise with such high stakes. These are decisions that really mattered to people. As a result, they paid a lot of attention to their decisions, and those decisions reflect profound truths."

Men who served in war companies with men who shared similar characteristics — a common religion, race, ethnic group, socioeconomic status, hometown or even plantation — behaved markedly differently from their counterparts in more diverse companies. For starters, they had much lower desertion rates than the norm of one desertion per 10 Union soldiers. Union soldiers who served alongside men from the same occupations deserted at one-third the rate of counterparts in more diverse companies, as did former slaves who served with former slaves from the same plantation.

"Economic theory says you would look out for your own welfare," Kahn said. "Yet we kept seeing all these very moving examples of people wanting to sacrifice for others."

And the absence of diversity actually outranked other potential sources of loyalty. Costa and Kahn expected to find that company morale or commitment to the cause played as big if not a bigger role in keeping a soldier on the battlefield. Yet they found that companies with the lowest amount of diversity — such as companies in which friends, relatives or neighbors served together — had the lowest desertion rates. The least diverse companies had one-third fewer desertions than more diverse companies with high morale or strong ideological commitments.

"This was one of the most ideological wars in U.S. history," Costa said. "If we find that serving with similar people or buddies matter the most here, then we know the effect is big, and in fact, that's what we found. Even when the ideological stakes are huge, it's serving alongside comrades that keeps you in the war — not commitment to cause."

In addition to inspiring enlistees to persevere, peers also proved important to surviving such grim prisoner of war camps as Andersonville, a Confederate death trap that claimed the lives of close to 40 percent of its captives. Survival rates for Union soldiers born in Ireland, for instance, improved only from 60 percent to 64 percent if they were incarcerated with 15 comrades from their original company, Costa and Kahn found. But the soldiers' survival rates shot up to 90 percent when those 15 comrades were not only from the same company but were also fellow Irish immigrants.

"We believe that your comrades would help you get healthy if you got sick and share their food rations," Kahn said. "So in P.O.W. camps, diversity actually turned out to be a bad thing. It hindered survival rates."

But diversity was not without its benefits. Costa and Kahn focused on three separate kinds of African American troops: troops that consisted primarily of freed slaves, troops that consisted primarily of freemen (African Americans who were not born into slavery) and troops that mixed both kinds of African American enlistees.

While companies with both former slaves and freemen had higher desertion rates than units that consisted primarily of one group or the other, former slaves in the diverse companies learned to write at higher rates than their counterparts in more homogenous units, Costa and Kahn found. In companies comprised almost exclusively former slaves, only 16 percent of soldiers learned to write during their tour of duty. That number nearly doubled in companies in which former slaves mixed with freemen.

"For former slaves who had been prevented from learning to read and write in slavery, diverse companies were almost like a job training program, preparing them for improved economic opportunities down the line," Kahn said.

In addition to being more likely to learn to literacy skills, former slaves in diverse companies also were more likely than their counterparts in more uniform companies to change their slave names, the researchers found.

"Adopting a new name was a measure of taking on the identity of a free person," Costa said. "These former slaves started to see themselves differently as a result of the more expanded horizons of the men with whom they served."

African American soldiers from diverse companies also were much more likely than their counterparts to move away from home after the war. This was especially the case among illiterate soldiers for whom word of mouth was their primary source of information. The economists found that for every 10 percentage-point increase in comrades who hailed from a particular state different than the home state of an illiterate solider, the likelihood of that illiterate soldier ultimately relocating to that state jumped by more than 30 percent.

"We call this 'the Zagat Guide effect,'" Kahn said. "So if we're in the same company, and I'm from New Jersey, you are more likely to move to New Jersey after the war. We believe that I taught you about the benefits of New Jersey. Serving in a diverse unit helped open horizons for men who had previously enjoyed no mobility whatsoever."

"Heroes and Cowards" represents the most extensive use to date of Civil War data amassed by the University of Chicago and Brigham Young University under the direction of Nobel laureate economist Robert Fogel. For the past decade, Costa has been second-in-command on the unparalleled NIH-funded project.

Compiled from records in the National Archives, the Union Army Dataset focuses on Union rather than Confederate records because of the North's famously generous medical benefits for veterans, the authors explained. The most widespread form of assistance to the elderly before the advent of Social Security, these benefits ultimately extended to 90 percent of Union Army veterans. Records for these benefits allow researchers to track the health, whereabouts and other outcomes of Union veterans into old age and to link them to census data for the men, which in turn revealed even more information.

After losing the war, Confederate states offered benefits to veterans on a state-by-state basis, and they did so only when they could afford them, the authors explained. With the South's financial collapse following the war, many Confederate veterans did not receive medical benefits, confounding any attempt to track them through public documents as they aged.

"Confederate records just aren't as comprehensive or useful for this kind of analysis as the Union records," Costa said.

The confluence of increased privacy protections on medical, military and census data and the replacement of a draft armed forces with a voluntary one make military records from later U.S. conflicts less valuable for research purposes than the Civil War documents, the authors contend.