July 21, 2009

Can Science Rule Society?

The Obama administration has been greeted with enthusiasm by scientists who see the potential for “research-based policy.” Reason, not ideology, will govern. The New Scientist, among other zines, headlines “Let Science Rule: the Rational Way to Run Societies.” (May 28, p. 40-43) This is part of a larger theme: Behavioral economics is taking off.

One commonly offered example of policy fixes that are crying out for a research-based approach is sex education. Abstinence-only sex education programs are well-taxpayer--funded at the federal, state, and international level. And they don’t work, either for HIV prevention or pregnancy prevention. What advocates of abstinence perhaps forgot is that the social context in which abstinence was preached with some success to upper-middle-class Victorian young ladies (not the young men, even the Victorians had more sense than that) were perpetually accompanied by adult chaperones. (The cost was horrendous… the innocent Victorian young ladies would be infected with venereal diseases by their husbands, and they and their babies would suffer and often die without ever being told what was wrong or how it could have been prevented). Done. Criminal law also could reap substantial benefits from a research-based approach. I have written elsewhere about the problems of ignoring deterrence research in copyright.

But it gets harder. The key problem: There is research, and then there is research. Much of it is done by advocates or just people who are careless with their assumptions. Some of these people might not even be aware of the extent to which they are advocates.

Examples: One is the EU FLOSS Report, which I was reading the other day. Some interesting data, but the authors are so busy making the case for open source they neglect key questions. Such as, just for example, how do you measure the contribution to GDP of volunteer labor? Can one simultaneously express concern (as the FLOSS report does) about the EU’s low levels of investment in software, and delight that open source reduces the amount that firms must spend in software research or on software? If one wishes to make the cheery prediction that increasing the take-up of open source software within Europe will close their innovation gap with the United States, shouldn’t one consider that perhaps the United States might simultaneously increase it’s take-up rate if it appears to be a good idea? Just why does the United States seem to be more innovative than Europe anyway?

And so on. On the problem of result-oriented studies, New Scientist quotes Laurence Moore of Cardiff University in the United Kingdom: "They're almost designed to show that the idea is a good idea … Rigorous evaluations are perceived as threatening rather than supportive of better policy."

Sound studies that run contrary to popular ideas are often simply ignored. Abstinence policy is one example. But the problem is not confined to conservatives. Another example is Head Start. This is the classic 1960’s early childhood research-based triumph. Except the only study showing lasting results from Head Start was an study designed by the originator of the program. Later studies continue to cite the original study, and to cite studies citing the original study, and so on. The results showing long term gains have never been replicated. This problem, too, is recognized: “Assessing social policies using randomised controlled trials did start to take off in the US from the 1960s to 1980s. But the practice has declined, partly because policy makers became disenchanted when the trials did not endorse their brainwaves, according to Sheila Bird, a statistician at the Medical Research Council Biostatistics Unit in Cambridge, UK.”

This raises the question of why politicians and many other policymakers stick with such determination to their agendas even in the face of contrary evidence. Another way of asking the same thing: What *is* ideology and what does it do for us? I offer some thoughts. Are people just dumb? Is it ego? People must “save face” and are unwilling to back down from a position once taken publicly?

Some of our willingness to go with general principles is a good thing. A vast amount of human experience gets summed up and expressed in the form of ideology. The United States Constitution is an ideological document. Yet it is also based on human experience with hundreds of years of monarchy, condensed into few words. One does not, and ought not, lightly set such things aside. Example: There is a not-well-enough-known Supreme Court case, Buchanan v. Worley. At issue was the constitutionality of racial segregation laws. The supporters of the laws were filed many research reports from highly progressive social scientists, purporting to prove that segregation was good for people. Quite rightly, the Supreme Court dismissed the research and favored the principle of equality in the eyes of the law. Segregation statutes were unconstitutional. Without the Court’s willingness to declare them so, the United States could have developed a full-blown apartheid system along the lines of South Africa.

ertainly it would be good to scrutinize the human experiences that go into our ideologies and rules of thumb carefully. But this cannot always be done. I do not wish to be a gulag guinea pig.

Also, there are general, as well as particular lessons to be drawn from research-based policy. One general lesson appears to be that many bright ideas fail. The excellent book Seeing Like A State elaborates on this theme. But there is another. Why is it the need for research-based policy so pressing and not, say, just for example, research-based ideas for small business or how to cook a good hard-boiled egg? Why is the need usually in the public sector, not the private sector? When research is needed in the private sector, such as medicine—why is it taken up readily, egos set aside, while the public sector has been so stubborn?

This is not merely an accident. In the private sector, failure often has natural and severe consequence for those who support or act on a bad idea. In government, failure often has no consequences except embarrassment for those who act on or support a bad idea. Research will continue to be ignored without accountability. Which brings us back to ideology; a gentle rule of thumb favoring action through the private sector, not the public sector, may be more research-based than some would like to think after all.

Comments

You can follow this conversation by subscribing to the comment feed for this post.

Abstinence-only sex education programs are well-taxpayer--funded at the federal, state, and international level. And they don’t work, either for HIV prevention or pregnancy prevention.

You repeat that claim several times. Do you have an actual site to back that claim up, or is it just such "common sense" that it doesn't need any research?

Because my memory is that the research showed that "pro-abstinence" sex education programs do at least as well as the "screw everything you want, just wear a condom" approach.

And are you simply talking about sex ed classes whose sole content is "don't do it"? Or are you talking about classes that that teach everything, while saying "the best method of birth control and STD prevention is to not have sex"?

The prospect of domination of the nation's scholars by Federal employment, project allocations, and the power of money is ever present and is gravely to be regarded. Yet, in holding scientific research and discovery in respect, as we should, we must also be alert to the equal and opposite danger that public policy could itself become the captive of a scientific technological elite. It is the task of statesmanship to mold, to balance, and to integrate these and other forces, new and old, within the principles of our democratic system -- ever aiming toward the supreme goals of our free society.
Military-Industrial Complex Speech, Dwight D. Eisenhower, 1961

The primary thing to remember is that scientists are human too. And humans can pervert the vaunted "scientific process" to come up with predetermined results. Too many people claim that science can't be perverted and use that to not only bludgeon people who disagree with their preferred political solutions to the problem but to shut down research that might prove the original problem was not a problem at all.

I believe Marxism is about the application of science to the governance of society. I also believe that the Nazis attempted to use science to control their society. The problem is that politicians attempt to justify their actions with science. Science and politics do not mix.

I think you are confusing Head Start (a federal program) with the Perry Preschool program (a small program in Ypsilanti Michigan) whose experimental (more or less) evaluation is often used to justify funding for Head Start, even though the two programs are very different. This is not to say that Head Start has a great evaluation track record; it does not. But they are different programs with different issues.

The Job Corps might actually serve as a better example, as it has a halo of good karma about it and has its very own experimental evaluation showing that while it has some positive effects on earnings (a rarity for youth programs) it does not come close to passing a social cost-benefit calculation for most groups.

Aside from the detour into sex ed (I believe I've read a study recently showing that Greg O above is correct), your post is very interesting and puts into words a notion I've had, mainly that it's ridiculous to attribute "science" some noble, impartial quality when the people who claim the mantle of "scientist" in a debate are often at least as partial and ignoble as their opponents.

The case Buchanan v. Worley is of particular interest. It would be nice to see it become a counter-meme if Obama's declaration catches on. Not to invoke Godwin's Law too readily, but there *have* recently been societal regimes that couched their raison d'etre in scientific terms, and I don't think many today consider their accomplishments worthy of imitation.

"I believe that a scientist looking at nonscientific problems is just as dumb as the next guy."
-Richard Feynman

I'm a physicist myself (albeit hardly one of Feynman's achievments or stature in the community), and I've recognized that scientists make rather poor leaders of nations. That's because that something we scientists have a hard time understanding is that in policy decisions, none of the variables are controlled. This means that when it comes to policy, all of us are smarter than any of us. In science, on the other hand, it only takes one person with the evidence to support their claim to completely overturn centuries of accepted knowledge (the classic "It's not what we don't know that's the problem - it's what we know that just isn't so" issue). One only need look at the societies that claimed the mantle of scientific planning, and the bloodbaths they perpetrated (on their own citizens, or on outsiders as defined by the state), all across the 20th Century, to recognize that. Any historian could tell us better people to pick for political leadership than a scientist.

"In a survey of more than 100 studies covering the past 20 years of research in the social sciences, the Institute for Research and Evaluation concluded that “…when measured by the same standards of effectiveness, comprehensive sex education programs in America’s classrooms do not show more evidence of success than abstinence education programs.” Quite the contrary, in fact: when the criteria applied to the programs include measurements of changed behavior lasting more than a year following teens’ program participation, practically none of the comprehensive sex education (CSE) programs produced any measurable change, whereas at least three of the abstinence programs sustained significant reduction in teen sexual activity more than a year after the teens finished the program."

Abstinence education actually does work, some. Comprehensive sex education does not, and receives far more funding. And the general misinformation regarding the two, illustrates why it's not a great idea to try to let "science" rule policy -- because in fact, it's the beliefs of the policy-makers that end up ruling science. Research tells us things we need to know, but partisans see in the research only what they expect to see. There is no cure for this, it's the human condition. That's why a small government, limited by checks and balances and affecting as little as possible, is the best approach.