The comments on a recent post raised the question of how much we can trust drug studies sponsored by the very companies that manufacture the drugs in question. On that point, a recent New York Times article may be relevant:

In what may be among the longest-running and widest-ranging cases of academic fraud, one of the most prolific researchers in anesthesiology has admitted that he fabricated much of the data underlying his research, said a spokeswoman for the hospital where he works.

The researcher, Dr. Scott S. Reuben, an anesthesiologist in Springfield, Mass., who practiced at Baystate Medical Center, never conducted the clinical trials that he wrote about in 21 journal articles dating from at least 1996, said Jane Albert, a spokeswoman for Baystate Health. . . .

The drug giant Pfizer underwrote much of Dr. Reuben’s research from 2002 to 2007. Many of his trials found that Celebrex and Lyrica, Pfizer drugs, were effective against postoperative pain. . . .

“When researchers are beholden to companies for much of their income, there is an incredible tendency to get results that are favorable to the company,” said Dr. Jerome Kassirer, a former editor of The New England Journal of Medicine and the author of a book about conflicts of interest.

Is it naive to be astonished that it took this long for someone to notice that 21 scholarly articles had been published about clinical trials that had never occurred?

A recurring theme here is the difficulty in knowing whether some (much?) of modern healthcare is actually beneficial or not. A couple of recent links that add support to that theme:

1. From JAMA, a new study analyzes more than two decades of heart care guidelines (that is, the guidelines that your doctor might follow in deciding how to treat you) from the American College of Cardiology and the American Heart Association. The study found that the overwhelming majority of recommendations are not supported by good evidence:

Level of evidence provides the link between recommendations and evidence base. Although there is significant variation among individual guidelines in available evidence supporting recommendations, the median of level of evidence A recommendations [i.e., those supported by more than one randomized trial] is only 11% across current guidelines, whereas the most common grade assigned is level of evidence C, indicating little to no objective empirical evidence for the recommended action. . . . Interestingly, our findings are reflective of a specialty — cardiology — that has a large pool of research to draw on for its care recommendations. Guidelines in other medical areas in which large clinical trials are performed less frequently may have an even weaker evidence-based foundation.

2. In this post, Dr. Eades criticizes (convincingly, I think) a recent study purporting to show that statins reduce mortality.

I think what religion and politics have in common is that they become part of people's identity, and people can never have a fruitful argument about something that's part of their identity. By definition they're partisan.

Which topics engage people's identity depends on the people, not the topic. For example, a discussion about a battle that included citizens of one or more of the countries involved would probably degenerate into a political argument. But a discussion today about a battle that took place in the Bronze Age probably wouldn't. No one would know what side to be on. So it's not politics that's the source of the trouble, but identity. When people say a discussion has degenerated into a religious war, what they really mean is that it has started to be driven mostly by people's identities. . . .

The most intriguing thing about this theory, if it's right, is that it explains not merely which kinds of discussions to avoid, but how to have better ideas. If people can't think clearly about anything that has become part of their identity, then all other things being equal, the best plan is to let as few things into your identity as possible.

Most people reading this will already be fairly tolerant. But there is a step beyond thinking of yourself as x but tolerating y: not even to consider yourself an x. The more labels you have for yourself, the dumber they make you.

In a presidential campaign that promises straight talk and no gimmicks, why do both candidates champion one of medical care’s most pervasive myths? The myth is that like magic, preventive medicine will simultaneously reduce costs and improve health. . . .

The term "preventive medicine" no longer means what it used to: keeping people well by promoting healthy habits, like exercising, eating a balanced diet and not smoking. To their credit, both candidates ardently support that approach. But the medical model for prevention has become less about health promotion and more about early diagnosis. Both candidates appear to have bought into it: Mr. Obama encourages annual checkups and screening, Mr. McCain early testing and screening.

It boils down to encouraging the well to have themselves tested to make sure they are not sick. And that approach doesn’t save money; it costs money.

Taubes has several overarching themes; he contends, for example, that eating refined carbohydrates is what makes you obese, and that refined carbohydrates contribute to many of what used to be called "diseases of civilization" (such as heart disease, which seems to have been less common in traditional cultures that ate less processed food, including Northern cultures that ate almost exclusively meat). (These arguments are still controversial, although new evidence continues to support them.)

The most important theme, however, suffuses the entire book: bias in scientific inquiry.

One logical conjecture would be to expect this form of political behavior to be much like any other. In other words, it would be disproportionately the province of well-educated, high-income populations. Indeed, the frequency of general political discussion tracks closely with these characteristics of high socioeconomic status. But the correlates of cross-cutting conversation are strikingly different. As shown in Figure 2.3, there are clear patterns of difference with respect to race, income, and education, but they are not in the usual directions. Nonwhites are significantly more likely to engage in cross-cutting political conversation than whites. And as income increases, the frequency of disagreeable conversations declines. Exposure to disagreement is highest among those who have completed less than a high school degree and lowest among those who have attended graduate school.

I can testify to how easy it is for conversation among academics, the most educated group of people, to turn into a one-position echo chamber. Liberalism is taken to be an IQ test, and the rare conservative is encouraged to be quiet or go elsewhere. For political disagreement I go to the coffee house, which in our town draws a broader range of people than the faculty club contains.

Of course, one explanation would be that what looks like herd behavior and social conformity is really just what happens when a bunch of superior intellects independently settle on the objectively correct viewpoint. But that’s rather a self-serving explanation, isn’t it?

Brain images are believed to have a particularly persuasive influence on the public perception of research on cognition. Three experiments are reported showing that presenting brain images with articles summarizing cognitive neuroscience research resulted in higher ratings of scientific reasoning for arguments made in those articles, as compared to articles accompanied by bar graphs, a topographical map of brain activation, or no image. These data lend support to the notion that part of the fascination, and the credibility, of brain imaging research lies in the persuasive power of the actual brain images themselves. We argue that brain images are influential because they provide a physical basis for abstract cognitive processes, appealing to people’s affinity for reductionistic explanations of cognitive phenomena.

As the BPS blog elaborates:

David McCabe and Alan Castel presented university students with 300-word news stories about fictional cognitive research findings that were based on flawed scientific reasoning. For example, one story claimed that watching TV was linked to maths ability, based on the fact that both TV viewing and maths activate the temporal lobe. Crucially, students rated these stories to be more scientifically sound when they were accompanied by a brain image, compared with when the equivalent data were presented in a bar chart, or when there was no graphical illustration at all.

This fits in with the theme of how people tend to overvalue something that is dressed up in the attire of science.

I would be with a bunch of Kennedy fans watching the debate and their comment would be, “He’s really slaughtering Nixon.” Then we would all go to another apartment, and the Nixon fans would say, “How do you like the shellacking he gave Kennedy?” And then I realized that each group loved their candidate so that a guy would have to be this blatant — he would have to look into the camera and say: “I am a thief, a crook, do you hear me, I am the worst choice you could ever make for the Presidency!” And even then his following would say, “Now there’s an honest man for you. It takes a big guy to admit that. There’s the kind of guy we need for President.”

2. Tyler Cowen has an article in The New Republic that is rather cynical about the value of most published research:

The sad truth is that "non-fiction" has been unreliable from the beginning, no matter how finely grained a section of human knowledge we wish to consider. For instance, in my own field, critics have tried to replicate the findings in academic journal articles by economists using the initial data sets. Usually, it is impossible to replicate the results of the article even half of the time. Note that the journals publishing these articles often use two or three referees–experts in the area–and typically they might accept only 10 percent of submitted papers. By the way, economics is often considered the most rigorous and the most demanding of the social sciences.

3. Seth Roberts points out that the value of data is not binary, i.e., either convincing or worthless:

A vast number of scientists have managed to convince themselves that skepticism means, or at least includes, the opposite of value data. They tell themselves that they are being “skeptical” — properly, of course — when they ignore data. They ignore it in all sorts of familiar ways. They claim “correlation does not equal causation” — and act as if the correlation is meaningless. They claim that "the plural of anecdote is not data" — apparently believing that observations not collected as part of a study are worthless. Those are the low-rent expressions of this attitude. The high-rent version is when a high-level commission delegated to decide some question ignores data that does not come from a placebo-controlled double-blind study, or something similar.

So considering links 2 and 3, should we really downgrade the evidentiary value of published studies and upgrade the evidentiary value of anecdotes? (That wouldn’t mean treating them both as equal, of course.)

How do you tell whether something is good or bad? Human judgment is surprisingly swayed by contextual cues, rather than by the actual attributes of the thing being judged. As a recent Boston Globe article pointed out:

SCIENTISTS AT CALTECH and Stanford recently published the results of a peculiar wine tasting. They provided people with cabernet sauvignons at various price points, with bottles ranging from $5 to $90. Although the tasters were told that all the wines were different, the scientists were in fact presenting the same wines at different prices. The subjects consistently reported that the more expensive wines tasted better, even when they were actually identical to cheaper wines.

The experiment was even more unusual because it was conducted inside a scanner – the drinks were sipped via a network of plastic tubes – that allowed the scientists to see how the subjects’ brains responded to each wine. When subjects were told they were getting a more expensive wine, they observed more activity in a part of the brain known to be involved in our experience of pleasure.