Science isn’t as solid as it should be – but science can fix it

SCIENCE, like any walk of life, has its share of bad apples. Scientists are under pressure to produce results and for some the temptation to massage data or make it up is too much to resist. But the rotters are rare: scientific fraud is sufficiently unusual and shocking to make headline news when uncovered.

Science cannot afford to be complacent. Over the past few years there has been a creeping realisation that while bad apples are few and far between, there is a deeper problem. The barrel itself may be rotten.

Those at the scientific coalface strive for objectivity, but like all of us they have unconscious biases that can lead them astray. And the scientific method is not robust enough to catch all the errors – statistics in particular can be used to prise a significant result out of almost any data set, a practice known as “torturing the data until it confesses”. This means that all too often scientists embarking on research projects are going to sea in a sieve (see “Why so much science research is flawed – and what to do about it“).

Advertisement

These weaknesses have troubling consequences for the reliability of our knowledge base. One analysis has claimed that more than half of published research is wrong. A widely reported study published in the journal Science last year found that of 100 important psychology experiments, more than 60 couldn’t be replicated. Similar problems have been uncovered in medical science, biology and economics.

“Science’s weaknesses have troubling consequences for the reliability of our knowledge base“

That sounds like a crisis in the making, not just for our ability to discover things but for the reputation of scientists and science. One leading psychologist has compared it to the sub-prime mortgage crisis which did so much to disgrace the bankers. There are some signs that he is right. A UK poll carried out at the end of 2015 found that people’s trust in scientists had fallen over the previous year.

Nonetheless, 79 per cent of people polled still said they trust scientists to tell the truth. That is down from 83 per cent a year ago, but it hardly constitutes a crisis. Bankers scored 37 per cent and politicians 21 per cent.

What about the problem of reliable knowledge? On this front, things might also not be as bad as they seem. Last month, Science published a follow-up to the reproducibility paper arguing – ironically – that it used flawed statistics. Correct for these, and almost all 100 studies were reproducible, its authors claimed.

That, of course, may be just another case of torturing the data; the authors of the original paper have accused those of the new one of selectively interpreting the numbers. And it doesn’t absolve other problematic branches of science.

But it does demonstrate science’s willingness to face its own problems. In fact, we wouldn’t know about them at all were it not for scientists turning their tools on themselves. Metascience – the science of science – is a growing field that is increasingly discovering the loopholes in the system and closing them.

For that, science should be congratulated. Which other field of human endeavour would scrutinise itself so publicly, find itself wanting, and then set about putting its house in order?

If there is a sub-prime problem in science, then scientists are doing their best to fix it before it brings the whole edifice down. Unlike those politicians or bankers, they are not turning a blind eye, covering their own backsides or simply hoping to get away with it.

This article appeared in print under the headline “Crisis, what crisis?”