These issues are the product of a number of underlying problems. A study published recently in the journal PLOS Biology presents evidence showing that a very cheap solution could help to patch up one of them: the sharing of data that underlies published research. It's a solution that isn't out of place in a video game—all it takes is a digital badge or two to encourage researchers to be more transparent.

It’s clear that although the scientific method is still the best tool we have at our disposal, there are some situations where it desperately needs sharpening. One thing that has dulled the tool is the damaging incentives often faced by scientists. They’re under pressure to publish exciting, positive results in order to keep their jobs and be promoted, which can push some people to do awful things like fake data. But on a more mundane level, the pressure often means that only the most exciting, whizzbang studies see the light of day.

The nonprofit Center for Open Science is the organization responsible for the headline-grabbing finding that fewer than half of all psychology studies replicate. The center has previously made some suggestions for how the scientific publishing process could change to rearrange incentives and make it easier to do robust science. One of the things it highlights is how vital it is to improve scientific transparency. If one team of scientists doesn’t have all the details about what another team did, it’s really hard to replicate their results.

The LaCour scandal saw a widely publicized paper on gay canvassers retracted due to fabricated data. The falsification was rooted out by a different research team trying to replicate the finding and poking around in the data. It’s a beautiful illustration of how problems with research can be found more easily when everything about the research is shared and made public—other people can check the original analyses, look for problems, and conduct their own versions of the research much more easily.

Right now, though, there aren’t too many incentives for researchers to share their data. It’s kind of like a politician who quietly works away at fixing problems for the people he represents, without much song and dance: it’s the right thing to do, but it’s not the thing that will get him the publicity that spurs a re-election.

Researchers are keeping this one on hand at the LHC, just in case...

In January 2014, the journal Psychological Science introduced a small incentive to encourage researchers to share their data: badges appearing at the top of the published research paper, signaling that the research comes with shared data or shared research materials. “Badges acknowledging open practices signal that the journal values transparency, lets authors signal that they have met transparency standards for their research, and provides an immediate signal of accessible data, materials, or preregistration to readers,” write Mallory Kidwell and the other authors of the new paper.

The Center for Open Science took a look at publications in Psychological Science to see if this incentive was making a difference and found strong evidence that it was. In 2012 and 2013, before the incentive was introduced, an average of 2.5 percent of articles in Psychological Science reported that their data was available. After the badges were introduced, this number quickly shot up to 22.8 percent. In the first half of 2015, it had reached as high as 39 percent. Badges also seemed to increase sharing of research materials like surveys, although the results weren't as dramatic.

Results from the Center for Open Science's analysis of data sharing with and without badges. Data sharing at the journal Psychological Science saw a sharp increase when badges were introduced.

Badges also seem to have increased sharing of research materials like surveys, although the picture is much less clear.

Research that had earned a badge for open data was more likely to have usable data than reported open data without badges.

Research that had earned a badge for open materials was also more likely to have usable materials than reported open materials without badges.

This phenomenon could be explained by cultural changes at the time that were increasing the popularity of open data. So the authors looked at other, similar journals that didn’t introduce a badge system. These journals had an average open data rate of 3.3 percent prior to January 2014 and an average of 2.1 percent after—nothing like the huge jump seen in Psychological Science.

There’s an important caveat here: just saying your data is available to other researchers doesn’t mean it's true, and it doesn’t mean it’s in a format that's actually useful. So the researchers checked whether the complete materials were actually available where they should be and whether they were understandable and usable. They found that Psychological Science articles with badges were more likely to be reliably stored and were more complete than open data from other journals. “When badges were available, rates of actual availability, correctness, usability, and completeness were dramatically higher than the comparison journals, but not close to perfect,” the authors write.

Overall, it really does look like the badges help, not just with increasing sharing rates but with making sure that shared data is helpful to the research community. Of all the 2,478 articles used in the study, those without badges were very weak about sharing: “Just six of 37 articles from journals without badges and two of 10 articles from [Psychological Science] before badges that reported available data had accessible, correct, usable data,” write the authors. By contrast, of the articles with badges, “actual sharing was very similar to reported sharing.”

It’s not clear why this incentive is so effective. It could be to do with the journal signalling that transparency is an important value, or it could be that the guidelines for earning a badge make the steps to transparency simple to achieve—we just don’t know. These results also don't tell us whether sharing ultimately affects the ease of replication and the reliability of science. Future research will need to try to answer these questions.

However promising these results are, though, badges don’t come close to solving the overall problem. “Sharing rates increased dramatically, but not all data or materials that could be shared were shared," the authors write. "Moreover, even with badges, the accessibility, correctness, usability, and completeness of the shared data and materials was not 100 percent.”

Ultimately, badges are just one strategy to attack bad science; other approaches will be needed as well.