Group calls for more transparency in science research, announces guidelines

An international group of academic leaders, journal editors and funding-agency representatives and disciplinary leaders, including Rick Wilson, the Herbert S. Autrey Chair of Political Science and professor of statistics and psychology at Rice University, has announced guidelines to further strengthen transparency and reproducibility practices in science research reporting.

The group, the Transparency and Openness Promotion (TOP) Committee at the Center for Open Science in Charlottesville, Va., outlined its new guidelines in a story published in this week's edition of the journal Science.

While transparency, openness and reproducibility are readily recognized as vital features of science and embraced by scientists as a norm and value in their work, a growing body of evidence suggests that those qualities are not necessarily evident today.

"A likely culprit for this disconnect is an academic reward system that insufficiently incentivizes open practices," Wilson said. "In the present reward system, the emphasis on innovation undermines practices that support openness. Too often, publication requirements—whether actual or perceived—fail to encourage transparent, open and reproducible science."

TOP's objective is to translate scientific norms and values into concrete actions and change the current incentive structures to drive researchers' behavior toward more openness.

"We know the disciplines differ in what is emphasized, so we sought to produce guidelines that focus on what is shared across disciplines," Wilson said.

Each of eight standards has three levels of adoption in the TOP guidelines; each moves scientific communication toward greater openness, according to the article in Science. These standards are modular, facilitating adoption in whole or in part. However, they also complement each other; commitment to one standard may facilitate adoption of others.

Two standards reward researchers for the time and effort they have spent engaging in open practices. Citation standards extend current article citation norms to data, code and research materials. Regular and rigorous citation of these materials credits them as original intellectual contributions. Replication standards recognize the value of replication for independent verification of research results and identify the conditions under which replication studies will be published in the journal.

Four of the standards describe what openness means across the scientific process so that research can be reproduced and evaluated. Reproducibility increases confidence in findings and also allows scholars to learn more about what results mean. Design standards increase transparency about the research process and reduce vague or incomplete reporting of the methodology. Standards for research materials encourage the provision of all elements of that methodology, and data-sharing standards give authors an incentive to make data available in trusted repositories.

The final two standards address the values resulting from preregistration. Standards for preregistration of studies facilitate the discovery of research, even unpublished research, by ensuring that the existence of the study is recorded in a public registry. Preregistrations of analysis plans certify the distinction between confirmatory and exploratory research, or what is also called hypothesis-testing versus hypothesis-generating research. Making the distinction between confirmatory and exploratory methods transparent can enhance reproducibility.

"The guidelines are sensitive to concerns by both journals and researchers," Wilson said. "For example, we encourage journals to state exceptions to sharing because of ethical issues, intellectual property concerns or availability of necessary resources. We encourage journals to pick and choose among the different levels and standards in order to define what they expect of the researchers.

"We acknowledge the variation in evolving norms about research transparency. Depending on the discipline or publishing format, some of the standards may not be relevant for a journal. Journal and publisher decisions can be based on many factors—including their readiness to adopt modest to stronger transparency standards for authors, internal journal operations and disciplinary norms and expectations," Wilson said.

The present version of the guidelines is not the last word on standards for openness in science, according to the report. "As with any research enterprise, the available empirical evidence will expand with application and use of these guidelines," the TOP Committee wrote. "To reflect this evolutionary process, the guidelines are accompanied by a version number and will be improved as experience with them accumulates."

An information commons and support team at the Center for Open Science is available (top@cos.io) to assist journals in selection and adoption of standards and will track adoption across journals. Adopting journals may also suggest revisions that improve the guidelines or make them more flexible or adaptable for the needs of particular subdisciplines.

Related Stories

A group of Cambridge computer scientists have set a new gold standard for openness and reproducibility in research by sharing the more than 200GB of data and 20,000 lines of code behind their latest results - an unprecedented ...

Peer reviews in science, in which independent scientists who are experts on the subject assess the paper, is the current strategy for ensuring quality and control in scientific research and, therefore, it is essential for ...

In the Friday (Jan. 3) edition of the journal Science, an interdisciplinary group is calling on scholars, funders, journal editors and reviewers to adopt more stringent and transparent standards to give social science research ...

(HealthDay)—An abridged version of the 2015 Standards of Medical Care in Diabetes has been produced for primary care physicians. The condensed guidelines were published in the April issue of Clinical Diabetes.

An international partnership of the Northwestern University/Art Institute of Chicago Center for Scientific Studies in the Arts (NU-ACCESS), the Art Gallery of Ontario (AGO) and the National Gallery of Art, Washington, has ...

Musee national Picasso-Paris and the Northwestern University/Art Institute of Chicago Center for Scientific Studies in the Arts (NU-ACCESS) have completed the first major material survey and study of the Musee national Picasso-Paris' ...

Fossils that preserve entire organisms (including both hard and soft body parts) are critical to our understanding of evolution and ancient life on Earth. However, these exceptional deposits are extremely rare. The fossil ...

denglish: it's not the scientists that are withholding their data. It's the fact that the "free-market" solution to communicating science has been for-profit journals. In order to fact-check and peer-review articles before publication, that costs money. So journals charge money for access. And since a journal article is usually only accessible to people with a high degree of scientific training to comprehend (one must use dense scientific jargon to compress an article into a reasonable reading length), the sales of such journals have naturally small audiences. Thus, the cost for each journal is fairly high.

I think what would be better, of course, is to publicly fund open-access journals and their editing. But I'm sure then you'd accuse *them* of operating with a pro-big-government bias.

So there are your options: expensive "free" journals, or free "public" journals.

So there are your options: expensive "free" journals, or free "public" journals.

...or one could look for a preprint version on arxiv...or one could write to the author and ask them for an electronic copy (this has worked for me 100% of the time. For some reason scientists actually *like* it if someone wants to read their paper. Go figure.)

Oh, yeah I agree that there are often ways "around" the journal system. But Arxiv doesn't work as a preprint server if there isn't, somewhere, some form of publication. You still need someone to do peer-review (maybe not the most perfect system, but better than nothing at all). Otherwise you get people publishing their "theories" online... *cough every crackpot ever cough*

antigoracle: Why do you think that? Nearly all the scientists who currently have access to each others' data and methods agree on AGW. How would more people having access to the same information "end" that?

The anti-AGW side already cherry picks data and arguments. Would access to more data make them any less likely to cherry pick? Any less likely to take data out of context, or take analysis techniques out of scientific context? I doubt.

Open access won't do anything to change either side of the AGW debate. One side is scientific in nature already, and increased journal access won't help them. One side is economic and political in nature already, and increased science is irrelevant to their arguments in the first place.

Shavera, perhaps you should have a look at the Climategate emails to see why I make that statement. Those emails are littered with admissions of how they would subvert the scientific process. The following are just a few:- redefine what peer-review is- destroy data, acquired with public funding, rather than make it available- refuse to divulge their method- threats to scientists and publishers who dare to have an opposing view

AGW is painted as such a global human tragedy, you would imagine that they would open their doors and beg for you to come in and have a look at their evidence.

For years I bought into the AGW scam, until I listened to a few scientists on the other side, and they all made one very compelling statement - Don't take my word for it, educate yourself and look at the evidence.So, open access would let you see that evidence and not take either side just on their word.

Oh, yeah I agree that there are often ways "around" the journal system. But Arxiv doesn't work as a preprint server if there isn't, somewhere, some form of publication. You still need someone to do peer-review (maybe not the most perfect system, but better than nothing at all).

I was just referring to papers that are published in peer reviewed journals. You can often get preprint versions of them at arxiv. Arxiv isn't a first go-to for research papers. (Note that you should be *very* careful about citing an arxiv paper in your own publications. Citing the published version is infinitely preferrable). Papers that appear only on arxiv and never anywhere else? Not a good source.

Please sign in to add a comment.
Registration is free, and takes less than a minute.
Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.