Cautiously Open to Open Science

Last week I was asked to peer-review a manuscript for a top academic journal in my field of study. I accepted the invitation and spent several hours of my weekend reading and critiquing 30 pages of dry technical writing. Did I get paid to do this? Nope. Did I get credit for it? Not really (reviewers are typically anonymous). So why would I waste a precious Saturday for this? I did it for the reason I think most of my peers accept invitations to review: It is my obligation as someone with academic credentials to participate in the scientific process and ultimately advance the science. Serving as peer reviewers is the unspoken contract we all agree to when we gain expertise in a field.

The work of scientists has progressed through the scrutiny of fellow scientists for hundreds of years. Photo: Flickr User: Pierre Metivier

The peer-review system provides a way to ensure the quality of science, and it safeguards against accidental or intentional inaccuracies in scientific literature. The process is imperfect, of course, but it works impressively well for a system that relies on anonymous volunteers who donate their time and intellectual thought for little in return. It’s quite remarkable really.

But recently the traditional peer-review process has been complicated by a widespread push for more open science. The two concepts aren’t necessarily at odds—plenty of open science is peer reviewed—but let me explain. The goal of the open science movement is to make science—including data, publications, computer codes, etc.—available and accessible to all, experts and non-experts alike. This includes initiatives like the Open Data Policy released by the White House last week, and a growing number of open-access academic journals. Even more, there have been success stories for crowd-funding of research and crowd-sourcing of peer review itself. The open science movement is here now and it looks like it’s here to stay.

The power of peer review

Here at the Center for Science and Democracy, we have always been fans of transparency, especially when it comes to government science. The more transparency we have in how science is (or isn’t) being used in the policy-making process, the better we know and can hold our leaders accountable to making important decisions based on evidence.

But when it comes to science-based policy, we are also big fans of the peer-review process as a means to obtain the best available science and to help non-experts distinguish science from its impostors. As the push for open science grows, it has placed strains on the traditional structure of academic publishers that use revenue from journal subscriptions to finance the peer-review process and ensure the quality of the articles they publish. In last week’s public comment meetings on the White House’s open access initiatives, several groups expressed this concern. For example, the American Physical Society (APS) stated,

“… we want [to] re-emphasize, in strong terms, the crucial contributions of scientific publishers to the research enterprise, and the need for sustainable funding to support these contributions. Peer-reviewed journals are, if anything, even more essential in our internet-enabled environment. In an era in which a vast amount of un-refereed scientific literature is freely available on the web, refereed journals take on special importance and their publishers perform critical services.

Open access publishing: science versus spin

Some have argued that the subscription-based (non-open-access) publishing industry overcharges authors for these editorial and peer-review services and that open-access journals can offer these same services at a fraction of the cost. But the APS statement also touches on another concern about open-access publishing: the “vast amount of un-refereed scientific literature [that] is freely available on the web.”

Open-access online journals can be a great thing. For scientists, they often mean peer-reviewed publication can happen faster and cheaper and your scientific work will be more widely read and cited. I myself enjoyed these benefits in a 2011 research paper I published in an open-access peer-reviewed journal. While I appreciate that my article is freely available even to my mom (she claims to have read it), I worry that the abundance of free scientific literature online makes it challenging for non-experts like her to distinguish science from what just looks like science. And there’s a lot of the latter out there, as documented in a recent New York Times piece.

Many online-only open-access journals (like the one I published in) are well-respected academic publications that maintain traditional peer-review processes. But as the NYT article describes, this new online journal environment also has given rise to a new breed of pseudo-journals—disguised as scientific publications but without the rigorous peer review and editorial process. Such journals are simply pay-to-publish operations designed to dupe academics into paying high fees for publication in these non-reputable “journals.” And many academics have fallen victim. If even professors are being duped by these pseudoscience journals, how will interested citizens (who aren’t scientists) be able to find reliable scientific information online?

Expanding the access, preserving the process

Last week, I wrote that the administration’s new initiatives on open access were a step in the right direction, and I think they are. The new demands for more open science are a good thing. We can collectively share more, know more, make new connections—and ultimately advance the science faster. After all, we know science works best when it is open to wide criticism from peers.

But as we move forward, we need to ensure that the systems that produce high-quality science and help to enable non-experts to discern science from pseudoscience are maintained. New models for the production of scientific literature should account for these essential features. I think we can get there but we need to carefully navigate this new publishing environment. For now, I remain cautiously open to open science.

Support from UCS members make work like this possible. Will you join us? Help UCS advance independent science for a healthy environment and a safer world.

Show Comments

Comment Policy

UCS welcomes comments that foster civil conversation and debate. To help maintain a healthy, respectful discussion, please focus comments on the issues, topics, and facts at hand, and refrain from personal attacks. Posts that are commercial, self-promotional, obscene, rude, or disruptive will be removed.

Please note that comments are open for two weeks following each blog post. UCS respects your privacy and will not display, lend, or sell your email address for any reason.

I understand your concerns, but I see them just a special case of general question of how can we humans navigate through more and more information available online. Luckily, there are solutions, Reddit and Google are showing examples of this for global Internet, where the question is exactly the same: how to find relevant and reliable information. So we are already working on this issues. What is missing is that science community has not joined it. Once this will happen, through open access, we will be able to apply same or similar tools to the science as well. Even more, once all papers will be available in the open access manner, arguably more good quality papers (if the argument is that currently closed access journals have good quality papers) will be available as well. The issue is thus only that currently general public does not have access to this good quality research papers. This is why it is easier for them to find some fake science papers. Open everything and then we will be all able to find good research, learn from it and also learn to distinguish good research from bad. How do you expect general public to know the difference now, when most of the good research is not available to them?

Thank you for your thoughtful comment. I agree that scientific studies are a special case of our bigger challenges around the new information environment that we are currently trying to work through. Crowd-sourcing can be helpful for ensuring quality but the peer-review process represents a specialized crowd-source, so-to-speak. One concern that I think may be more unique to science is the combination between technical writing and potentially ill-intentioned actors. Climate change is the best example of this. Without the hurdle of peer-review by experts in the field, its easier for information to look like science through use of scientific language. I’d love to hear more about new tools and ways of helping us navigate these issues for the case of science. Keep the ideas coming!