Seldom original. Often wrong. Occasionally interesting.

I recently learned about Peer Community In (PCI), a new system for reviewing and recommending preprints. I’m really intrigued. It’s true that I’m an old fuddy-duddy who’s on record as saying that we often exaggerate the problems with the status quo, and as not liking to think outside the box. And yet there are good reasons to think it might be good to have other ways beyond traditional journals to disseminate science. We should experiment with a variety of new systems, and PCI seems like one well worth exploring. Read on to learn more!

We believe that the current system of publishing with academic journals suffers from four crucial problems. First, it is incredibly costly for research institutions. For example, in France, the cumulative cost of subscriptions and page charges exceeds 145 million euros/year (sum for universities, CNRS and INRA). These funds are unavailable to further research or teaching/training efforts. Second, for-profit publishing companies rely upon the work of researchers as authors, editors, and reviewers, who are not paid, and are not doing research or teaching while they work for journals. Third, the system is not transparent enough: editorial correspondence and reviewers’ critiques of accepted articles are rarely published and those reviewers’ critiques are often far too succinct to ensure scientific quality. Fourth, the shift towards the economic model of open access means that almost anything can get published, whether it is of high quality or not. The open access movement is laudable in many respects – research findings should be a greater good, accessible by anyone. But the system is intrinsically corruptible: for-profit publishing companies have an economic interest in accepting as many articles as possible, regardless of their worth.

A new development makes changing this system more feasible than ever before. Manuscripts, called preprints, are increasingly commonly deposited by researchers in open archives such as bioRxiv.org or preprints.org, making research results available more quickly than when submitted to a journal, as well as free of charge. This immediate availability also allows the use of social networks to comment on the results, thus promoting contact between science and the public. However, preprints are not formally evaluated by the scientific community, making it difficult to discern whether the studies are valid. This is a problem.

We propose a completely free solution for authors and readers, called Peer Community In (PCI).

This project aims to establish communities of researchers (Peer Communities) in different fields of science to evaluate articles. If the research is deemed to be sound and worthy, it is then validated with a public recommendation to the broader field.

The authors of manuscript deposit it into an open archive such as bioRxiv.

The authors then submit a request that the manuscript be reviewed by a relevant Peer Community. As evolutionary biologists ourselves, we have started with the creation of a Peer Community in Evolutionary Biology.

The only condition for requesting review by a Peer Community is that the preprint is not already published or submitted for review to a journal.

A researcher within the relevant Peer Community (called a “recommender”) handles the preprint and obtains at least two reviews of the manuscript (e.g. their own, and one other), and the standard revision and re-review occurs.

If the recommender deems the work to be valid, as well as interesting to the field, they will publish a recommendation of the manuscript. Recommendations are short essays that highlight the merits of the manuscript (here’s an example). Along with the recommendation, the reviews (anonymous or signed, as the reviewer prefers) and response to reviews are also published. The recommendations themselves have a DOI and can be cited.

Importantly, evaluation and recommendation of a preprint by a Peer Community will not prevent subsequent submission of the article for publication in a journal.

This new system has some similarity with conventional academic journals. However Peer Communities do not publish scientific articles. They do publish recommendations, reviews and commentaries on preprints posted in open archives. As a way to engage in the broader academic research process, they also publish evaluations of already published articles (in this case, the authors do not request an evaluation – a recommender simply highlights a particularly interesting paper that is already published, similarly to F1000 recommendation).

Because recommendation of a preprint by a Peer Community does not prevent it being submitted for publication in an academic journal, the Peer Community In project can coexist with the current journal system. Indeed, most journals today accept submission of articles whose preprints have been deposited in open archives (to check your favorite journal, use sherpa/romeo’s handy service). Furthermore, reviews and recommendations by Peer Community in can be used by journals, avoiding duplicate review efforts.

We launched the first Peer Community in January 2017: The Peer Community in Evolutionary Biology (PCI Evol Biol). PCI Evol Biol currently brings together more than 300 evolutionary biologists to handle pre-prints. This Peer Community has already posted recommendations of 20 postprints and 3 preprints, and 16 preprints are currently in evaluation. The editor-in-chief of Ecology Letters (Tim Coulson) has indicated interest in using PCI reviews and recommendations to streamline their review processes. It seems likely that other journals will follow suit.

In summary, the idea behind the Peer Community In project is to establish a free and public review system to evaluate preprints and to validate and highlight preprints of good quality. The ultimate goal is that the Peer Community In recommendations will be recognized by the scientific community, including research institutions and funding agencies, as a high-quality label, thus rendering submissions to conventional journals optional.

We encourage evolutionary biologists to submit their work for evaluation by PCI Evol Biol, and we encourage others to consider forming Peer Communities of their own. Discussions about the launch of a PCI Ecology, a PCI Paleontology and a PCI Applied Statistics have already started. Contact one of us if you’re interested in exploring how to initiate a Peer Community in your field or sub-field.

A few thoughts from Steve – and please leave your own in the Replies. I’m much less pessimistic about the flaws in the journal-publication model, but I think we should be experimenting with other possibilities. I’m especially intrigued with PCI because I think it has the potential to fix one major shortcoming of the preprint-server system: its lack of an attention-leveling mechanism. Journals are one system for assigning review for, and garnering reader attention to, papers without regard to the pre-existing fame of their authors. (Yes, I know they aren’t a perfect system for that.) PCI seems like it could do the same, as long as all requests for review are honoured. I also like the fact that PCI neither requires nor prevents later submission of a paper to a conventional journal; this solves the bootstrapping problem of how we shift to a new publication medium before we have assessment systems (like tenure committees) that value that medium. Will PCI need some tuning as it grows? Probably, like anything else. I wonder, for example, how it will keep up if the volume of authors requesting review grows very large. But since that’s a problem that comes with success, having to solve it might be a good sign.

I have been interested in the whole pre-print idea but have a couple questions regarding this model.

1. Is it assumed that the “system/community” will take care of conflict of interest situations? For example, one could submit their paper, have a colleague quickly grab it as a “recommender” and then send it out to “friendly reviewers”. It seems that this is the job of an official editor and that check may go missing in this sort of model.

2. The idea that it can later be sent to a journal is great, and the idea that an editor may use the reviews is even greater (minus my concern above), but as an early career researcher I would and am worried that its an extra step (read time) in the publication processes. Unfortunately I feel that I need to bolster my publication record in hopes of landing that sought after permanent job. Do you think faculty search committees or Executive Directors of NGO’s are starting to take pre-prints as seriously as published journal articles or is this a long way down the line?

Even in the early days of my own research career I find myself frustrated by the publication process for many of the same reasons written about here. So, I am greatly appreciative to those taking the time to work on solutions.

Hi Travis
Thanks for your message. To complete the answer of Ruth about parallel submissions, here’s why we do not like this idea (as many of our colleagues):
-We received 15 preprint recommendation requests during the last month. An extrapolation to the end of 2017 would lead to 80 submitted papers, a really good achievement for PCI Evol Biol, without parallel submissions.
-Many of our colleagues consider that simultaneous submissions will not increase their incentive to submit to PCI Evol biol, because 2 months is nothing compared to the amount of time needed to get the results and to write the article
-The goal is to increase the quality of the preprints and PCI Evol Biol reviews can be very helpful in this perspective.
-Parallelizing the reviewing work may lower the role of PCI Evol Biol reviews. Authors may not respond to requests for changes if their paper is accepted for publication in a journal prior to PCI Evol Biol’s recommendation.
-Many recommenders simply think that reviewing a preprint that is submitted in parallel to a journal is just a waste of time.
As indicated by Ruth, after a year or so, the managing board of PCI Evol Biol may reconsider this option if needed.
Cheers
Denis, Thomas, Benoit

Hi again Travis.
About moral hazards, There are 2 important points:
-this is the role of the managing board to ensure that there are no conflict of interests. The MB uses to check if recommenders and reviewers have published or collaborated with the authors of preprints they handle.
-reviews and recommendations are public: this is a good protection against such problems. Bias, cronyism, retaliation or flattery are limited by i) the transparency of the reviews, which are freely available and possibly signed, and ii) the transparency of comments and recommendations, which are freely available and signed.
Cheers
Thomas, Benoit, Denis

Thank you, Thomas and Ruth for responding. This whole concept is very fascinating to me and I appreciate the work you all are doing. Seems to me, if publish or peril is the system, there will always be cheaters. I do think the transparency of your system will help with self-policing. As for recruiting ECR, seems like a conundrum. All of your points seem perfectly valid to me, thanks for thinking about it.

I concur with Travis that there is a danger of moral hazard. But I find the whole concept FAR more appealing than the commercial efforts put in place now to reward reviewers through Publons (probably others are jumping this bandwagon as well). Thanks for thinking outside the box with PCI.

Hi Koen and Travis-
I think that issues of moral hazard are important, and also found in the traditional publishing model. There are very real issues of cheating, and then there’s also the more slippery slope of bias of all sorts. Here is PCI Evol Biol’s code of conduct: https://evolbiol.peercommunityin.org/about/ethics
Recommenders can be kicked out, essentially, if they don’t behave ethically.

For the lag time of two submissions – Travis – I completely agree with you that it’s a challenge, especially for young scientists. I and some on the board of PCI Evol Biol have argued that we should allow simultaneous submission to journals to minimize any “cost” of submitting to PCI. Others on the board of PCI Evol Biol and some recommenders are concerned that an author will respond to reviews from a traditional journal rather than to PCI reviews, and that it will frustrate recommenders and reviewers who are volunteering their time. For the moment, Denis, Thomas and Ben have decided to stick with the original plan of not allowing simultaneous submission. If it seems like it is prohibitive, however (e.g. we don’t get enough review requests) then after a year or so, the board will reconsider.

Privacy

I participate in the Amazon Services LLC Associates Program, an affiliate advertising program, and its equivalent at Chapters.ca. If you follow a link from my blog to Amazon or Chapters, your origin here will be tracked only for the purpose of paying me a pittance (with no effect on pricing for you). If you prefer not to be tracked, you can always visit an online or bricks-and-mortar bookseller directly. I promise I won't follow you there.