Saturday, August 15, 2015

Earlier this week I received an unsolicited email message
from a company called Cyagen Biosciences
inviting me to cite its “animal model services” in my scientific publications. By
doing so, I was told, I could earn a financial reward of $100 or more. And since
the amount would be based on the Impact Factor (IF) of the journal in
question, the figure could be as high as $3,000 — were I, for instance, to cite
Cyagen in Science (IF of 30).

Austin Jelcick

The email surprised me for a number of reasons, not least because
I am a journalist/blogger not a scientist. As such, I have never published a research
paper in my life, and have no plans to do so. Moreover, I have only the vaguest
idea of what an “animal model service” is, let alone how I would cite a company
selling such a service in a scientific paper.

Curious as to why I had received such a message I searched on the
Web for the company’s name, only to find that the link from Google to Cyagen’s
home page delivered an error message.

Eventually locating an email address I contacted the company and
asked if it could confirm that the message that I received had been sent on its
behalf (It appeared to have come from a direct marketing company called Vertical Response).

The next day I received a reply from Cyagen product manager Austin Jelcick, who explained
that I had received the message “as part of our marketing campaign which is
currently seeking to raise awareness within the scientific community for our
citation rewards program.”

As I was associated with “several blogs and articles related to
open access journals and publishing” he added, it was assumed I would be
interested in “our newly launched campaign to actively reward scientists for
citing us in their materials and methods section while simultaneously
encouraging them to submit into higher impact journals for increased awareness
of both their study and our services offered.”

He added: “we felt that it would be beneficial to the researcher
to receive a sort of ‘store credit’ for doing something they already must do as
part of the publication process.”

Now intrigued, I invited Jelcick to do an email Q&A so that
he could explain in more detail who the company was and why it had launched this
campaign.

Very surprised by the offer

While I was swapping questions and answers with Jelcick by email
the company’s campaign was starting to attract a good deal of commentary on the
Web.

Yesterday, for instance, high profile physician and science
writer Ben Goldacre published
a blog
post entitled, “So this company Cyagen is paying authors for citations in
academic papers”.

Goldacre concluded, “Perhaps my gut reaction — that this feels
dubious — is too puritanical. But I am certainly very surprised by the offer.”

Goldacre’s intervention also sparked a post
over on Retraction Watch entitled, “Researchers,
need $100? Just mention Cyagen in your paper!”

By now there was also a steady stream of comments from scientists
on Twitter, expressing everything from puzzlement to outrage — see this
for instance.

By late yesterday Cyagen clearly felt the need to make a public
statement, which it did by means of a
Q&A on Facebook, explaining: “Please find below some of the questions
which were asked of us and our response which should help clear up the
misunderstanding which has occurred about this promotion.”

The post went on to list seven questions and answers. What the
company did not explain, however, is that these had been extracted from the interview
I was still in the process of doing with Jelcick. That is, Cyagen did not cite
me!

What has become clear is that the company believes that its email
invitation has been misunderstood. Linking to the Facebook post from a comment
on Goldacre’s blog, Jelcick went so far as to complain that Cyagen has become a
victim of “some gross miscommunication”.

Richard Van Noorden appears to agree, saying on
Twitter that the story has been “gleefully badly reported”. He explained: “you
can’t get $100 by citing them. You get a discount voucher for their products”. He
nevertheless suggests
that Cyagen should withdraw the offer “pronto”.

It would seem that the mistake Cyagen made was to link its
promotion to the much-maligned Impact Factor, which has become a red rag to
many scientists. (See also the first comment below).

Anyway, below is the full list of 17 questions and answers that
make up the interview I did with Jelcick. Some of the answers are a little repetitive,
but given the confusion surrounding Cyagen’s email I have chosen not to edit
them.

Wednesday, August 12, 2015

Earlier this
year University of California Press (UC Press) launched a new open access mega journal called Collabra. Initially focusing on three
broad disciplinary areas (life and biomedical sciences, ecology and
environmental science, and social and behavioural sciences), the journal will
expand into other disciplines at a later date.

One of the
distinctive features of Collabra is that its authors can choose to have the peer review reports signed by the reviewers and published alongside their papers, making them freely available for all to
read — a process usually referred to as open peer review.

This contrasts
with the traditional approach, where generally the reviewers’ names are not
disclosed to the authors, the authors’ names are not disclosed to the reviewers, and the reviewers’ reports are not made public (commonly referred to as “double-blind”
peer review).

Since Collabra is offering open peer review on a voluntary basis it remains unclear how many papers will be
published in this way, but the signs are encouraging: the authors of the first
paper published by Collabra opted for open peer review, as have the majority of
authors whose papers are currently being processed by the publisher. Moreover,
no one has yet refused to be involved because open peer review is an option, and
no one has expressed a concern about it.

So how does open
peer review work in practice and what issues does it raise? To find out I emailed some questions to UC Press Director Alison Mudditt, whose answers are published
below.

Alison Mudditt

RP:
Presumably both the author and all the reviewers have to agree to open peer
review before Collabra can publish the reviews? What percentage of the papers
it publishes does Collabra expect will have the reviews published alongside?

AM: Authors
choose open peer review as an option upon submission, so it is always their
decision and as such they have already agreed in advance. Reviewers are made
aware that authors have chosen this option and could opt to decline the review
if they are unwilling to have their review comments made publicly available.

As a secondary option, whether or not open review has been chosen
by the author, reviewers can sign their reviews. So it is possible to have
reviewer comments be open, but the identity of the reviewer remain anonymous.
Or, for that matter, have closed review, but reviewers sign their reviews. This
is all described here.

With only one published article it is hard to project what the
percentage will be, but at this point the majority of authors—for the papers
currently being processed in our system—have opted for open review.

We are not targeting certain percentages, but rather want to put
new options in front of people, especially given the numerous critiques of
traditional closed peer review systems. This will not be for everyone, but we
believe there’s much to be learned from experimentation with new models.

RP: Will
Collabra make any effort to seek out reviewers who are comfortable with open
peer review?

AM: The
academic editors are selecting reviewers, and their top consideration will of
course be the reviewer’s expertise for any given paper.

We make all the options and elements of Collabra clear when
inviting external editors to be involved. Some editors are particularly
interested in the open review option, and other editors have not commented on
it.

No one has refused to be involved because it is an option or
expressed a concern about this option.

RP: I
assume that not all the correspondence is shared when Collabra publishes the
reviews, and perhaps they might be edited in some way first (at least
sometimes)? If so, what considerations/editorial rules are applied before
making reviews public?

AM: Currently,
the “open review file” is constituted by the reviewers’ comments on the
reviewer form, the editor’s comments to the author based on the reviewers’
comments, and the author’s response—all as captured in our editorial system.

It is clear on the review form that there is an area for
confidential comments to the editor that would not be shown to the author nor
included in the openly available comments. But, for the remainder of the form,
it is made clear that comments may be seen by the author and used without
editing.

What is not currently shown is any earlier version of the
paper and any comments or tracked changes on that. We will continue to monitor
this policy and will consider other options, if it seems that useful or
important elements are being omitted by not including earlier versions/changes.

And, obviously, if any discussion occurs outside of the editorial
system between a reviewer and an editor, that will not be captured.

As regards editorial rules and considerations for any edits or
omissions, we would discuss that with the editors as they came up. It is hard
to say in advance what that might be (other than any information which is
confidential and not even being revealed in the paper), so we’ll deal with that
on a case by case basis.

Naturally, we would opt to be transparent about this happening
should it occur beyond normal confidentiality considerations. For now we will
see how it goes with it’s being clear on the form that comments may be used as
written.

RP:
Having started down this road (and so given concentrated thought to the
matter), what would Collabra say were the pros and cons of open peer review?

AM: Speaking
on behalf of UC Press (I’m not sure it’s appropriate to speak as “Collabra” in
this context), we think that the inner workings of the peer review process are,
purely and simply, interesting for any reader, but in particular for people who
would like to see more transparency in this process.

There is clearly an argument to be made that making things open
(rather than, for example, the double blind process) will help to reduce
biases, problematic opinions, or hierarchical sensitivities that can affect the
review process.

Equally importantly, open review starts to demonstrate the value
added by the review process and to recognize the contributions of reviewers to
scholarship

Finally, we all know that traditional peer review has not put a
stop to whole disciplines being rocked by scandals of fabricated data and
unquestioned results, and it’s possible that open peer review will actually
help to improve the scholarly record.

On a related note, one of our other aims with Collabra is to get
rid of the phrase “peer review lite” which has plagued the type of review that
Collabra (and other OA titles) employs.

We characterize our review criterion as being “selective for
credibility only”—checking for the scientific, methodological, and ethical
rigor of a paper, and removing, as much as humanly possible, more subjective
reviewing criteria for novelty or anticipated impact. Open reviews will support
this mission—to show that there is nothing “lite” about this kind of review
(and in fact, sometimes quite the opposite).

It’s too early for us to be able to identify specific problems
with open peer review for Collabra, although we are aware of studies suggesting
that it may be harder to get reviewers and it may lengthen the review time. Our
limited experience so far does not support either of these concerns.

The other cons of open peer (as opposed to double blind) review
are clearly to do with concerns about bias, the highly variable nature of peer
review, and the additional costs it could impose on an already overtaxed
system.

For example, a reviewer might be worried about openly and
critically reviewing a more senior author and believe there could be a negative
effect on her own career.

Our hope is that a more open system will improve the integrity of
the peer review process, but the reality is that any system will be subject to
the biases of human nature—we just think that this is more likely to be
surfaced through greater transparency.

RP: Does
Collabra think that there are occasions when open peer review is inappropriate?
If so, when and why?

AM: Anything
raised in peer review of a confidential nature which does not make it into the
published article should be carefully removed from any open peer review
comments that get published during open review.

That said, we (UC Press) are not really the drivers of how open
peer review will evolve in Collabra or elsewhere. Since Collabra works only
with external editors, editorial policies should emerge that are firmly based
on the standards of each research community that publishes in Collabra.

If a community-driven majority standard emerged which stated that,
in certain situations, open peer review was inappropriate, then we would
respect such a decision.

RP: Are
there any other learning points that have emerged as Collabra has sought to
implement open peer review?

AM: It’s too
early in the launch of Collabra to really be able to comment, although we have
been pleasantly surprised at authors’ and reviewers’ willingness to consider
the option of open peer review. That seems to be a great start for this concept.