Saturday, August 15, 2015

Earlier this week I received an unsolicited email message
from a company called Cyagen Biosciences
inviting me to cite its “animal model services” in my scientific publications. By
doing so, I was told, I could earn a financial reward of $100 or more. And since
the amount would be based on the Impact Factor (IF) of the journal in
question, the figure could be as high as $3,000 — were I, for instance, to cite
Cyagen in Science (IF of 30).

Austin Jelcick

The email surprised me for a number of reasons, not least because
I am a journalist/blogger not a scientist. As such, I have never published a research
paper in my life, and have no plans to do so. Moreover, I have only the vaguest
idea of what an “animal model service” is, let alone how I would cite a company
selling such a service in a scientific paper.

Curious as to why I had received such a message I searched on the
Web for the company’s name, only to find that the link from Google to Cyagen’s
home page delivered an error message.

Eventually locating an email address I contacted the company and
asked if it could confirm that the message that I received had been sent on its
behalf (It appeared to have come from a direct marketing company called Vertical Response).

The next day I received a reply from Cyagen product manager Austin Jelcick, who explained
that I had received the message “as part of our marketing campaign which is
currently seeking to raise awareness within the scientific community for our
citation rewards program.”

As I was associated with “several blogs and articles related to
open access journals and publishing” he added, it was assumed I would be
interested in “our newly launched campaign to actively reward scientists for
citing us in their materials and methods section while simultaneously
encouraging them to submit into higher impact journals for increased awareness
of both their study and our services offered.”

He added: “we felt that it would be beneficial to the researcher
to receive a sort of ‘store credit’ for doing something they already must do as
part of the publication process.”

Now intrigued, I invited Jelcick to do an email Q&A so that
he could explain in more detail who the company was and why it had launched this
campaign.

Very surprised by the offer

While I was swapping questions and answers with Jelcick by email
the company’s campaign was starting to attract a good deal of commentary on the
Web.

Yesterday, for instance, high profile physician and science
writer Ben Goldacre published
a blog
post entitled, “So this company Cyagen is paying authors for citations in
academic papers”.

Goldacre concluded, “Perhaps my gut reaction — that this feels
dubious — is too puritanical. But I am certainly very surprised by the offer.”

Goldacre’s intervention also sparked a post
over on Retraction Watch entitled, “Researchers,
need $100? Just mention Cyagen in your paper!”

By now there was also a steady stream of comments from scientists
on Twitter, expressing everything from puzzlement to outrage — see this
for instance.

By late yesterday Cyagen clearly felt the need to make a public
statement, which it did by means of a
Q&A on Facebook, explaining: “Please find below some of the questions
which were asked of us and our response which should help clear up the
misunderstanding which has occurred about this promotion.”

The post went on to list seven questions and answers. What the
company did not explain, however, is that these had been extracted from the interview
I was still in the process of doing with Jelcick. That is, Cyagen did not cite
me!

What has become clear is that the company believes that its email
invitation has been misunderstood. Linking to the Facebook post from a comment
on Goldacre’s blog, Jelcick went so far as to complain that Cyagen has become a
victim of “some gross miscommunication”.

Richard Van Noorden appears to agree, saying on
Twitter that the story has been “gleefully badly reported”. He explained: “you
can’t get $100 by citing them. You get a discount voucher for their products”. He
nevertheless suggests
that Cyagen should withdraw the offer “pronto”.

It would seem that the mistake Cyagen made was to link its
promotion to the much-maligned Impact Factor, which has become a red rag to
many scientists. (See also the first comment below).

Anyway, below is the full list of 17 questions and answers that
make up the interview I did with Jelcick. Some of the answers are a little repetitive,
but given the confusion surrounding Cyagen’s email I have chosen not to edit
them.

See what you think.

The interview begins …

RP: Can you say something briefly about
Cyagen Biosciences and your role in the company?

AJ:
Founded in 2005, Cyagen Biosciences Inc. is a 200-employee contract research
organization and cell culture product manufacturer with offices in Silicon
Valley, California and China, and production facilities in China.

As a leader in custom animal models and
molecular biology tools, Cyagen prides itself on the mission of bringing
outstanding-quality research reagents, tools, and services to the worldwide
biological research community at highly competitive prices.

My role at the company as Product Manager
ranges from marketing materials development, product/service seminars and
presentations, technical support and scientific support on our animal model
projects as well as our vector construction projects, and also general customer
service to ensure our customers receive the best support while simultaneously
delivering the highest quality products and services.

RP: In which country/state is Cyagen
registered and who owns the company?

AJ:
Cyagen is a privately held company with a US office and base of operations in
Santa Clara, CA as well an Asia Pacific office and base of operations in China,
with production facilities in China as well.

RP: Are you not able/willing to say who
owns the company or where it is registered (as in where it files its accounts)?

AJ:
Cyagen is registered in both the US (California) as well as China. I am unable
to provide further details of ownership aside from it being privately held.

RP: On 11th August I received an
unsolicited email message that appeared to come from a direct
marketing company called Vertical Response
saying that Cyagen was offering to pay me to cite its “animal model services”
in my scientific publications. If I did, I was told, I would get a payment based
on the Impact Factor of the journal in question. In the case of a journal
like Science, it added, this would be $3,000. Can
you point me to a web page containing the full details of the offer?

AJ:
The Citation Reward Program does not offer payment for citations, but rather a
store credit voucher good for future purchases of products and/or animal model
services from Cyagen for researchers citing us in their publications.

We are not asking researchers to do
something that they will otherwise not do. In fact, when a researcher publishes
a paper, they are ethically required to disclose in their publication the
outside reagents and services they received that contributed to their research
findings, including those purchased from commercial entities. Our voucher is
just a way of thanking them for using our products and services that ultimately
led to their published scientific findings.

It appears that our email has caused
misunderstanding as to the nature of, and how we wish the researchers to
respond to, the voucher. As such, we extend our full apologies to you and other
researchers for the misunderstanding caused by insufficient clarity in our
email. Full details on the promotion can be found here.

RP: Is there a marked difference between offering
to pay an individual for citing Cyagen and offering them a store credit for
citing the company?

AJ:
We believe so for two reasons. The first being that citing us in their
publication is a standard ethical requirement for any researcher should they
have chosen to utilize our products or services in their study initially and so
they are being rewarded by means of a discount on future purchases rather than
a personal incentive.

Expanding on this, the second reason is
that we offer an incentive that allows for future research/studies to be
performed rather than a personal incentive; this way it is the research
institution and their studies which are benefitted.

RP: How many scientists have shown interest
in/taken up Cyagen’s offer so far, and what are your expectations here
(presumably you have set aside a sum of money to cover the payments)?

AJ:
Since the promotion’s launch in July, we have had only a few researchers
contact us regarding the program. Again, no money is set aside as researchers
are not being given financial compensation but rather store credit for future
purchases.

Given that researchers who utilize our
services are required to disclose the source(s) of all reagents and services
used in their research when publishing a journal article, we would hope that
our existing customers would take advantage of the promotion and become return
customers, while new customers may be more likely to choose us over a
competitor for being rewarded for something they already must do as part of the
publication process.

Why did I receive this
invitation?

RP: I am actually a journalist/blogger
rather than a scientist and so have never written a peer-reviewed paper. Why
did I receive this invitation?

AJ:
Our marketing and business development teams do their best to find researchers
or associated individuals (i.e. Lab managers, purchasing personnel, etc.) who
are actively seeking and/or purchasing reagents and laboratory services for
their research.

Due to the nature of your blog (which
covers peer reviewed publications and open source journals) our staff assumed
that you were involved in active publications or research leading to
peer-reviewed publications.

RP: What are “animal models” by the way? Is
it that Cyagen breeds rats and mice and sells them to labs for experimentation,
or am I misunderstanding?

AJ:
Animal models are just this: model organisms for the study of diseases and
underlying biological mechanisms for the development of novel therapeutics for
both humans and animals (domestic, livestock, etc.). The animal models we
produce are custom engineered for the researcher based on their study.

For example, a researcher may be studying
the effects of a drug on weight loss during a high fat diet and may think that
Gene X may cause some sort of increased/decreased response to the drug. The
researcher would then look for a mouse/rat model where Gene X is missing or
overexpressed, and after contacting us, we would engineer a genetically
modified mouse/rat lacking Gene X or over expressing Gene X for their studies.

RP: So my understanding was right: When you
say “The animal models we produce are custom engineered for the researcher” you
are talking euphemistically. An animal model is a live rat or mouse, and when
you say you “engineer” animal models you are referring to a process that a
layperson would call “breeding” an animal?

AJ:
Yes and no. “Model” is not a euphemism but a standard term in the field, which
refers to modelling a disease state or condition.

Breeding the animals is a step of the
process however. To initially develop the animal models a variety of genetic
engineering techniques are used including constructing synthetic DNA fragments
and subsequently injecting these into an early embryonic stage of development
so that they integrate with the animals’ DNA, subsequently creating an animal
with foreign DNA or with portions of its DNA altered.

Also, the embryos and animals which are
originally used are inbred laboratory strains such as ones you would find at a
variety of animal providers or university core facilities. If you would like
more information on these animal models, there are a variety of good articles
on Wikipedia among others on transgenic and CRISPR mice.

RP: What is your understanding of the
Impact Factor, how it is calculated, and what it measures?

AJ:
Generally speaking, an impact factor (IF) is a metric used to gauge the amount
of times articles in a given scientific journal are cited, thereby serving as a
proxy for the journal’s importance within its field of study.

This can be calculated in two ways. The
first is the number of citations in a given year to articles published in the
previous two years (A) divided by the total number of articles published in the
prior two years (B). A/B = IF.

The second is similar, but removes the
number of self-citations (C) from (A) prior to division by (B). (A-C)/B = IF.

For example, if in 2012 Nature was cited 250 times for articles
published in 2010-2011, and it has cited itself 75 times, and a total of 100
articles were published in 2010-2011, then its corrected IF would be
(250-75)/100 = 1.75.

Citing products not papers

RP: To clarify: Cyagen is asking
researchers to cite its products, not papers written by its
employees? If that is right, then presumably this is not about traditional citation
boosting , but paying scientists for what amounts to
“product
placement”?

AJ:
The goal is not product placement but rather: (1) to increase the number of
publications featuring Cyagen as a product or service provider to add strength
to our company’s reputation and visible experience; (2) to reward researchers
for performing a task that is already required of them as an extra way of
saying thanks for choosing us as a service provider to begin with.

Researchers already must (ethically)
disclose in their publications any references cited, any sources of reagents
and services, and any collaborations. Should a researcher purchase cell media
from Company X, and mouse breeding services from Company Y, while collaborating
with Professor Z, they would have to disclose this in their publication so that
all entities receive credit for their contribution to the study.

Additionally, most studies are not
“endpoint” studies so to speak, with many having follow up studies to determine
further details.

As a result, it is beneficial to a
researcher and their laboratory budget if they receive a store credit for their
next purchase as a reward for their initial purchase and subsequent published
paper as this decreases the cost of their next study. This is similar to
receiving coupons after a purchase at a retail store good for use on the next
visit.

RP: Do you feel that there are any ethical
issues involved in offering to pay researchers to cite products in their
papers?

AJ:
Again, we are not offering to pay researchers to cite our products or services,
but rather we are offering them an incentive to become a repeat customer.
Rather than offering a discount on a current order (which we have done as part
of other promotions) we are offering a discount by means of a store credit on
future purchases.

Because of this, we do not feel that any more
ethical considerations are raised than if we were to offer discounted services
from the beginning to gain customers’ interest in choosing us over a
competitor.

RP: When I contacted you about the message
I was sent you replied that Cyagen was encouraging researchers “to submit into
higher impact journals for increased awareness of both their study and our
services offered”. The ethics of paying people to cite Cyagen products in
research papers aside, do you not feel it to be a somewhat retrograde step to
encourage researchers to chase after Impact Factors at a time when many are
calling for the downgrading or extinction of the Impact Factor? For instance,
over 12,000 researchers and nearly 600 organisations have signed the San
Francisco Declaration on Research Assessment recommending
that the research community should not “use journal-based metrics, such as Journal Impact Factors, as a
surrogate measure of the quality of individual research articles, to assess an
individual scientist’s contributions, or in hiring, promotion, or funding
decisions.”

AJ:
The encouragement of researchers to seek higher impact factor journals is based
on the existing academic thought that impact factor still is a somewhat
relevant metric for assessing the importance of a given journal or article,
simply because it is based on the number of times a paper/journal is cited by
other publications.

This being said, impact factor is just
that: a metric. The number of times a paper has been cited does not necessarily
reflect the quality or integrity of a publication or journal, as many
researchers are aware of pioneering and ground-breaking studies which were
looked over at the time of publication but years later came into popularity and
prevalence.

As many researchers still actively look to
publish in large, popular journals simply due to their exposure and subsequent
citations by other researchers in order for their own studies to gain exposure
and relevance, we felt that offering store credits to researchers for something
they already were doing or attempting to do was a sort of easy and simple
method for researchers to obtain discounts on future orders as opposed to discounts
requiring minimum purchases, quantities, etc.

Unscientific and deceptive?

RP: The point critics of the IF make I
think is that while it may say something about some of the papers that a
journal has published over a specific time period, it says very little (or
nothing) about an individual paper published in that journal. Is it not
the case, for instance, that a paper published in a high IF journal may receive
no more — and possibly fewer — citations than if it were published in a small specialist
journal with a low or no IF? Is that not the reason why there is now a strong
movement against the use of the metric: it tells you little or nothing about
the quality of an individual paper? If so, is not Cyagen encouraging
researchers to engage in a practice that it is now widely held to be unscientific
and deceptive. Would you agree with that?

AJ:
You are correct in your statement regarding a given article within a given high
(or low) impact journal. Ultimately the impact factor of the journal does not
directly relate to the quality or number of citations any given article in it
may have or receive.

As someone who has previously published in
open source journals (PLOS ONE) I would also add that there are a number of
high quality articles and studies in smaller open source journals, and that a
higher impact factor journal may have the same number of high quality studies
as any other journal.

The IF metric however is not deceptive or
unscientific, it is simply one measurement of a journals’ relevance to the
field it is a part of. It may be going too far to say it has no relevance to
the quality of the paper as high impact journals tend to have a higher “level
of entry” but is not a perfect metric. Similar to any survey or poll which
attempts to measure a general trend, opinion, or importance, it must be taken
with a grain of salt and in conjunction with other metrics for a true
observation to be drawn.

Our citation rewards program in its current
state is our first attempt at rewarding researchers for performing an aspect of
research which they normally do already as an attempt to gain repeat customers.
As we receive more feedback on the program, we may update and alter the program
so that the promotion details are clear and that all previous customers (who
actively publish studies) receive credit towards future purchases.

RP: Here is what I still don’t quite get:
you said that researchers are “ethically required to disclose in their
publication the outside reagents and services they received that contributed to
their research findings, including those purchased from commercial entities.”
You also said “We are not asking researchers to do something that they will
otherwise not do”. Why then would you go to the effort of emailing them to say,
“We are giving away $100 or more in rewards for citing us in your publication!”
And why would you say to me when I contacted Cyagen that the aim is to
“increase the number of publications featuring Cyagen as a product or service
provider”. I hear what you say about
wanting to thank people for being customers and encourage them to buy again,
but would it not be better to focus on improving your products and your
customer service than paying them for doing something you say they are bound to
do in any case? More pertinently perhaps, is it wise to link your offer to
something as controversial as the Impact Factor? As I think you acknowledged,
researchers have not responded too well to the offer. Here are some of responses
on Twitter: one,
twothree
and four.

AJ:
As I mentioned, we are constantly looking for ways to improve the promotion as
well as new promotions. As we receive feedback from the research community, we
will update promotions accordingly so that they are both beneficial and clear
without cause for any sort of ethical concern. Impact factor was initially decided
upon based on its historical usage by the research community. Based on
feedback, we may alter the promotion to simply offer a store credit for each
publication, independent of impact factor.

With regards to the previous statement
about our aim to “increase the number of publications:” We frequently receive
inquiries from new customers requesting references or citations from
researchers who have utilized our services in the past, and do our best to list
all publications (found through periodic PubMed queries) citing us on our
website as a reference. By offering this promotion researchers will actively
reach out to us to inform us of their latest study and publication allowing us
to update our citation/reference database more quickly, thereby increasing the
number of publications in which we are cited available for viewing and reference
by new customers.

Additionally, although researchers are
ethically required to disclose all sources of reagents, services, and
collaborators in their materials and methods, they may not always remember to this.
So the promotion serves a dual purpose as a reminder to do so.

Error message

RP: I received the marketing email two days
ago. When I searched on your company name I found the home page but when I
tried to access it I was told it was down. It is still down as I write this.
Were you aware that your home page was giving an error message?

AJ:
Our staff has been unaware of any down time suffered by our website as we have
not received any reports of downtime by either customers or staff members
utilizing the website this week including today.

RP: Perhaps it is because I am based in
Europe, but if you search “Cyagen Biosciences” on Google and click the first
hit that comes up you get the following result:

That is how it has been for the past few
days, and I am still getting the message as I write this follow-up question. I
am not sure this could be described as an improper link because the URL that Google
has indexed (and which is not working) is www.cyagen.com/.
This would seem to me to be likely to conspire against Cyagen attracting many
new customers to its services. Would it not be better to focus on improving the
way that customers find you than mass mailing them and offering to pay them for
something they have to do anyway?

AJ:
This is the first report of any downtime of our website in
Europe, and we thank you for providing a screenshot of the error which occurred
(we will forward this to our IT team to solve the issue as soon as possible).
This may be a regional DNS error as well as we have not had any reports of
errors within the US, Canada, or the Asia Pacific region (see below for
screenshot from US based inquiry).

Mass emailing is only one small aspect of
our marketing strategies. We are actively engaged in optimizing our website as
the newest revision just launched this summer, including SEO and relevant
keywords, in addition to various promotions.

The citation rewards program is just one of
several current promotions including free vector designs for nuclease based
animal projects, reduced pricing for more popular mouse strains, and more.

Additionally, we have regional sales
representatives who visit universities and companies on site for seminar
presentations as well as attend vendor shows to enhance our customer exposure,
and are constantly looking for new and innovative ways to expand our
visibility.

7 comments:

I had a comment posted on Twitter saying, "Despite what they say, this IS paid product placement and should be declared as a conflict of interest."

I don't disagree with that. And since Cyagen envisages issuing vouchers retrospectively it suggests that an additional complication could be that some authors would find they have to go back and issue an erratum.

In short, whatever one's views on the ethics of the scheme, this would seem to be a bird that will not fly.

If Cyagen Biosciences is as you say only a small company then their 'discount vouchers' might suggest they are struggling to compete with the big companies like JaxMice and Charles Rivers etc. Please set their activities in context with what is already known about preclinical research and you may see that it is only a small piece of the somewhat scandalous nature of the industry. I wrote about it here: http://www.biomedcentral.com/1472-6939/16/53

1. What you are saying about the impact factor being phased out is not quite correct. The journal impact factor is the average citation count of the articles in the journal. Citation counts themselves are of course not being phased out. It is relevant and important for many reasons how much an article (or an author) has been cited — or is likely to be cited. The average citation count (impact factor) of the journal in which an article is published is of course much less informative than the citation count of the article itself, but its predictive value is not zero either. If something important depended on sorting in advance the articles that are likely to have high, medium or low citation counts in two years’ time, the average citation count of the journal (together with the average citation account of the author) are the best predictors when it’s still too early to count the article’s eventual citations. Only an expert who could read, compare and sort all the articles could do better.

2. That said, what this offshore “animal model” vendor is doing with these financial incentives to flog his wares in journals — the higher-impact the journal, the better — is obviously revolting and ought to be illegal, or at least named and shamed, as you are doing. I can hardly understand why it is being treated as if there were any ambiguity whatsoever on this count. (And the fact that they want to give a bigger reward for being advertised in a high-impact journal is just evidence — if evidence was needed — that the impact factor still has some predictive value, not that this disreputable business practice is unwittingly hitching its wagon to the wrong metric. (I’m sure they’d be happy to give more vouchers to articles with higher citation counts, but it takes rather long to wait for those counts to accumulate, so it may complicate the business arrangement.)

3. Infinitely more important than any of this: the business of genetically engineering living, feeling creatures as a “product” is loathesome in its own right. If you think the “victims” of predatory Gold OA publishers are research and researchers, imagine how humane these genetic factories are for the real breathing feeling victims they design and sell, along with their sales vouchers for publishing plugs. Biomedical research, vital to the saving of lives, cannot be halted, but you can be sure that in a business like the one under discussion here, the life-saving criterion, for humans, is of no account, and the humanitarian one, for the animal victims, even less so. It’s all about cash: animal “models” for cash, and cash for whatever promotes the product. Give me predatory fool’s-gold OA publishers any time: at least there, nobody’s bleeding.

Thanks for this follow-up, Ricky. I think where the messed up was tying the promotion to the Impact Factor. It's common for companies to want to know how often they're being mentioned in the literature, and lacking the ability to text-mine all publications, most companies have to depend on authors reporting those mentions to them. A financial incentive, which works like a reminder, to let company X know you've mentioned them in a paper is just one way to get authors to report to them the usage. I don't find that bit objectionable. It also suggests that one potential market for altmetrics is companies that want to know about the "impact" their services have had.

Just to briefly respond to Dr. Harnad, he's demonstrably wrong about point 1, and arguably misled about point 3. (I guess I've said what I think about point 2 above).Regarding point 1: The Impact Factor is of no predictive value regarding an individual article in an issue of a journal. The reason for this is that citations are distributed among articles in an issue in a pseudo-logarithmic fashion. It's not at all true that the average number of citations per article in an issue of a high impact journal is higher than the average citations per article in a issue of a journal with a lower impact factor. More to the point, citations/article isn't the proper metric to use for the distribution of citations per article. Average citations per article would only be appropriate to use if the citations each article received were normally distributed, which they most certainly are not. What Cyagen should have done, if they were to do anything at all, is to use an article-specific metric. Since they can't use number of citations, because they take too long to accumulate, they could have use the number of downloads of the article, the number of Mendeley readers of the article (which correlates reasonably well with future citations - disclosure: I work for Mendeley) or the altmetric.com score or, you know, ANYTHING other than the impact factor.

Regarding point 3: Animal models save lives.

I am aware that all of the above could use citations in various places, but please forgive this oversight. I am writing this around midnight on Sunday after a 10 mile trail run and a few drinks. The alternative would not be writing it at all. Please inquire if you need further information.

"Animal models save lives" may or may not be true, but Mr Gunn completely misses Dr Harnad's point, which is ethical. Even if Mr Gunn's assertion is correct, it does not change the fact that animals are being treated as less important than humans, and are suffering and being manipulated to help humans - which is Dr Harnad's point.

Mr. Gunn, I think we can agree that Point 1 could only be settled by evidence: One could take, say, 10 articles from 100 journals at publication time T, and rank them by the impact factors of their journals. Then, at Time T + 2 years, rank them again by their accrued citation counts. Would there be a significant positive correlation between the two rankings or not? That would not depend only on the normal distribution of citations within and between journals. It would also depend on the correlation between journal impact factor and journal quality (as ranked by submission/rejection rates or by peer quality ratings). The latter correlation may not be high, but I bet it would be positive and significant too. Where I expect distribution would play a role is among the journal impact factors rather than just the article citation counts: The correlation between impact factor and quality may differ in different impact factor bands. And of course this would all have to be done within fields, not across them. -- But I don't deny that the correlations, even if significant, would be small, and the error variance huge. Hence the predictive power of impact factors would be small.

About Point 3 I would have to say that the issue is far, far deeper and more complicated than something that can be resolved with the statement "Animal models save lives." That correlation, like the putative correlation between journal impact factor and quality, can lie anywhere between 0% and 100%, and what is at stake in the error variance is animal lives and suffering, not useless articles, useless journals or useless quality rankings.