Scientists are universally familiar with the Impact Factor, even if they’re often frustrated with how it can be manipulated and misused. More recently, Ferric Fang and Arturo Casadevall have introduced the idea of the Retraction Index, a measure of how many papers journals retract for every 1,000 they publish. As science journalists who have spent the last 2 years closely monitoring retractions, we think this is a great idea.

Last year, in a post on our blog Retraction Watch, we recommended that journals publicize their Retraction Indices just as they trumpet their Impact Factors. It’s unlikely many will take us up on the suggestion, but we’ll go once more into the breach anyway and suggest another metric of journal performance: the Transparency Index.

Regardless of what metric scientists use to rank journals, one of the reasons they read the top-ranked journals is their sense that the information is reliable. We believe, and we’re not alone here, that journals become more trustworthy when they are open about not only their successes, but also their failures.

We understand—in theory, at least—why some journals and editors might be reluctant to share the details of a retraction with their readers. Sometimes the problems involve shoddy reviews, failure to check a manuscript for evidence of plagiarism or duplicate publication, or other avoidable mistakes.

But lack of transparency serves only to reinforce a sense of incompetence. Journals and editors willing to pull aside the curtain to show readers what went wrong with a particular article or group of articles send the messages that 1) they care about conveying truth to their audiences; 2) they are committed to producing a high-quality publication; and 3) potential fraudsters are not welcome in their pages.

Our hope is to turn the above criteria into a numerical metric that can give authors and readers a sense of a journal’s transparency. How much can they trust what’s in its pages? Help us refine the Transparency Index at retractionwatch.wordpress.com/transparencyindex. The number, however, will just be an indicator. Scientists’ judgment will still be the most important factor.

What sort of factors might go into the Transparency Index?

In no particular order:
• The journal’s review protocol, including whether or not its articles are peer-reviewed, the typical number of reviewers, time for review, manuscript acceptance rate, and details of the appeals process

• Names and expertise of editorial board members, including whether or not they know they’re on the board (that last part shouldn’t be controversial, but we’ve spoken to a number of alleged editorial board members who are surprised to hear about such appointments. The publishers of many of those journals show up on University of Colorado Denver librarian Jeffrey Beall’s List of Predatory, Open-Access Publishers. Read his Critic at Large, “Predatory Publishing.”)

• How authors are asked to disclose conflicts of interest, and the threshold for sharing those conflicts with readers

• Contact information for the journal’s editor-in-chief, including an e-mail account that someone actually reads

• Costs, not only for authors, but for readers. How much does it cost someone without a subscription (personal or institutional) to read a paper?

• Whether the journal requires that underlying data are made available

• The journal’s mechanism for dealing with allegations of errors or misconduct, including whether it investigates such allegations from anonymous whistle-blowers

• Whether corrections and retraction notices are as clear as possible, conforming to accepted publishing ethics guidelines such as those from the Committee on Publication Ethics (COPE) or the International Committee of Medical Journal Editors (ICMJE)

Ivan Oransky is former deputy editor of The Scientist. He and Adam Marcus are cofounders of Retraction Watch: retractionwatch.com.

Add a Comment

Comments

Transparency Index (TI) is a great idea which should promote and encourage best practice, while exposing publication malpractice. In order to be useful TI should reflect predominantly one thing which is: â€œDid the journal (editor and publisher) do the right thing?â€쳌

Retraction can happen due to variety of different reasons.Many retractions might indicate poor reviewersâ€™ and/or editorâ€™s practice. However, in other cases a retraction might indicate the opposite â€“ that the editor did the right thing after discovering irregularities about which s/he was unaware at the time of publication.

So, TI based only on No. of retractions from the total No. of publications will blur the real picture.

On the other hand, a journal (editor and publisher) might have brilliant policies/frameworks/procedures to deal with misconduct, which, however, are never (or seldom) used in practice. Such duplicity would indicate malpractice which intends to cover up previous malpractice. Thus such state of affairs would be even more dangerous as it intends to further deceive the readers.

Therefore, I think that TI should have as one of its main components:Do-the-Right-Thing (DRT), which can be measured by:No. of retractions from No. of cases (with evidence about misconduct) brought to the attention of the journal.

In ideal world DRT should be 1.Example 1:8 cases about irregularities, all backed up with evidence were reported to the attention of the journal, and the journal retracted all 8 articles (8/8=1).

Example 2: 8 cases about irregularities, all backed up with evidence were reported to the attention of the journal, and the journal retracted only 2 articles (2/8=1/4=0.25).

Provided that both journals have in place nice Frameworks to deal with misconduct, the later example would present the journal in a better light, while in reality the second journal is much worse, since it refuses (for whatever reason) to do the right thing.

We absolutely need retractions to clear the record and must avoid doing anything to inhibit retractions. For example, it can be difficult to publish a result different than what has been previously published. Not only must you be ultra-sure of your result (which should be true for any publication), you then need to establish why it is different than the precedent result. There are many retractions that are a result of sloppy science and/or statistical anomalies. One can argue that the report should never have been published. But it was. And the authors and journals are doing the right thing by retracting it. One wants to devise a retraction index that does not hinder the authors and/or journals from publishing those retractions. Then there are the 'forced' retractions resulting from an ethical irregularity. Unfortunately, those retractions rely to a certain extent not just on the honesty of the original authors and the journals, but on the authors' home institution as well. If a typical journal receives a report of an irregularity, the procedure for most is to bring this report to the authors' attention. If not answer to the journal's satisfaction, most journals do not have the resources to conduct an indepth investigation. The original report and the authors' response instead will be forwarded by the editor to the institution. There are large disparities in the extent to which those institutions follow-up. Perhaps the 'transparency index' most appropriately should be targeted towards the institution's responses to enquiries???Take home message. Make sure your index doesn't hinder what is necessary and good. For targeting that which is genuinely bad, make sure you target all of the right contributors if your goal is to improve behavior.