Improving Transparency in Dental Research by Making the Raw Data Available

Fraud and misconduct in medical science research are an unfortunate reality. We have seen several cases in the last few years, giving rise to the perception that such misbehaviour is increasing.1 In one extreme example, Dr. Scott Reuben, a once-prominent researcher in pain management, fabricated data on painkillers and reported these results in more than 20 journal articles.2 Many people were placed at risk of potentially serious side effects, including increased cardiovascular complications.3 Furthermore, Dr. Reuben’s fraudulent findings compromised the results of any secondary research conducted with his data, such as systematic reviews and meta-analyses.4 Clinical decision-making based on the results of these fraudulent studies may very well be flawed.

It would therefore seem logical to make every possible effort to reduce the chances of false or unreliable data being published in the scientific literature. The challenge is that identifying such fraud is not an easy task. The peer-review process can assess the research methods used by an author, but it can’t determine whether the procedures themselves were actually conducted. Yet manuscripts which are peer-reviewed and get published in the scientific literature are generally assumed to be high-quality articles.

Publishing the original data (known as the “raw data”) used by authors in preparing a manuscript could be an effective means of deterring or reducing scientific misconduct. The raw data would allow other research groups and interested readers to reproduce or verify the analyses used in an article.5 Registering the trial protocol in a public clinical trial registry is another way to monitor deviations between what is reported in the trial methodology and the final published paper.

To determine how frequently these 2 measures are used in current editorial policies, I examined whether dental publications suggest or require the publication of raw data and the registration of the clinical study protocol in the public domain. The results of my online review of such policies may generate debate among all parties involved in dental clinical research, including journal editors, researchers, patients and JCDA readers.

Not All Editorial Policies Are Created Equal

I reviewed the guidelines to authors of the 64 dental journals that had an impact factor in the 2009 Journal Citation Reports, published by Thomson Scientific, to determine whether the journals had a policy suggesting or requiring publication of the original data from a randomized controlled trial (RCT). I also examined if the journals suggested or required pre-registration of RCT study protocols in a clinical trial registry. Similar editorial policy information was obtained from the 10 highest ranked general and internal medical journals, along with a random sample of 40 other medical journals, for comparison purposes. I chose to assess these lower ranked medical journals to allow for a more equal comparison between dental and medical publications with relatively similar impact factors.

The publication policies of the 10 highest ranked medical journals are more explicit than the dental journals and lower ranked medical journals. No dental journal suggests or requires the publication of the raw data with manuscript submission (Table 1). However, some dental journals allow the submission of supplementary material, such as data sets or additional figures or tables. Almost one-third of the dental journals (29%) require or suggest the pre-registration of the trial protocol in a public entity.

Although some of the high-ranked medical journals suggest that the raw data be published, most do not make it a requirement. (Table 2). However, most high-ranked medical journals require the pre-registration of the trial protocol in a public entity. None of the lower ranked medical journals suggested or required the publication of the raw data with manuscript submission (Table 3). Similar to the dental journals, less than one-third of these journals (24%) require the pre-registration of the trial protocol in a public entity.

Why Transparency in the Publication Process Is Important

The results of my review demonstrate that the editorial policies of dental journals could be improved. With no policy even suggesting or requiring the publication of the original data, monitoring the reliability of the data by editors and reviewers is a daunting challenge. A recent paper6 suggests measures that could be taken to reduce the chance of scientific misconduct in dentistry. Although this article recommends other interesting measures to improve transparency in the editorial process, such as software to control plagiarism and detect image fakery, the publication of raw data and the registration of the study protocol are not discussed.

Sharing the original data can be beneficial in many ways, most notably by enabling other researchers to replicate the study findings. Replication is one of the cornerstones of experimental studies,7 and the repetition of a study under similar conditions should increase the reliability of the results. Publishing raw data also opens up pivotal parts of the research, like detailed statistical assessments and complete results of the primary research.5,8,9 Such elements allow researchers to thoroughly interpret the findings. Publishing the research protocol could reduce cases of selective outcome reporting (where researchers report only some of the analyzed outcomes) and generate less biased meta-analytic estimates.10 In fact, there is some evidence that trial outcomes can sometimes be inconsistent with protocols.11 This would justify having journals adopt a policy of demanding the research protocol of a clinical trial with the final manuscript.

If other journals adopted some of the editorial procedures of the high-ranked medical journals, transparency in research would improve. For example, for industry-sponsored trials, these medical journals ask that the complete data set (protocol, raw data and paper) be assessed by an independent statistician to check the reliability of the data. This is requested because industry-funded trials are more likely to be associated with statistically significant pro-industry findings.12,13Authors could also be asked to report details of any possible previous or duplicate publication in a covering letter, which would reduce the chance of publishing duplicate data and would optimize the secondary research process in systematic reviews and meta-analyses.

This article provides suggestions for changes to editorial policies that would improve transparency in the sharing of data in dental publications. Having authors publish the study protocol in the public domain would be an improvement of the editorial process. But the dental community should now decide if it is time to go further and start requesting that authors share the raw data from clinical trials. I hope this article will stimulate constructive discussion of this topic.

THE AUTHOR

Dr. Faggion is a research fellow in the department of prosthodontics, University of Heidelberg Dental School, Heidelberg, Germany.

Acknowledgements: The author was partially funded by a postdoctoral fellowship from the medical faculty of the University of Heidelberg.

aManagement of raw data: Does the journal suggest or require submission of the raw data with the manuscript?bResearch protocol in public domain: Does the journal suggest or require pre-registration of the trial in the public domain? It was considered a suggestion when the guideline used the word “encourages,” but did not explicitly demand the registration.cThe journal only publishes papers by invitation (reviews).dMandatory for industry-sponsored trials.

Table 2 Editorial policy of the 10 highest ranked general and internal medicine journals regarding data sharing, as explicitly stated in the guidelines to authors section

Journal

2009 impact factor

Management of raw dataa

Research protocol in public domainb

New England Journal of Medicine

47.050

No

Requirement

Lancetc

30.758

–

Suggestion

Journal of American Medical Associationd

28.899

–

Requirement

Annals of Internal Medicine

16.225

Suggestion

Requirement

British Medical Journal

13.660

Suggestion

Requirement

PLOS Medicine

13.050

Suggestion

Requirement

Annual Review of Medicinee

9.940

–

–

Archives of Internal Medicine

9.813

No

Requirement

Canadian Medical Association Journal

7.271

No

Requirement

Journal of Internal Medicine

5.942

No

No

a Management of raw data: Does the journal suggest or require submission of the raw data with the manuscript?b Research protocol in public domain: Does the journal suggest or require pre-registration of the trial in the public domain? It was considered a suggestion when the guideline used the word “encourages,” but did not explicitly demand the registration.c Authors may be asked to provide the raw data for research papers when they are under review and up to 10 years after publication in the journal.d The journal will publish raw data as a supplementary online file if such information is important for readers wishing to interpret a study or for others to replicate the study.e The journal publishes only reviews.

a Management of raw data: Does the journal suggest or require submission of the raw data with the manuscript?b Research protocol in public domain: Does the journal suggest or require pre-registration of the trial in the public domain? It was considered a suggestion when the guideline used the word “encourages,” but did not explicitly demand the registration.c The journal suggests submitting the study protocol together with the paper.d The journal does not publish clinical trials.e The journal publishes reviews only.f Authors may be asked to provide the raw data.

JCDA.ca Tweets

Essential Dental Knowledge

Disclaimer: CDA Oasis supports clinical decisions; however, it does not provide medical advice, diagnosis or treatment. CDA Oasis is intended to serve as a rapidly accessible, initial clinical reference resource and not as a complete reference resource.

All statements of opinion and supposed fact are published on the authority of the author who submits them and do not necessarily express the views of the Canadian Dental Association.

The editor reserves the right to edit all copy submitted to CDA Oasis.

Publications of an advertisement does not necessarily imply that the Canadian Dental Association agrees with or supports the claims.