Reporting on Trials Better, but Still Not Good

Action Points

Explain to interested patients that this study evaluated the effects of guidance on reporting trial results on actual reporting.

Point out that the study analyzed the effect of the 2001 version of the CONSORT statement and that it has been updated recently.

Guidance that outlined the best ways to report randomized trial findings resulted in improvements, but the overall quality of reporting remained inadequate, researchers found.

From 2000 to 2006, there was an increase in the number of trials for which researchers reported details of the primary outcome (risk ratio 1.18, 95% CI 1.04 to 1.33), sample size calculation (RR 1.66, 95% CI 1.40 to 1.95), and the methods of random sequence generation (RR 1.62, 95% CI 1.32 to 1.97) and allocation concealment (RR 1.40, 95% CI 1.11 to 1.76).

"However," Sally Hopewell, DPhil, of the University of Oxford in England, and colleagues reported online in BMJ, "the quality of reporting remains well below an acceptable level."

In an accompanying editorial, Gerd Antes, PhD, director of the German Cochrane Center in Freiburg, agreed, noting that the proportion of trials reporting sample size estimation (27% versus 45%), description of the randomization procedure (21% versus 34%), or description of the concealment of treatment allocation (18% versus 25%) increased, but from a very low level to begin with.

"An optimistic perspective would be to view the glass as half full," he wrote. "A realistic one would have to admit that the glass is more than half empty."

Hopewell and colleagues also found that there was no progress in the percentage of trials that gave specific details on blinding (RR 0.91, 95% CI 0.75 to 1.10).

Their goal was to assess the effect of the 2001 Consolidated Standards of Reporting Trials (CONSORT) statement. The statement, a revision of the original 1996 version, establishes rules authors should follow when reporting trial results in order to enhance the clarity of the findings and better enable readers to assess the validity of the methodology and conclusions.

"Unfortunately attempted assessments frequently fail because authors of many trial reports neglect to provide lucid and complete descriptions of that critical information," according to Kenneth Schulz, PhD, of Family Health International in Research Triangle Park, N.C., and colleagues.

Their comments were made in a paper detailing the 2010 update to the CONSORT guidance, which was published with the Hopewell report in BMJ and simultaneously in The Lancet, Obstetrics & Gynecology, PLoS Medicine, Annals of Internal Medicine, Open Medicine, Journal of Clinical Epidemiology, BMC Medicine, and Trials.

"Poor reporting allows authors, intentionally or inadvertently, to escape scrutiny of any weak aspects of their trials," Schulz and colleagues wrote.

The CONSORT statement has been officially endorsed by more than 400 medical journals and three editorial groups -- the International Committee of Medical Journal Editors, the Council of Science Editors, and the World Association of Medical Editors.

The 2010 update aimed to improve the specificity and clarity of the 2001 version.

"The revision process resulted in evolutionary, not revolutionary, changes to the checklist," the authors wrote.

The new version expands the checklist from 22 to 25 items, adding sections on the requirement to register trials, the inclusion of information about accessing the trial protocol, and the inclusion of funding sources.

In his editorial, Antes said the guidance is clear but that awareness of and adherence to the recommendations are lacking.

In 2006, according to the study by Hopewell and colleagues, only 44% of the included randomized trials were published in journals that endorsed the CONSORT statement.

Antes said the reason there has not been better acceptance of the guidelines is unclear.

"Concerns have been raised that such publication guidelines may be too prescriptive, impede the creativity of research, and lack authorization of the self-selected author groups who write them," he wrote. "However, these concerns are clearly outweighed by the harm caused by poor reporting."

A fear on the part of editors about an increased workload needed to implement the changes could also contribute, he said.

"However, much of the workload could be shifted to authors," he said, adding that using the guidance could eventually streamline the peer review process.

"This would establish a solid communication structure between authors, reviewers, and editors, but this approach has not been implemented on a large scale so far."

Hopewell and her colleagues acknowledged that their study was limited by the use of single data extraction and the use of different teams of researchers and slightly different sampling techniques to extract 2000 and 2006 data.

In addition, "it is important to recognize that poor reporting does not necessarily mean poor methods," they wrote.

The CONSORT group received financial support from the United Kingdom National Institute for Health Research and the Medical Research Council, Canadian Institutes of Health Research (CIHR), Presidents Fund of the CIHR, Johnson & Johnson, BMJ, and the American Society of Clinical Oncology (ASCO).

Members of the writing group reported relationships with Johnson & Johnson, BMJ, ASCO, the EQUATOR Network, American Board of Obstetrics and Gynecology Foundation for Excellence in Women's Health Care, Ortho-McNeil Janssen Scientific Affairs, American College of Obstetricians and Gynecologists, and Wyeth.

The study by Hopewell and colleagues was carried out as part of a larger study funded by a grant from the U.K. National Institute for Health Research to support the work of the CONSORT group.

Accessibility Statement

At MedPage Today, we are committed to ensuring that individuals with disabilities can access all of the content offered by MedPage Today through our website and other properties. If you are having trouble accessing www.medpagetoday.com, MedPageToday's mobile apps, please email legal@ziffdavis.com for assistance. Please put "ADA Inquiry" in the subject line of your email.