The Clinical Impact of Published Trials

Ashwini R. Sehgal1,2,3

Objective A key goal of medical journals is to influence clinical practice. However, there are no objective, reproducible, or comprehensive measures of the clinical impact of articles published in medical journals. UpToDate is an online, continuously updated information resource used by more than 1 million clinicians to obtain specific recommendations regarding diagnosis and treatment. Each UpToDate chapter is generally written by 1 to 3 physician authors who are also responsible for selecting articles to cite. A section editor, deputy editor, and peer reviewers provide additional input. We reasoned that citations in UpToDate may provide a useful measure of the clinical impact of published articles. In previous work, we ranked journals based on their citations in UpToDate and compared this type of ranking with journal impact factors. We found little relationship between journal rankings based on UpToDate citations and those based on impact factor, which indicates that these are distinct markers. We then sought to use UpToDate citations to assess the clinical impact of specific trials 2 to 10 years following publication.

Design We selected all 1527 clinical trials published in 3 general medical journals (the New England Journal of Medicine, JAMA, and the Annals of Internal Medicine) and 3 nephrology journals (the Journal of the American Society of Nephrology, Kidney International, and the American Journal of Kidney Diseases) in 2006, 2008, 2010, 2012, and 2014 and then determined how many of these published articles were cited in UpToDate in March 2016.

Results Of all 1527 clinical trials published in the 6 journals, 76% (95% CI, 74%-78%) were cited in UpToDate in March 2016. Trials published in the 3 medical journals were more likely than trials published in the 3 nephrology journals to be cited in UpToDate (85% vs 49%; difference, 36% [95% CI, 31%-42%]). From 2006 to 2014, the proportion of medical journal trials cited in UpToDate showed little variation (range, 84%-89%). From 2006 to 2014, the proportion of nephrology journal trials cited in UpToDate increased from 37% to 57% (difference, 20% [95% CI, 5%-33%]).

Conclusions A large proportion of clinical trials published in these 6 journals may have influenced clinical practice by informing specific recommendations that guide clinicians regarding diagnosis and treatment. The proportion varies by journal type and year of publication. The limitations of this study include the small sample of journals and the lack of a gold standard of clinical impact.