Differences in Readership Metrics and Media Coverage Among Negative, Positive, and Mixed Studies Published by the New England Journal of Medicine

Objective Negative studies are defined as reports where there is no statistical difference between groups in the primary outcome. These studies may not be published owing to several factors, including hesitation by authors to submit negative studies for publication. When negative studies do proceed to publication, it is unclear how much attention they receive from readers and the media. We analyzed whether there were differences in readership metrics and media coverage among negative, positive, and mixed studies published by the New England Journal of Medicine (NEJM).

Design NEJM.org tracks and displays metrics on readership and media coverage using 3 online analytic sources: Atypon.com, Crossref, and Cision. We retrieved information on page views (number of times the article was accessed online), citations (number of citations by peer-reviewed journals), and media coverage (number of unique media mentions) for all reports of clinical trials published in NEJM between 2012 and 2015. Readership and media coverage metrics were collected from date of publication through January 2017. The papers were labeled as negative, positive, or mixed (discordant coprimary end points) based on results for the study end points. Readership metrics and media coverage means for the 3 groups were assessed by analysis of variance (ANOVA), and adjustments for multiple comparisons were made using the Scheffé method.

Results A total of 338 articles were included in the analysis, of which 73 (22%) were negative studies, 224 (66%) were positive, and 41 (12%) were mixed. There was 100% agreement on classification of articles by 2 authors (S.D. and P.M.). For the 73 negative studies, the article metrics were as follows: page views, 58,728; citations, 93; media coverage, 101. For the 224 positive studies, the article metrics were as follows: page views, 64,364; citations, 88; media coverage, 120. For the 41 mixed studies, the article metrics were as follows: page views, 43,810; citations, 62; media coverage, 65. There were no statistically significant differences across the 3 groups in mean page views, mean citations, and mean media coverage (Table). Following adjustment for multiple comparisons, there were no statistically significant differences in readership or media coverage metrics among the 3 groups. A larger study would be required to assess the generalizability of these findings to other journals and to evaluate other factors that influence postpublication metrics.

Conclusions There was no difference in mean page views, citations, and media coverage among positive trials, negative trials, and trials with mixed outcomes published at NEJM.org.