Ray Moynihan: Let’s stop the burning and the bleeding at Cochrane—there’s too much at stake

Comment SIN-NL
A most important acknowledgment of the fact that medical harms (iatrogenic harm) is underreported and under-investigated in medical studies.
Quote…..”This is particularly true when it comes to harms such as autonomic dysfunction syndromes and other syndromes that are not reported” …..
It is a very sad and wrong development that Peter Gøtzsche, a highly critical scientist and supporter of evidence-based medicine, has been expelled from Cochrane.
As known Sophie Hankes LLM Chair of foundations SIN-NL and IEU-Alliance suffers from autonomic dysfunction caused by experimental neurosurgery on her brainstem, without informed consent, serious medical harm!
Please read statement by Peter Gøtzsche sept. 14 2018: Moral crisis in Cochrane
Please read website www.deadlymedicines.dk

To see the future of Cochrane threatened foreshadows a disaster for all of us

Sometimes it feels necessary to state the bleeding obvious—particularly when there’s blood on the floor. The tens of thousands of people around the world who create Cochrane and its summaries of evidence are contributing to one of humankind’s greatest scientific endeavors.

For 25 years researchers, clinicians, consumers, policy-makers and others have been using the rigorous new tools of an evidence-informed approach, to collectively produce systematic reviews about what works and what doesn’t in healthcare. Notwithstanding limitations and uncertainties, the Cochrane Review has rightly become something of a gold standard in evidence, and whenever a friend or family member has a question about their care—including questions of life or death—it’s the first place I send them.

What’s at stake in the current bloodyfight unfolding within Cochrane’s Governing Board, is not just the credibility of individuals or organisations, it’s the future of reliable trustworthy evidence in a world of increasing falsity and fake news. [1,2] To see this future threatened foreshadows a disaster for all of us.

Unlike almost everything I’ve ever written, this piece is a desperate personal plea to the many clever and cool heads within the extended Cochrane family to try and calm this crisis, heal the rifts, and turn this challenge into an opportunity to enhance public trust, not squander it. No one asked me to write this opinion piece and writing it may mean I lose my role hosting The Recommended Dose podcast—funded by Cochrane Australia and co-published with The BMJ—but I’ve spent too long watching this tree grow to stand by and watch it burn to the ground.

When I first started reporting on healthcare almost 25 years ago, reading Archie Cochrane’s simple book Effectiveness and Efficiency was life-changing. [3] The searing scrutiny of evidence-based medicine was starting to reveal that the benefits of many medical interventions were being routinely exaggerated, and their harms played down. Twenty years ago the television documentary that accompanied my first book, took me to Welsh coal mining districts where Archie cut his epidemiological teeth, to the tiny first Cochrane centre in Oxford, and to McMaster in Canada, where this radical new approach was incubating.

This piece is not about who is right in the current fight, but about reinforcing the rightness of the giant global collaborative project that is Cochrane. My personal views on the dispute over the HPV vaccine review are unimportant and beyond the scope of this opinion piece—suffice to say what appears to be some overstatement of criticism has precipitated what looks like an overreaction. [4] I would respectfully suggest that if there was a pill—or a course of cognitive behavior therapy—that meaningfully reduced the symptoms of hubris and promoted humility, I would recommend key players take a strong dose. Starting yesterday.

More seriously, I would wish every success to those at the Cochrane Colloquium in Edinburgh this week who must be striving through late-night and early-morning meetings to try and heal the wounds as quickly as possible. Potential short-term solutions to the crisis will be complicated and of uncertain benefit—familiar challenges for those dealing with healthcare evidence.

What’s more certain is that the current challenge can serve as an opportunity. The much bigger crisis here is the threat to the reliability of healthcare evidence and public trust posed by the unhealthy financial entanglement between industry and those who evaluate and use its products. [5]

Fifteen years ago, the Cochrane Collaboration was at a crossroads in its relationship with pharmaceutical companies. [6] It opted then to tighten its policy and firmly reject the idea of companies sponsoring Cochrane reviews. Yet Cochrane policy, renewed again in 2014, still allows individuals with financial ties to pharmaceutical companies to review evidence about those same companies’ products—if they constitute a minority of the review team.

Given what we know about the systemic bias introduced into industry-sponsored studies, the egregious nature of much of industry’s marketing behavior, including its work with key opinion leaders, it’s an anathema that conflicted individuals should be reviewing what is often conflicted evidence to start with. Cochrane has an opportunity to provide global leadership by cleaning up this mess—as TheBMJ is attempting to do with its new policy of seeking out non-conflicted researchers to author influential educational material. [7]

Related to this reform should be explicit new ways to address and investigate the under-investigation and under-reporting of harms in medical studies—already flagged by Cochrane leadership in recent correspondence* about the current controversy. [8] And thirdly, as part of a drive to enhance public trust and get closer to the truth about which particular groups of people might benefit from medical interventions, Cochrane reviews might throw a lot more explicit scrutiny on the sometimes controversial definitions of disease on which they rely—highlighting the inappropriately lowered diagnostic thresholds which can drive overdiagnosis and overtreatment.

These words are offered respectfully, not as partisan criticisms to support one side or the other in the current dispute—but rather with a hope for calm, cool, conflict-resolution—from a long-term observer with a strong interest in seeing this global collaboration survive and thrive. Like many of us, I want to see this tree continue to grow and bear fruit for a long-time to come.

Ray Moynihan, senior research fellow, Centre for Research in Evidence Based Practice, Bond University, Australia.

Conflict of interest statement: Ray Moynihan has a contract to present The Recommended Dose podcast, funded by Cochrane Australia. He is also a long-time contributor to BMJ.

deaths in the older vaccinated women, in both relative and absolute terms, within the Abstract of

the review as well as in the main body of the text. We judged it important to present the data

transparently, but also to provide further context to ensure responsible reporting.
The assessment by World Health Organization experts and the

data onthe causes of death provide no clear causal mechanism or link with the vaccine. We judged that readers would find this information useful and that its inclusion was appropriate.

Otherwise the reporting of other harms was, as described in the protocol, limited to the published

peer-reviewed reports from randomized controlled trials. This is not unusual for systematic reviews

from Cochrane or elsewhere.

In relation to harms more generally, we acknowledge that there is a case for including other forms

of evidence. The ‘Discussion’section of theCochrane Review and the accompanying Editorial both

noted the importance of national surveillance programmes to identify and report harms. 2,8

This is particularly true when it comes to harms such as autonomic dysfunction syndromes and other syndromes that are not reported (positively or negatively) in most of the journal-published reports, but about which concerns have been raised subsequently from observational reports.

This underlines the importance of systematic reviews being used in conjunction with the evidence from national surveillance programmes.

Finally, we believe that this Cochrane Review has raised broader questions for Cochrane in relation to reporting harms. We propose to initiate work aimed at providing updated guidance for

author teams on identifying and reporting harms in the current and future data and research

environment, as part of our ongoing implementation of Cochrane’s content strategy.

—————-

Trish Greenhalgh: The Cochrane Collaboration—what crisis?

Last week, the Governing Board of the Cochrane Collaboration voted to expel one of its members, Peter Gøtzsche, for activities which allegedly threatened to bring the organisation into disrepute. Four of the 13 members resigned in solidarity with Professor Gøtzsche, and two appointed trustees (who did not support Gøtzsche) volunteered to step down for administrative reasons (essentially, to allow the reduced Board to continue to function). [1] In a single day, the Collaboration lost over half its Board, four of whom (including Gøtzsche) are leads of national Cochrane Centres. Is there a crisis—and if so, of what nature?

The dispute is still unfolding and involves legal material that is not in the public domain. With that important caveat, the “crisis” is both philosophical (relating to the nature of facts) and political (relating to organisational governance).

Gøtzsche’s three-page statement did not name a specific incident. [2] Rather, it appears that his overall approach to overseeing and commenting on Cochrane reviews was deemed (by some but not all members of the Board) unacceptable.

Gøtzsche might be classified as an evidence-based medicine purist. Described by one publication as “a fearlessly outspoken defender of integrity in medicine,” he is a co-author on the CONSORT, PRISMA, STROBE and SPIRIT statements on how to undertake and publish research. [3] He has argued—controversially—that content experts may not be required on systematic review teams since assessing methodological quality is fundamentally a scientific task. [4] He campaigns passionately against bias in research and against policies that rely on what he views as biased evidence. [5,6]

Why would Gøtzsche be asked to leave the Cochrane Collaboration, of which he was a founder member and whose mission and values appear so closely to reflect his own?

The philosophical explanation is that facts are not self-interpreting; they are theory- and value-laden. [7,8] Even when there are agreed criteria for including or excluding a study or for assigning a particular score to an aspect of the methods, multiple subjective judgments need to be made. [9] What if one trial used a slightly different version of a vaccine, or a new (better) test for a primary end-point, than the one named in the protocol? What counts as a placebo? Because of the need for judgement on such questions, two systematic review teams can produce different findings even when both teams are expert and use identical checklists and statistical methods. [10] This important point is often overlooked by those who view systematic review as an entirely technical process (and for whom there are only “good” or “bad” reviewers). [11] One person’s intellectual rigour is another’s intellectual rigidity. Meticulous application of the Cochrane Handbook is thus likely to generate new kinds of disagreements rather than a single, uncontested truth. [8] Some methodological hard-liners view such disagreements as errors to be corrected rather than as a philosophical inevitability. And this, I think, is the stone in Cochrane’s shoe.

Gøtzsche and colleagues recently published a detailed critique of a newly-published Cochrane review, claiming that the authors had failed to identify numerous studies and misapplied the risk-of-bias tools, resulting in a review that was itself biased. [12,13] This prompted a news report in The BMJ which suggested that the famous Cochrane kite mark had been tarnished, as well as an editorial defending Gøtzsche’s team in BMJ Evidence-Based Medicine, though at least one leading scholar considered their analysis flawed and Cochrane’s Editor-in-Chief ruled that what had been described as “omissions” were actually the result of defensible judgements that took account of clinical, scientific and policy realities.[9,14,15,16]

The political explanation for Cochrane’s crisis relates to the tension between governing an organisation and respecting individual members’ academic freedom to express dissent. The four Board members who resigned have claimed the moral high ground, but if they do not accept the statutory decision-making processes of the organisation they agreed to govern, why did they cast a vote at all? [17]

In the 26 years since its inception, the Cochrane Collaboration has grown from a tiny academic network run on collegiality and small donations to a sprawling trans-national bureaucracy with numerous sub-committees, a thick tome of standard operating procedures and a multi-million dollar annual turnover along with an address book of philanthropists on whose contributions its work depends. 21st-century science is an intersectoral endeavour that necessarily occurs in dialogue with society. [18] Maintaining—and funding—the “view from nowhere” requires delicate navigation of tricky political spaces and sometimes accepting hard-won compromises. [19] Board members are presumably expected not to spit in the soup (especially when using Cochrane letterhead).

At this stage in a fast-unfolding story, I am not convinced that the Cochrane Collaboration is experiencing a crisis of either morality or democracy. Its brand, now as ever, stands for rigour, independence, and a commitment to using science to achieve high-quality patient care and social justice. We should cut it some slack while it gets its house in order.

Trish Greenhalgh is Professor of primary care health sciences at the University of Oxford.

Competing interests: TG works in the same department as Carl Heneghan who is editor of the journal BMJ Evidence Based Medicine. In writing this article, she drew on comments and links posted on social media and circulated via an email list of academics who are members of, or connected to, the Cochrane Collaboration. She has attended the Cochrane Colloquium as a keynote speaker in the past, claiming economy class air fare or second-class train fare, but not receiving a fee. Her work is part-funded by the National Institute for Health Research Oxford Biomedical Research Centre, grant number BRC-1215-20008 to the Oxford University Hospitals NHS Foundation Trust and the University of Oxford.