Is Big Data Eclipsing the Role of Randomized Controlled Trials?

Boston, MA—May 24, 2017—The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) hosted its third and final plenary session this morning focusing on the usefulness of big data in health care policy decisions. In the final plenary session at ISPOR’s 22nd Annual International Meeting in Boston, MA, USA, renowned panelists debated the reliance on randomized controlled trials when big data can generate evidence in observational study designs to support health care decision making.

While randomized controlled trials are considered the “gold standard” in research, big data is changing the strength of evidence in observational studies and is influencing the types of studies where randomization is necessary. The panel of leading researchers and policy experts shared their views on the usefulness of big data for health care policy decisions and discussed the many challenges remaining in the evidentiary requirements for health care decision making.

William H. Crown, PhD set the context for the discussion by providing an overview of the topic and noting that more than 80% of biomedical data are unstructured (e.g., genomics, curated medical literature, health care claims, medical device feeds, electronic medical records, etc.). Dr. Crown pointed out that the proposed new evidence-based medicine pyramid now reflects “wavy lines” between the study design segments, indicating fuzzy boundaries between the different levels of evidence. Recognizing the increasing interest in the use of real-world evidence for health care decision making, Dr. Crown indicated that the 21 Century Cures Act section 3022 now requires the US Food and Drug Administration to develop a framework and guidance for evaluating real-world evidence in the context of drug regulation.

Robert Califf, MD presented a very different view of the use of data in health care. He believes that health care needs to look to companies such as Google, Apple, and WalMart to see how they leverage data. Although they’re operating outside of the health care industry, these companies represent what a learning health care system should be—a system where research influences practice and practice influences research. Dr. Califf pointed out that the United States currently has a highly disaggregated health care system that does not allow for this type of approach. However, when we consider the evidentiary needs that will shape health care decisions, Dr. Califf stressed that our approach to future research should not be viewed as an “either / or” option (i.e., real-world evidence OR randomized controlled trials), as they are not polar opposites.

Steven Goodman, MD, MHS, PhD asserted that analytic methods need to “catch up” with observational studies and big data. Dr. Goodman is a proponent of transparency. He thought that asking, “Which is better: observational studies or randomized controlled trials?” is like asking, “Is it better to use a knife or a hammer?” The answer depends on what problem is being solved. Dr. Goodman believes that big data is rendering the “observational versus randomized” question quaint. He noted that while we often speak of moving toward a learning health care system, we also need to consider developing a learning research system.

Sebastian Schneeweiss, MD, ScD posed the question, “How confident are we that the study we plan will ‘get it right?’” His remarks addressed the need to develop guidelines to help increase our confidence in the study design and methods to avoid the potential mistakes and pitfalls in research, (e.g., conducting studies with known design flaws). Dr. Schneeweiss echoed a theme many of the speakers, noting that transparency is the bedrock of good research.

Additional information on the ISPOR 22nd Annual International Meeting can be found here. Released presentations from the conference can be found here. Interested parties can follow news at ISPOR’s press site and on social media using the conference hashtag #ISPORBoston.