Our Projects

Projects

Supplementary Resources

Evaluating Reproducibility and Robustness of Real World Evidence from Large Healthcare Databases (ENCePP registration number: EUPAS19636)

Our objective is to produce empirically based recommendations on how to measure, document and achieve fully reproducible and robust findings from healthcare database studies.

We will start by identifying a random sample of 250 large healthcare database studies published in a leading clinical or epidemiology journal within the last 5 years. For this sample of 250 studies, we will evaluate transparency of reporting based on a catalogue of specific parameters identified by a joint task force between two leading research societies who focus on using databases to evaluate the use and value of drugs (International Society of Pharmacoepidemiology, International Society of Pharmacoeconomics and Outcomes Research). We will then attempt to replicate 150 studies from this sample using the same methods and data sources as reported in the original publication. After replicating these studies, we will evaluate the robustness of evidence in a subset (N = 50) of studies by applying a range of plausible alternative design and algorithm choices and varying assumptions regarding confounding and misclassification.

We will 1) describe how closely studies are replicated, 2) describe areas where insufficient detail in reporting on scientific decisions are most common, 3) identify operational parameters where insufficient reporting has the greatest impact on findings, and 3) explore predictors of insufficient reporting. Recommendations on how to improve transparency and reporting will be developed collaboratively with a Scientific Advisory Board and closely aligned with professional society guidance and journal editor policies.

Vertical Divider

Graphical Depiction of study design to assess the effectiveness of medical products with healthcare databases