Drivers of effectiveness: a case study in diabetes

Context

In practice, an efficacy-effectiveness gap (for a definition, see here) is likely to occur if a key characteristic (such as a patient-related or disease-related characteristic) is distributed differently in randomised controlled trials (RCTs) compared with real life.

No efficacy effectiveness gap or drivers of effectiveness were identified: no clear differences in effect estimates across study designs were observed. Therefore, no evidence of an efficacy-effectiveness gap was identified. Also, no clear differences in patient characteristics across study designs were observed, meaning no potential drivers of effectiveness were identified.

However, based on the available studies, it is not possible to fully rule out the existence of an efficacy-effectiveness gap.

What are the limitations?

There are two possible reasons why an efficacy-effectiveness gap was not detected by the literature review:

The observational studies may have been designed to mimic the RCTs. However this was not explicitly stated nor could it be deduced from inclusion criteria.

Quality of the observational studies was poor (for example, no control for confounding) so the study effects may not reflect the actual effectiveness of the medicines.

The drivers of effectiveness investigated were limited to information available in both RCTs and observational studies (age, sex, BMI, time since diagnosis, baseline HbA1c). Therefore, other potential drivers of effectiveness (for example exercise and diet, comorbidity, co-medication, delivery of and adherence to treatment) could not be investigated.

Future studies based on patient-level data could investigate other potential drivers of effectiveness based on analyses of effect-modification.

The literature review compared relative effect estimates across study designs, by identifying studies where glucose-lowering medicines were compared with each other. An alternative is to compare the absolute effect of a specific medicine across study design. This would include studies not identified in the present review (e.g. the active treatment arm of a placebo RCT, and the relevant treatment arm of studies with a comparator not in the scope of this review).

Funding

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115546], resources of which are composed of financial contribution from the European Union's Seventh Framework Programme (FP7/2007-2013) and EFPIA companies in kind contribution.