Measuring Success for Health Care Quality Improvement Interventions

Efforts to formally evaluate quality improvement (QI) programs and understand their successes and failures have been limited by the lack of standard measures of success and the use of subjective or self-reported measures. This article compares self-reported and externally rated measures of QI success to understand how best to measure QI success.

The retrospective evaluation included 30 organizations participating in QI training during 2006-2008 who, subsequently, used their training to carry out QI initiatives within their organizations. The analysis evaluated the Perfecting Patient Care (PPC) University, operated by the Pittsburgh Regional Healthcare Initiative (PRHI). Two measures of self-reported QI success based on survey responses and four externally rated measures of QI success based on organization outcome data were assessed.

Key Findings:

Within organizations, there were low to moderate correlations between self-reported and externally rated measures of QI success.

The weakest correlation between self-reported and externally rated measures was between project successful (self-reported) and sustainable monitoring.

While overall training dosage had a positive, statistically significant association with externally rated measures, there was no statistically significant association with the self-reported measures of QI success.

This article shows that different results may be obtained depending on how QI success is measured and rated. Ideally, evaluations of QI efforts should use externally rated measures, suggests this study.