Abstract

The considerable gap between what we know from research and what is done in clinical practice is well known. Proposed responses
include the Evidence-Based Medicine (EBM) and Clinical Quality Improvement. EBM has focused more on ‘doing the right things’—based
on external research evidence—whereas Quality Improvement (QI) has focused more on ‘doing things right’—based on local processes.
However, these are complementary and in combination direct us how to ‘do the right things right’. This article examines the
differences and similarities in the two approaches and proposes that by integrating the bedside application, the methodological
development and the training of these complementary disciplines both would gain.

Introduction

Those working in healthcare are aware of the considerable gap between what we know from research and what is done in clinical
practice.1 For example, enforced bed rest is ineffective in several conditions where it is still used; exercise reduces mortality in
heart failure but is rarely used; and brief counselling after trauma is fashionable but ineffective. The response to this
well-documented problem includes both Evidence-Based Medicine (EBM)23 and Clinical Quality Improvement.45

The term EBM was coined by Gordon Guyatt in 1992 for the JAMA ‘User Guide’ series to describe the bedside use of research
to improve patient care. At the time it was common at McMaster teaching hospitals for patients' notes to include a key research
paper relevant to their care, and for this to be discussed at ward rounds (personal observation—PG). Improved information
technology allowed Dave Sackett's team to use an ‘evidence cart’ on ward rounds at the John Radcliffe in Oxford; the team
asked and answered two questions for every three patients, with the searches changing 1/3 of clinical decisions.6 Different specialties and different individuals have adopted the principles of EBM to different degrees and in vastly different
ways (Interviews on www.cebm.net/index.aspx?o=4648 illustrate this diversity).

In parallel, the Quality Improvement (QI) movement emerged to address similar problems, but with a focus on recurrent problems
within systems of care. The first methods used in the National Demonstration Projects in the 1980s were adopted from those
introduced by Deming into industry.4 But the methods soon evolved to the different environment of healthcare, with the founding of the Institute for Healthcare
Improvement, and the development of the breakthrough series collaborative, the model for improvement, and other modifications
and extensions of QI methods.

EBM and QI have had overall similar goals but focus on different parts of the problem. EBM has focused more on ‘doing the
right things’: actions informed by the best available evidence from our clinical knowledge base (figure 1), whereas QI has focused more on ‘doing things right’: making sure that intended actions are done thoroughly, efficiently
and reliably; however, these are complementary (figure 1) and in combination direct us how to ‘do the right things right’.7

Before ‘fixing’ an evidence–practice gap, we would be wise to ask ‘is this the right thing to do?’ Is there really a problem?
Is there something effective we can do about it? For example, many QI initiatives in diabetes had aimed to achieve low HbA1c
levels8 for which the evidence was weak, and subsequent large scale randomised trials (ACCORD and ADVANCE) have suggested may be
unhelpful or even harmful.9

The ‘right things right’ process might be illustrated by a familiar clinical example. In the clinic of one of the authors,
the primary care team was considering whether to try to wean elderly patients from long-term benzodiazepines being given for
insomnia. We and the patients seemed reluctant. However, the team reviewed the evidence10 and found the risk of falls on benzodiazepines was higher than we'd expected and other adverse effects, such as cognitive
loss, added to the problems. So cessation was the ‘right thing’, but how best to achieve this? A review of the controlled
trials showed weaning was possible, but that simple (but very slow) tapering was as effective as more intensive methods such
as cognitive therapy counselling. Finally, sending a structured letter to the patient (on why and how to withdraw) had also been shown to be effective.11 Without this evidence review by the clinical team, we might have wasted a lot of effort on ineffective means to achieve our
goal. But without a clinical improvement process to change our practice, this knowledge might not have provoked action. Of
course, QI would also suggest additional questions such as how many patients are on which benzodiazepines and for how long?
How many falls or other adverse events have occurred on the benzodiazepines? What will occur with our patient's anxiety once
we remove the medication?

This article aims to look at what each of these disciplinary areas might learn from the other. The difference in approach
of the two disciplines may be better understood by looking at the problem each perceives it is addressing.

The EBM perspective

One cause of the evidence–practice gap is information overload, for example, approximately 8000 references—including around
350 randomised trials—are added to MEDLINE each week. But only a small fraction of this is research that is sufficiently valid
and relevant to change practice. So keeping up to date with new developments and information is problematic. One arm of EBM
has been to synthesise and summarise this flood of research, and be able to access evidence wherever and whenever it is needed.
To achieve this requires both ready access (to resources such as MEDLINE and the Cochrane Library) and skills (in finding,
appraising and applying evidence) that few healthcare workers currently have. The EBM movement has focused on developing both
the skills and tools to better connect research and clinical practice, with some12 but not universal successes.

A particular focus of EBM has been to take a more sceptical approach to innovation, asking for clear evidence before changing
practice. Given that few innovations represent a real advance, this cautious approach means less disruption arising from unnecessary
changes in practice.13

The QI perspective

The problem addressed by the QI approach might be characterised as the ‘knowing–doing’ gap: we know (individually and collectively)
what to do, but fail to do it or fail to do it correctly. The gap has many causes, from simple lack of (some individuals)
knowledge about what should be done, to uncertainties about the ‘how to’. For example, we may know that certain groups of
patients should receive influenza vaccine but fail to do this because the system does not encourage reliable administration
of the vaccine.

For example, at one institution, the electronic medical record (EMR) was fully implemented for many years, so the physicians-in-training,
the staff physicians and nurses trusted the EMR. On discharge from the hospital a prompt in the EMR enquired whether a patient
needed to receive the influenza vaccine. No one realised that this prompt led to a blind alley—no vaccine was ordered and
no vaccine was given. The individuals (and the EMR) knew that influenza vaccine was the ‘right thing to do’, but the EMR and the culture of trusting the EMR inhibited ‘doing the
right thing’. Fixing this problem proved to be a challenge—requiring several Plan–Do–Study–Act iterations. The simple change
in the EMR system and the influenza vaccine ordering were low on the priority list for the IT support group. This necessitated
that the care team create a work-around and learn how to reliably ‘do the right thing’ in the context and setting of care.14 Even an intervention as simple as administering influenza vaccine in a fully integrated EMR environment required a thorough
knowledge of the system and then creativity in dealing with its limitations—much more than just the knowledge of the ‘right
thing’.

The techniques of QI have focused on integrating what we know and moving it to how we should be doing it. Common techniques
to achieve this are reminder systems (paper or computer), simplification of processes (reducing unnecessary steps), structuring
the system to do the right thing at the right time and place (putting hand-washing equipment and/or instructions in the right
place) and testing new interventions to determine what works in this particular setting. This task is often a creative and
iterative process of identifying barriers, and working out solutions to overcome these.

Marrying EBM and QI

As illustrated by the above examples, EBM is involved in the early stages of checking the validity and applicability of the
available evidence to the clinical problem. This involves the traditional ‘four steps’ of EBM illustrated in figure 2. QI processes15 may be triggered at the fourth step if it seems likely that the clinical problem is a common one for which the current system
of practice is not optimal.16 Similarly, in the planning stage of a QI project there may be several questions that trigger an EBM cycle to check for evidence.

The evidence for EBM and QI

A common criticism levelled at both EBM and QI is that there is only weak evidence that either process makes a difference
to patient outcomes. However, neither EBM nor QI are a single ‘thing’ and cannot be evaluated in the same way as a fixed dose
of a single chemical entity. Rather they are disciplines with multiple techniques that may be used to address the research–practice
gap. For example, we know from large randomised trials that aspirin lowers mortality after myocardial infarction, but is underused;
we would therefore want to improve appropriate usage. EBMers might focus on the ‘appropriate’ part (subgroups, balance of
benefits and harms, etc); QIers might focus on the usage part (barriers, prescribing systems, etc). But success here could
be largely judged by process measures—an increased use in aspirin—rather than in-hospital mortality. The link to mortality
has already been proven in the trials. Hence, rather than enquire, ‘what is the evidence that EBM (or QI) are beneficial’,
we should instead ask what are the best techniques within each discipline for achieving better practice? In order to improve,
we need to know in what circumstances do those techniques work and how we can we disseminate those techniques?

The problems with assessing interventions by process or outcome measurements is illustrated by a recent systematic review17 of QI ‘collaboratives’ that focused on increasing the use of surfactant in premature infants. A collaborative (sometime called
‘breakthrough collaborative’) is a consortium of 20–40 healthcare organisations using QI methods on a specific healthcare
quality problem. The review found nine eligible studies, including only two randomised trials. The reviewers concluded that
the evidence for collaboratives was ‘limited’ because the first trial showed no effect, and the second trial showed ‘significant
improvement in two specific processes of care but no significant improvement in patient outcomes (mortality and pneumothorax)’.
However, their conclusions may ask too much of a trial's ability to detect real changes in outcomes. The improvement in processes
of care included a substantial increase in surfactant use from 18% to 54% (a 36% increase). However, given the pooled trials
of surfactant, which included 1500 randomised infants,18 was barely enough to demonstrate the mortality reduction, expecting to detect a mortality reduction by the collaborative
is unrealistic. With the 36% improvement in surfactant use seen, to reliably detect the predicted mortality reduction, we
would require a trial of collaboratives about nine times larger (9×1500), that is at least 13 500 infants individually randomised.
This may be infeasible and unnecessary. If both steps (surfactant effectiveness and the QI process improvement: see figure 1) separately have strong evidence, then this represents a ‘complete causal chain’ whose evidence is equal to the evidence
of the weakest link.19 The more important question here then is ‘What elements of the neonatal collaboration were important?’ and ‘How well will
that transfer to other settings?’

By shifting the focus to specific methods we may ask more focused and answerable questions. For example, how effective are
reminder systems to reduce ‘slips’? A systematic review of trials of reminders is a valuable resource for QI practitioners
wanting to know when and how they do or don't work?20 Similarly, having to support a team's evidence-based practice, a ‘clinical informaticist’ is intuitively appealing and ‘do-able’,21 but evaluative trials, which show a positive impact of clinical informaticists on clinical decision making, are relatively
recent.22 Some techniques, such as a QI collaborative, may be a complex mix of techniques and therefore intrinsically more difficult
to evaluate. Evaluation is still worthwhile, but may shift focus to understanding what methods a particular collaborative
used, and what seemed to work or not and in what circumstances.

Finally, the epistemology of both disciplines is evolving. A better understanding of the science and scientific rules of both
areas will be important for their continued growth and impact. For example, an early but simplistic interpretation of EBM
was that all interventions required randomised trial evidence. While important, we now recognise that different types of questions
need different types of evidence,23 and that even for treatment questions occasional evidence from non-randomised studies, including some case series, can be
sufficiently compelling.24

A way forward

Early QI methods in healthcare incorporated a link to evidence, but this connection seems to have faded over the years. In
the early 1990s, The Hospital Corporation of America (HCA) developed and used FOCUS-PDCA that explicitly included a detailed
analysis of the evidence for proposed changes, the processes and the data about local performance.25 This methodology developed into the PDSA cycle,15 a common, simple and effective technique, but one where the connection to evidence is less clear. We propose that re-establishing
a clear connection between EBM and QI will benefit both disciplines and, ultimately, benefit patients and families. For those
engaged in either QI or EBM (or hopefully both!) there are several implications, both epistemological and practical, of the
complementary focus and methods of the two disciplines.

Those working in QI teams, before taking on a change should routinely check the validity, applicability and value of the proposed
change and should not simply accept external recommendations. (Corollary: At least some members of a QI team must have high
level skills in EBM.)

Those working in EBM should recognise that it is not sufficient to simply appraise the evidence, but at the end we should
ask ‘what is the next action’16 (and sometimes enter a PDSA cycle) (Corollary: At least some members of an EBM team will need high level skills in QI.)

Those working on the methods of QI and EBM should stop being so concerned about whether the abstract concepts of EBM or QI
‘work’, and instead focus on development and evaluation of specific methods of each that sheds light on what elements are
most effective in what circumstances. This evaluation should involve two related processes. First, recognise that ‘experiential
learning’ is a cyclic process of doing, noticing, questioning, reflecting, exploring concepts and models (evidence), then
doing again—only doing it better the next time (PDSA cycles).26 Second, when new potential generalisable techniques are developed, then these should be subjected to a more formal evaluation.
Recently, several stages of evaluation specific to surgery have been proposed,27 which recognise the development and learning needed before a full evaluation. Related problems have been recognised in applying
the Medical Research Council (MRC) complex interventions framework for health improvement.28 However, some creative tension between doing, developing and evaluating will always exist.

Finally, those teaching the next generation of clinicians should value both disciplines, which should be taught, integrated
and modelled in clinical training.29 Medical curricula, undergraduate and postgraduate, and healthcare organisations should incorporate both EBM and QI training
and these should be taught as an integral whole. Such training requires learning background skills and theory, but also ‘bedside’
teaching and modelling of how EBM and QI are applied in real clinical settings. By integrating the bedside application, the
methodological development, the training and the organisation support of these complementary disciplines, hopefully, we can
ever more frequently do the ‘right things right’.