Abstract

Background Differences in contexts (eg, policies, healthcare organisation characteristics) may explain variations in the effects of
patient safety practice (PSP) implementations. However, knowledge of which contextual features are important determinants
of PSP effectiveness is limited and consensus is lacking on a taxonomy of which contexts matter.

Methods Iterative, formal discussions were held with a 22-member technical expert panel composed of experts or leaders in patient
safety, healthcare systems, and methods. First, potentially important contextual features were identified, focusing on five
PSPs. Then, two surveys were conducted to determine the context likely to influence PSP implementations.

Results The panel reached a consensus on a taxonomy of four broad domains of contextual features important for PSP implementations:
safety culture, teamwork and leadership involvement; structural organisational characteristics (eg, size, organisational complexity
or financial status); external factors (eg, financial or performance incentives or PSP regulations); and availability of implementation
and management tools (eg, training organisational incentives). Panelists also tended to rate specific patient safety culture,
teamwork and leadership contexts as high priority for assessing their effects on PSP implementations, but tended to rate specific
organisational characteristic contexts as high priority only for use in PSP evaluations. Panelists appeared split on whether
specific external factors and implementation/management tools were important for assessment or only description.

Conclusion This work can guide research commissioners and evaluators on the contextual features of PSP implementations that are important
to report or evaluate. It represents a first step towards developing guidelines on contexts in PSP implementation evaluations.
However, the science of context measurement needs maturing.

Introduction

Many practices have been implemented to improve the safety of patients, such as use of a checklist to prevent blood stream
infections. The effectiveness of these patient safety practices (PSPs) can vary markedly between different settings. Such
variations in effectiveness are likely to be attributable to variations in a range of contextual factors affecting the implementation
of PSPs, such as policies, regulations, or organisational characteristics.1–10 That is, some contexts are likely to affect the effectiveness of PSP implementations. However, as Eccles and colleagues (2009)
noted, despite the effect that context can have on PSP implementation, ‘the role of context in intervention development needs
to be better understood’.1 The lack of understanding in the field of patient safety is especially evident when considering the role that particular
contextual factors have on PSP interventions. Our lack of understanding could in part be due to the complex nature of contexts.
That is, the importance of particular context features might depend on the infancy or maturity of the PSP being implemented,
the stage of the PSP implementation, the specific component when considering multi-component PSPs, the level at which the
PSP is targeted (eg, national-level, hospital-level or unit-level efforts), and the type of PSP (ie, some contexts may be
somewhat PSP specific). Our poor understanding of the effects of contextual factors on PSP interventions also could be because
of the paucity of empirical evaluations of the issue, although contextual factors have been widely examined in the organisational
science and business management fields.1112

For us to better understand the effects of context on implementations, some contexts should be measured and analytically assessed.
However, as Øvretveit and colleagues report, few studies assess the effect of context on PSP implementation and many studies
only describe a few contexts.13 Some exceptions exist. For example, as part of the evaluation. For example, as part of the evaluation of the Agency for Healthcare
Research and Quality's (AHRQ) patient safety contract portfolio, one study found that AHRQ grantees of health information
technology implementations tended to report several contextual factors as contributing to their implementations' success:
commitment from top management, dedicated staff and financial resources, an open process to encourage buy-in and enthusiasm
by end users, and sheer persistence.14 Another study conducted as part of that evaluation reported on several structural components that AHRQ patient safety grantees
considered essential to being in place prior to implementation, such as an institutional commitment and leadership, trust
among participating stakeholders, a supportive organisational culture, and prior experience with performance improvement.15 Although a few additional studies specifically focused on patient safety16 or other types of intervention implementations have examined the effect of context,1718 most do not.

Another potential reason we understand less than we do about the role of contextual factors on PSP implementations is that
many evaluations report only basic information on contextual features. For example, a review of health information technology
studies found that the interventions' contextual features were poorly described.19 Additionally, a recent paper noted that many published reports of clinical interventions in general fail to adequately describe
the context in which interventions are implemented such that there is inadequate information for replication and/or implementation.20 Similarly, an analysis of quality improvement strategies identified a need for more detailed descriptions of both interventions
and contextual features to inform interpretation, replication and local adaptation when choosing potential intervention strategies
for particular circumstances.21 This would allow readers to answer the question, ‘Is this intervention likely to work in our circumstances?’

However, it is difficult to prioritise which contextual features to assess or only describe if there is a poor understanding
of which contextual features are likely to modify the effectiveness of a PSP intervention. Given that such a wide range of
contextual features exist, it is important to avoid calls for the reporting of more and more contextual features to an excessive
level of detail, which risks paralysing researchers and readers alike. Therefore, the challenge is to identify which contextual
features—given our current level of understanding—are most likely to inform understanding and generalisability. Admittedly, some contextual features
are generally thought to be important (eg, having a culture of safety and leadership support) but the patient safety field
is far from in agreement on whether these are specific to safety issues or any organisational change effort in general and
what other types of patient, hospital unit, hospital, or external contextual features are important enough to assess or at
least describe in research papers.

In sum, the role of contextual features in PSP implementations appears important, although there is little evidence or consensus
around which contexts are the most salient for PSP implementations, subsequently hindering the generalisability and reporting
of evaluations. The authors addressed this gap by presenting the conceptual domains of contexts and the specific contextual
features within those domains that experts considered high priority for assessment or description. Experts were asked to distinguish
between assessing the effects of contextual features and only describing contextual features in research papers. If researchers
more richly describe the contexts in which PSP interventions are implemented, it might enhance readers' ability to determine
if the study is likely to work in their own set of circumstances. However, assessing some potentially important contexts might
help move the field forward by building the evidence for the importance of some contextual features to PSP implementations.

Methods

Throughout the study, the authors used an iterative process of formal group discussions with a 22-member technical expert
panel composed of patient safety experts, clinical and healthcare system leaders, and methodological experts.These discussions
were based on the results of targeted literature reviews and two surveys, with the purpose of identifying which contextual
features are likely to have most relevance to and impact upon a diverse range of five PSP's. We used an expert panel, given
that the issue of the importance of context in patient safety is relatively nascent. It was formed as part of a larger study
and was composed of international experts in specific patient safety practices; methodologists from fields including epidemiology
and statistics, program evaluation, organisational behaviour and human factors engineering; senior health system executives
responsible for implementing safety practices; and leaders of national and international patient safety organisations. Throughout
this study, an iterative deliberative process with the panelists was used, involving structured discussions (first by telephone
and then face-to-face) and two surveys.

Selection of PSPs

The authors grounded their efforts to select contextual features using five diverse and representative PSPs. To select the
PSPs, a list of possible PSPs was generated from a variety of sources, including the AHRQ Evidence-based Practice Report Making Health Care Safer22, the National Quality Forum's Safe Practices 2009 update,23 and input from relevant organisations including AHRQ and the Institute for Healthcare Improvement.24 Through an iterative process of developing a framework to classify PSPs based on their diversity and representativeness,
and based on input from the expert panel, five PSPs were selected:25

use of a checklist to prevent blood stream infections. These were representative of PSPs in different settings (eg, hospitals,
clinics and nursing homes), and addressed both PSPs that were subject to regulatory authority and those that were not, and
those that were aimed at preventing very rare and those that targeted more common safety events.

Selection of initial set of contexts and first survey

To guide the panelist discussion and surveys, first, a list was compiled of contextual features reported in seminal PSP papers
and these were grouped into nine broad conceptual domains. The authors recognised that some contexts might affect the implementation
of only some PSPs while other contexts might affect the interventions of most PSPs. Given the long list of potential contexts
that could have been examined, the authors tried to examine those contexts that might be important to many PSPs. (However,
the panelists were specifically asked in the second survey (table 1) which of the five PSPs each context might be important to.) Also, some of the context features could be considered interventions
(eg, the presence of information systems and decision support). When these already exist across both intervention and control
sites in an experimental design, and the intervention being evaluated is independent of or additive to them, such features
represent part of the contextual background to the evaluation. Given the purpose of this study was to examine how context
might affect PSP implementations, the panelists’ discussions and survey responses of contexts were based on the features as
contexts and not part of the intervention.

Contexts judged as high priority for assessing (A) their effects on PSP implementations or only for describing (D) in papers

The authors then discussed the domains and the specific contextual features during the first meeting with the panelists to
help determine on which features panelists should be surveyed. After the meeting, a web survey was used to ask the panelists
to rate the relative importance of the contextual features to each of the five PSPs, from ‘not very important’, ‘important’,
‘extremely important’, to ‘important or extremely important only in some settings’, and to ask them to consider nursing home,
inpatient, outpatient, and community settings. The purpose of the first survey was to guide the development of the second
survey and the context taxonomy (as described below).

Development of the context taxonomy

A taxonomy of important contextual features was developed using the features that were rated in the first survey as ‘important’
across all five PSPs or ‘very important’ across four or five PSPs. Lower scoring contextual features, while acknowledged as
potentially important, were not discussed further. This taxonomy was first proposed by the project team and then discussed
and agreed upon by the panelists as an easy way to conceptualise the various types of contexts that might affect PSP implementations.
The panelist exchanges also resulted in some contextual features being divided into more homogenous groups and other new features
being added and included in the second survey.

Second survey

The authors held a second intensive discussion about contexts with panelists and based on this discussion and the results
of the first survey, a second web survey was designed for panelists to prioritise the list of contextual features. For each
feature, the following question was asked: ‘Is it a high priority to collect data on [context] for describing the context and/or for assessing the effect of context on this PSP implementation?’ Response options were: 1. yes, it is a high priority for
assessing the influence of context on the success of that implementation, 2. yes, it is a high priority only for describing the contexts, and 3. no, it is not a high priority. Both this survey and the first survey had a 95% (n=22) response rate.
The authors conducted a descriptive analysis (frequencies) on the results of the surveys.

Results

Box 1 shows the 42 contextual features mentioned in the seminal PSP papers and how the project team organised them into nine conceptual
domains. One of these domains was labeled ‘cultural organisational factors’, and one of the contexts within that domain was
‘safety culture’. It is worth noting that there are numerous definitions of organisational culture in general,2627 including the informal ‘the way we do things around here’. On safe culture specifically, Pronovost and Sexton28 suggest that, ‘In a safe culture employees are guided by an organisation-wide commitment to safety in which each member upholds
their own safety norms and those of their co-workers.’ When the panelists reviewed this list, they had a lively discussion
on what contexts might affect PSP implementations. Some argued for using theory on which to select contexts because so little
work has been done to determine which contexts are important; others acknowledged that some of the contexts mentioned could
be part of the intervention in some circumstances, but recognised that our discussions would focus on the situations when
those context features were external to the intervention; others mentioned that work has been done in other fields such as
human factors and business management to examine the effect of contexts; and another panelist felt that ‘context’ should include
anything one needs to know to replicate the study. The panelists agreed that our survey should include most of the contexts
on our proposed list and they suggested a few additional contexts.

The results of the first survey showed the panel considered 19 of the 42 contextual features as important or extremely important
across the PSPs. The results of this survey and discussion with the experts were then used to shape the second survey (as
noted below). The results were also used to develop a taxonomy of four conceptual domains of contextual features that are
important for PSP implementations, as shown in box 2. When the panelists were presented with the results of the first survey, they agreed with the results. However, based on
the results, the panelists also thought that the second survey should ask them about a few additional contexts and that the
group of contexts should continue to be diverse in terms of their degree of universality, how tactical, psychosocial or mutable
the contexts are, how much empirical evidence exists for the contexts' importance, and the number of PSPs to which the contexts
related.

Box 2

Results of first survey: four domains of contexts judged as important for patient safety practices (PSPs)

The results of the second panelist survey on the priority of a modified list of contextual features are shown in table 1. Panelists tended to rate the six patient safety culture, teamwork and leadership contexts as high priority for assessing their effects on PSP implementations rather than for only describing these features in evaluations or not a priority at all. One exception to this pattern is noteworthy: panelists tended to
report that patient safety culture at the level of the hospital or clinic organisation is important only for describing in studies (not for assessing their effect). Moreover, in post-survey discussions with the
panelists, they recognised that the items in the patient safety culture/teamwork/leadership domain likely represent distinct
attributes of context.

Panelists tended to rate the 12 structural organisational characteristic contexts (eg, size or location), as high priority
only for describing in PSP evaluations. However, again, two organisational contextual features were noteworthy exceptions: at least a third reported
it was a high priority to assess the effects of existing quality/safety infrastructure and organisational complexity on PSP implementations.

For the five external contexts (eg, regulatory requirement and payments or penalties) and the 11 implementation or management
tools (eg, internal incentives and staff training), panelists appeared somewhat split overall on whether the contexts are
a high priority for assessment or a priority only description in evaluations.

Finally, of all the contexts examined, panelists tended to report that only two are not a high priority at all (not for describing
or assessing): teamwork at the organisational level (for bloodstream infection checklists and for the universal protocol)
and marketplace competition (for bloodstream infection checklists).

Discussion

This paper reports on the use iterative, formal discussions with a 22-member technical expert panel comprised of experts or
leaders in patient safety, healthcare systems, and methods to determine what contexts are likely to have major influences
on PSP implementations and which should be described in PSP evaluations. Given a long list of potentially important contextual
features, the experts were able to reach agreement that four conceptual domains of contexts, and some specific contextual
features within those domains, are important.

This study has several strengths, including the use of a heterogeneous, interdisciplinary panel of experts and the opportunity
to have both face-to-face and telephone meetings with them to iteratively and intensely deliberate the contexts that might
be important for patient safety interventions. Perhaps the biggest strength of this paper is that it aims to help build the
consensus that has been lacking, given the infancy of the field, and creates categories of context that can be the focus of
future research.

The limitations of this study include our potential omission of some contextual features that others consider important for
assessing PSP interventions or for describing in evaluations. Admittedly, the process by which the contexts were selected
and the opinions of the panelists were subjective, such that another set of experts might have derived a different set of
context domains than the ones derived here. It is likely that the panelists' academic background, degree of medical experience,
and other factors affected their decisions. However, the authors intentionally selected a heterogeneous group of panelists
so that their overall opinions would be balanced. In addition, the taxonomy was not validated. However, the efforts presented
in this paper should be considered the first step in a longer-term, iterative process of subsequent research. Of course, there
is as yet scant evidence that the contexts selected actually do influence the effectiveness of patient safety interventions.
However, the purpose of this study was to try and get the field to focus their measurement and reporting on those contexts
believed to be potentially the most important. It is only by getting patient safety evaluators to focus on a common set of
contexts that an evidence base can be built about which contexts really are important—the authors expect a future version
of this process, informed by better evidence, to refine this list of contexts, possibly dropping some and adding new ones.

Several contextual factors within four conceptual domains of contexts were found to be important. Readers familiar with the
evidence-based medicine literature might wonder about the applicability of the ‘PICO’ framework29 for categorising contexts as an alternative. In that framework, study questions are categorised into population, intervention,
comparison group, and outcome groups (PICO). That appears to be a useful framework for categorising some contexts. For example,
the ‘population’ category could include contexts that are characteristics of the organisational target group (eg, culture,
teamwork, leadership) and setting (eg, structural organisational characteristics and external factors). However, the ‘intervention’
category in the PICO framework could include many of the contexts examined here because many could be the focus of an intervention.
Likewise, the ‘comparison’ category in the PICO framework could include almost any context in the group not receiving the
PSP intervention. The ‘outcome’ category might be less applicable to the contexts examined here. As such, the PICO framework
might not lend itself to the efficient categorisation of contexts.

The authors suggest future PSP implementation studies measure, describe, and perhaps assess the effects of the contextual
features that experts reported in this paper as high priority. Assessing some potentially important contexts would help build
the evidence for the importance of some contextual features. If future evaluations more richly described the contexts in which
PSP interventions took place, using some of the contextual features mentioned here, readers might be better able determine
if the study is likely to work in their own set of circumstances. Guidelines such as CONSORT30 and SQUIRE3132 have already been developed for reporting interventions. Perhaps these or other existing guidelines can be modified to include
the reporting of contextual features when PSP implementations are reported. The results presented here could be the first
step towards that effort.

Readers of research papers on PSP implementations need to know about context factors but also about the rigour of study design
to draw conclusions about effectiveness. The technical expert panel and this paper focused on the first of these considerations,
however additional research is needed to address the second. Also, despite consensus on the context domains presented here,
the existing literature suggests the ability to accurately measure the contextual features comprising these domains is immature.
Although the authors encourage researchers to describe or assess several contextual features in evaluations, they recognise
the need for ‘basic research’ in patient safety to advance the science of measuring the contextual features.

. How does context affect interventions to improve patient safety? An assessment of evidence from studies of five patient safety
practices and proposals for research. BMJ Qual Saf2011;Published Online First: 13 April 2011. doi:10.1136/bmjqs.2010.047035.