Triggers of potential safety risks were reported in the anaesthesia literature 20 years ago.1 Trigger tools are sets of easily identified flags, occurrences or prompts that alert reviewers to situations where harm is thought to be more likely than in routine care.2

Where there are electronic health records, applying both prospective and retrospective computer search algorithms for various triggers has been proposed as a method of identifying error and adverse events, especially in hospitals.3 Such searches provide a reasonably unbiased, systematic method of reviewing patient records to alert doctors and nurses to potentially risky situations and to provide measures of safety improvement as harm avoidance measures are implemented.

The usefulness of identifying harm is that processes and systems within practices that may lead or contribute to harm, can be analysed and changed, if we knew what they were. To be effective in this role, triggers should be sensitive (i.e. identify all occasions of the trigger event occurring) and specific (i.e. not identify situations that seldom result in harm to patients). There are some reports of proposed triggers having sensitivity and specificity problems.4 This makes their use inefficient as on each occasion a trigger occurs, a manual review must be done to assess whether harm has occurred, and (if it has) its type and severity.

If the potential for harm associated with a trigger is seldom realised and the trigger identifies a common situation, the labour associated with reviewing “triggered” cases may be a cost that overwhelms possible benefits. Reports of trigger tools being tested in UK primary care practices show that it is possible to review up to 20 records in a 2–3 hour session, and that 8–12 triggers may provide optimal balance between sensitivity, specificity, and feasibility for using as a routine safety improvement tool.5–7

Despite reports of the development of primary care trigger tools, little is yet known about the practicalities of using them in practice and in New Zealand there are no reports of their uptake. We could find no research showing the role of trigger tools in documenting the underlying harm arising from care provided in general practice settings. As a result it has been difficult to extrapolate these trigger tools to our clinical context, understand the proportion of harm that might be identified if we used one of the existing trigger tools, and inform our decisions about making our primary care safer for patients.

Because of the potential importance of triggers in protecting patient safety, we decided to test their use in a large general practice (>12,000 enrolled patients) situated in provincial New Zealand. The practice’s patients are mainly New Zealand European but Māori comprise 18% of its enrolled population. Its catchment includes both urban and rural areas.

We aimed to establish what trigger tool worked for us, which triggers were most useful, and whether we could derive a process that would be practical for us to use routinely.

Methods

Possible triggers were identified from reviewing the literature of triggers tested in primary care and a focus group of two general practitioners, two pharmacists and one practice nurse decided on the 36 triggers for initial use (Table 1). The focus group was facilitated by the local Primary Healthcare Organisation’s (PHO’s) quality improvement leader. In New Zealand, PHOs are responsible for the funding, quality improvement and clinical governance of primary care.

We calculated that we needed to review the records of 170 patients, based on an assumption that the background harm rate in primary care is 5% and with 90% power to detect harm. To be included in the review, patients had to be registered with the practice for ≥12 months and have at least one visit with a general practitioner in 2011. We decided to include all ages in the cohort (other studies of primary care trigger tools had excluded children) and that 50% of reviews would be of Māori patients’ records. Records were reviewed from patients randomly selected from the practice’s January 2011 patient register.

The trigger tool was applied by two teams of reviewers. One team consisted of a general practitioner and a community pharmacist and the other team was a general practitioner and a practice nurse. The teams separately reviewed each patient record for the presence of a trigger. If one was present, indication of harm relating to that trigger was then sought.

Each record was then reviewed for the presence of any harm that was not related to the trigger. Harm was defined according to the Medication Error Index adopted by the National Coordinating Council for Medication Error Reporting and Prevention.8

Harm was classified according to the WHO National Coordinating Council for Medication Error Reporting.8 Following each session a reconciliation of findings between teams ensured consistency of interpretation of triggers and harm. If there was a difference between the two teams then a decision was made based on consensus.

The analytic plan was first to measure the harm events associated with each trigger and calculate the sensitivity and specificity of each trigger. We then carried out logistic regression analyses, adjusting for sex, ethnicity and age to estimate the odds of harm associated with each trigger and with the 36 triggers combined.

Using a consensus approach between members of the research team, triggers with the lowest specificity were then excluded and a refined trigger tool derived and tested for its ability to identify harm, using a further age-sex-ethnicity-adjusted logistic regression analysis.

The study was reviewed and approved by the Northern X Ethics Committee (NTX/11/EXP/298).

Results

The records of 170 patients were analysed for both the presence of a defined trigger and the presence of harm – see Table 2 for demographics and Figure 1 for a flow chart of the analysis process and results. Thirteen patients had no trigger in their records.

Table 2. Demography of patients whose records were reviewed

Variables

Male

Female

Total

Age (years)

<18

18–65

≥65

24

37

17

17

55

20

41

92

37

Māori

44

41

85

Non–Māori

34

51

85

Total

78

92

170

A total of 1033 triggers were identified over a total of 40,030 days of follow-up in which 637 consultations were recorded. In these consultations, 44 harms were picked up by 62 triggers and 1 harm was not picked up by any triggers. All harms identified were medication related.

Figure 1. Flowchart of analysis and results

Table 3 lists triggers associated with harm. The rate of harm per consultation was 0.07 (95%CI 0.05–0.09) or 7 occurrences of harm per 100 consultations. The rate of harm per 100 patient years was 41 (95%CI 29–55).

Of the 45 occurrences of harm:

34 (76%) were classified as Category E – temporary harm to the patient and required intervention;

8 (18%) were classified as Category F – temporary harm to the patient and required initial or prolonged hospitalisation;

2 (4%) were classified as Category G – permanent patient harm; and

1 (2%) were classified as Category I – patient death.

The odds ratio of harm occurring using 36 triggers was 0.78 (95%CI 0.5–30) with a sensitivity of 0.98 and a specificity of 0.08.

The refined primary care trigger tool included only 8 triggers: adverse drug reaction documented in the record, ≥2 consultations with a GP in the same practice in a week, cessation of medication, reduction in medication dose, ≥6 medications prescribed, attending the emergency department or an after hours provider within 2 weeks of having seen a GP, eGFR <35, and death.

The odds ratio of harm occurring if one of the reduced set of triggers was present was 3.4 (95% confidence interval 1.7–7.1) when adjusted for age, sex and ethnicity. The sensitivity of this refined trigger tool was 0.81 and the specificity was 0.51. The odds ratio for harm occurring among male patients was 0.59 (0.32–1.10) and for Māori was 0.96 (0.48–1.93). The correlation coefficient for the refined primary care trigger tool, was 0.4 between the two groups of reviewers.

Table 3. Number of consultations with a trigger and number (percentage) associated with harm)

Trigger

Number of consultations with triggers

Number (%) of triggers associated with harm

Adverse reaction

18

15 (83.3)

≥2 consultations in a week

27

2 (7.4)

Telephone prescription for new medication and no appointment

40

1 (2.5)

Cessation of medication

45

19 (42.2)

≥6 medications prescribed

38

1 (2.6)

Change of medication

25

6 (24.0)

Reduction in medication dose

17

6 (35.3)

Hospital discharge

67

4 (6.0)

Accident and medical clinic or emergency department after GP consultation within 2 weeks

18

2 (11.1)

Attended outpatient clinic

266

5 (1.9)

Estimated glomerular filtration rate <35 mL/min/1.73m2

5

2 (40.0)

Death

1

1 (100.0)

Discussion

In this study we showed that 27.1% of the study sample of 170 patients experienced at least one of the 36 triggers we identified from the literature, within the time their electronic records were held by the study general practice. The only other study we could find using a random sample of patients found a slightly smaller proportion (21.1%) experiencing some sort of safety incident (not necessarily associated with harm).9

The per consultation rate of harm we found (0.07 per consultation) is comparable to other reported rates of harm of 0.1 per consultation. The main type of harm in this cohort was adverse events from medications which are often an expected occurrence. Most harm was minor and temporary.

The refined primary care trigger tool we developed is a compromise between reaching high sensitivity and making the tool practical to use in primary care by limiting the triggers to those that have high specificity. The final list of 8 triggers balances practical considerations (not being too arduous to use when reviewing patient records) and providing some assurance that most harm will be identified. It is possible that a different practice population may have a different set if triggers and further work is needed to confirm the validity of the 8 triggers we finally arrived at.

There was relatively low correlation between decisions made by the two sets of reviewers. This can be explained in a number of ways. First, there was no training on reviewing the record for triggers. This is mainly because trigger tool use in general practice is a novel concept in New Zealand and there has been no previous work to enable training. Essentially the training occurred “on-the-job”. Second, during the process of developing consensus between the two groups, it became apparent that the triggers lacked a tight definition. This resulted in each group having a different concept of what was a trigger and what was not. As a result, different triggers were identified. Thirdly, the makeup of the two groups of reviewers differed. The group that included a pharmacist picked up more triggers relating to medication (adverse reaction, cessation of medications, change of medication and reduction of medication).

All of these factors resulted in the groups identifying different patient records with triggers. To improve validity we recommend that triggers are well defined, that training occurs for reviewers (by attending workshops run by quality and safety organisations such as the New Zealand Health, Quality and Safety Commission) and that consideration is given to composition of the review team.

Previous papers, on primary care trigger tools, have used similar methods with the exception of looking for the occurrence of harm when a trigger is absent, as was done in this study.5–7 Although only one harm was identified that was not associated with a trigger the actual harm may be higher as the study protocol excluded more subjective harm that might have arisen from delayed diagnosis. In addition harm rates might be under-represented in the number of patients selected as other papers have had greater numbers of patients reviewed.5–7 Further research would therefore be required on a larger population.

This study was designed to inform the researchers about measurable harm relating to triggers that have already been proposed by international researchers. However, in the process of examining the randomly selected electronic records, we also identified errors in the process of care that probably also resulted in patient harm, undocumented in the records. These errors included problems with telephone prescriptions that obviously resulted in financial and time costs to patients but were due to the practice’s internal systems, poor continuity of care as patients moved through different care settings, and failure to document received care in the appropriate place in the record. The one death in this study was due to an inadvertent failure to continue a medication that had been initiated in hospital.

In summary the final 8-trigger trigger tool shows promise as a practical mechanism to identify harm in general practice although the time this review takes means that only a small subset of the patient records for each practice can be reviewed. Our sample provided generalisable information for our practice but the relatively small sample size combined with the low correlation between reviewers, means that inter-practice comparison of harm would be invalid.

The primary care trigger tool offers an opportunity for pharmacists, nurses and other primary care providers to work collaboratively with general practitioners and could initiate further work on medication reconciliation and better defining the roles of different health professionals working in general practice.

Further study is required on the primary care trigger tool to assess the generalisability across other practices and to determine what quality improvement initiatives occur within practices as a result of using this tool.

Summary

This paper looked at how much harm occurs in general practice and developed a tool, called a ‘Trigger Tool’, which could assist primary care in measuring harm. According to the findings harm occurs in about 7% of consultations, although it possibly could be higher, and is predominately due to side effects of medication. The tool that was developed identifies warnings events (triggers) that general practice teams can investigate to see if harm has occurred. The tool is very good at picking up the majority of harm that occurs in general practice and not as good at excluding events in which harm did not occur.

Abstract

Aim

Using triggers to identify adverse events is proposed as an efficient means of consistently measuring, and tracking events that result in harm to patients. We aimed to test whether using triggers in our large provincial general practice could provide meaningful directions for improving safety.

Method

A literature review identified potential triggers and established the number of patients whose records we should review. Two teams independently reviewed 170 randomly selected patients’ records for trigger presence and for evidence of harm relating to that trigger. All triggers were tested for sensitivity and specificity: triggers with low specificity were removed. Logistic regression was used on both initial and refined trigger sets to measure the odds ratio (OR) of harm occurring if a trigger was present.

Results

Initially 36 triggers were identified. Applying these to 109.6 patient-years of records for 170 patients, we identified harm in the records of 46 (27.1%) patients. There were 7 occurrences of harm per 100 consultations (harm rate per consultation=0.07 (95% confidence interval [CI] 0.05–0.09) and 41 per 100 consulting patient years (95%CI 29–55). All harms related to medication use. The initial triggers were sensitive (0.98) but non-specific (0.08): removing triggers with low specificity left only 8. The OR for harm occurring using the initial triggers was 4.0 (95% 0.5-30) and using the refined trigger set OR=6.3 (95%CI 2.7–14.8).

Conclusion

8 selected triggers are a useful way of measuring progress towards safer care for patients in primary care practice.

Author Information

Kyle S Eggleton, Senior Lecturer, Department of General Practice and Primary Health Care, University of Auckland, Auckland; Susan M Dovey, Professor, Department of General Practice and Rural Health, University of Otago, Dunedin