ABSTRACT

BackgroundElectronic prescribing systems with decision support may improve patient safety in ambulatory care by offering drug allergy and drug interaction alerts. However, preliminary studies show that clinicians override most of these alerts.

MethodsWe performed a retrospective analysis of 233 537 medication safety alerts generated by 2872 clinicians in Massachusetts, New Jersey, and Pennsylvania who used a common electronic prescribing system from January 1, 2006, through September 30, 2006. We used multivariate techniques to examine factors associated with alert acceptance.

ResultsA total of 6.6% of electronic prescription attempts generated alerts. Clinicians accepted 9.2% of drug interaction alerts and 23.0% of allergy alerts. High-severity interactions accounted for most alerts (61.6%); clinicians accepted high-severity alerts slightly more often than moderate- or low-severity interaction alerts (10.4%, 7.3%, and 7.1%, respectively; P < .001). Clinicians accepted 2.2% to 43.1% of high-severity interaction alerts, depending on the classes of interacting medications. In multivariable analyses, we found no difference in alert acceptance among clinicians of different specialties (P = .16). Clinicians were less likely to accept a drug interaction alert if the patient had previously received the alerted medication (odds ratio, 0.03; 95% confidence interval, 0.03-0.03).

ConclusionClinicians override most medication alerts, suggesting that current medication safety alerts may be inadequate to protect patient safety.

Adverse drug events (ADEs), defined as injuries due to medications, are common in primary care.1- 4 In a study5 of 4 Boston adult outpatient practices, one-quarter of patients who received a prescription experienced an ADE during the 3 months of follow-up, and more than one-third of these injuries were preventable. Organizations such as the Institute of Medicine and the National Patient Safety Foundation indicated that electronic prescribing is an important tool for improving medication safety, and outpatient clinicians are increasingly adopting its use.6- 8 Although rudimentary electronic prescribing can prevent errors through more legible and structured orders, experts believe that integrated clinical decision support, which can alert clinicians about inappropriate medications or harmful medication combinations, is needed to reduce the number of ADEs in ambulatory care.9- 13

To date, studies10- 12 that have examined outpatient electronic prescribing and clinical decision support alerts have been limited to a few clinicians at teaching hospitals. In 1 study12 of 5 academic primary care practices, clinicians accepted less than 10% of medication alerts, and physician reviewers concluded that more than one-third of all alerts lacked an adequate scientific basis or were not clinically useful. If the threshold for sending alerts to clinicians is set too low, prescribers may develop “alert fatigue,” which can lead them to override important warnings.14

As clinicians increasingly adopt electronic prescribing, the development and dissemination of decision support systems will depend in part on whether clinicians find medication safety alerts valuable. To our knowledge, no studies have examined the behavior of clinicians regarding medication alerts in a large number of community practices or across clinicians of different specialties. This information may provide perspective on the perceived usefulness of electronic alerts in outpatient care and insights into how to design better alert systems, which, in turn, may reduce the number and severity of ADEs. Therefore, we studied the electronic prescribing behavior of clinicians in 3 states to address the following questions: How often do clinicians encounter medication safety alerts? How often do clinicians accept alerts, aborting a potentially dangerous prescription? Do accepted medication alerts vary depending on alert type (allergy vs drug interaction), alert severity, or the classes of interacting medications? Do certain characteristics of patients and clinicians affect the decision to accept an alert? Finally, what can we learn from clinician behavior regarding medication safety alerts that could be used to improve alert quality?

METHODS

STUDY SAMPLE

We studied electronic prescriptions and associated medication safety alerts generated by all clinicians who used a specific electronic prescribing system in Massachusetts, New Jersey, and Pennsylvania from January 1, 2006, through September 30, 2006. We included clinicians who wrote at least 50 prescriptions during the study period. These 2872 clinicians were distributed across 862 practices; 69% of eligible clinicians were from Massachusetts, 20% from New Jersey, and 11% from Pennsylvania. The study was approved in advance by the Dana-Farber Cancer Institute institutional review board.

ELECTRONIC PRESCRIBING SYSTEM

All study participants used PocketScript, an electronic prescribing system (Zix Corporation, Dallas, Texas), which enables clinicians to transmit prescriptions electronically to a pharmacy via a desktop computer or handheld device. The system creates a profile of a patient's active medications based on previous electronic prescriptions written by any member within the practice. The system also references dispensed drug history from the patient's pharmacy benefit management records, which has the added benefit of including information from handwritten prescriptions and prescriptions from other clinicians within the state. A patient's medication allergies must be entered into the system by the clinician or a member of the clinician's office staff. When an electronic prescription is written, the system alerts the clinician if the patient is allergic to the medication or if there is a drug-drug interaction (DDI) with one of the patient's active medications. The system uses a set of alerts maintained by Cerner Multum (Denver, Colorado), a health information technology company. Pharmacists at Cerner Multum use a consensus process to grade the severity of each potential DDI on a 3-tier scale (1 indicating high; 2, moderate; and 3, low) based on information from pharmaceutical companies and evidence in the scientific literature. When a DDI alert is generated, a warning banner with the severity category is displayed to the clinician. The prescriber may view a detailed description of the interaction, although this information does not appear on the initial screen. A prescriber may respond to an allergy or a DDI alert by (1) overriding the alert and continuing to prescribe the medication, (2) canceling the prescription altogether, or (3) rewriting (ie, changing) the prescription for a different medication.

PRESCRIPTION, CLINICIAN, AND PATIENT CHARACTERISTICS

We abstracted the following information from the electronic prescription database: prescription date and time, prescribed generic drug name and class, alert type (allergy or interaction), interacting generic drug name and class, alert severity level, action taken in response to an alert (override, cancel, or change), binary variable indicating whether the same generic drug had been prescribed to the patient previously, and device used to enter the prescription (handheld vs desktop computer). We abstracted deidentified clinician and practice identification numbers, clinician type (physician vs nonphysician), clinician specialty, number of electronic prescriptions written by each clinician during the study period, and length of time the clinician had been using an electronic prescribing system to write prescriptions. We also abstracted patient age, sex, and number of active medications at the beginning of the study period.

PRESCRIPTION ATTEMPTS AND MEDICATION ALERTS

If a patient takes multiple medications, several interaction warnings can arise from a single prescription attempt. Duplicate alerts can also appear for the same prescription if a clinician views an alert, cancels the prescription, and then reattempts to order the same medication.12 To eliminate duplicate alerts and to focus on the clinician's final decision of whether or not to prescribe after generating an alert, we selected the last observation of alerted prescription data sorted by physician, patient, prescribed generic drug name, date, and time.12 This eliminated 235 525 duplicate alerts, leaving 3 570 378 prescription attempts and 233 537 alerts in our final data set.

STATISTICAL ANALYSES

We performed analyses using the prescription writing attempt and the alert as units of analysis. We calculated alert rates for each clinician type and specialty by dividing the number of alerted prescription attempts by total prescription attempts. We examined a clinician's response to an alerted order as a dichotomous variable, accept vs override. Accepted orders included canceled prescription orders or a change of the alerted medication to an alternate drug. We calculated accept rates for each alert type, DDI severity, clinician type, and clinician specialty by dividing the number of accepted alerts by the total number of alerted prescriptions. We used the Fisher exact test to examine differences in alert rates and accept rates in each category. We also examined accept rates for interacting medication classes, limiting our analysis to combinations of drug classes that generated at least 50 alerts to achieve more stable estimates. Some drugs did not have a drug class specified, and in these instances we used the prescribed generic name in lieu of the drug class.

To examine characteristics of accepted alerts, we excluded allergy alerts (<2% of all alerts) because of variability with which practices entered patient allergy information into the electronic prescribing system. For adjusted analysis of DDI alerts, we excluded patients seen by a pediatrician or those younger than 18 since the safety concerns of medications for children differ from those for adults. We fit logistic regression models with generalized estimating equations using the response to the alerted prescription (accept vs override) as the binary outcome. We accounted for clustering nested within the practice at the clinician level and used an exchangeable correlation structure.

The multivariate model included patient age, sex, and number of active medications as independent predictors. We examined for differences in accept rates among patient age groups using a Wald test, and we examined for a linear trend across number of patient medication categories. We also examined the relationship between the decision to accept a DDI alert and clinician type, specialty, and experience with electronic prescribing. We examined number of prescriptions written during the study period and duration of time using electronic prescribing as indicators of electronic prescribing experience. Using Wald tests, we examined for differences in accept rates across clinician types, specialties, and categories of time since becoming an active prescriber and examined for a linear trend across categories of number of prescriptions written. Finally, we examined the relationship between the decision to accept an alert and prescription characteristics, including the type of device used to enter the prescription, severity level of the interaction, and whether the same generic drug had been prescribed to the patient previously. Since classes of prescribed medications are highly related to provider specialty and may therefore act as confounders, we included approximately 200 variables to adjust for the fixed effects of the most common class-class interactions. These variables accounted for more than 70% of all alerted orders. All analyses were conducted using a commercially available software program (SAS, version 9.1; SAS Institute, Cary, North Carolina).

RESULTS

CLINICIAN CHARACTERISTICS

Physicians accounted for 82.0% of the 2872 prescribing clinicians and made 86.7% of all electronic prescription attempts. Most clinicians were in primary care fields, with 21.9% in family medicine, 19.5% in internal medicine, 17.4% in pediatrics, and 3.3% in obstetrics-gynecology. Clinicians wrote 78.9% of electronic prescriptions using a desktop computer and 21.1% using a handheld device. Each clinician wrote a median of 754 electronic prescriptions (interquartile range, ±1423), with a range of 50 to 10 687 prescriptions. Clinicians had a median of 352 (interquartile range, ±426) days of experience with electronic prescribing, with a range of 0 to 1987 days.

ACCEPT RATES OF INTERACTING MEDICATION CLASSES

Among high-severity interactions, accept rates for classes of interacting drug combinations ranged from 2.2% to 43.1%. Table 2 lists the high-severity interactions that clinicians accepted most and least frequently. Among high-severity alerts, clinicians frequently overrode alerts for medications used in combination to treat certain diseases, such as corticosteroids and antirheumatic agents for rheumatoid arthritis. Clinicians commonly accepted alerts that involved combinations of antibiotics and antiarrhythmic medications or antibiotics and warfarin sodium.

Table 4 gives the results of the multivariable model, which examined the factors associated with the clinician's decision to accept 20 180 of 219 517 DDI alerts. We found no difference in DDI acceptance for patients of different ages or sexes (P = .09 and .69, respectively). We did not find a linear trend between alert acceptance and number of active medications (P = .15), although clinicians were approximately half as likely to accept alerts of patients who took 2 or more medications compared with patients who took a single medication. Clinicians had higher odds of accepting high-severity alerts compared with low severity alerts (odds ratio, 1.43; 95% confidence interval, 1.10-1.87). Clinicians had much lower odds of accepting DDI alerts if the patient had previously received the alerted medication (odds ratio, 0.03; 95% confidence interval, 0.03-0.03). In contrast to the unadjusted analyses, we did not find any differences in acceptance of alerts between clinician types (physician vs nonphysician; P = .10) or among clinician specialties (P = .16). Clinicians who wrote more electronic prescriptions were less likely to accept DDI alerts than clinicians who wrote fewer prescriptions (P for trend <.001). However, clinicians who used the electronic prescribing system for a longer period were more likely to accept DDI alerts (P for trend <.001).

COMMENT

In this study of more than 3 million electronic prescriptions, we found that clinicians overrode most high-severity DDI alerts and allergy alerts. Clinicians distinguished among different alerts, accepting some DDI combinations more frequently than others, such as interactions between antiarrhythmic agents and antibiotics. Among high-severity DDI alerts, a 20-fold difference was found between alerts with the highest accept rate and lowest accept rate. In multivariable analyses, we found no difference in response to alerts among clinicians of different specialties.

This study offers several advantages over previous analyses of medication safety alerts in ambulatory care. We examined the behavior of a large group of community-based practitioners in response to medication alerts, and the sample size allowed us to examine responses to alerts for classes of interacting drug combinations and patient and clinician characteristics associated with alert acceptance. We found that clinicians discriminated among alerts based on their severity, the classes of interacting medications, and the patient's previous experience with the medication. Given the high override rate of all alerts, it appears that the utility of electronic medication alerts in outpatient practice is grossly inadequate. For active clinicians, most alerts may be more of a nuisance than an asset.

Our results are consistent with those of previous small studies12,15 of practice sites affiliated with academic medical centers, which have shown that DDI and allergy alerts are commonly overridden. Several observers argue that to reduce the burden of outpatient ADEs, high-quality decision support must be an essential element of electronic prescribing systems.1,9,11,16,17 However, few studies have examined ways to improve alert systems. In a survey of Veterans Health Affairs physicians, clinicians recommended that DDI alerts provide suggestions for alternative medications and that previously overridden medication alerts should not reappear for the same patient.18 Clinicians who use commercial electronic prescribing systems have suggested that extant decision support systems are inadequate.17

In a 2005 national survey, only 14% of outpatient physicians reported using electronic prescribing.8,12 However, several government and private initiatives are under way to promote adoption. The E-prescribing and Medicare Modernization Act of 2003 created standards for electronic prescribing system data and subsidizes third parties to offset implementation costs.19 The National ePrescribing Patient Initiative, a coalition of private technology companies and insurers, offers free software to participating practices.12

Despite these efforts, the only clear evidence that computerized prescribing reduces ADEs is limited to the inpatient setting; only a few studies20- 24 have examined its effectiveness in ambulatory care. Gandhi and colleagues11 found that basic electronic prescribing systems with structured orders and pick lists, but no advanced decision support, reduced the number of prescription writing errors but did not reduce the number of ADEs in 4 adult outpatient practices. Unless designers integrate high-quality decision support into electronic prescribing systems, substantial improvements in outpatient medication safety may not occur.

Our findings provide some insights into ways that designers may improve medication safety alerts. Although most DDI alerts were classified as high severity, clinicians accepted these alerts only slightly more frequently than lower severity alerts. Designers should consider reducing the proportion of alerts classified as high severity by reexamining and potentially reclassifying the alerts that are frequently overridden. By prioritizing a subset of high-severity alerts and only allowing these alerts to interrupt workflow, Shah and colleagues10 improved acceptance of these alerts to 67%. Designers should also provide an option for clinicians to suppress alerts for medications that patients had previously received, since most of these alerts are overridden. Finally, designers should consider customizing alerts depending on the clinician's specialty. For example, since psychiatrists encounter alerts in one-quarter of prescription attempts, they may be at higher risk for overriding important warnings. The usefulness of alerts may vary by clinician specialty, and reducing the number of frivolous alerts for each specialty may help improve alert adherence and enhance care.

Examination of the relationship between clinician experience and alert acceptance yielded mixed results. High-volume prescribers accepted medication alerts less often than lower-volume prescribers, raising concern that high-volume prescribers may become habituated to alerts and neglect potentially useful warnings. In contrast, clinicians who used electronic prescribing systems longer accepted more alerts than novice users. These findings are amenable to several possible interpretations. Experienced electronic prescribing clinicians may know more about alternative medications that can be prescribed after an alert is generated. Alternatively, experienced clinicians may learn to trust certain alerts over time. Further study is necessary to examine these possible relationships.

Our study has several limitations. First, we examined prescriptions written using 1 electronic prescribing system. However, the alerts we examined are typical of those encountered by clinicians, and the drug interaction database from Cerner Multum is used by many commercial electronic prescription systems. Second, our inference about an alert's utility was based on the clinician's decision to accept or override an alert. However, overridden alerts may be useful to clinicians if alerts prompt clinicians to change their behavior (eg, by observing a patient more closely or by ordering additional studies). Third, we assumed that a clinician's decision to override an alert was based on sound clinical judgment, although we were unable to validate this by reviewing the medical record. Previous studies25 have shown some examples of unjustified or dangerous prescribing behavior by clinicians. Finally, we could not examine several patient and clinician characteristics, such as patient comorbidities and clinician familiarity, with the use of electronic medical records.

In conclusion, clinicians' responses to medication safety alerts have much to teach us. Clinicians override most alerts, suggesting that existing alerts may provide little value to practitioners. Unless designers take steps to improve prescription alert systems, the potential benefits of electronic prescribing may not be realized.

Correspondence

The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.
The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with
the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.

Indicate what change(s) you will implement in your practice, if any, based on this CME course.

Your quiz results:

The filled radio buttons indicate your responses. The preferred responses are highlighted

For CME Course:
A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes

Indicate what changes(s) you will implement in your practice, if any, based on this
CME course.

Instructions

Thank you for submitting a comment on this article. It will be reviewed by JAMA Internal Medicine editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.

Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.

* = Required Field

Comment Author(s)* (if multiple authors, separate
names by comma)

Example: John Doe

Affiliation & Institution*

Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.

This feature is provided as a courtesy. By using it you agree that that you are requesting the material solely for personal, non-commercial use, and that it is subject to the AMA's Terms of Use. The information provided in order to email this article will not be shared, sold, traded, exchanged, or rented. Please refer to The JAMA Network's Privacy Policy for additional information.

Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.

What is this ?

Article rental gives users the ability to access the full text of an article and its supplementary content for 24 hours.
Access to the PDF is only available via article purchase.