Devices & Technology

October 15, 2010 - Computerized provider order entry (CPOE) systems are becoming more and more common, but, like all technology, they have their pros and cons. Even if improvements are seen in some areas, clinicians must not come to totally rely on the technology to catch problems. This is evidenced by a recent Leapfrog Group study that found CPOE systems failed to alert prescribers to more than half of the common medical errors entered.

Leah Binder, CEO of The Leapfrog Group, an employer-based health advocacy organization based in Washington, D.C., calls the findings “disturbing.”

“Our report highlights that the lack of testing and monitoring of such systems could potentially jeopardize patients,” she said. “But plenty of solutions are available.”

Leapfrog asked hospitals participating in its annual survey to test their computerized provider order entry (CPOE) systems to determine their ability at catching common medication errors, using a simulation tool. Between June 2008 and January 2010, 214 urban, rural and pediatric hospitals used the tool. Leapfrog gave adult hospitals scenarios with 10 test patients and 50 medication orders that would result in an adverse drug event. Some of the test orders would result in a potentially fatal medication error.

Fifty-two percent of medication orders at adult hospitals did not receive an appropriate warning from the CPOE system, and 42 percent of CPOE systems at pediatric hospitals failed to give the proper alert.

“Of potentially fatal orders, 32.8 percent of those in the adult setting and 33.9 percent among pediatric order sets did not receive appropriate warnings,” Binder said. “Roughly a third would have likely resulted in death of the patient.”

The tests were simulations. No real patients received any medications during the tests.

“This [report] revealed something very interesting, that CPOE if not done correctly can have some unfortunate consequences,” said Rich Temple, executive consultant with the healthcare management consulting firm Beacon Partners in Weymouth, Mass.

“No CPOE system works as well as people would like it to,” said Thomas Payne, M.D., FACP, medical director of information technology services at University of Washington Medicine in Seattle and one of the developers of the simulation tool. “Many systems do not pick up on [certain] things. You shouldn’t assume your population of patients is protected. That’s the idea of testing.”

While the study showed flaws in the systems, they still performed better than paper-based systems, said Graham Hughes, M.D., chief medical officer and vice president of product strategy for GE Healthcare's Enterprise IT Solutions division in Seattle.

“Leapfrog is to be applauded for calling attention to the fact we are not done,” Hughes said. “We think it’s a continual quality improvement process combined with organizational change.”

Hughes said hospitals that have used the GE system for more than a decade are still refining their systems and workflow.

“The Leapfrog report highlights the complexity of CPOE and in particular medication order and fulfillment,” said Edna Boone, senior director of healthcare information systems for Healthcare Information and Management Systems Society (HIMSS) in Chicago. “Those hospitals who scored poorly on the test were given specific information on areas in need of focus and improved patient safety issues by 94 percent when addressing these concerns and re-testing.”

Opportunities to grow

After a six-month interval, 102 hospitals took the test again, after modifying their systems. Nearly all performed better.

“That tells us the test is useful in improving the quality and safety of CPOE systems,” said Binder, adding that Leapfrog gave the hospitals information about where the systems failed to alert. “All of them found mistakes in their systems. All of them were surprised by the mistakes they found. They did not know of these hazards in their systems.”

Temple added that he felt the improvements validated the premise of setting up and testing to develop a more robust system that will deliver as anticipated.

“This study really shows the value of doing your homework on CPOE,” Temple said. “When they made the fixes, based on the early unfortunate results, they had the positive impact that was desired.”

The University of Maryland Medical Center (UMMC) in Baltimore was one of the hospitals that upgraded its CPOE system after taking the initial Leapfrog simulation test. Agnes Ann Feemster, assistant director of pharmacy at the medical center, reported on its program at the Maryland Patient Safety Center 2010 annual conference.

Data from the 2008 survey revealed opportunities to improve UMMC’s electronic medication ordering system, so the medical center developed a multidisciplinary work group, which identified and addressed areas in need of alerts, such as detecting therapeutic duplications, contraindicated routes, drug interactions, and contraindications for therapy or dose limits based on radiology results. After implementing the additional decision-support tools, the system now alerts for about one-quarter of the orders written, most commonly for therapeutic duplication.

“We believe that this team’s efforts resulted in a safer medication ordering system as evidenced by achievement of ‘fully implemented’ on the 2009 and 2010 Leapfrog CPOE tests,” Feemster said. “The purpose of alerts is to create an awareness that the potential exists for patient harm. The majority of the orders that create alerts are indicated and safe based on the clinical situation. Our prescribers act on alerts by changing or discontinuing the original order anywhere between 12 percent and 24 percent of the time depending on the type of alert. We are comfortable with this rate.”

Hughes considers a 25 percent alert rate typical. The GE system and some others allow hospitals to prioritize alerting, for instance, notification of something severe would show up in a big pop-up window and attract more attention than one for a minor interaction that would appear on the dashboard.

“It doesn’t interrupt you but is available as a warning light that there are drug interactions at a low level,” Hughes said. “You can click through if you choose to.”

Hughes reported that hospitals also can change the alerts by setting, so a medication given on a med/surg unit might alert but not in the intensive care unit where patients are monitored more closely. They also can program the system to not continue alerting on the same item if the prescriber has ordered that drug for that patient before.

Temple said the Leapfrog report argues for making sure there is a lot of physician and other key clinical stakeholder involvement in defining which alerts pop up and which don’t, and the overall flow, so prescribers can get through the order in not much more time than scribbling on paper.

Payne cautions that it is important that systems not alert too often, because then providers will simply become overwhelmed with all of the alerts and override the notification. That’s called alert fatigue.

The Leapfrog Group is calling for the federal government to invest not just in technology but also in the testing and monitoring of the systems. Binder recommends hospitals test their systems monthly and whenever a medication error occurs.

“CPOE systems are not plug-and-play,” Binder said. “There is complexity to the adoption process that must be honored and then they must be tested over time.”

Payne agreed about the need for frequent testing, at a minimum every time something new is added. He also suggested that testing should assess what actions clinicians take in response to the alert. Then the hospital should dig deep to understand any reasons for not acting on the information.

Ongoing periodic reviews also should include verifying that the evidence attached to the alerts remains accurate and order sets are working as desired, Temple said.

GE recommends testing the system after every software release or after reconfiguring the system.

Leapfrog also encourages sharing of information about best practices in health information technology (IT) and CPOE implementation.

“This is a competitive environment,” said Leapfrog Chairman David Knowlton, president and CEO of the New Jersey Health Care Quality Institute. “Competition is healthy, but in the case of IT, collaboration is far better.”

HIMSS’ Enterprise Systems Steering committee is developing a CPOE wiki to be released this fall, Boone said, adding that the online tool is intended to assist hospitals and providers in improving care delivery and achieving federal government incentives and will include the stages of implementation, building blocks and stories from the field.

“The HIMSS CPOE wiki will share all lessons learned by hospitals and health systems in order to improve patient safety concerns as CPOE is deployed,” Boone said. “Leapfrog hospitals have been invited to submit best practices and lessons learned.”

Even though the study found room for improvement, The Leapfrog Group encourages hospitals to use CPOE, because the systems can save lives and reduce adverse drug events.

“As medicine grows more complex, it will not be adequate to rely on the individual memories of each and every clinician to ensure a plethora of medication errors are avoided,” Knowlton said. “We will need to rely on advancing technology to support the mission, and we will need to improve on the performance of that technology over time.”

HIMSS also supports greater adoption of CPOE, as does Payne.

“CPOE is very important and should be used more, but it’s not the last chapter in how to automate the system to protect patients,” Payne said. “CPOE is no panacea. It is one of the things we should be doing.”