Study highlights 'lurking question' of measuring EHR effectiveness

Hospitals' use of electronic health records has had just a limited effect on improving the quality of medical care nationwide, according to a study by the nonprofit RAND Corporation.

The study, published online by the American Journal of Managed Care, is part of a growing body of evidence suggesting that new methods should be developed to measure the impact of health information technology on the quality of hospital care.

Most of the current knowledge about the relationship between health IT and quality comes from a few hospitals that may not be representative, such as large teaching hospitals or medical centers that were among the first to adopt electronic health records.

The RAND study is one of the first to look at a broad set of hospitals to examine the impact that adopting electronic health records has had on the quality of care.

The research included 2,021 hospitals – about half the non-federal acute care hospitals nationally. Researchers determined whether each hospital had EHRs and then examined performance across 17 measures of quality for three common illnesses – heart failure, heart attack and pneumonia. The period studied spanned from 2003 to 2007.

The number of hospitals using either a basic or advanced electronic health records rose sharply during the period, from 24 percent in 2003 to nearly 38 percent in 2006.

Researchers found that the quality of care provided for the three illnesses generally improved among all types of hospitals studied from 2004 to 2007. The largest increase in quality was seen among patients treated for heart failure at hospitals that maintained basic electronic health records throughout the study period.

However, quality scores improved no faster at hospitals that had newly adopted a basic electronic health record than in hospitals that did not adopt the technology. In addition, at hospitals with newly adopted advanced electronic health records, quality scores for heart attack and heart failure improved significantly less than at hospitals that did not have electronic health records. EHRs had no impact on the quality of care for patients treated for pneumonia.

Researchers say the mixed results may be attributable to the complex nature of healthcare. Focusing attention on adopting EHRs may divert staff from focusing on other quality improvement efforts. In addition, performance on existing hospital quality measures may be reaching a ceiling where further improvements in quality are unlikely.

"The lurking question has been whether we are examining the right measures to truly test the effectiveness of health information technology," said Spencer S. Jones, the study's lead author and an information scientist at RAND. "Our existing tools are probably not the ones we need going forward to adequately track the nation's investment in health information technology."

New performance measures that focus on areas where EHRs are expected to improve care should be developed and tested, according to researchers. For example, EHRs are expected to lower the risk of adverse drug interactions, but existing quality measures do not examine the issue.

"With the federal government making such a large investment in this technology, we need to develop a new set of quality measures that can be used to establish the impact of electronic health records on quality," Jones said.

Support for the study was provided by RAND COMPARE (Comprehensive Assessment of Reform Efforts). RAND developed COMPARE to provide objective facts and analysis to inform the dialogue about health policy options. COMPARE is funded by a consortium of individuals, corporations, corporate foundations, private foundations and health system stakeholders.

Other authors of the study are John L. Adams, Eric C. Schneider, Jeanne S. Ringel and Elizabeth A. McGlynn.