Special Edition: The ONC/NIST Workshop on Creating Usable EHRs — Part 2

If you ask clinicians which aspects of their EHRs drive them nuts, many can tell you in some detail. On the other hand, if you ask them how to improve those EHR designs, most cannot articulate the issues in ways that would lead to fundamental change. Relying on focus groups and implementing user requests turn out to be similarly unproductive.

If these methods don’t work, how should one design EHR software that meets the goals and needs of its users and thereby improves healthcare?

There are no simple answers. After all, EHR software is a very new cognitive tool.

An alternative to asking users for advice and feedback is to apply rational design methods collectively referred to as User-Centered Design (UCD). This was the focus of last month’s ONC/NIST Workshop.

Since my last post, I’ve been thinking a lot about the term User-Centered Design because it has two distinct definitions.

On the one hand, it can mean design based on our understanding of how the human brain best takes in, organizes, and processes information — in other words, Human-Centered Design. By this definition, UCD encompasses not just usability testing, but also the findings and methods of a number of related fields, including interaction design, data visualization, cognitive science, and human factors.

On the other hand, the term User-Centered Design can refer to a relatively codified method of software design that places emphasis on setting user performance objectives, conducting iterative user testing during development, and ultimately performing formal summative usability testing to evaluate the end product.

I prefer the first definition because it places more emphasis on the design process itself. A design process that brings together the findings and methods of several fields is more likely to foster innovative solutions. One comprehensive design approach I particularly like is Goal-Directed Design, as described by Alan Cooper, Kim Goodwin, and colleagues in their complementary books About Face 3 and Designing for the Digital Age.

The next question is what role, if any, should ONC play in regard to User-Centered Design and EHR usability. There are two basic philosophies on how to improve EHR design and safety.

One approach is to encourage innovation by allowing market forces, including those created by disruptive innovation, to work. The other approach is to regulate the evaluation process — for instance, to require summative usability testing, to have the FDA regulate EHRs as medical devices, and so forth.

While everyone wants safer EHR designs, in practice it’s not clear to me that more regulation will help. Because of the complex and interactive nature of software user interfaces, evaluating the safety of EHRs is orders of magnitude more difficult than evaluating the safety of physical devices.

An EHR can follow a long list of guidelines, pass all kinds of usability testing, and still present the user with terribly problematic interfaces. After having studied the NIST, AHRQ, and HIMSS documents related to EHR usability, I don’t see how mandating formal usability testing is going to make EHRs safer.

For one thing, one usability guideline inevitably conflicts with another. Furthermore, while summative usability testing is reliable and yields quantitative data, exactly what gets tested is highly subjective. Third, evaluating the safety of EHR software is a moving target, as the software development tools, the design patterns, and the platforms are all changing rapidly.

It is clear that ONC has been considering the role it should play in regard to EHR usability. While we don’t know what ONC’s final rules on User-Centered Design will be, we can glean some information from last month’s workshop.

In their presentations, National Coordinator Farzad Mostashari and ONC’s recently appointed acting Chief Medical Officer, Jacob Reider, made the following points:

The UK model, mandating a particular EHR design, clearly didn’t work.

Getting feedback from clinicians is generally a poor way to improve EHR design. As Henry Ford remarked about his cars, “If I had asked people what they wanted, they would have said faster horses.” The UCD process, broadly defined, is a better way to improve design.

Market forces should work. The more usable EHRs will be the successful ones. Vendors who understand these issues will make User-Centered Design a high priority instead of focusing on new "bells and whistles."

It has taken the aviation industry a hundred years to learn how to build safe planes. Health Information Technology (HIT) is a young industry. Transformation will not occur overnight.

ONC does not see its role as defining how an EHR should look and feel. Rather, its main concern regarding usability is safety.

The tradeoff between innovation and safety is not a "zero-sum game." With more usable designs, everybody wins.

It would appear this same perspective is reflected in ONC’s March 2012 Notice of Proposed Rule Making (pp. 13842-3). First of all, ONC proposes to limit the UCD process to eight certification criteria, all related to the high-risk area of medications. Secondly, the notice states:

… we believe that a significant first step toward improving overall usability is to focus on the process of UCD. While valid and reliable usability measurements exist … we are concerned that it would be inappropriate at this juncture for ONC to seek to measure EHR technology in this way … Presently, we believe it is best to enable EHR technology developers to choose their UCD approach and not to prescribe one or more specific UCD processes that would be required to meet this certification criterion.

Unless innovative designs are allowed to emerge, the next generation of EHR user interfaces will continue to have all the major usability problems of our current ones. From my perspective as a physician EHR user who also thinks and writes about EHR design, I’d say that ONC got its User-Centered Design policy just about right.

Rick Weinhaus MD practices clinical ophthalmology in the Boston area. He trained at Harvard Medical School, The Massachusetts Eye and Ear Infirmary, and the Neuroscience Unit of the Schepens Eye Research Institute. He writes on how to design simple, powerful, elegant user interfaces for electronic health records (EHRs) by applying our understanding of human perception and cognition. He welcomes your comments and thoughts on this post and on EHR usability issues. E-mail Dr. Rick.

HIStalk Featured Sponsors

Currently there are "13 comments" on this Article:

In practice, these are the people that give us CHPL and some of the most unusable FAQ processes I’ve ever seen when it comes to clarifying the regulations.

That, plus, they want to publish new sweeping regulations with detailed requirements and wonder why it would take some one more than 90 days to interpret them, design them, program them, test them, usability test them, deliver them, integration test them, train them and go live on them.

Government gone too far. This is why an administration that started off with so much HOPE is about to experience a whole lotta CHANGE.

Let the doctors and nurses decide the winners and losers. Not ONC Grantees and Government welfare recipients (Altarum, MITRE, etc.).

How do you balance the comment about physician input being a poor guide for future design with Jayne’s suggestion on this site that having doctors as part of your design team is a sign of a good process (paraphrase)?

His two “definitions” of UCD are not really different things. The former focuses on the early stages of the process. The latter the entire process.

All designs are wonderful and innovative – at least in the eyes of the developers – until they meet the reality of testing.

He’s similarly off base when contrasting approaches to encourage usability. It is a red herring to erect innovation or regulation as alternate approaches to this issue when they can (and do) work together in many other markets. Market forces rely on transparency – the ability for prospective purchasers of systems to know ehther they’re usable (and/or safe) in some sort of comparable, rigorous ways.

That transparency does not exist today. NIST’s guidance (which is different from regulation) provides a framework for conducting valid, repeatable comparable usability tests, one piece of the information that the propsective purchasers (or vendors) of EHR systems could use to ensure that their systems are safe and usable before they’re used for patient care.

Shining a light on the shoddy development practices of many EHR vendors isn’t something that either ONC or the EHR industry wants to happen. But it’s an opportunity for those vendors who truly have built UCD – respect for users – into their development and validation processes to differentiate themselves in the marketplace (and hopefully sell more).

The question is: What is the mechanism by which the (probably many) vendors who have no idea of the “use error” risk of their systems (or worse, know about risks but continue to sell / don’t fix problems in implemented systems) get pulled off the market / fixed?

I have a feeling that many CIOs and vendor selection committees are loathe to admit that they should have asked for the same types of usability test results that NIST has specified.

How ironic that decades after EMRs/EHRs were initially implemented, we are now talking about useability and how to design systems that actually support clinicians. The fact is that virtually every major clinical system on the market today was designed to automate manual workflows, not change them–arguably the worst possible approach to getting value out of your IT investments–and because these systems are so complex, expensive and take so long to implement, they have (as a Forbes columnist noted recently) a very long tail. Also, the workflows they automate are pretty much circa 1975-1990. Medicine has changed (and is changing) dramatically, but our systems are changing only incrementally to meet certification, MU and other mandated standards (and who knows what will happen if ACOs really do take off!). As Christansen notes, disruptive innovation requires the formation of new value networks, and fiddling with IT is in some ways marginal to the foundational health care challenges facing this country today. Certainly developing more “useable” systems will move the ball a little further down the field, but it won’t affect the rule book, the size and shape of the field, nor the fact that in the future we may have to play in a different stadium.

Reality Check — As I wrote in my post, I too am concerned that if ONC were to mandate formal usability testing, the law of unintended consequences might come into play.

With regard to MITRE, I am not familiar with the specifics of the particular grant you are referring to, so I can’t comment on its merits. I do know several researchers at the Center for Transforming Health at MITRE. They are a thoughtful, hard-working, dedicated group who do excellent work.

Ross Halfin — EHR designers need a deep understanding of healthcare issues, but the major EHR software design issues are not unique to medicine. While it’s fine to have physicians on the design team, physicians frequently do not fully understand what will actually help their workflows. In my opinion, physicians should not serve as substitutes for trained usability, human factors and interaction design specialists.

rm — I’m basing my second, more specific definition of UCD in large part on the way it’s described in NISTIR 7741 and NISTIR 7742.

NISTIR 7741 states that “usability testing is a core component of user-centered design.” Other authors place more emphasis on the design process itself. For instance, in About Face 3, Alan Cooper writes that “when project constraints force us to choose between ethnographic research and usability testing, we find that time spent on research gives us much more leverage to create a compelling product … we’ve found that spending time on design provides more value to the product design process than testing.”

NISTIR 7741 further states that “usability is not graphic or visual design” and would seem to relegate graphic and visual design to a secondary role of providing a “pleasing aesthetic.” In my opinion, attention to the visual display of information needs to be a cornerstone of any well-designed EHR user interface.

NISTIR 7742, the Customized Common Industry Format Template for EHR Usability Testing, states that its intention is to help vendors demonstrate evidence of usability in their final product. This document is essentially a format for reporting summative usability tests.

I agree that, in theory, innovative designs and regulation do not need to be mutually exclusive. I can envision, however, many scenarios where the specific content of a set of regulations would have the unintended effect of stifling innovative designs.

Lynn Vogel — As you note, current EHRs and other HIT systems certainly have “a very long tail.” I am a little more optimistic than you about the positive effect that can be achieved by improving EHR designs, even if they don’t change the foundational health care challenges you refer to. From my perspective, improved EHR design is the last free lunch in our healthcare system.

Dr. Rick said:
“For instance, in About Face 3, Alan Cooper writes that “when project constraints force us to choose between ethnographic research and usability testing, we find that time spent on research gives us much more leverage to create a compelling product … we’ve found that spending time on design provides more value to the product design process than testing.”

I agree that it makes sense to focus on ethnographic research and design (over usability testing) in the development of most systems.

But EHRs aren’t most systems.

Like airplanes and medical devices, they’re mission critical. People can die when there are usability issues with them.

The current situation is that few vendors spend much effort an ANY portion of UCD. It’s a really good thing if they start understanding who their actual users are and how they work. But it’s not enough. And why the testing phases of UCD are essential.

I’m surprised that you mention focus groups not being particularly helpful is ironing out the poor usability of some EHRs. What past experience led you to this conclusion? I find it interesting that so many are touting physician/clinician input when it comes to designing EHRs for usability, but yet focus groups aren’t considered effective.

Regarding market-driven usability innovation versus alternatives, Clay Shirky was spot on over a decade ago when he wrote:

“There is a dream dreamed by engineers and designers everywhere that they will someday be put in charge, and that their rigorous vision for the world will finally overcome the mediocrity around them once and for all. Resist this idea – the world does not work that way, and the dream of centralized control is only pleasant for the dreamer.”

I still wince every time I read this, since I think of myself as an engineer. And engineering design was one of my favorite courses in school. Nonetheless, I apply Shirky’s market-driven usability ideas to improving EHR usability in “Efficient and Moral Market-driven EMR and EHR Usability Innovation”

Present day EHRs have many usability problems, documented here and elsewhere. However, their single most important usability problem is lack of flexible workflow and lack of competition based on workflow. More specifically, many EHR and HIT systems have “frozen” workflow. Users can’t change it. Therefore they cannot improve it. The workflow is locked up in hardcoded Java, C#, and MUMPS if-then and case statements.

Changing and improving EHR workflow today is expensive and unreliable. It requires expensive programmers who do not understand, cannot understand, and should not be expected to understand what users understand without effort: their work.

Life will be better, for all concerned (physician, patient, programmer) when modern workflow technologies (workflow engines, process definitions, process mining, BPM suites, adaptive case management) finally come to healthcare. Users will be able to improve their workflows without having to confront the programmer bottleneck. So-called “codeless” or “zero-code” programming is not yet a reality, but it is closer to reality in workflow management systems and business process management suites than anywhere else.

The problem with EHRs is workflow. The solution is workflow technology, the research behind this technology, and the decades of experience with this technology in other industries.

rm — The issues you raise are very important. I agree that HIT applications are different because of the ever-present safety concerns. I also agree that many vendors seem to have little in the way of a UCD process in place.

For these reasons, it seems all the more important for EHR developers to devote adequate time and resources to the design process itself. In my opinion, if the high-level EHR design is poorly conceived, usability testing can have limited impact at best.

Mrs. Blackwell — It does seem paradoxical that focus groups are not felt to be a good method for getting input for software design, but this has been the experience of many researchers in the field. This may be due, in part, to the fact that what users think they want and what will actually help can be very different.

In general, ethnographic research and contextual interviews, performed by experienced interaction designers, usability specialists, or systems analysts, turn out to be more productive for the design process.

Chuck Webster — Thanks for your comments and links. I agree that workflows and operational problem solving are of critical importance and are frequently neglected in EHR design. You write about the issues very cogently.

Dr. Rick said: “For these reasons, it seems all the more important for EHR developers to devote adequate time and resources to the design process itself. In my opinion, if the high-level EHR design is poorly conceived, usability testing can have limited impact at best.”

I’m in 100% agreement, Dr. Rick! And summative (validation) testing will be a big waste if the developers haven’t conducted formative testing (and don’t have a good understanding of the users, their tasks and workflows). It all works together and it is all necessary (especially for mission critical systems).

I also agree with Chuck Webster that EHRs should resemble (or be) BPM systems. Unfortunately this would require total re-imagination and development of today’s EHR systems…and the purchasers of EHR systems to take out the workflow-slowing stuff they bought (customized, interfaces, trained everyone on, etc.) and replace it with something new (albeit potentially better in many ways).

HITECH ensured that healthcare organizations (especially health systems) will have the systems that they have now for many years and real platform-blowing-up innovation took a back seat to unimaginative (but on time!) roll-out of MU requirements.

“EHRs should resemble (or be) BPM systems. Unfortunately this would require total re-imagination and development of today’s EHR systems…and the purchasers of EHR systems to take out the workflow-slowing stuff they bought (customized, interfaces, trained everyone on, etc.) and replace it with something new (albeit potentially better in many ways).”

I agree with every word, except “Unfortunately”. “Re-imagination” and “something new” are exactly what the EHR and HIT industry needs.

Fortunately…

1. Business process management (BPM) and adaptive case management (ACM) vendors are eager to partner with healthcare organizations and vendors. Many already do substantial healthcare business, though usually not at the point-of-care (yet).

2. EHR users are restive, increasingly critical of the workflow-challenged systems they feel forced or bribed to use.

3. And some EHR & HIT vendors have more customizable workflows than others. They may not think of themselves as EHR BPM systems (yet), but in effect they are, or at least becoming so.

As a matter of fact, BPM vendors are further along than HIT vendors in use of cloud, mobile, and social technology. As such, cloud, mobile, and social will be important “vectors” for transport of BPM’s process-aware ideas and technologies into healthcare.

I agree, Chuck – I only used “unfortuntely” because what’s being implemented today is far from what I think both of us imagine that is needed / wanted.

I’m less optimistic than you that markedly better systems will supplant what’s being implemented, even with widespread end-user dissatisfaction. Over time, end users will learn to live with the pain (or have others do the typing) and the systems will be used administrivially instead of for all of the things that could actually make a difference in patient care.

Participate

HIStalk has been bringing the healthcare IT industry together since 2003 with reader-contributed material such as interviews, guest articles, news and rumor reports, and Advisory Panel participation. We gratefully solicit your involvement in our efforts to educate and inform. Please see how you can become involved.

Navigate

Sponsor

HIStalk reaches a huge daily audience of provider and vendor executives, technologists, clinicians, consultants, journalists, investment professionals, professors, government officials, and other influencers. Of those, 99 percent say HIStalk influences the industry, 92 percent say it helps them do their job better, and 83 percent say it influences how they perceive companies and products. Interesting in helping both our work and yours? Contact us for an information packet.