I. Introduction

A. Why CPOE Now, After All
These Years?

"What is your CPOE strategy?" Over several months of
posing this question to hospital information technology executives, we
have yet to run into one who did not have at least the beginnings of a
plan. In recent meetings we have attended, the Healthcare Information
and Management Systems Society summer meeting in Las Vegas in June
2002, heavily focused on computerized physician order entry (CPOE),
and the Association of Medical Directors of Information Systems
meeting in Ojai, Calif., in July 2002, focused almost entirely focused
on CPOE.

Physicians are acutely aware of CPOE, and, as might be expected, are
deeply divided in their views. While researching this study and
writing the report, we’ve asked virtually every physician we’ve
encountered in both our business and personal settings what they
thought of CPOE. A clinical professor we visited at UCLA said she’d
be thrilled to have a CPOE system, citing ease in entering complex
orders and having them available and universally understood. On the
other hand, a highly respected community-based specialist expressed
the view that CPOE is a management tool to control physicians’
decisions (a topic discussed in this report). Also, as reported in
Inside Healthcare Computing (IHC, 10/28/02), we ran into a physician
who seems to dislike computers so much that he refused to accept one
of the authors of this report as a patient because the only available
formats for the relevant diagnostic images were digital–as a CD
or as files on the computer hard drive of a diagnostic center 50+
miles away from his office. This M..D. demanded “the” film
(of which there is one) or no visit.

In October 2002, Sheldon I. Dorenfest & Associates, which
surveys hospitals, said that in interviews of the first 441
interviewed for the 2002 survey, patient safety and CPOE initiatives
are "motivating large growth in plans to purchase new clinical
systems." Regarding CPOE systems, Dorenfest says these first
surveys show:

Why did this focus on--and possibly rush to implement --
computerized physician order entry (CPOE) come about? After all,
physician order entry has been around since El Camino Hospital went
live with the Technicon system in 1971 with physician order entry for
pharmacy orders, and has been in wide use in major teaching hospitals
for well over a decade.

An often-cited reason is that the heat has been on to reduce medical
errors since November 1999, when the Institute of Medicine released
what is widely perceived to have been a landmark report, To Err is
Human. This report codified what has been increasingly obvious to
perceptive patients for a decade or more -- that there are too many
medical errors despite the dedication of care professionals. While the
report calls for a “systems approach” to reducing medical
errors, we believe its facts lend themselves equally in support of our
view that the more fundamental issue is this: as the financing of
health care has deteriorated since the late 1980s, so has the quality.

However, regardless of whether a systems approach, or reversing the
slow fiscal starvation of caregivers, is the more fundamental
response, the release of the report was extremely timely.

Soon after the IOM spoke up, an organization called the Leapfrog
Group was funded by the Business Roundtable, a trade association of
leading industrialists. Leapfrog leapt to, and has acted on
conclusions including these--

medical costs can be significantly reduced by reducing medical
errors; and

CPOE is a central, valid tool to reduce medical errors.

Leapfrog began economically pressuring health care providers in 2001
to implement CPOE. Its method was to seek to steer patients covered by
its sponsor industrialists to hospitals with information systems which
provide CPOE with clinical alerts to physicians.

We believe that there is ample reason to be skeptical that
Leapfrog's core goal is to improve care -- if, for no other reason,
than the fact of who is paying for it to speak up. The Business
Roundtable, as far as we can tell, has never left the forefront of
those entities which, in the political arena, wage constant war
against corporate spending on health care benefits for employees. We
believe that--

Leapfrog's focus on medical errors is, quite validly, a focus on
reducing the high costs resulting from medical errors; but that--

A likely greater interest of Leapfrog’s sponsors in CPOE is
as a management tool to clamp down on physicians’ consumption
of hospital resources covered by health insurance; and that

There is good reason for provider organizations to embrace this
motive – to make care more effective and also to head off its
potential for abuse.

For more on this view, see the "Our
Observations" section at the end of this report. However,
regardless of its motivations, Leapfrog has emerged as a highly
visible, and therefore important, source of pressure to implement
CPOE.

However, many CIOs tell us they are implementing CPOE primarily
because good patient care is their organizations’ mission, and
they simply cannot tolerate the notion of inflicting avoidable harm.
User reports gathered for this study strongly support the view that
the reduction of medical errors and faster treatment turnaround (and
thereby, less avoidable harm to patients) are both motivating factors,
and indeed likely, if you implement CPOE well.

Some CIOs also see CPOE as a cost of staying in business. If you
don't implement CPOE now or in the near future, you won't have to
worry about finding funds for that new building, because you won't
have any patients to put in it, said Ken Clarke, CIO at Central Maine
Medical Center, which is in the selection process. "Who," he
asks, "wants to go to a hospital that is unsafe?"

Another cited reason: in 5-10 years, hospitals will be bursting at
the seams even more so than they are today, said Greg Walton, CIO for
Carilion Health System. Hospitals that do well financially will make a
priority out of shortening lengths of stay. A way to reduce lengths of
stay is making treatment and information available to the patient as
soon as possible. A well-functioning CPOE system helps meet that goal,
he believes.

However, building the perfect system is, in the words of First
Consulting Group's medical errors guru David Classen, M.D., "a
very high-risk, difficult initiative." You need deeply committed
top executives, and close synchronization between your top clinical
leadership and your top executive leadership. You need to go to great
pains to engineer your software for flexibility, so that if, during
the course of implementation, you realize that the net effect of your
efforts is to dump heaps of new work on someone, you can apply a
just-in-time corrective measure, so you don't suddenly have a
revolution on your hands. You need to decide up front how you will
measure success, so that down the road when the going gets tough, as
it surely will, you will have a way to convince your clinical staff to
continue.

Moreover, Dr. Classen and others note, making the process of care
faster and more "efficient," like a computer-managed
industrial production line, can be a direct pathway to worse, not
better, outcomes. See "Many Hospital Pharmacies Shut Off Patient
Safety Software Alerts" below, and the "Our
Observations" discussion at the end of this study, for
additional views on this topic.

B. Questions And Issues
Addressed In This Report

This study has revealed to us that while institutions have succeeded
at CPOE have taken varied paths to successful CPOE implementations,
most have done a great many things similarly, and all have had to find
ways to surmount the same hurdles.

A small portion of this report focuses on the "why" of
CPOE. To a degree, your course of action will vary depending on
whether or not you're planning to use CPOE to influence or control
resource consumption decisions by physicians in situations in which
such decisions are not, strictly speaking, questions of erroneous or
correct medical practice, but of resource consumption. As we have
said, we believe that addressing resource consumption is the hidden
agenda behind some CPOE advocacy. It is also a significantly more
ambitious undertaking than that of reducing errors. If you take it on,
it will put CPOE implementation smack in the middle of major
governance issues, and political issues both internal to your
organization and national in scope.

However, in at least the early going, judging from what implementers
have told us for this report, your course of action (that is,
successfully getting CPOE working and stable without a physician or
nursing rebellion) will be largely the same regardless of how far you
intend to eventually push the potential of a CPOE system.

Most of the content of this report deals with this early-going "how-to,"
and the "how to avoid our mistakes" commentaries of
organizations which needed to make mid-course corrections in their
CPOE implementations. These are the issues and questions this study
seeks to address:

The real costs of CPOE above and beyond software, hardware, and
implementation.

How many order entry devices you will really need.

Who should be on the CPOE physician advisory team.

Challenges and hidden costs of CPOE in best-of-breed information
systems

Why CPOE may be easier to implement at a rural hospital.

User self-assessments of what they did right

What went wrong at some sites, emphasizing avoiding troubles
they encountered.

Implementers' views on why and how they succeeded, and benefits
derived.

How much physician order entry is done on wireless devices.

Whether care providers are using departmental and individual
order sets.

How and how widely "live" on-line decision support is
used.

How many FTEs and which job skills are needed to implement a
CPOE system.

What early implementers would do differently if they were to do
it again.

Closing Notes:

(1) This report steers clear of evaluating or specifically comparing
vendors. Some of the comments herein may be useful to you in drawing
conclusions about individual vendors and systems, but this is not
intended to be a vendor-to-vendor comparison of capabilities. Most of
those interviewed for this report are early adopters. Thus, they both
had to discover and deal with unforeseen new-system problems, and had
the benefit of extra vendor attention. Neither a glowing report nor
report of trouble should be assumed to be a likely universal
experience with that particular system.

(2) This report is written without footnotes. We are journalists,
not academicians. We write in news style, in which most of the time,
the attribution is contained within the paragraph in which a source is
quoted. We believe that this makes for easier, less distracting
reading.

II. Overview: Key
Lessons Learned On Implementing CPOE

This overview quickly summarizes some of the broad lessons your
colleagues have drawn from their implementations of computerized
physician order entry:

1. Be prepared to have to tailor CPOE screens and work flow to
some departments' special needs or face system rejection in those
departments.

A key crossroad: when you witness physician resistance, it may well
be just resistance to change by physicians who are accustomed to doing
things their own way. However, a highly likely alternative possibility
is that the CPOE system, as designed or implemented, truly interferes
with the ability of physicians to get their jobs done.

Asif Ahmad, CIO at Ohio State University Medical Center, winner of
the 2001 Davies Award for outstanding computer-based patient record
implementation, goes so far as to say that more frequently than not,
when a CPOE implementation runs into trouble, "It is the CIO
[who] is the problem." Most CIOs don't understand clinical
process well enough to design a workable implementation. (For more
information, see the Ohio State Case Study below.)

Mr. Ahmad suggests including ancillary department physicians, not
just surgeons and internists. You need physicians who think
analytically and understand workflow, not necessarily MDs who are
technically proficient with a computer.

Gary Strong of Fairview Health Services in Minneapolis, Minn.,
has recruited early users by specialty area, focusing those
specialties which, by their nature, have the greatest potential for
disastrous medical errors.

Holland, Mich., Community Hospital recruited physicians who are
closely aligned with hospital management to test the system. They
are likeliest to speak directly about what they think of the system,
and to be forgiving if does not work perfectly right out of the box.

Meditech's Hoda Sayed-Friel recommends including a champion, a
pessimist, and a devil's advocate, because that way, all physicians
on your staff will feel represented. The IT department may find it
easiest to communicate with physicians who are in roles of
administrative leadership. They are likeliest to be willing to tell
you if your system stinks, and you may find it easiest to tell them
if their expectations are unrealistic.

Bill Van Dornik, director of information systems at Holland
Community, used physician administrators to beta-test the first
version of new QuadraMed CPOE software. They didn't like it, and
they told him so, without walking away from the project.

However, if your committee is comprised only of physicians who
are in positions of administrative leadership, you could be in
trouble, because rank-and-file physicians do not necessarily respect
administrative leaders. Similarly, you want to send your top
clinicians on site visits, said Bart Lally, an MD with El Camino
Hospital, Mountain View, Calif., which was shopping for a system to
replace its 31-year-old TDS (now Eclipsys) installation as this
section was being written. "They are your real users." And
if they resist? Tell them that if they don't like the change, this
is their chance to be part of the process.

3. Clinical alerts should not be too weak, but should not be too
intrusive.

Plan on testing them with caregivers to achieve a balance.

Ohio State has implemented the Siemens Healthcare Dashboard, which
fires alerts for abnormal results. The OSU implementation "triages"
the alerts, so the MD can tell when an out-of-range result or other
event is truly urgent.

Here's a yardstick: look at your organization's track record for
medical chart completion. Look at your rules, and look at what
actually occurs. The greater the gulf between the two, the harder time
you will have enforcing CPOE. (This comment came from an audience
member at a seminar. We suspect that he was onto something, because he
drew wide applause from a mostly physician audience.)

The group of physicians staffing the emergency room at Mount Carmel
St. Ann's, Columbus, Ohio, picked up physician order entry on the A4
HealthMatics system easily, said Frank Orth, DO, group president. All
19 ED docs enter their own orders, along with 8-10 physician
assistants. Training was fairly easy. Today, physician order entry is
the single most popular application among St. Ann's ER physicians. One
reason is that an interest in using cutting- edge technology to
improve patient care is within the group's culture. But getting your
average hospital's general medical staff on board with CPOE? He thinks
that would be a much greater challenge. (See more details below in our
Mount Carmel Case Study below.)

6. Build on what you already have--and especially what you
understand well and can adjust in house.

If you have a long relationship with a vendor, consider sticking
with it. Most implementers say that the success of a CPOE
implementation is one-quarter the system you choose, and
three-quarters the way you implement it. To the extent that you
already know the vendor and the system, you may be way ahead of the
game. See, for example, the Case Studies below of Lehigh Valley
Hospital and Health Network, Allentown, Pa.., and St. Francis Care,
Hartford, Conn.

Your vendors will not be able to make major system modifications as
rapidly as your medical staff wants. Therefore, you will be much
better off if your information technology department can make some
changes in house and promise quick turnaround. For example, perhaps
partly because of its extensive experience with IDX LastWord, Lehigh
Valley Hospital and Health Network's IT department is able to respond
rapidly and effectively to physician requests.

Many organizations which have CPOE manually re-enter their orders
into their pharmacy information systems, said William Bria, M.D., an
Assistant Professor of Medicine at the University of Michigan School
of Medicine in Ann Arbor, and a longtime leader in medical
informatics. That is inefficient, less safe, and obviously best
avoided.

David Classen. M.D., of First Consulting Group offers another reason
to avoid non-integrated systems: they lack integrated decision
support. Bruce Elkington, CIO at Overlake Hospital Medical Center,
Bellevue, Wash., adds: maintaining interfaces is difficult and
expensive, and many organizations will install Meditech CPOE for the
simple reason that "it's installable."

8. Be sure that sure you have enough data entry devices.

Physicians are angered if they want to make rounds, and can't place
orders because there's a line at the terminal. Remote access, allowing
physicians to place orders from out of the ward and even from out of
the hospital, is a big plus. However, there is no hard-and-fast figure
on how many terminals make an ideal number; see the discussion of the
numbers in case studies and in the survey response section.

9. Wireless data entry is widely used, but far from universal;
PDAs are not a device of choice for wireless CPOE.

Most of the wireless devices used are PCs or laptops on carts which
are moved through hallways. Physicians widely use personal digital
assistants (PDAs), but the use of PDAs for CPOE is virtually
nonexistent, probably because as devices, they're too simplistic for
complex order sets. Microsoft’s Table PC software and the various
hardware packages that go with it are getting some attention from a
couple of CPOE vendors, however.

10. Get the strong commitment of executive and clinical
leadership.

Be sure there is a single, shared vision for CPOE. We would add:
vision is only 20-20 from the street. Share it with the troops–your
clinical users, from physicians and nurses to departmental allied
professionals--and listen to what they have to say about it.

Some institutions, with the power do to so, don't win it--they force
CPOE on physicians. See the section below, "Six Views on Winning
Physician Acceptance of CPOE" below.

12. Return on investment: hard, replicable data is lacking; big
savings are quite possible but present political and governance
challenges.

Much of the discussion of benefits to date has to do with reduction
of errors due to legibility of orders, error reduction due to
decision-support in medication ordering, and improved time turnarounds
to treatment. It is our view that these improvements, while clear-cut
and useful, will eventually be seen as among the relatively trivial
functions of clinical systems with CPOE.

CPOE does offer a clear and unambiguous pathway to significant
hard-dollar savings. The chief benefit of CPOE may well not be in
reducing errors and speeding turnaround of treatment, but as a
management tool to standardize and limit resource consumption by
physicians. However, a key question looms: to what degree does your
organization's management have the clout with physicians to take this
path? A second question: who will ultimately control the
resource-consumption decision-tree--the caregiver organization or the
payer?

More information: See the "Return on Investment"
subsection immediately below. See also the benefits discussed by some
early users in Case Studies and in the survey responses in Section
IV, and especially the Vanderbilt
responses at the end of that section. See the "Our
Observations" section for our discussion of the political,
governance, and resource-control issues.

Return On
Investment From CPOE: Anecdotal Information Is Real, But $$ Data Is
Soft

Big Possible Savings:
Provider Management CAN Use CPOE To Change Physicians' Resource Use

Most of the information we have seen to date on cost savings
directly attributable to CPOE is either soft on numbers or anecdotal.
Some savings are based on a single-department system with more limited
costs; some stated savings are for single applications which comprise
only part of the overall CPOE or clinical system package. In many
cases, improvements in turnaround time on care and reduction in errors
can be documented, but no one has enumerated the savings.

Some savings and returns on investment from CPOE systems discussed
in this report are:

Several survey respondents report payoffs and benefits in both
the aggregated survey responses and in the
Vanderbilt survey response in the
Survey section below.

See the Mount Carmel St. Ann's case study for some details on
savings. Officials there have estimated that direct savings will pay
for their Emergency Department CPOE system in two years. The savings
listed seem to be quite concrete and measurable. Note, however, that
this is a single-department system.

Gary Strong, CIO of nine-hospital Fairview Health Services in
Minneapolis, Minn., has said that Fairview expects a return of 27%
per year on its $27 million investment in a clinical system with
CPOE. (We did not obtain details for this report.)

A rare event to date–time saved in entering orders: at
Lehigh Valley Hospital and Health Network, Allentown, Pa., IDX
LastWord, seems to work well. The word in "hallway
conversations," according to Allentown's Physician Liaison Don
Levick, M.D., a pediatrician, is that proficient physicians can
enter orders in LastWord in slightly less time than it takes to
hand-write orders.

This seems to be significantly better
than what users are reporting from other CPOE systems, so it is a
valid question to ask: Is this really likely to be replicated by
other LastWord users? We will ask IDX for other sites that might be
having a similar experience; we hope to provide an update in a
future edition of this report and in an issue of Inside Healthcare
Computing.

Ken Clarke, CIO at Central Maine Medical Center, predicts that
his organization will see these three kinds of cost savings:

a reduction in adverse drug events;

a reduction in drug costs; and

a reduction in losses from payer audits.

Here is his logic:

An adverse drug event costs $2,000-$4,700 (depending on which
study you believe). Other studies suggest the number of adverse drug
events an organization is likely have in the absence of CPOE. Five
hundred per 25,000 is a pretty good number, he said. It is a
reasonable assumption that CPOE will reduce adverse drug events.

He notes that studies at Partners Healthcare have shown that
physicians will choose the less costly alternative if presented with
equally beneficial drugs. Central Maine plans to seize the
opportunity to capture these savings by inviting physicians from
Partners Healthcare to come to Maine and discuss how they approached
medication substitution.

You should boost insurance pay-up rates because the system will
capture documentation on orders.

Mr. Clarke does not anticipate savings in FTEs, although he does
expect to free up nurses to spend more time on patient care.

III. Case Studies

These 14 Case Studies of individual implementations are largely
based on interviews which Inside Healthcare Computing Editor Suzanne
Corrales conducted in 2002. For these Case Studies, all of those
interviewed were some identical basic questions. However, a core
purpose was to let interviewees discuss what they, not we, deemed
significant. As a result, the Case Studies are wide-ranging, do not
offer a common data set or answers to all the same questions, and
follow no specific presentation template.

We sought to augment these Case Studies with a more formal survey
asking exactly the same questions on 15 CPOE topics. The survey
respondents were offered anonymity, and were allowed to respond in
writing as well as in voice surveying, conducted by researcher Rachel
Ross. The survey section appears below the 14 Case Studies.

The #1 reason that CPOE projects fail has nothing to do with
shortcomings in the technology or the medical staff. According to Asif
Ahmad, CIO at Ohio State University Medical Center, winner of the 2001
Davies Award for outstanding computer-based patient record
implementation, "It is the CIO that is the problem."
Most CIOs, analysts and other IT employees do not understand clinical
processes well enough to design a clinically workable implementation.
When problems do arise, they chalk them up to physician resistance,
rather than recognizing that they have a real issue on their hands.

Mr. Ahmad cites a problem in his own implementation: the CPOE needs
of the medical ICU and the surgical ICU turned out to be so different
that it was necessary to take the medical ICU down for six months and
redesign the implementation (see below). If he hadn’t understood
the problem, "I would have had a failure." To help
understand, 40% of the people in Mr. Ahmad's 170-person IT shop have
clinical backgrounds. Mr. Ahmad's background is in biomedical
engineering. He also has an MBA.

2. Which physicians to involve in managing an implementation.

Physician support is a different story, he said. The physicians in
the project don't need to be technical at all. They must be
open-minded, analytical people, who can discuss workflow and care
paths in a global way. "These are often some of the best
physicians." Often hospitals box themselves into a corner with
physician teams that are not sufficiently broadly based. "Too
many surgeons and internists, not enough ancillaries," he said.
OSU went live March, 2000 on a pilot in the transplant unit. It went
live in all units in a "big bang" in May, 2000. The beauty
of a so-called “big bang” is that it saves the cost of
running double systems and the adoption rate is higher because, from a
physician point of view, there is no switching between systems.

Implementation was preceded by two years of awareness training. Mr.
Ahmad estimates that he personally participated at least monthly.
Physician training was mandatory for the medical staff, and was a
condition of medical center privileges.

Mr. Ahmad discounts the lore suggesting that CPOE implementation is
relatively easy for academic medical centers because they have more
power over their staffs than community hospitals. OSU's largest-volume
service by DRG, women and infant care, is wholly dependent on
community Ob/Gyns, he said.

3. Clinical alerts: should you? How and how many?

OSU's system is Siemens character-based Invision. As an older
system, it is more stable than some newer-generation products.
However, it is also based on a financial system, which makes it less
flexible, he said. OSU has implemented the Siemens Healthcare
Dashboard, which fires alerts for abnormal results. The OSU
implementation "triages" the alerts, so a physician can tell
when an out-of-range result or other event is truly urgent.

As the recipient of last year's award, Mr. Ahmad sat on this year's
Davies award committee. His opinion: so far, there aren’t yet any
new systems that fill the bill. "I don't see huge success stories
out there," he said. "I see huge promise."

Buried in all of the well-publicized successes of the Ohio State
University Hospital's implementation of Siemens Invision CPOE was one
significant initial failure: the medical ICU.

The system went up house-wide in June 2000, following pilot testing
on a transplant unit. It came down in the medical ICU one month later,
after medical staff complained that it interfered with patient care so
badly that it was creating a potentially dangerous situation.

This is the system that worked well enough to earn OSU a Nicholas E.
Davies award, the most prestigious award in healthcare information
technology. How could something work so well elsewhere throughout a
586-bed hospital, and fail so miserably in a 37-bed ICU?

Inadequate training was one problem, said Steve Hoffman, M.D., a
pulmonologist who prefaces every conversation on this topic by noting
that he does not consider himself computer-smart (but who was
obviously up to his neck in this project). Insufficient hardware was
another.

However, the biggest problem was one that many organizations could
slam into: the workflow in University Hospital's medical ICU was
fundamentally different from the workflow in most other inpatient
areas. In most areas of the hospital, physicians evaluate one patient,
then sit down and enter as many as 10-12 orders for that patient, then
go on to the next patient. However, in University Hospital's MICU,
care is far more random, and is more likely to be based on multiple,
disjointed orders, rather than order sets given together.

Patient care was being delayed because it was taking too long to get
orders into the system. Dr. Hoffman and others hoped that
administration would relent, and decide that they did not have to use
the system after all. Instead, they were given six months and
considerable support from the information technology department to fix
the problems.

Here are some steps that helped:

They created order "pick lists." For example, a
physician can choose "antibiotics with renal failure" and
call up a list of appropriate orders.

They created protocols. Rather than write a large series of
orders, a physician now writes a single order to place a patient on
a protocol–for example, a ventilator-weaning protocol.

They created a direct route to patient orders of a particular
type. Physicians can now call up a patient in the system, see all
current ventilator orders, and modify them.

The re-implementation came off on schedule and with minimal
difficulties, he said. Vastly more data entry hardware was crammed
into the ICU, and 24 nurses were trained as "super-users,"
compared with the three who had been trained six months earlier.

However, even these fixes did not completely solve the problem
of time-consuming order entry, he said. Residents and interns were
still hand-writing orders, and then transcribing them into the system
as of mid-2002. They still spent more time with the computer, and less
with the patient.

Second problem: changing orders with transfers in or out of the
ICU

A second significant problem was found: transfers in and out of the
ICU were not handled well within the system. Treatments change when a
patient transfers from one unit to another. That means that some
nursing orders become inappropriate or irrelevant because the
patient's physical surroundings have changed or because the medical
interventions they support have been changed or stopped.

On a paper-based system, the orders are taken off the clipboard, and
that's the end of that. However, in an electronic system, open order
go into a queue and sit there until someone cancels them. The system
does include commands for canceling orders upon transfer, but the
sequence of steps was complex, and in the early days, interns and
residents rarely got it right. As a result, nursing was beset with
open orders that did not make sense and could not be carried out. That
problem was still not entirely solved as of mid-2002, he said.

His attitude toward the system is best described as lukewarm. He
said he does believe that CPOE can work in a medically complex
environment, such as an ICU. There is even some evidence that it
speeds up patient treatment in the ICU:

Turnaround time on radiology orders appears to have been cut
from about 7.5 hours to about 4.5 hours. (That figure is based on a
comparison of the electronic system with a retrospective review of
only about 20-25 paper charts on which the time or the order was
noted, but even the small sample size may point to a valid
conclusion.)

Turnaround time on lab orders was reduced from a 31-minute wait
from time of order to time of draw, to 24 minutes elapsed. That
number is based on a comparison of 683 handwritten lab orders
generated by the medical ICU after the system was taken down in the
Medical ICU, and 1,100 electronic orders generated in the same time
period by the Surgical ICU, which had kept it running.

Turnaround time on medication orders was reduced from five hours
to two hours, based on a comparison of 46 paper records from prior
to the implementation and 70 after. (NOTE: this study was conducted
on the transplant unit by the nurse who championed the project.)

These studies were not done prove the system's value, but to attempt
to dispel provider concerns that it was slowing treatment down, he
said. He also said he can tell empirically that turnaround times on
X-ray and echocardiograms is vastly improved under the new system.
However, "We have no idea why." As to improvements on
laboratory and medication order turnaround times, he was guarded,
saying only, "We clearly confirmed" that "things did
not slow down."

"I think we can say for sure
that the system did not add to the time," he concludes. If there
was an improvement, it came "with a good deal of angst."

Gary Strong, CIO of nine-hospital Fairview Hospital & Healthcare
Services in Minneapolis, Minn., is leery of turning on physician order
entry for 1,700-bed Fairview-University Medical Center, because he
believes that he knows what is coming: a thousand residents will want
full use of every speck of functionality that Eclipsys' Sunrise
Physician Order entry offers, and want it now. "These people are
computer-literate," he said. Also, they are accustomed to the
bugs, annoyances, and freezes of the Windows operating system. For
them, "We have to be so ready."

At Fairview's community hospitals, he expects a different response.
Community physicians tend to have a greater depth of medical knowledge
than hospital interns, and they tend to be less tolerant of the
headaches computer system dish out. They will have to expect to spend
two minutes electronically entering orders that might take one minute
to hand-write, Mr. Strong said. That presents a bigger challenge.

But their appetites are being whetted by an interesting source.
Fairview also operates a large ambulatory care practice, which is
rolling out Epic Systems' EMR. Physicians who have used Epic on the
outpatient side are likely to be more receptive to Sunrise on the
inpatient side because they are already familiar with the benefits of
an order entry system.

Mr. Strong has given sales pitches for the system at numerous
physician gatherings. He figures he's made at least 100 presentations
and recruited a core group of 125 early adopters. He pitches CPOE on
two grounds:

that it will make patients safer; and

that it will make care more efficient.

He has recruited early users by specialty area, focusing those
specialties which, by their nature, have the greatest potential for
disastrous medical errors.

The early adopters are "anxious to get on," he said. They
seem to agree that the extra time they will spend on order entry will
be compensated in the long run. "But we haven't hit the masses
yet. That's where the walls will go up."

He mentions two factors that will help tear the walls down:

First, and foremost, is the systems' ability to generate on-line
alerts. For example, Fairview expects to set up its Eclipsys system
to alert physicians if a piece of equipment a test order requires is
out of order.

Second, Fairview is rolling out first by giving physicians
access to electronic results, then taking away paper results.
Forcing physicians to look up results on line may help encourage
them to enter orders electronically.

The estimated cost of the project is $27 million, including
hardware, software, installation, and training across four
organizations. The anticipated return on that investment is 27% per
year. "We are talking about very direct savings," he said.

If CPOE were generally faster than handwritten orders, implementing
it at 230-bed Holland Community Hospital, Holland, Mich., might be a
no-brainer. But MIS Director Bill Van Doornik has concluded that it
will take physicians an extra 30 minutes per day to enter their orders
electronically through the Per-Se physician order entry system--or,
for that matter, any electronic order entry system. And, unlike
teaching hospitals, Holland cannot compel physicians to do much of
anything. "I don't have the tools to ram this down anybody's
throat," Mr. Van Doornik said.

With one failed beta project under its belt, Holland's IT staff is
avoiding coming out a "loser" if its medical staff
ultimately rejects the system by taking a fairly neutral position on
whether it should be adopted. IT will provide "the best system we
can," he said. Whether or not it is rolled out will be entirely
up to the discretion of executive management and the medical staff. "Hopefully,
that is an empowering message," he said.

In a departure from the philosophy followed by many organizations,
Holland deliberately recruited physicians who are closely aligned with
hospital management to test the system. They are likeliest to speak
directly about what they think of the system, and to be forgiving if
does not work perfectly right out of the box, Mr. Van Doornik said.

So far, the approach seems to be working. Holland Community
beta-tested an earlier version of the Per-Se physician order entry
system, found the screen flow cumbersome, and sent it back for a
rewrite, he said. Holland is working with Per-Se on a branching
structure for physician order sets, so that a basic order template
will customizable for multiple orders, Mr. Van Doornik said. The
project is still alive and Holland plans a second beta test.

When a best-of-breed shop implements computerized physician order
entry with on-line decision support, how does it integrate clinical
decision support?

For example, suppose that the physician enters an order, deals with
a series of decision- support questions, and then passes the order to
the pharmacist on line. Using an entirely different system, the
pharmacist is confronted with a different series of decision-support
questions, and presumably no history of how the physician acted on
them. The pharmacist then releases the order to the nurse, who is
confronted with yet a third set of decision-support questions.

This concern has been raised by, among others, David C. Classen,
M.D., who discussed it in a presentation at the June, 2002 Healthcare
Information and Management Systems Society (HIMSS) meeting in Las
Vegas. Although orders can be integrated among disparate systems, Dr.
Classen said he has not found any examples of integrated alerts,
saying, "Technically, it is much more difficult."

Dr. Classen, a vice president with First Consulting Group, is a key
architect of the CPOE testing protocol of the Leapfrog Group, and is
an associate professor of medicine at the University of Utah, giving
him both credentials and clout on CPOE. Dr. Classen implied in his
HIMSS presentation, that vendors are putting significant pressure on
First Consulting Group to remains silent on the issue of the
difficulty of integrating clinical alerts in a best-of-breed
environment. "Despite many pleas to the contrary," FCG is
discussing it, he said.

We raised the question with Alberto Kywi, CIO for Santa Barbara
Cottage Health System, which uses IDX radiology and McKesson pharmacy,
Misys laboratory, and will launch a pilot for Eclipsys Sunrise CPOE in
December. His answer: Eclipsys Sunrise will become Cottage Hospital's
definitive decision support system.

Mr. Kywi said that pharmacy decision support is being built into
Eclipsys Sunrise 5.3, and at this writing, it was due out at the end
of 2002. He believes that eventually, the functionality in Sunrise
will allow Cottage to use it in place of the McKesson pharmacy system.

Cottage will also implement the McKesson robotics system, which
includes the Robot-Rx distribution system, the AcuDose-Rx cabinets,
and the AcuScan-Rx Dose medication administration record. The plan is
for orders to flow from Sunrise to pharmacy to the robot to AcuScan,
which feeds back to Sunrise Clinical Manager.

Implementation to start in post-surgical unit

Santa Barbara Cottage Health System expects to begin piloting its
Sunrise Clinical Manager with CPOE in a post-surgical unit by the end
of this year.

Why post-surgery? CIO Kywi explains: SBCH surgeons like to
try new things. Post-surgical patients usually don't go anywhere from
the unit except home, so a patient's record will not be as fragmented
between electronic and non-electronic systems as it might otherwise
be. Finally, surgeons use a significant number of standing orders.
From a physician’s point of view, the workload tends to be
lighter if he/she can reuse orders.

For Cottage, CPOE represents the next logical step in a long-term
information technology plan approved by the Board of Directors in
1996. The plan, which includes 16 initiatives, was developed by Mr.
Kywi the year after he became CIO. He avoided using consultants in
order to build credibility with management and the board. In earlier
phases, Cottage upgraded its network, replaced an outdated McKesson
Series patient financial management system with Eclipsys Financial
Management (formerly SDK), and installed a successful picture
archiving and communications system (PACS.)

The Cottage Board is driven by a need to both improve care and cut
costs. (Because it's in a California earthquake zone, the health
system is faced with seismic retrofitting requirements so large that
it expects to entirely rebuild its flagship hospital, Santa Barbara
Cottage). The board is buoyed by the success of its other IT projects,
particular its DR Systems PACS. It has committed to fully funding the
project, he said. Cottage is a teaching hospital, but most physicians
are not employed by the health system.

Following the pilot, all three hospitals will go up on Sunrise
results, nursing documentation, and CPOE for laboratory, medications,
and radiology, he said.

Additional reasons for moving to CPOE:

The HIPAA Data Security and Privacy rules. An order is more
secure if it flows from the physician directly to the ancillary
department than if it is handwritten and passes through other hands
before reaching its destination.

California law. A new law requires hospitals to adopt error
reduction plans which make use of technology. CPOE is one component
of the SBCH plan.

Convenience. If the MD writes an order, he or she must turn it
in to the nursing station and hope it gets entered. With Sunrise,
physicians will be able to enter orders and monitor results from
anywhere.

Speed. When a handwritten order is turned in, there can be a
significant delay before it is relayed to its destination. All
physicians round at about the same time, and unit clerks are
sometimes inundated. This can increase length of stay, and is a sore
spot with physicians. If they can enter their own orders on their
own timetables, the delay can be eliminated.

Chart availability. With Sunrise, multiple users can look at the
chart at the same time.

Mr. Kywi characterized Eclipsys Corp. as a firm that is "struggling"
with issues of integration, of both its products and its business
units. On the whole, they are succeeding, he said. "They are a
very good company to work with," he said.

M.D. Anderson Cancer Center, Houston, Tex., went live in the fall of
2002 on a pilot of IKnowMed EMR, a new product designed to support
cancer treatment. A month into it, Anderson had a five-physician
IKnowMed CPOE project at a breast clinic; nurses also use the product
to document chemotherapy administration at an outpatient center.

"We're very pleased," said Richard Pollack, CIO. "It's
been well-received." Anderson plans to roll IKnowMed out across
its other outpatient clinics and to integrate it with its Per-Se
Patient1 inpatient data repository. IKnowMed will be used for
inpatient chemotherapy orders and for structured physician
documentation; all other ordering will be through Per-Se, he said.

What's particularly interesting here:

In a world of tight budgets and cutthroat competition, Per-Se and
IKnowMed are stepping up to the plate with a level of systems
integration that many vendors cannot achieve even within their own
organizations. Both vendors are rewriting code so their systems will
parse each other. Instead of interfacing the EMR components on select
data elements, Anderson expects to end up with a system that allows
any data element to be pulled into either system, he said. The effort
will be commercialized and available to other sites, he said.

M.D. Anderson was a very early Cerner Millennium site at a time when
Cerner was having a hard time meeting its vision. That project failed
three or four years ago (IHC,10/12/98; 5/15/00). The staff has bounced
back and is "more than ready" to get back into the EMR
implementation business, Mr. Pollack said. The experience defies
popular wisdom that it takes many years for organizations to recover
from a failed implementation. Mr. Pollack noted that the project
predates his arrival, so possibly new blood at the top was part of the
solution.

Working with Per-Se, Anderson has embarked on a project to unlock
some 25 years of data in generations of legacy laboratory information
system databases. The project involves exporting the original data in
flat files, building screens in Per-Se to reflect the original
presentation, and having Per-Se map the legacy repository. Anderson
will eventually replace the legacy repository with Per-Se's Patient1,
an M-based (MUMPS) clinical data repository, and its Decision1
Oracle-based research database. Anderson started work in the spring of
2002 and expects to finish by late 2002 or early 2003. The pace
depends on the speed of the legacy repository and on Anderson's
decision to clean data as it goes.

Why choose Per-Se? Mr. Pollack offered three reasons. First, it runs
on MUMPS, allowing it to handle large volumes of data with an
efficiency and reliability other database management systems can't
match. Second, the Per-Se system is extremely flexible, which meant
M.D. Anderson could replicate its legacy data presentations by
creating hundreds, even thousands, of different screens in Per-Se. A
less flexible system would have meant "tremendous compromises."
Finally, Anderson happened to have several people who were extremely
experienced in Per-Se systems on its staff.

Just how do you persuade community-based, attending physicians to
enter their own orders?

Lehigh Valley Hospital and Health Network, Allentown, Pa., set an
expectation that all physicians would be trained on the new system,
with the understanding that no one would be forced to use it. Unit
clerks continue to be available for physicians who are unwilling to
enter their own orders. However, physicians are under constant, gentle
pressure to get with the program. When they do, they can count on a
lot of support.

Suppose you have a physician who just completed training, and it's 7
p.m., and he's writing orders. A trainer approaches him and says, "Wouldn't
you like to enter them electronically?" He says, "No. I want
to go home." So the trainer says, "OK – we'll do it in
the morning."

So far, it is working pretty well at this 644-bed community
hospital. A trauma step-down unit went live on IDX LastWord in late
2001; three medical surgical units have gone live beginning in
February, 2002. As of this report, about half the physicians are
entering their own orders. About 60% are entered by community- based
attending physicians; 40% are by residents.

Other factors that help:

Lehigh Valley has been a LastWord site since 1993. Physicians
are already accustomed to looking up results in the system. Labs and
transcriptions are printed out and placed on the chart, but it is
often easier to use the ubiquitous on-line record. Also, electronic
results are available first. Medication administration records and
vital signs are also available on line.

Perhaps partly because of its extensive experience with
LastWord, Lehigh Valley's IT department is able to respond rapidly
and effectively to physician requests. For example: a physician
wanted an order change for supplemental oxygen to include a
titration order and the ability to choose a target level to wean the
patient to. The department had the change ready within a few days.
(Implementation took longer because the change had to be cleared
with affected ancillary departments, and support staff and educators
had to be notified, so they could communicate the change to the
medical staff.)

Medical staff leadership paved the way for the CPOE project with
multiple meetings, presentations, and demonstrations. Physicians on
the design team met twice per month, from 6:30-8 a.m. Coffee,
muffins and bagels were always available. Free bagels and donuts
were also available to nurses and physicians in the kitchen area of
the unit as units went live. Project supporters received recognition
at meetings.

The project is well-staffed. Since the inception of the CPOE
system in October, 2000, Dr. Levick has devoted about half of his
time to it. A senior information systems manager spends most of his
time on the project, as do four analysts and three educators. The
team provided support 24 hours/7 days for several weeks after each
unit went live, as well as ongoing support.

All training is one-on-one. It takes about two hours.

The system itself, IDX LastWord, seems to work well. The word in
"hallway conversations" is that proficient physicians
can enter orders in LastWord in slightly less time than it takes to
hand-write orders.

Data entry is by desktop computers, wireless laptops on carts,
or wireless sub-notebook computers that are carried by some
physicians. Lehigh has also made LastWord available on Citrix,
meaning that it can be used with a thin-client terminal.

There's "no question" that Great Plains Regional Medical
Center will get good cooperation from physicians on its computerized
physician order entry project, said Robert Buckland, MD. How can the
physician chair of the information system committee for a 116-bed
hospital in the middle of nowhere make an assertion like that?

"Middle of nowhere" may be precisely the point. Big-city
physicians may feel no ties to the hospitals they practice at, and be
less willing to extend themselves for the good of the organization.
But the 60 or so physicians who practice in the North Platte area
understand that their fates are wrapped up with the future of Great
Plains Regional.

Some 20 physicians turned out for a meeting at which QuadraMed
Corp., the vendor, showed a prototype of the system. Half a dozen have
agreed to essentially donate the time it will take to get the
prototype ready for a beta test; 12-15 are likely to participate in
the six-month test. "They're excited," he said.

Geography is facilitating the project in another way as well. With
such a small medical community, the overwhelming majority of
physicians' offices are a stone's throw from the hospital, and are
already tied to it by fiber optic cables. Adding order entry and
results retrieval from offices should be relatively straightforward.
Viewing is through QuadraMed's browser-based results viewer, Clinician
Access.

But the main reason he expects the project to succeed is his belief
that QuadraMed is poised to do things right functionally. "If
you've got a good system, getting cooperation is easy," he said.

A longtime Affinity client, Great Plains is the development site for
QuadraMed Affinity physician order entry. Applications are being built
on the Affinity Cache database; a change to Oracle at a later date is
likely, he said.

Great Plains is installing medication order entry, laboratory, and
radiology, and it won't roll out until the entire system is built,
said Kim Dyer, VP, operations. The reason: part of the problem with
offering computers to physicians is that they are often also compelled
to consult paper medical records as well. "Until we have it all
electronic, they are not going to be all that interested in using it."

Great Plains is also installing Antibiotic Assistant, a decision
support system from Theradoc, Salt Lake City, Utah. The system
recommends appropriate antibiotics on the basis of lab values and
gives underlying literature to support its recommendation. QuadraMed
could be set up to provide the same decision support, but the purchase
saves Great Plains the work of setting it up. Ms. Dyer said she was "leery"
of licensing software "telling physicians how to do clinical
practice," but found that they see it as "a great tool."

Organization: several, based on literature review and survey
by the head of a HIMSS committee. CPOE System: various.Sources
of information for this report: Joe Miller, chairman of a HIMSS
Special Interest Group for the Computer-Based Patient Record and
manager of IS for Christiana Care Health System.

Few organizations had pharmacy systems working when they started
their computerized physician order entry projects. Also, the most
common decision support rules among early CPOE sites had more to do
with efficiency and data compliance than they do with prevention of
adverse drug events. For example, alerts on incomplete orders, or
incomplete data are fairly common.

This information comes from an early 2002 review of 10 CPOE projects
that have passed the pilot stage. The results are surprising because
reduction in errors, particularly medication errors, is one of the
most frequently cited reason for implementing physician order entry.
However, most decision support to reduce medication errors can work
only if the organization also has the patient's medication available
electronically.

Organizations thinking they need a full electronic medical record in
place prior to implementing CPOE "may be setting the bar to high,"
said Joe Miller, chairman of the Healthcare Management and Information
Systems Society's special Interest Group for the Computer-Based
Patient Record and manager of information services for Christiana Care
Health System.

Four are academic medical centers, five are teaching hospitals, and
one is a community hospital. In a world in which many CPOE projects
never make it past the pilot stage, this list includes some roaring
success stories. All are past the pilot stage and more than half claim
100% CPOE, he said. Sarasota, the community hospital, has about 25%
physician order entry. He based his conclusions on literature about
the projects, and on his own discussions and surveys. Not all of his
questions were answered by all organizations, he said.

Organizations implemented CPOE for a variety of reasons. Some were
worried about medical errors. Others were responding to competitive
pressures. Still others saw is as part of a broader re-invention
effort that accompanied a merger. All their projects pre-dated
Leapfrog.

Successful organizations varied in their basic approach to CPOE.
Some scrapped existing systems entirely in favor of implementing
totally new, integrated system. Others upgraded what they had. Boston
Medical Center went straight from 100% paper-based orders to CPOE
without ever having electronic orders entered by nursing or unit
clerks. Abington Memorial achieved 40% CPOE, waited a while, then made
a strong push for 100% compliance.

Here are other details he learned:

Several organizations warned that the physicians who are most
attracted to a CPOE project sometimes lack clinical influence. It is
crucial to involve physicians who are respected clinical leaders
among their peers, he said. Leaders of successful projects "selected
very carefully who was involved," he said.

Virtually all of these organizations offer physicians order
sets, which are groups of orders that can reduce the amount of time
it takes the physician to enter his or her own orders. Academic
institutions tend to do departmental order sets, he said. Teaching
hospitals and community hospitals often allow personal order sets as
well.

Classroom training was common among early adopters. (This flies
in the face of advice that says "physicians won't show up for
classroom instruction.") Only one institution used one-on-one
training, and one used computer-based training. Most offered
around-the- clock support for two to four weeks following go-live.

Several organizations reported that additional training sessions
were necessary weeks or months after go-live. The reason: nurses
were back to entering orders because physicians were demanding it.

User access to input devices was a big concern. Many started out
with one PC for every three to four beds on medical surgical units,
and ended up with two beds per PC.

Few had wireless systems when they started their projects, but
most have wireless today. Wireless is a great convenience in CPOE
because it allows an organization to easily expand the number of
data entry units in the early stage, when users tend to be slow with
their orders, he said. Later, as people speed up, the devices can be
transferred to the next new unit.

Many did studies highlighting "early wins" to their
organizations. Several focused on reductions in time elapsed between
the time the order is written and the time the order is filled.

Virtually all said that top-level commitment and strong clinical
leadership were essential to project success. They "did not
locate the authority for the project in information technology."

Many found it useful to create a simple, direct vision statement
of 10-20 words which summarized the goals of their electronic
medical records projects. Vision motivates people to behave in ways
that are not necessarily in their own interest, and it helps
coordinate the actions of hundreds of individuals who are involved
in a project, he said. Mr. Miller, (302) 324-3523.

Mount Carmel St. Ann's of Columbus, Ohio, anticipates that it will
take less than two years to pay back the cost of its A4 HealthMatics
emergency department system, said Frank Orth, D.O., president of the
physician group which staffs the emergency room, in an interview with
the Inside Healthcare Computing staff for this report. Here are the
returns on the investment:

Reduced transcription costs. It formerly cost
$450,000/year to transcribe reports that originated in emergency.
Now, 95% of that cost is saved because physicians document on line
using Fujitsu hand-held notebooks.

Reduced insurance denials. Mount Carmel set its system
up to force physicians to make coding choices that carry Medicare
reimbursement. Test denials, which previously cost tens of thousands
of dollars per month, have dropped to zero, he said. Dr. Orth said
he guesses that the savings on denied tests will at least equal the
savings in transcription costs.

Improved charge capture. As with reduced insurance
denials, dropped charges for nursing supplies have become less
common.

Here's what's so interesting: St. Ann's implemented an emergency
room system as a way to gather data to support performance
improvement, not to save money, said Dr. Orth, who is also physician
facilitator for performance improvement for the emergency department.

A4's HealthMatics ED went live at St. Ann's in mid-2001. St. Ann's
chose the system first for its reporting capabilities and second for
its flexibility. "We can customize it on site without spending a
lot of money," he said.

St. Ann's deployment includes all major modules: nurse triage,
nursing documentation, patient tracking, physician documentation,
order entry, and results reporting.

Screen design/flow issues did cause some pain

Installation was "painful" because of unpredictable issues
with interface design and workflow. The only real remedy would have
been a full-blown test environment, with the ability to mock up all
interfacing ahead of time. That would have been prohibitively
expensive, he said.

The system will benefit by some tweaking, but all modules are in
production and working well, and the vendor has been remarkably good
at taking suggestions from the medical staff and incorporating them
into upgrades, Dr. Orth said. Allergy checking, order screening for
completeness and logic, and certain changes to the way the chart is
presented will be available in the future. Overall, support has been "very
good," he said.

Physician acceptance within St. Ann's busy (40 beds, 65,000 annual
visits) emergency department has not been a huge obstacle, principally
because of the group's culture. All 19 ED physicians enter their own
orders, along with 8-10 physician assistants. Physicians groups of 3-4
were trained on order entry. Getting ER physicians to attend the
sessions was not terribly difficult, he said, but "I'd hate to
try" to persuade a general medical staff, he said.

Physicians place lab, x-ray and pharmacy orders electronically. A
year after installation, physician order entry is the most popular
component of the system with emergency physicians, nurses and
secretaries. The reason: orders are no longer lost or delayed before
being entered in the system, and secretaries no longer track down
physicians with questions about handwriting ambiguities or disallowed
orders.

When a physician places a lab order, it appears immediately on the
ED tracking board, where it notifies nursing that a blood draw is
needed, and a label prints out of the printer. Sometimes the nurse is
there with the needle before the physician has left the patient's
bedside, Dr. Orth said.

It is almost impossible for structured data templates to capture the
nuances of every possible patient scenario. To overcome this
limitation, St. Ann's allows dictation of parts of the history and
physician if the case just won't lend itself to the system. Physicians
use this feature in about five percent of cases. Over-use is
discouraged by the six-hour or so delay before reports are transcribed
and appear back in the system.

St. Ann's also uses HealthMatics to force selection of appropriate
dosages. Amounts are set by the medical staff.

One argument which helps ease acceptance of order entry: when
reluctant physicians complain about becoming "clerks," Dr.
Orth reminds them that they are judged by the quality of the data
going in to the system, and asks whether they wouldn't prefer to be in
control of it themselves.

Also, St. Ann's was able to redesign order entry screens for the 12
most common diagnoses to "be at least as fast as dictation."
The system administrator, a nurse who was trained by HealthMatics but
has no IT background, did the work. St. Ann's emergency department
uses departmental templates, but physicians are free to develop their
own.

Problem: rest of the medical staff dislikes the long reports

HealthMatics is available system-wide through Mount Carmel's
browser-based front end for dictated reports and certain other patient
data.

If the system is so terrific and so cost-effective, then why isn't
every hospital in the country rolling it out? One factor is the
resistance of other hospital physicians. They complain that ED
records, which formerly comprised a page and a half of dictation, now
run eight pages or more. Their resistance has been "a little
messy," Dr. Orth acknowledged. A4 has helped address the problem
by re-organizing the data. A true fix, putting a summary report at the
top of the chart, was to become available within a couple of months
after the interview for this report.

Another hurdle for non-emergency physicians is that if an emergency
patient is to be admitted, and an attending physician wants to order a
medication or test for that patient, the order must be entered by an
ED physician or other trained system user.

An organization which is moving to its second generation of CPOE has
chosen the relatively new IDX CareCast system to replace its old
Eclipsys (formerly TDS) 7000. St. Francis Care, Hartford, Conn.,
talked to "a lot of vendors" and "a lot of clients"
before settling on IDX.

One reason for the decision was that CareCast offered a platform
and an architecture which the hospital concluded would provide fast
response times and high reliability, said Catherine Szenczy,
senior vice president and CIO. CareCast, introduced in February, 2002,
runs on Tandem Nonstop Himalaya, just as its predecessor, IDX
LastWord, does. The old Eclipsys 7000 runs on an IBM mainframe, which
is also fast and highly reliable, she said. "Several-second
response times" were simply out of the question, she said. Cerner
Corp. was the other finalist.

Another reason for choosing CareCast: integration was crucial,
primarily because of the importance of integrating clinical decision
support with order entry.

St. Francis is serving as the development site for Care Paths, a
clinical pathways system, in CareCast. The system St. Francis
envisions will allow physicians to change the patient's clinical
pathway and adjust the underlying order without having to leave the
care path to go back into order entry, she said.

St. Francis will use Cap Gemini Ernst & Young's Rapid Design
Methodology to develop the Care Paths, she said. The development plan
is for the entire 150-member end-user design team to meet in three
3-day sessions to plan the entire system. At each session, they're
being asked to make a new set of decisions, she said. The system will
be prototyped based on these suggestions to be sure it is really what
the users had in mind, she said.

The meetings, held by IDX, are spaced about six weeks apart. Cap
Gemini's approach was first used by St. Francis two years ago with
great success to plan its e-health strategy, she said. While getting
people to a meeting for three consecutive days requires enormous
planning, it is more effective and less difficult than coping with the
constant interruption of a long series of meetings, she said. IDX and
CGEY were to attend all sessions and provide a transcriptionist to
document all decisions.

Roll-out is to three facilities: St. Francis Hospital Healthcare
System, St. Francis Hospital, and the Medical Center Mount Sinai
Campus.

Bristol Hospital, a 108-bed affiliate in nearby Bristol, Conn., with
no history of CPOE, will also participate in the project, replacing
its McKesson Series 2000 with IDX Carecast. Bristol will go up on
order communications for nursing and unit clerks with electronic
resulting. The idea, Ms. Szenczy said, is to ease physicians into CPOE
through the availability of an electronic medical record. Bristol will
participate side-by-side with St. Francis in the nine days of planning
meetings, but there is no firm timetable for implementing CPOE at
Bristol.

St. Francis is a teaching hospital, staffed by community physicians
and 100 employed physicians, who train some 200 residents. It
installed the original predecessor to the EC 7000 about 30 years ago.
What made for a successful CPOE program 30 years ago:

Extremely strong top-level support. The CEO and senior
management made it clear that they were fully committed to physician
order entry, and that if physicians intended to admit patients to
St. Francis, they would have to go along with the program.

The first thing senior administrative support did was make sure
that senior medical management wholeheartedly endorsed the project.
Whether they talked to administrative management, or medical
management, physicians heard exactly the same message.

System selection involved a very large group of physician who
participated in the advisory committee, the selection process, and
site visits. They willingly gave time to the project because they
understood that if they did not participate, they might be stuck
with a system they did not like.

The caliber of St. Francis physicians made a big difference.
They are highly motivated to partake in cutting-edge technology. "That
is very relevant to our success."

Montefiore Medical Center111 East 210th StreetBronx,
NY 10467(718) 920-4321Size: 1,060 total beds CEO
or Administrator: Spencer Foreman, M.D.Chief MIS
Executive: Jack WolfMedical Director: Brian Currie,
M.D.CPOE System: IDX LastWordSources of
information for this report: Dorrie Napoleone, Director of
Clinical Systems, (718) 405-4092. (This report is based on both an
in-depth interview and Ms. Napoleone's responses to our survey; most
other user reports in this study are based on one or the other.)

Teaching hospital Montefiore Medical Center just finished rolling
out the IDX LastWord system with CPOE to the last of its nearly 1,100
beds in the fall of 2002.

The implementation, which began in March, 1999, has cost $25
million, including hardware, software, and out-of-pocket vendor
implementation fees. That amount doesn't include the substantial
amount of in-house effort.

The first unit was a general medical unit, chosen because there was,
and still is, good physician support on that unit. It "went up
and stayed up," said Dorrie Napoleone, Director of Clinical
Systems.

We asked what she would have done differently. Her answer: "It
took three years to roll out the system to serve 1,100 beds. I would
have liked to have hired more staff so that we could have done it in
two years. [But] [o]ur [core] strategy we would repeat again: unit by
unit--being successful as you go."

System selection approach

Montefiore's management assembled a committee of 25 people to select
a system. About half were physicians. This committee also included
nursing, ancillary department staff members, and administrators. The
physicians were from a diverse variety of services--surgery,
pediatrics, internal medicine--who were well-respected by colleagues,
and even a new graduate from a residency program who had just joined
the staff. Management wanted the decision to carry weight after it was
made.

The entire group went on "road trips" to learn about
systems. They had marching orders to look for a system that would
provide high quality, reduced cost, and high physician utilization of
the system.

They saw five vendor systems in action: IDX LastWord at Michael
Reese, TDS at New York University, and Cerner, HBOC (now McKesson),
and the SMS (now Siemens) system at a time when these last three
really have much.

The eventual choice was LastWord. This may or may not have been the
entire group's basis for making a selection, but Ms. Napoleone says,
"For hospitals approaching this project, I recommend that you
purchase and install a system that has 100% up time. This is critical
when you're trying to go electronic. You can’t afford down time
at all."

(One of the benefits which IDX touts about LastWord and its
successor, CareCast, is that they run on high-reliability Tandem
mainframe hardware.)

After selection: talk it up to build support

After the selection of LastWord, two well-respected physicians were
chosen to be champions to speak up for the system. They spoke at as
many existing Montefiore staff and employee forums as
possible--perhaps--25 times. They went on grand rounds and attended
departmental meetings, giving updates.

Going live, getting physicians to do CPOE: remote access a plus

Montefiore forces physicians and nurses onto the system. "We
take away the paper, so there's no choice for the physicians but to
use and master the system. This rule went into effect: a physician
could not use the system if he/she did not sign on for training, and
couldn’t place an order without training. A benefit was provided
at the same time: the physician received remote access and the ability
to place orders remotely -- a big plus for the physician. "They
really like remote access." They now give very few verbal orders.

Another important success factor, Ms. Napoleone said, is this: start
at a location where you will have the greatest chance of success. Once
you get 1-2 units under your belt, it is hard for a nay-sayer to claim
that the system "never would succeed."

Also, it's very important to pay attention to the details as you go
live. For example, the first nursing unit had a lot of dialysis
patients. The implementers had to be closely engaged to review orders.
"People don’t like to tell you what they have been doing
wrong."

Building and implementing the system--more how-to details:

The in-house "build" and implementation team is about 13
FTEs. "We employ a couple of physicians for about 20% of their
time..."Because they're still practicing medicine, they have more
clout within the physician community. We don't want them to become the
"IS doctors" and lose their standing in the practice of
medicine. We supplement their salaries for the time they put in with
IT, but they can continue to practice. We have five nurses, one
pharmacist, and the remainder of staff go in the programmer/analyst
bucket...We found that it is easier to teach clinical people about
technology than the other way around."

The group included 1-2 people from nursing whose principal tasks
were to do workflow analysis for nursing units. They would identify
the uniqueness of the department's or unit's needs, and send the
screens back to a couple of analysts to do screen tailoring.

Then, at go-live, they would take away paper from nurses. They gave
support 24 hours a day, 7 days, for at least two weeks after each
go-live. In the ICU, typically that support was for 3 weeks; it varies
by number of beds and census.

The clinical CPOE system was a bigger change for nursing than for
physicians because of all the products that were brought up for nurses
at once. Each nurse had a four-hour class; physicians had 1.5 hours.

After going live, you are still implementing

Montefiore goes back 3-4 months after each unit is live and asks
what users need and want. "You need to solicit user feedback."
LastWord has a powerful screen builder tool, so they can make 95% of
needed modifications without going back to the vendor.

Integration with pharmacy, nursing helps build support

Another success factor: integration. Montefiore brought up the
LastWord pharmacy system before implementing CPOE.

Your best chance for success, Ms. Napoleone said, is to have
order entry completely integrated with pharmacy and with medication
administration so that the physician, the pharmacist, and the nurse
are always on the same page when looking at a medication order. "I
feel very strongly" that integration there is a key. Montefiore
currently has an older Community Health Computing (CHC) lab system,
which it's replacing with a Triple-G system. The radiology system is
IDXRad.

In failed implementations she has studied, they didn't provide the
clinical data feedback to physicians. They'd have to place orders and
drop to the paper world for results.

Ratio of data entry devices to patient beds

Montefiore has about one device for every 2.5 beds in all units
except the ICU, where it's one-to-one. "It's adequate, but I
would love to have more devices," she said. "Part of the
issue is what else you use the devices for. If it's only for CPOE,
then the ratio is OK. But if the staff is doing e-mail and research on
these same devices, then the ratio needs to be higher."

Decision support: another big benefit

"We use
a combination of out-of-the-box rules checking for allergies, drug
interactions, and duplicate checking," Ms. Napoleone said. "We
write some rules beyond these just mentioned using the rules engine
that comes with the IDX product."

She said that Montefiore has found "tremendous benefits"
in reducing prescribing errors through the system's allergy alerts,
drug interactions, warnings on incorrect routes, and dosing
alerts--along with the fact that orders were perfectly legible.

"With
each unit we brought up, we've seen more than a 50% reduction in
medication errors. That's immediate payoff. We've also seen
considerable reduction in turnaround time (approx. two hours less)
from order placement to the patient receiving the medication or
procedure. This time savings translates into short LOS because we're
treating patients faster, and they're getting better quicker."

Two things went right for Stanford in the implementation of
computerized physician order entry in its IDX LastWord system,
according to a 2002 conference presentation by Russ Peckenpaugh,
Stanford's VP of information systems:

LastWord has an integrated pharmacy information systems which he
said was "absolutely key" to the project.

Data entry was through a wireless network. Physicians' use of
low-end laptops on carts during rounds has worked well, he said.

The list of things Stanford would do differently was longer.

The good news: a year after the go-live, most physicians have
accepted its benefits and are entering their own orders. However, the
beginning was bumpy. In a commendable act of bravery, aimed at helping
other organizations avoid repeating Stanford's mistakes, Mr.
Peckenpaugh has gone public with the "do this differently"
list:

Stanford did not mandate training. The result was "residents
on units who did not have a clue" what to do with the system,
which was not intuitive.

Stanford didn’t deal well with nursing expectations.
Nursing believed that nursing entry of verbal orders would decline
once physician order entry was live. In fact, physicians continued
giving nurses verbal orders to enter at the usual rate. The problem
was particularly acute in surgery, where physicians would announce
their orders, then rush off to meet a surgery schedule.

Stanford did not go live on nursing documentation prior to
physician order entry. Without nursing documentation active, it is
difficult to manage the nursing work list, and it limits the amount
of is feedback physicians receive about the orders they have placed.

Stanford rolled out the entire organization in approximately six
months, at times bringing one unit per week live. "We could
have easily spent twice that long" building order sets and
stabilizing the units, he said.

Stanford had difficulty balancing the tension between ancillary
departments, which wanted as much order detail as possible and
physicians who wanted to keep screens bare-bones. "We ended up
with too much information on the screens and too many detail screens
required."

There were also some IT-related bumps. For example, if an order is
not added to the patient profile in the Pyxis drug dispensing software
by the time the nurse needs to administer the medication, the nurse
will perform what is called a manual override. The nurse then needs to
go into Pyxis or IDX (the systems are interfaced) and manually assign
the order to the physician. If the nurse chooses the wrong physician,
the physician will rightfully refuse to sign the order. The order must
then be researched and resolved to get it signed by the right
physician. "This required a well thought-out plan," which
was not in place initially.

LastWord screen flow was a big challenge for Stanford physicians.
For example, Stanford's system did not include a screen a physician
could consult to see all of the orders he had placed on a patient
before finalizing them, he said. The problem has been addressed in
CareCast, IDX's next-generation clinical system, he said, but Stanford
runs on Version 4 of LastWord. "We are working with IDX to bring
the functionality" to V4, he said.

Don Levick, M.D., physician liaison for the CPOE project at LastWord
shop Lehigh Valley, had quite a different experience. The majority of
Lehigh physicians find LastWord screen flow "intuitive and
acceptable," he said. He described Lehigh's order entry process
as follows:

First the physician chooses orders from multiple lists. The lists
depend on how a system is configured: Lehigh's are divided into
medication, laboratory, imaging, and nursing. Once orders are
selected, they appear in an "unprocessed orders list." The
physician then goes back in and fills in order detail. For example, he
or she would give the reason for an order for a radiology study. Then
the physician enters the order. At that point in the process, the
system issues any decision-support flags.

"There is no final window" for reviewing all orders, but
if the physician wants to see what he or she just did, there is
another route for it. All active orders on a patient appear in an "order
profile screen," which, when chronologically sorted, gives a view
of all ordering activity. Lehigh Valley Hospital and Health Network
physicians find that the system gives them "adequate control"
over order entry flow and display, he said.

CIO Rena Burkholder of Cambridge Memorial Hospital figures that she
recently witnessed the end of an era: the last physician to refuse to
use a computer to look up patient medical information has finally
retired, five years after Cambridge began its electronic medical
record project.

What worked best to encourage physicians to do computer lookups? "...bring
them on when they are comfortable, and not fight," she said.

For Ms. Burkholder, the event held an important message that is
relevant to Cambridge's CPOE project: left to their own devices,
physicians will get there eventually. "What works best is
bringing them on as they are comfortable, and not fighting them."

Cambridge plans to involve seven hospitalists in a two-year pilot
project on Meditech Magic Patient Order Management (POM). The project
is supported by an $89,000 CDN grant from the Ontario Hospital
Association's Change Foundation. As in the United States, CPOE is
widely seen in Canada as a way to reduce medication errors. Medication
errors are believed to kill 500-700 Canadians annually. Placing
decision support at the physician's fingertips is the principal allure
of the project, Ms. Burkholder said.

Capital costs are $91,600, including $89,000 for software. The
two-year operating expenses are $152,400, including the cost of
evaluating the project. The system goes live in November. Initial
rollout is pharmacy order entry only, with lab and radiology to
follow.

Meditech Patient Order Management (POM) allows physicians to create
their own custom order groups with no support from the information
technology staff. Meditech has also added fuzzy logic to help speed
order entry. Physicians have only to input a few letters of a
patient's name, before the system will start guessing at whose record
the MD wants to see. "I have been thrilled," she said. Her
comments seem to challenge the view that Meditech has a very limited
physician order entry capability.

Roll-out will be by speciality. Each speciality will be evaluated
prior to rollout to address gaps between what physicians require and
what the system can do. If a speciality is not satisfied, the project
will not proceed to another specialty area until it is -- “period”
-- she said.

Do lab and radiology order entry fit into the plans? Yes. "[w]e
hope to move to lab and radiology once medication ordering is
complete, said Ms. Burkholder.

These steps should also help ease physician acceptance:

Cambridge will pay hospitalists at their regular rates of pay
for the extra time they devote to developing the system.

In support of the project, the pharmacy department has agreed to
manually document the amount of time it spends on the telephone with
physicians clarifying orders. Ms. Burkholder believes that these
logs will help prove to physicians the utility of CPOE.

Cambridge pulled together some 50 relevant articles on the
benefits of CPOE and presented its medical staff with a summary and
the original research so that they could see for themselves the
risks and benefits. It was very well received.

Here are the areas in which the challenges lie:

Working out a process for getting patient allergies entered into
Meditech by someone who understand the difference between a
sensitivity and a true allergy. Ideally, allergies will be noted by
a nurse and affirmed if necessary by a physician or a pharmacist.

Making sure that staff support is in place and that the system
is not missing some critical component before it is rolled out to
each specialty area. A researcher will conduct focus group sessions
with physicians to ensure that they are satisfied with the results
of development efforts.

Overlake Hospital Medical Center launched a pilot test of CPOE using
Meditech Magic CPOE in October, 2002. The original July starting date
was delayed while Meditech did some upgrades recommended by clients
who were CPOE early adopters, said Bruce Elkington, CIO. The module
will “snap on” over Overlake’s existing pharmacy and
medication administration record systems.

Overlake is pilot-testing the system with hospitalists, who write a
significant portion of Overlake's medication orders. The
implementation staff will include a pharmacist, a nurse, a half-time
IS person, and a physician, who will devote eight to ten hours per
week to the project.

The nurse, who is still being recruited, will help with order set
development and workflow issues, and participate in training, he said.
One key workflow issue: blending CPOE with a paper chart. One knotty
question raised by the dual system is this: if some physicians place
electronic orders and others hand-write orders, how will the nurse,
the respiratory therapist, and others on the care team know whether an
order has been placed on a patient?

Training will be by the pharmacist and the nurse, who will provide
some instruction and will also follow physicians around until they get
the hang of the system.

Data entry will be by wireless device; Overlake has considered
Pocket PCs, Microsoft Tablet devices, PCs on carts, and wall-mounted
PCs he said. The hospital has no solid answer, and may end up using
more than one type of device.

Mr. Elkington firmly believes he is at a natural advantage in his
CPOE project because Overlake years ago chose Meditech Magic, a
closely integrated system. Other organizations are spending "many
times" what Overlake is because they face the problem of
integrating the underlying systems, he said. He believes that CPOE is
more common among Meditech sites than it is among hospitals that are
building on competing legacy systems “because (Meditech is)
installable."

Overlake's main motive is improvement of patient care, he said.
Overlake is situated in Bellevue, Wash., where major employers are
driving participation in Leapfrog Group reporting. “Leapfrog is
there,” he said, but we’d be doing this anyway...My family
comes here too.”

IV. Additional User Reports:
Survey Results

As noted above: most of those interviewed by Editor Suzanne Corrales
for the Case Studies above were asked some identical basic questions.
However, a core purpose was to let interviewees discuss what they, not
we, deemed significant. As a result, the Case Studies are
wide-ranging, do not offer a common data set or answers to all the
same questions, and follow no specific presentation template.

However, for the section below, we asked system implementers to
respond to one standard set of questions in e-mail and voice
interviews (with some, but little, opportunity to move off-topic).

Researcher Rachel Ross obtained survey results from clinical
implementers or information technology executives at 10 hospitals. The
majority asked not to be identified; therefore, in this section, none
is identified, with these exceptions:

Information from her interview with Montefiore
Medical Center is combined with the information obtained
by Editor Suzanne Corrales in the Montefiore Case Study above and is
not included in the aggregated responses below.

Vanderbilt University Medical Center spent years
developing its own unique and, judging from what we have heard about
it, highly capable system, and then licensed the system to vendor
McKesson for an ongoing commercial adaptation. Because no other user
information is available yet about the system, we concluded that the
Vanderbilt survey response may be of great interest to potential
buyers of a CPOE system. Therefore, the survey
response from Karen Hughart, Vanderbilt's director, systems
support services, is not included in the aggregated responses, but
is presented separately below them.

As some vendors have few CPOE clients, it was necessary to avoid
associating the exact size and CPOE vendor with a particular
institution’s response in order to better preserve their
anonymity. This does, unfortunately, make some responses a bit less
useful than they might be (for example, in making judgments about the
size of an implementation team). However, given the limitations of
promised anonymity, this is the best we could do for this edition.

Survey Questions And Aggregated Responses:

Questions asked on the survey are in boldface. Responses are
in regular type.

One respondent is a community hospital with fewer than 300 beds,
with no residency program. One is a community hospital with fewer than
200 beds, with a residency program. One is a community hospital with
more than 400 beds, with a residency program. One is both a community
hospital and an academic medical center with more than 500 beds. One
is a military hospital with more than 400 beds, with a residency
program. Three identify themselves as academic medical centers; of
these, the smallest has more than 500 beds, and the other two have
more than 800 beds.

2. Status of implementation:

The responding community hospital without a residency program is
just planning its pilot test. One of the community hospitals with a
residency program was just going live on its pilot test at the time of
the survey response. All of the others are live in all or most
departments.

3. Which systems these respondents are using:

Two community hospitals are using the Meditech Provider Order
Management System; two are using Per-Se Patient1. The military
hospital has the CliniComp system, CIS. The academic medical centers
are using Eclipsys Sunrise Clinical Manager, Per-Se Patient1, and two
use Siemens Invision. As noted, Vanderbilt, for which responses are
broken out separately, uses Whiz, an in-house-developed system first
rolled out in 1994.

4. What department did you use for pilot testing? Was it the
right choice? Why or why not? How did you chose it? How long did your
pilot test last?

5. When did you go live on your first non-pilot unit?

Community hospital with no residency program, with fewer than
300 beds, planning pilot: "Proposing the Hospitalist Group -
(provide care to patients who have no family physician)." Planned
go-live of pilot is April, 2003.

Community hospital with residency program, fewer than 200 beds,
currently live on its pilot program: "We selected one patient
floor and one physician. The physician was selected due to the variety
of patients, the close proximity of patients, and most importantly his
willingness to participate in the pilot. Selecting one physician
permitted us to focus in one area and correct problems as they
occurred in a central area. We feel this worked well as the pilot
uncovered some nursing procedure changes. These were easier to manage
on one floor than if we had attempted numerous nursing areas. Our
pilot is ongoing for the last week." Go-live on other units was
two weeks away at the time of this survey.

Community hospital with residency program, over 400 beds: "We
didn't have a pilot unit. We've had CPOE since 1996, and it was always
"live." We started with resident order entry in 1996, and
the resident population (medical and surgical) now average 95% direct
order entry.

"For the attending population, we concentrated on physician
direct order entry in September, 2000 for a specific unit in our
Inpatient Behavioral Health Division...They were a smaller, more
controlled group of physicians (mainly psychiatrists) whose order sets
were standardized according to their specifications, and the
medication order sets were not as complex as the medical or surgical
attendings.

"...We (IS Department) provided a
minimum of 3 weeks of 24x7 support for both the physicians and the
nurses. We had an issues list, and we worked with those groups to
correct their issues for at least the next 6 months."

Community hospital with residency program which is also an
academic medical center, 500+ beds: "Pilot test with Lab
system about 10 yrs. ago. Lab dept. was chosen because their contract
with CHC (Community Health Computing, a lab system vendor at the time)
was ending and it was a logical jump at the time. Test lasted for 6
months."

Go-live beyond the pilot: "Six months after lab dept. went live
with CPOE, we went live with the order entry piece and then the
pharmacy piece."

Military hospital with residency program (respondent had worked
on a CPOE implementation at a prior military hospital): "At
the previous facility, I used L&D (labor and delivery) & NICU
(neonatal intensive care) for 6-weeks. However, the hospital stopped
deploying CPOE after I left. Nurses are still, for the most part,
transcribing from paper orders. The NICU & L&D are still the
only units using CPOE. These units were chosen because the population
is somewhat contained, and business processes don't filter to the
other units with transfers, etc. Reasons are patient care continuity
and SAFETY!

"...[M]y current employer (also a
military hospital) initiated CIS with the providers entering orders
from the very beginning. When CIS was implemented, so was CPOE."

Academic medical center with 500+ beds: "Piloted on a
general medicine unit to ensure the largest breadth of goods/services
ordered (particularly medication orders). It was the right choice.
Pilot lasted about 3 weeks." This hospital went live on first
non-pilot unit in February, 2001.

One of the academic medical centers with 800+ beds: "We
did our pilot test in the transplant unit because it represents a good
mix of all kinds of patients. It's like a mini-hospital that enabled
us to address all ancillary areas, and iron out related medical
issues. It was the right choice because of the variety of patients.
Pilot lasted a good three months before rolling it out to rest of our
departments...We went live in all units in May 2002.

"After
our success with the pilot in the transplant department, we didn't
want to go unit by unit in a piecemeal way. We went fully live in all
units, all ancillary areas the same day. Sort of a big bang approach!
Part of the reason people fail at implementation is because they do it
unit by unit and the system has less adoption. If you go unit by unit,
doctors have to order meds in two different kinds of systems. The
frustration in the healthcare environment is stressful enough, but if
you put two different processes in front of a physician, forget it,
they won't use it. Once burned, twice shy, so to speak. They won't use
it. We have to keep it simple."

The other academic medical center with 800+ beds: "We
piloted using the inpatient General Medicine Unit. This unit
represented a wide cross-section of ordering patterns that allowed us
to identify and remediate the majority of issues quickly. Pilot
testing lasted for approximately one month, followed by a phased
implementation in other inpatient care areas.

"The project was initiated in Spring 1993 and went live on our
first non-pilot unit in Summer 1996."

6. When you went live, did you have --

A--a pharmacy system that was integrated with your physician
order entry system?

B--a medication administration record that was integrated with
your CPOE system?

Community hospital with no residency program with fewer than
300 beds, planning pilot: does have integrated pharmacy, does not
have integrated meds administration. it's "planned for future but
not feasible now since we need hand-held devices for nursing."

Community/military hospitals with residency programs:

Three did have integrated pharmacy, one did not.

Three did have integrated meds administration, one did not (a
slightly different subset than the three above).

7. Of the total number of physicians who practice at your
hospital, what percentage currently enter their own orders through
your CPOE system?

Community hospital with no residency program with fewer than
300 beds, planning pilot: "too early to tell."

Community hospitals with residency programs: the one which
has fewer than 200 beds is "still piloting with one physician,"
and at the 400-bed hospital which has been live since 1996, "90-95%
of docs do order entry online."

Community hospital with residency program/academic medical
center, over 500 beds: "Residents are at 95% and the
attendings are at 40%."

Military hospital with 400+ beds: "100% for inpatient.
There are areas which have such complex order sets [that] they remain
on paper (ex: Chemo, Dialysis). We are trying to upgrade to accomplish
this also."

8. What is your ratio of data entry devices to patient beds? Is
it adequate?

Community hospital with no residency program, fewer than 300
beds, planning pilot: "We have almost one device for every
patient bed and anticipate an increased demand once we move beyond the
pilot group, which is a very small group of physicians."

Community hospital with a residency program, fewer than 200
beds, currently in pilot: this hospital is using the pilot project
to test the appropriate number.

Another community hospital with a residency program: "1
PC for every 2 beds at our 600-bed hospital. But that is just in the
hospital. There are more PCs in the clinics."

Community hospital with residency program, over 400 beds: "We
have bedside, point of care devices in every patient room in addition
to having several devices at every nursing station and substation. We
can use more at the nursing stations and areas where the MDs
congregate."

The military hospital with 400+ beds: "1 to 1 on all
inpatient units, plus some! Providers want more. Not all physicians
have access to CIS on their office PCs and/or in the outpatient clinic
areas. However, they DO desire this. We are working on it."

Academic medical center with 500+ beds: its ratio is 1 to
2.5 beds. Its response did not comment on adequacy.

One academic medical center with 800+ beds: "Hard to
say because our stationary PCs are used for so many different
computerization systems. But at every nursing station there are a
least a minimum of two PCs, that are used for CPOE, among other
things."

The other academic medical center with 800+ beds: "The
ratio of data entry devices to patient beds is approximately 1:4 on
the inpatient units. However, a positive aspect of CPOE is off-unit
access to orders which allows physicians to manage patient care from
their office computers, or any network-connected computer."

One community hospital with a residency program: "100%--although
they could use desktop if desired."

Another community hospital with a residency program: "Only
10%. Wireless devices used mostly in surgery dept."

Community hospital with residency program, over 400 beds: "We
do not use PDAs for order entry, but we do have our home health RNs
set up to do documentation and order entry on laptops that use CDPD
wireless technology."

The military hospital with 400+ beds: currently, zero.

Among the academic medical centers:

The 500+-bed medical center said zero.

One of the two very large medical centers with 800+ beds said, "128
wireless laptops that roam around the hospital for physician order
entry. We have 128 dedicated wireless devices to 650 staff beds."

The other: "On the inpatient units, approximately 40% of
the devices are wireless- enabled laptops affixed to rolling carts.
In our outpatient setting, only a small percentage of devices in use
are wireless. At this time, we do not support wireless hand-held
devices such as PDAs or tablet computers."

10. What kind of training did you use for physicians--Computer
based? Classroom? One-on-one? Combination? And what worked best?

Community hospital with no residency program, with fewer than
300 beds, planning pilot: "A combination of classroom,
one-to-one and printed manual is planned."

A community hospital with a residency program: "We did
one-on-one training. This provides a better training method and works
well trying to accommodate physician schedules."

Community hospital with residency program, over 400 beds: "A
combination of classroom and one-on-one. Training is a hurdle. Docs
seem to be embarrassed to be seen by peers as "not knowing"
something in a classroom situation. They don't want to be seen as
being behind in any area. It doesn't matter what schedule of classes
we offered--morning, noon, night, the docs stopped coming after a
while.

"We found we got the best results from private sessions with
docs. Also, we make rounds once a day to try to catch docs actually
doing their order entry at the computer. Then we approach them, make a
friendly connection through conversation, and try to create a "teachable
moment" when they are on the job, so to speak.

"We
also provide a separate phone line, just for docs, to get help for
non-urgent questions. They can leave voice mail if they are just
wondering how something works. We e-mail them back (documentation of
response) and/or call. Docs seem to like the conversation piece. For
urgent needs, we have a 24-hour beeper help line. If they are stuck on
the system and need to prescribe meds right away, they beep us and we
help them on the spot."

The military hospital with 400+ beds: "A combination of
classroom and a lot of e-mail infomercials--physicians are extremely
reluctant to come to a classroom. At the last hospital, I had
super-users (nurses), who understood the system due to the
transcription of paper, teach the physicians for 6-8 weeks prior to
going live."

The community hospital which identifies itself as also an
academic medical center, 500+ beds: "We used a combination of
all three formats. Sometimes we have no other choice due to the
doctor's schedule. We usually need to do one-on-one with ER doctors,
for example, because they're working around the clock. We also
provided Internet self-paced learning. But the classroom format proved
to be the most popular, and worked best because it offered more
hands-on training."

The academic medical center with 500+ beds: "a
combination of classroom and 1:1."

One of the academic medical centers with 800+ beds: "There
were mandatory computer classes for doctors to retain privileges at
the hospital. We staggered the training throughout the implementation
period. We used a combination of mandatory computer classes and we
also trained "super users" from each department, and several
were trained to roam the halls and find out if a physician was having
trouble, and provide one-on-one help to physician. So we always had
one super user per floor but within a month, the demand plummeted
because the system is so easy to use, people got used to it right
away. A lot of hospitals that provide only classroom training make a
mistake. It's the real-life scenarios that require the most help, and
that is the best place to apply the learning emphasis."

The other academic medical center with 800+ beds: "A
combination of classroom and one-on-one. Medical students are taught
CPOE in classroom sessions as part of their introduction to UTMB. We
also arrange for one-on-one training at the convenience of the medical
staff, either as full training sessions or "refresher"
courses."

11. Did you go live with departmental order sets? Departmental
and personal? Has this changed since you went live? Does it depend on
the department? Which uses which?

Community hospital with no residency program, with fewer than
300 beds, planning pilot: "Both-- we will go live with
personal (physician created) order sets, and some order sets that have
been predetermined."

The community hospital with a residency program which was in its
pilot test at the time of the August, 2002 survey: "We are
using both departmental and physician order sets. It's to early in our
process to tell if changes will be necessary."

Community hospital with residency program, over 400 beds: "We
use a combination of departmental and personal order sets. Any order
sets are kinetic and we make adjustments to personal order sets when
requests come through. Revisions are very common requests from the
physicians. As they use the system, they are finding what works easily
for them versus what is too cumbersome. Overall, many of the changes
made for departments are because of regulatory requirements,
especially in the pharmacy area."

Community hospital with residency program which is also an
academic medical center, 500+ beds: "Combination. It really
does depend on the department. Changes are bases on the coding books.
If they change or add new codes, we've got to update the order sets.
If you're a doctor that's a specialist, you have your own personal
order sets designed. We accommodate them based on policy and
procedure. It's really very user friendly, that is the key. Otherwise
doctors won't use it at all."

Military hospital with 400+ beds and one academic medical
center: both replied that they did go live with departmental order
sets, but didn't elaborate.

The academic medical center with 500+ beds: “yes”
to departmental order sets.

One of the two very large academic medical centers with 800+
beds: "Order sets are organized to be service-specific and
then they're ailment-specific. So there's specific order sets for
cardiology, and another one for NICU. We have about 400+ different
order sets for various ailments and conditions. We encourage
practicing evidence-based medicine, best-practices medicine to contain
costs and ensure the patient gets the best possible care. Physicians
do have ability to modify, but that doesn't alter it permanently. We
have accountability to our patients to ensure they get the best
possible care so when a physician wants to change something, we go
back to evidence-based medicine. If they really believe that their way
of practicing medicine is better, they present it to the ongoing order
set evaluation committee. So, if it makes sense, we change it for the
baseline."

The other academic medical center with 800+ beds: "Yes,
we utilize departmental order sets extensively. We do not support
personal order sets. The use of departmental order sets and protocols
was a key factor in acceptance and subsequent success of CPOE."

Community hospital with no residency program with fewer than 300
beds, planning pilot: "Our organization will do some tweaking
of the rules" (presumably a standard set of vendor-supplied
decision-support rules).

The community hospital with a residency program which is still
in its pilot: "We are using the vendor-supplied interactions
and adverse drug checking."

The community hospital/academic medical center, 500+ beds: "Most
rules are for efficiency and data compliance. (e.g., Is the order
complete? Logical? For example, system might flag pregnancy test on a
man because it is illogical.) "We use Medi-Span, which is
integrated with Patient1 pharmacy module. It has, for example, IV
interaction alerts. Also, if a patient is lactose-intolerant, it will
signal if a drug has a lactose derivative. But you have to set the
different settings. If you want it at the most sensitive, then it will
bleep at anything. But if you just want the most critical, then it
will only bleep at the most critical reaction."

Community hospital with residency program, over 400 beds: it
is using out-of-the-box medication checking rules provided by a
vendor, its own adverse drug-event checking rules, and also something
called "criteria engineers," which the respondent describes
as "a kind of program logic that we build into the system as part
of our HIS system."

One of the academic medical centers with 800+ beds: "We
jointly developed rules with Micromedex. We were one of the
development sites. Most of our rules were developed by us. Siemens
didn't provide any rules in its software. We developed all of our
checking rules internally, as a partner with Micromedex."

The other very large academic medical center: [the medical
center] uses a combination of vendor and locally-written rules for
medication interaction/allergy checking; ADE checking; medication
dosage, timing and scales per unit validation."

Community hospital with no residency program, with fewer than
300 beds, planning pilot: it is planning on these figures:

Physician: .48 FTE total, varies over 20 weeks;

Nurse: "to be determined once rollout begins, initial
impact will be very minimal;"

Pharmacists: .53 FTE total, varies over 20 weeks

Programmer/analysts: .53 FTE total, varies over 20 weeks.

The community hospital with residency program which is in its
pilot program: "We had a team for the implementation but
primarily our clinical analysts worked on the project. To estimate, I
would say 1.3 FTE. The project length is hard to estimate since we had
to wait mid project for our vendor to make some necessary changes.
Total would be about 8 months."

The community hospital/academic medical center with 500+ beds:
"On a go-live basis, 4 to 5 of each of the above. We did what we
call "train the trainer." We train key personnel intensively
so that when we're not there, they can troubleshoot and support the
others in their department using it. We're a team, in a sense, but we
don't have to be there because these users are the first line of
defense. They can usually handle any IT or help a doctor when he gets
stumped, but if they can't figure it out, then they're usually the
ones to call us. It works well with these super-users because without
them, the others would be calling us all the time and we wouldn't get
anything done."

Community hospital with residency program, over 400 beds:

-Physicians: none;

-Nurses: 4 IS RNs;

-Pharmacist: one part time in IS

-Programmer/analyst: 1 lab analyst.

"There was no real time line - in 1996, when the
residents began order entry, that project lasted about 9 months."

The military hospital with 400+ beds: "Initially, there
were 5 assigned to deploy CIS, then reduced to 4. Now, we are hiring
the database administrator (a 1st) to review order entry processes
(overrides, etc.). We handle all our own training because there are so
many ongoing changes to CIS, which have an impact on everyone."

One of the academic medical centers with 800+ beds: "During
implementation, we had about 20 FTEs, ongoing super user groups, and
about 10 consulting physicians who met with us about once every two
weeks. Now that we're fully live, our support personnel is much less."

The other academic medical center with 800+ beds:

Physicians "participated as members of Steering Committee."

Nurses "participated as members of Steering Committee,
varied as new areas were brought up on POE."

Pharmacists "participated as members of Steering Committee."

Programmer/analysts: "initially, a team of 12
programmer/analysts developed, implemented, and supported the
system. Currently, a team of six provides enhancement and support."

14. If your project has been live long enough for you to see
payoffs (benefits), what were the first you saw? How much time went by
before you saw them?

Community hospital/academic medical center: "The
payoffs are immediate in dollars saved. That shows within 6 months to
a year. Everything is documented, [and so the] onus is on everyone who
has access to the system. It documents and tracks what they do. The
doctor can't say that the nurse didn't understand his writing. Even if
the doctor can't spell, he selects from clear options, so there's no
spelling errors. There's not a lot of room for major mistakes, so if
one is made, it's clear who's responsible. The individual
accountability is right there; each person in each department is held
accountable. Some people are afraid of that, but it's good for the
hospital, and especially for the patients."

Community hospital with residency program, over 400 beds: "The
RNs like the fact that they do not have to deal with the bad
handwriting or calling for clarification. Pharmacy likewise, but they
are also more concerned with seeing fewer medication errors. They also
like the fact that the docs see the drug interaction alerts and the
duplicate check messages when they direct-order."

Military hospital with 400+ beds with residency program: "There
is a more integrated operation with nursing, physicians, pharmacy, and
even medical records and JAG--much more of a team effort in patient
care and better understanding of how we all impact each other's
processes."

[Editor's note: while this respondent didn't give an indication of
the cost significance of conversion of intravenous medication to a
patient taking a medication orally, we did find this piece of
anecdotal information at the University of Kentucky Hospital's web
site: "The conversion of each patient receiving IV famotidine to
the oral form just one day earlier is projected to save the hospital
approximately $8,000." --
http://www.mc.uky.edu/pharmacy/dic/interchange/famotidineIC.htm.]

One of the academic medical centers with 800+ beds: "We
saw payoffs in the pilot phase. They were better access to care and
length-of-stay reduction (LOS). At 6 months post-live, we saw more
benefits:

Turnaround time from prescribing a medication to getting it into
the patient improved by 66%, reducing it from about 7 hours to about
3 hours.

Radiology exam turnaround improved by 45%. Finishing the tests,
getting the results into the system improved 25%.

In crucial areas, like cardiology, LOS reductions were favorably
measured. There was a 25% reduction in medical errors
administration.

Transcription errors went to ZERO."

The other academic medical center with 800+ beds: "Initial
payoffs seen almost immediately were:

use of printed requisitions to ancillaries resulted in less
confusion and time spent transporting requests;

standardized data collection resulted in fewer call-backs to the
floor for additional information;

15. All things being equal, knowing what you know now, what
would you have done differently?

Community hospital with residency program, over 400 beds: "We
should have had a better rollout plan. The hospital administration
mandated to the "captive" MDs that they would need to do
direct order entry hospital-wide beginning in January, 2002, but we
weren't prepared with all the doctors' order sets or education.
Doctors resisted--there is a large cultural change that must be taken
into consideration--and you need enough resources to be a presence on
the units to support the docs."

Community hospital/academic medical center: "The
initial idea was to copy and scan all charts on line, but because
we're in NY, there are strong unions, and this would have eliminated
lots of medical clerks' jobs. There was a big fight about that, so we
decided to do it slowly. But eventually, it will be paperless--that's
our goal. You won't need the paper chart. It will all be electronic.
We have a back-up generator, we've got the necessary safety mechanisms
in place."

Military hospital with 400+ beds, with residency program: "I
would have, if empowered to do so, insisted that the first hospital I
worked at with CIS implement the system the way [the second one] did,
and direct the physicians to put the orders in."

One of the academic medical centers with 800+ beds: "We
used a mainframe-based, non-Windows system, which took almost 3 years
of planning because we wanted to go live in all departments at once.
Today's systems are equipment are much more flexible. Adoption would
have been easier. Selling the idea and regular use to doctors with
today's technology, which is web-based, robust, user-friendly, would
have been easier. Vendors have matured a little. They used to build
clinical systems on business platforms but now they're building them
from scratch and designing them clearly for clinical settings. That's
an improvement."

The other academic medical center with 800+ beds: "We
would probably do very little differently. I would recommend
compressing the implementation schedule as much as possible, as it was
difficult for medical staff during transition between "POE Units"
and "non-POE Units."

"An
in-house system called Whiz, which the medical center sold to
McKesson. The medical center will be installing the McKesson Horizon
version of the system within the year."

4. What department did you use for pilot testing? How did you
choose it? Was it the right choice? Why or why not? How long did your
pilot test last?

"Our pilot units for Whiz Order were the Cardiac ICU and
Stepdown units. We had strong physician leadership, a homogenous group
of patients, and strong nursing leadership on the units. The patients
on the two units transferred between each other so we didn't have such
a big problem with patients going from implemented to non-implemented
units by starting there.

"We also got very early
experience with both ICU and lower acuity patients. In hindsight, this
was a good choice, because we had an excellent chance for success. We
had assurances that the Nursing and Physician leadership would not
fold at the first signs of adversity. They made a commitment to work
with us till we got it right. We implemented the next unit (Adult
Myelosuppression unit) approximately 2 months after the go-live date
in Cardiology."

6. When you went live, did you have --

"No. We had an MAR computer
generated from the Pharmacy Computer system that was manually
completed. (We still do.)"

7. Of the total number of physicians who practice at your
hospital, what percentage (estimate) currently enter their own orders
through your CPOE system?

"There are approximately 600 attending physicians and an
additional 300 residents and fellows. Approximately 68-70% of all
orders are entered by physicians. Because we are a teaching hospital,
the overwhelming majority of all orders entered by physicians are
entered by house staff.

"The 30% of orders
entered by others are entered by nurse practitioners and staff nurses
(as verbal, telephone, protocol, or nursing orders not requiring MD
countersignature; by pharmacists, respiratory therapists, dieticians,
and other professional disciplines who are permitted by policy to
enter verbal, phone, and protocol order for their own disciplines; and
by clerical personnel who may enter written orders."

8. What is your ratio of data entry devices to patient beds? Is
it adequate?

"We have 1-2 patients per computer in our ICUs, approximately 3
patients per computer in Stepdown areas, and 4-5 patients per computer
in general care areas. There are also computers in ancillary
departments, the cafeteria, etc. We do use some wireless laptops on
rolling carts to enter orders during rounds in a few units.

"Especially on medicine units, which tend to have physicians
present more hours of the day, we do have some wait times for
computers during busy times of the day. We would put more devices in
these areas if we had sufficient space. Except for these few areas,
the numbers are generally adequate.

9. What percentage of clinician order entry is on wireless
devices?

"We do use some wireless laptops on
rolling carts to enter orders during rounds in a few units."

10. What kind of training did you use for physicians? What
worked best?

"A combination. Except during
initial rollout, we've trained all house staff in classroom training
sessions. They give us one day during house staff and fellow
orientation and we train 50-150 people each day once per year. We like
to train groups of attending physicians, too, but don't have much luck
with this. Most of them get trained in one-on-one or very small groups
of 2-3 physicians at a time."

11. Did you go live with departmental order sets? A combination
of departmental and personal order sets?

"Yes, we started from the beginning with 'best practice order
sets' that an entire group was encouraged to use. We also had 'Private
Whiz,' which allowed each user to develop up to ten

personal order sets for use when a 'best practice' order set was not
available or to enter orders for patients not yet registered that
could be 'played back' once the patient was registered."

Has this changed since you went live?

"We are still doing the same."

Does it depend on the department?

"Some
departments are more proactive in maintenance of best practice order
sets than others, so in cases where the order sets are out of date,
they are not used as often."

12. What use are you making of on-line decision support?

"We have access to commercial (AHFI and Moseby) Drug References
and hospital-developed monographs from within the order entry system.
We have dose range checks for certain classes of drugs for adults and
most all pediatric medications. We also have more sophisticated drug
dosing decision support for titratatable drips, antibiotics, and
several other types of drugs used by children that consider weight,
age, and creatinine clearance (if lab data is available). There are
lots of other drug-specific decision support tools incorporated into
Whiz to address specific issues (e.g., an empiric antibiotic advisor
that shows cost adjusted rankings for therapeutically equivalent
therapies, interventions to advise users away from drugs that are
non-formulary, etc.)

"Our radiologists have
worked out several decision support tools to help users order the best
test to diagnose a problem to help steer users away from ordering both
CAT scans and MRIs when only one or the other is warranted. We have
decision support to help users order restraints in a compliant
fashion, easily initiate appropriate Latex precautions orders,
implement fall prevention orders appropriately, target skin prevention
and skin care orders based on skin risk assessment, specify reasons
for blood transfusion, and special preparation of blood bank products,
etc. "There's a LOT of decision support built into Whiz."

13. How many full-time employees were assigned to your project?
For how long?

A--Physician: "For most of the rollout, the
physician-programmer who created Whiz was on the project full-time.
For the past two years when we have not been actively implementing new
users, we have had several part-time MDs involved in the project, but
none focus on POE full-time."

B--Nurse: "There have been 5-6 nurses focusing on Whiz
as their sole or primary project since the beginning. This group is
responsible for implementation planning, training, implementation
support, and on-going support for all Whiz users. They also develop
order sets and do some simple systems maintenance."

C--Pharmacist: "Two pharmacist spend very significant
percentages of their time helping maximize pharmacy impact from Whiz.
A third, a chemotherapy pharmacist, maintains all chemotherapy order
sets."

D--Programmer/analyst: "The numbers have varied, and
there has been regular involvement from biomedical informatics
students throughout the project, but there are an average of 2
programmers and 1 analyst that work full-time on POE. Part-time help
that might add up to another FTE develops modules and does other
smaller pieces of work."

14. If your project has been live long enough, what were the
first "payoffs" (benefits) you saw, and how much time passed
before you saw them?

"We realized some benefits almost immediately. We are able to
deliver first doses of medications 17% faster for ICU and 38% faster
for general care patients if orders are entered directly by
physicians. Orders are legible and we've been able to alter cumbersome
work processes by taking advantage of the computer's ability to
produce an accurate "snapshot" of active orders on demand.
Nurses don't have to manually update Kardexes. We can review current
orders when patients change service or level of care and make changes
as needed instead of starting from scratch. Physicians, nurses, and
other members of the team are all working from a common set of orders
rather than from individual perceptions of what's active and what's
not.

"Other benefits came along somewhat later after we had a solid
user base. We generally think it's best to get people using the
system, then give them data, and then work on interventions rather
than start with new users with interventions to change practice
dramatically from day #1.

"We have used data from
Whiz to identify areas with over- or inappropriate utilization of
services. We've then been able to target interventions to address
these situations without adversely impacting quality of care. For
example, our clinical experts have looked at patterns for ordering lab
tests, portable chest x-rays, and other heavily-used services, and
made changes in the system to curtail this. You can no longer order
open-ended orders for these targeted services, but must instead review
orders every 24 hours in most cases. This targeted way of cutting cost
has been done with no identifiable impact on the quality of care, but
has saved hundreds of thousands of dollars. We've been able to provide
decision support to address quality and cost issues for a variety of
services (some mentioned previously)."

15. All things being equal, knowing what you know now, what
would you have done differently?

"We would have been more realistic about ongoing support needs.
It takes a lot of time, expertise, and effort to support clinicians
and to maintain up-to-date clinical systems. The more users and areas
implemented, the heavier the ongoing support burden.

"We underestimated how different pediatrics was, and spent over
3 years trying to develop the medication ordering infrastructure to
support their needs.

"We've been slow to respond to the increasing blur between
inpatient and outpatient status and to develop the application to
support both. Now that we are working with the ED, the Clinical
Research Center, and other patient care areas that treat large numbers
of outpatients, we are finally making system changes we've needed
earlier to better support appropriate ordering for outpatients that
are in inpatient beds for observation."

V. Additional Topics:

Six Approaches To Winning Physician Acceptance of CPOE

Hand-Helds: Many Physicians Do Like Them--But Only for Limited
Purposes

A Cautionary Tale: Pharmacy Alerts Shut Off

Six Approaches To Winning
Physician Acceptance of CPOE:

Five of the six organizations discussed below use persuasions based
largely on helping physicians to find and appreciate the benefits of
CPOE. In the sixth--a prominent hospital loaded with "physicians
to the stars," one of whom made the cover of Time Magazine--the
approach is, surprisingly, "You WILL use it." These
hospitals are:

In most situations, entering most orders into a computer will take a
physician extra time, which physicians will be loath to give up. So,
if winning physician acceptance of computerized physician order entry
(CPOE) is tough in a teaching hospital, where many physicians are
employees, what should a community hospital do, when most physicians
aren't employees?

Bill Van Doornik, MIS director at 213-bed Holland, Mich., Community
Hospital, plans to install the Per-Se Patient1 CPOE system. He agrees
with estimates that CPOE will take physicians an additional 30 minutes
per day. Gary Strong, CIO at nine-hospital Fairview Health System,
Minneapolis, Minn., guesses that it will add one minute of physician
time to an order that would normally take two minutes to write. He's
installing Eclipsys Sunrise.

The solution, they and other project leaders agreed, is to make the
technology so compelling, the order sets so easy to set up, and the
decision support so useful, that physicians begin to demand it. "You
have to make the benefits visible," said Bruce Elkington, CIO,
Overlake Hospital, Bellevue, Wash., who is installing Meditech Magic.

In some cases, circumstances will help. Overlake has a staff of
hospital-based intensivists and hospitalists who are responsible for
writing a significant portion of Overlake's orders. For some reason,
it also has a disproportionate number of technically-oriented
physicians on its OB/Gyn staff. It will beta-test physician pharmacy
order entry in the ICU, and probably roll out first to Ob/Gyn, Mr.
Elkington said.

Fairview Health System operates a large ambulatory care practice,
which is rolling out Epic Systems EMR. Physicians who are already
familiar with the Epic system come to the community hospital more
willing and eager to computerize than they otherwise might be, because
they are familiar with the benefits, Mr. Strong said. But he is also
out there doing old-fashioned sales pitches. He estimates that he has
participated in at least 100 meetings with physicians and physician
groups about CPOE.

Mr. Elkington figures that he has a natural advantage because
Overlake chose Meditech Magic, a closely integrated and relatively
inexpensive platform, years ago. Other organizations are spending "many
times" what Overlake is because they face the problem of
integrating the underlying systems, he said. If you look at the First
Consulting Group Leapfrog report, you notice that a lot of Meditech
sites have CPOE, he said. "That's because it's installable."

Great Plains Regional Medical Center, 116 beds, North Platte, Neb.,
has a physician project champion who just happens to be married to the
woman who was, until recently, the Great Plains CIO, Bonnie Buckland.

Not surprisingly, Robert Buckland, MD, sees plenty of advantages for
physicians in CPOE. One obvious one: they'll be spared the annoyance
of puzzling over each other's handwriting. As a surgeon, Dr. Buckland
often walks into a patient room cold and picks up the threads of what
another physician has started. Being able to consult a legible,
well-organized medical record and tell at a glance what is going on
will be a big improvement, he said.

Not being interrupted during the day for order clarifications, and
not having to repeat orders or prolong patient stays because someone
didn't understand what the MD meant in the first place are other major
benefits for physicians, he said.

277-bed Cambridge Memorial will try to document that benefit for its
pharmacy and medical staffs. Cambridge pharmacists will manually log
the amount of time they spend on the phone with physicians, both
before and after installation of Meditech Magic CPOE, said Rena
Burkholder, CIO.

Cheryl MacInnis, Cambridge director of clinical pharmacy, also
conducted an extensive literature search on the benefits of CPOE. Ms.
MacInnis summarized some 50 CPOE research articles and presented them,
along with the original articles, to the medical staff. It was
well-received, she said.

Cambridge will also try to boost physician acceptance by paying
physicians at their regular rates of pay, for time they spend on the
hospital's Meditech Magic CPOE project. Cambridge received an $89,000
grant from the Ontario Hospital Association's Change Foundation. The
technical quality of underlying systems is the one critical success
factor that is heavily within the control of the IT department, so
project leaders are focusing on it. For example, Holland Community
beta tested an earlier version of the Per-Se order entry system and
found the screen flow cumbersome, Mr. Van Doornik said. Per-Se has
rewritten the module using Java, and Holland plans a second beta test.
"We are building them the very best system we can."

Integration of the underlying system is also key. Dr. Buckland, who
is installing QuadraMed Affinity, and Mr. Van Doornik said having a
unified, underlying system architecture is a crucial first step. "Without
that, you just couldn't do this," Mr. Van Doornik said.

Cedars Sinai Policy Suspends
Physicians Who Don't Train For CPOE

In what seems to us to be a bold move, Cedars Sinai Medical Center
is planning to suspend physicians if they don't finish either a 2-hour
classroom training course or a self-paced computer assisted learning
module on the hospital's new computerized physician order entry (CPOE)
system before their units go live.

Physicians do receive a benefit--CME credits--for the training,
which is set into stone as a requirement in Cedars Sinai's medical
staff bylaws.

The strategy seems to be working. Virtually all physicians had
completed the course by the time a pilot test went live in the fall of
2002. Of two who had not, one was retiring and the other had moved
away, said Michael Shabot, M.D., medical director of enterprise
information systems.

The new system, Patient Care Expert (PCX), is a home-grown patient
management, patient accounting, and clinician order entry system.
Physician use of the system is mandatory. Paper orders will not be
accepted once the system goes live. Physicians who have not taken the
training will not be able to give telephone orders, because their
names will not be in the system. Cedars officials said that medical
staff cooperation on the training requirement is actually fairly high
because the system was built largely to their specifications.

Hand-Helds: Many
Physicians Do Like Them, But Only for Limited PurposesMost Are
Using Devices To Consult EPocrates

Just because many members of your medical staff carry Palm Pilots
doesn’t meant that they are necessarily ready to accept hand-held
applications or would vote for hand-helds as the device of choice for
CPOE. Those are our conclusion after listening to a presentation by
Neil Smithline, M.D., and conversing with him briefly.

Dr. Smithline consults on hand-held deployments. He says the
overwhelming majority of physicians who use hand-held computers use
the simplest of all applications, EPocrates, the Physician Desk
Reference, with automated drug interaction checking and regular
on-line updates via the Internet. EPocrates is simple, convenient, and
free to physicians. Some 170,000 physicians, or one out of five in the
U.S., use it. Patient Keeper, another vendor, offers a $35 application
that allows physicians to manually enter and store patient
demographics, histories and physicals, lab data, and other clinical
data on a Palm. Patient Keeper claims 45,000 users of that
application.

Now take all other major vendors of wireless applications:
Allscripts, MDeverywhere, e-Physician, and OnCallData--applications
which do more and cost more. They may have 10,000-15,000 users
combined, Dr. Smithline said.

His bottom line: physician adoption of hand-helds has been
slow because physicians are not yet convinced that the benefits they
offer are worth the money and the trouble. The more features
hand-helds have, the more complex they are, and with complexity come
costs and technical challenges.

Some physicians do use hand-helds for CPOE in a few hospitals.
However, given Dr. Smithline's observations, and the relative
difficulty of performing complex decision-tree tasks on a small
device, we believe that at present, synchronized wireless devices of
notebook size or smaller are more likely to end up being popular as
data retrieval, limited prescription-writing, and transcription
devices for physicians on the move than as full-blown order entry
units. (There are, of course, exceptions, as noted below.)

Operating system: The leaders are Palm and Windows CE. At
present, Palm is by far the most popular with physicians.

Wireless vs. cradle synchronization: at the time of his
presentation, Allscripts, MDeverywhere, and OnCallData offered
wireless synchronization; Allscripts, MDeverywhere, e-Physician,
MedAptus, and I-Scribe offered cradle synchronization. The
advantages of wireless are that it is real-time, and that it reduces
the load on devices with limited data storage. Wireless costs are
coming down and response times are good, but security is often only
so-so, he said.

Integration: if the hand-held is not integrated with the system
that stores patient demographic data, it will not be used, he said.

Here's his rundown on the major hand-held applications, what they
do, and what their limitations may be:

Prescribing: major vendors of hand-held prescribing
applications include Allscripts, Touch Works, OnCallData, and
e-Physician. While having physicians enter their own orders directly
into a hand-held can theoretically help reduce medical errors, the
system is only as good as the data that has been entered into it.
Hand-held applications can, for example, provide drug allergy
checking, but only if the patient's allergies have been entered into
the system in the first place. Similarly, drug interaction checking
works only insofar as patient medications are in the system.

Transcription: major vendors of hand-held transcription
applications include Allscripts, Touch Works, MDeverywhere, and
MedAptus. Early dictation systems were clumsy because the hardware did
not support four-button functionality (fast-forward, rewind, etc.);
more recently vendors have built this capabilities into the software.

Charge capture: major vendors of hand-held charge capture
applications as of June, 2002, included Allscripts Touch Works,
e-Physician, MDeverywhere, and MedAptus. Charges are less likely to be
lost with such a system. Also, to the extent that the system builds in
insurance company rules, charges are less likely to be rejected by the
insurance company.

Six to one return on investment claimed

Studies publicized by MDeverywhere on its own system claim returns
on investment of 6 to 1 and sometimes higher, but they have involved
academic medical centers and may not hold true for small practices,
Dr. Smithline said. His reasoning: while salaried physicians can
afford to be casual about billing, physicians whose livelihoods depend
on accurate billing are probably already doing a better job, and
therefore may have less room for improvement from solely using a more
convenient charge-capture device.

He predicts that hand-held applications will evolve from today's
stand-alone systems to a more integrated approach. The evolution is
already occurring. Examples include:

Patient Keeper has developed a common platform that includes modules
for charge capture, lab data review, and problem lists, and it
supports legacy applications. Cerner Corp. recently announced plans to
build on the Patient Keeper platform.

Epic: arguably the leader in electronic medical record systems, Epic
is developing a hand-held interface to its EMR. It will use either
cradle or wireless synchronization. Charge capture is the first
functionality it will offer.

Microsoft Believes It Sees A
Future For Tablet Devices

We do note, however, that just as we completed the first edition of
this study, Microsoft held a press conference in which it announced
the Tablet PC, a Microsoft name for its Windows XP Tablet PC Edition
software running on a variety of third-party wireless devices. Two
hospital software vendors, Allscripts and Eclipsys, announced
physician order entry applications on this operating system and
hardware. Here’s more information, from a Microsoft announcement
(see
http://www.microsoft.com/presspass/press/2002/nov02/11-07Tablet2002LaunchPR.asp):

"Tablet
PCs are available in two primary designs: One features an attached
keyboard and can be configured in the traditional laptop "clamshell"
mode, and the other uses a variety of detachable keyboard designs in a
so-called "slate" form factor. All Tablet PCs are designed
to be a user's primary business PC, recognizing input from a keyboard,
mouse or pen. Tablet PCs are powered by chips optimized for low power
consumption and longer life from Intel Corp., Transmeta Corp. and Via
Technologies Inc. Tablet PCs are available at retailers throughout the
United States. Final retail prices are being announced by each
manufacturer."

Whether the Tablet PC or something like it becomes a new CPOE device
of choice, or turns into the latest clinical computing device to find
its use as a paperweight like the AT&T PNut or a coat-rack like
the IBM Clinical Workstation, only time and use will tell.

In a test which may offer an implied warning for organizations
implementing any clinical decision-support system, it was found that
most pharmacists had turned off alerting systems and safety checks in
their software because it interfered with getting their work done.

Perhaps it is arguable whether the response of a pharmacist to
annoyances in his/her computer system necessarily imply that a
physician, or a nurse, would behave similarly. However, it is not
unknown for physicians to loudly object--and even to sabotage --
information systems which they perceive as a serious interference with
their ability to effectively treat patients. (One of the co-authors of
this report wrote about a case in which military physicians began
unplugging terminals and reporting them down when superiors tried to
force a faulty order entry system on them in the early 1990s.)

Here is what the test found:

Only four of 307 pharmacists found that their software issued alerts
on all 10 dangerous errors run through the software as medication
orders in the test.

"It was a real eye-opener," said David Classen, M.D., the
First Consulting Group medical errors expert who shared the results at
a recent meeting of the Association of Medical Directors of
Information Systems. Dr. Classen is also helping to shape the Leapfrog
decision support system test, which is expected to be available early
next spring.

In addition, Dr. Classen said that Leapfrog Group evaluations of
decision-support capabilities so far had turned up these observations:

Most medication errors that harm patients are caused by errors
in IV solutions, rather than errors in pill distribution. Therefore,
it is possible that bar-code-driven systems at the bedside will
reduce errors without actually reducing adverse drug events.

It is possible for automation to increase medication errors. Dr.
Classen said a new pharmacy robot at LDS Hospital somehow misfired,
and 273 patients received the wrong medication.

A study under way at Stanford and several other California
hospitals is helping define traits of what he called "high
reliability organizations" (basically, safe hospitals).

The most commonly observed characteristic of less safe hospitals: a
relentless pressure to produce. "Nothing had a greater impact,"
he said.

VI. Our Observations

Summary: Is Your Principal Goal To Reduce Errors Or Costs And
Resource Use?

CPOE can be principally a tool to reduce errors. Or, it can be
principally a tool to reduce physicians' consumption of resources. Or,
it can be a bit of both. However, control over physician use of
resources can also be a political battleground -- an "HMO through
the physician entrance"-- to the hospital. The authors argue
below that the principal reason that the large industrial entities are
expressing interest in CPOE via the Leapfrog Group is not to improve
care, but to seek to standardize physician actions for the purpose of
reducing resource use and costs. This is not necessarily an
undesirable purpose. However, ground on which you, as a caregiver
implementing software which these businesses will seek to appropriate
as a tool of their goals, should tread with eyes wide open.

Discussion: CPOE Is The Common Tool Of Two Groups With Divergent
Agendas

Two divergent core reasons exist for implementing CPOE. You -- that
is, the medical and bean-counting managements of your care-providing
entity--must decide which is/are yours.

As stated in our introduction above, the Institute of Medicine
report, "To Err is Human," and the Leapfrog Group's push for
CPOE are widely cited as mutually consistent, dovetailing pressures to
implement CPOE. It’s our view that this is seriously faulty
logic. Let us explain.

First, IOM and "To Err Is Human"...

The stated mission of IOM is to “advance and disseminate
scientific knowledge to improve human health.” That is, to use
science to improve medical care. "To Err Is Human" seeks to
do so by reducing medical errors.

However, error reduction is only a partial strategy if the core
purpose is to improve care. Vastly more care fails the sick and maimed
because it is not given than fails because it is given improperly.
Most of us understand intuitively that this is so. In fact, the IOM
now seems to be attempting to re-steer the debate away from CPOE as a
panacea in a new publication, “The Future of the Public's Health
in the 21st Century,” announced Nov. 11, 2002. “The report
calls for the federal government to lead a national effort to examine
options and develop plans for making comprehensive and affordable
health insurance available to everyone,” according to a National
Academy of Sciences announcement about it.

It also intuitively seems likely that more adverse treatment
outcomes arise in ways unlikely to be prevented by CPOE with decision
support than occur in ways which are headed off by CPOE. For example,
a patient the authors know had a tiny tumor which went undiscovered,
apparently because the reading physician was just too busy to notice
or to explore an anomalous spot on an MRI. In that same patient,
different physicians failed to spot a cancer metastasis–in part
because diagnostic activity for cancer is fragmented, and no one
compared a tiny anomalous glow-spot on a PET scan image with the
position of a lymph node on a CAT scan. These are two sets of
real-life examples with which the authors of this report are closely
familiar, and which would not be headed off by a CPOE system.

In another example, the final writing of this report was interrupted
in October, 2002–first for a week by an injury to one of the
authors, and then for another week by a series of treatment bungles.
These adverse outcomes (“mistakes,” to the mere layman) do
not appear anywhere in the medical records of the treating institution
from whence they arose, but their repair cost more than another full
week for a series of outpatient visits at a second institution (where,
obviously, a medical record was created, but with no cause-and-effect
link to the errors which made them necessary).

Thus, if there is one core activity needed to reduce medical errors,
it is that of judiciously spending more money on health care on an
ongoing basis. To reduce adverse outcomes, that money would likely
best be spent on improved access to care, better diagnostic equipment,
more intensive training of residents, more peer review and internal
quality-assurance watchdogs, and above all, more time for clinicians
to do their jobs well without being rushed. CPOE, while certainly
useful, logically falls quite low on the list as a strategy if the
objective is to improve care.

The fact is that CPOE is higher in priority as a tactic to “reduce
medical errors” (which is just a subset of the larger strategy of
improving care) because is offers that old (and as often discredited
as not) computer argument: it shows the potential to be the most
cost-effective, rather than the most actually effective, tactic for
reducing adverse outcomes.

Second, Leapfrog...

We doubt that this goal, or course of action–broadly improving
care--is within the marching-orders of the Leapfrog Group. The
sponsors of Leapfrog are industrial organizations which who have a
lengthy, unbroken, and, while quite legitimate, sometimes far too
effective track record of seeking to reduce their health insurance
costs. (Granted, to the extent that they are economically motivated,
they are also motivated by a desire to reduce lost employee work days
arising from medical errors. However, as in the cited situation
involving one of the authors of this report–fixing medical errors
is also usually an additional insurance-paid expense, and therefore
comes directly out of the employer’s pocket in the end, where “lost
productivity” due to absenteeism is one of those theoretical
numbers pulled out of the air for its own political reasons.)

Put bluntly, the larger motive of the industrialists behind Leapfrog
is to reduce consumption of health care resources, not increase their
use, which would be needed to turn around the decade-plus-long decline
into "sicker and quicker" discharges of hospital patients.

We do not mean to suggest that there is anything inherently
unethical about these cost-cutting motives. Rather, we think that as
one implements CPOE, it is extremely important to understand them for
what they are because caregiver organizations should use CPOE to adopt
some resource-saving strategies.

However, CPOE, in the hands of an overly money-focused management,
can take treatment decisions too far down that path. If it is fair to
characterize the worst bottom-feeders among HMOs (and other
managed-care entities) as those which cut expenditures and maximize
net income as the principal goal, and which elaborate medical
justifications for reducing resource use as the path thereto, then a
hospital management which over-uses CPOE to reduce physician
consumption of resources can be characterized as having sneaked its
own HMO in through the physician’s entrance.

In other words, a clinical system with CPOE can either focus solely
on improving care, or it can focus principally on limiting resources.
Or, third, it can provide half a loaf to each camp by catching the
mundane medical errors and speeding turnaround, but at the same time
strait-jacketing some physician diagnostic and treatment decisions
with built-in decision trees. A clinical system with CPOE cannot take
you all the way down both paths--improving care and limiting resource
consumption--at once.

Whose hand will be on the steering wheel? Yours? Washington?
Leapfrog? The HMO?

If a clinical system with CPOE and decision support can be used to
limit resources, a second question is: whose hand is on the steering
wheel?

We think the sponsors of Leapfrog view CPOE as a version of
industrial Enterprise Resource Planning (ERP) software. Reducing
errors does reduce costs--but reducing consumption of resources in the
course of production, in a standardized manner, on a day-in, day-out
basis--that is, requiring a hospital staff to behave like the
production line of an industrial organization -- has the larger
cost-reduction potential.

CIO Magazine describes ERP as --

"a sort of battering ram for improving the way your company
takes a customer order and processes it into an invoice and
revenue--otherwise known as the order fulfillment process. [ERP] takes
a customer order and provides a software road map for automating the
different steps along the path to fulfilling it.... With luck, the
order process moves like a bolt of lightning through the organization,
and customers get their orders faster and with fewer errors than
before." (http://www.cio.com/research/erp/edit/erpbasics.html)

Note the terms at the core of the ERP process: "order," "customer,"
and “battering ram.”

In the case of a care provider, the identity of the "customer"
is, obviously, a source of debate and confusion. The physician may be
the "customer" in that it is the physician who decides what
to order, places the order, incurs the cost, and demands fulfillment.
However, it is the need (or perceived need) of the patient which is
being fulfilled (or not fulfilled). As such, the patient is, in the
most real sense, the "customer." However, a synonym for “customer”
is the term, “paying customer.” Therefore, a third and even
a fourth real "customer" are in the equation–the payer
(an insurance company, Medicare, Medicaid. CHAMPUS, an HMO, etc.) and
the industrial corporation which pays for all or part of its
employees' health insurance. They are real "customers" of
health care to the extent that it is from their pockets from which the
principal payment arrives. This last group of "customers" is
Leapfrog's constituency.

In non-HMO (and pre-CPOE) medical practice, more of the power of “customer”
decisions lies with the patient and physician. The “paying
customer” has a small say through pre-authorization and
post-decision utilization review in the background, which modify the
physician's ordering practices. In managed-care situations, the HMO is
vastly more aggressive as the paying "paying customer," and
influences utilization with the now largely-discredited practice of
waving dollars in the face of the physician or his/her management for
choosing to use fewer resources.

CPOE, when coupled with decision support, offers the opportunity for
exactly the kind of programmed, cost-driven channeling of decisions
that the sponsors of Leapfrog have always had as their principal
health care agenda.

This use of software to guide, pressure, or force physicians into
resource-conscious decision paths is, as stated, already happening in
CPOE systems. It is prominent in Vanderbilt University Medical
Center's Whiz system. Other academic medical centers and community
hospitals with residency programs have also cited resource consumption
decisions being among the core functions of their CPOE systems. Two of
the respondent medical centers in the survey section of this report
allude to greater control over physician-ordered resource consumption,
one citing "clinical practice guideline enforcement" as a
benefit, the other referring to an "increase in IV to PO
(intravenous to patient oral medication) conversions," which can
save significant amounts of money and reduce the risks of IV-caused
infections.

In addition, we reported last year on presentations by John Halamka,
M.D., CIO of CareGroup Healthcare System, a large Boston, Mass.-area
physician group practice, who said that a software system which
physicians use in a manner similar to a CPOE (and which could likely
be integrated with CPOE) saves CareGroup $10 million a year by heading
off referrals to physicians outside the group practice.

That system, which CareGroup calls Service Tracking and Referral
System (STARS), is a CareGroup add-on to the New England Health EDI
Network (NEHEN), a network which allows its members to send
HIPAA-compliant electronic eligibility verifications, claim status
inquiries, referrals, authorizations, and specialty referrals. The way
it works is that all 535 of CareGroup's primary care physicians use
NEHEN to obtain payer approvals to refer patients. If the caregiver
seeks a payer approval to refer a patient to a specialist who is not a
CareGroup provider, a STARS screen pops up, warning the primary-care
physician that this out-of-network referral must be justified, that it
will be reviewed, and that it may be delayed. The system invites the
physician to refer to a CareGroup specialist. STARS issues a similar
challenge if the primary care physician requests too many specialist
referrals.

A referral, while not strictly speaking a medical "order,"
is, in effect, another kind of physician order. Thus STARS is a "decision
support" tool, interjected into the process of arranging for
further clinical care for purposes including maximizing revenues.

It is our view that as CPOE systems evolve, the error-reduction
functions (misunderstood handwriting, drug interactions, wrong
prescriptions, etc.) and the time-saving functions (faster movement of
prescriptions, tests, and faster resulting treatment) will be seen as
lower-level functions of CPOE. They'll always be quite important--but
they will be the small stuff. The emerging principal function of CPOE
systems will be as a software system to get more bang for the health
care buck by heading off uses of expensive medical resources which
health care management deems to be "unnecessary" or "cost-inefficient"
or "wasteful," or whatever term management applies to its
decision-making.

And as that happens, the core questions are: “Where do you draw
the line?” and “Just who are you–clinical management
the financial management, or an outside financial or regulatory
influence?” That is, a CPOE system with decision support can
readily be devised as a way to manage (and justify) a lower-quality,
denial-of-expensive-procedures, faster-discharge version of hospital
care. A clinical system with CPOE can be a system which not only turns
the hospital into an industrial production line, but builds a
bullet-proof record to medically justify it.

This emerging capability of clinical information systems with CPO to
make resource-use decisions will require decisions by the implementing
caregiver's money and medical managers:

What do you want your clinical information system to do?

Who is the "customer" of CPOE -- the patient, the
physician, or the payer, or (we hope) some careful balance of all
three?

This
report was written by William Donovan, publisher of Inside Healthcare
Computing, based on his own research, research by Suzanne Corrales,
editor of the biweekly hospital information systems newsletter, Inside
Healthcare Computing, research by Rachel Ross, an independent
researcher for Inside Healthcare Computing, and public presentations of
others; those outside sources of information are cited in each case.

Opinions and Conclusions In This Report

Conclusions
and opinions expressed in this report are those of directly quoted
individual interviewees where so attributed. Conclusions and
opinions not attributed to others are those of Inside Healthcare
Computing.

About
the Publisher:

Computerized
Physician Order Entry: How to Make It Work

This
publication is Copyright 2002, Inside Information Group, Ltd.

This
Report is Copyrighted Intellectual Property:

It is a violation of federal law
to make copies or to send this report to anyone without a license
to do so. That includes individuals in your own company. Each
person must be a licensed reader, or you must have an
organization-wide license.

Notice: the
publisher uses tracking systems to ensure delivery of electronic
issues to licensed persons. These systems can also record
forwarding of the publication to unauthorized readers.

Copyright 2002, Inside Information
Group, Ltd., 3600 S. Harbor Blvd., Suite 220, Oxnard CA 93035 USA.
All rights to make copies are held by the publisher. You may not
make a copy without the publisher's permission. Faxing or
e-mailing the report to anyone not licensed is making an illegal
copy. Additional copies and group licenses are offered at low
rates. Call (800) 294-6032 for information.