MIS Dashboards:Buy or Built

Author

The growing trend in Europe (and
much of the world) to corporatise hospitals and set performance targets has been
accompanied by an escalation in management information requirements. One answer
to this challenge lies in Management Information System (MIS) Dashboards.
However, as with much else in healthcare IT, there is a major debate about how
add-on MIS 'dashboards' compare to customised configurations built in house. Given
below is a whistle-stop tour by a specialist in predictive technologies in
healthcare management on key issues related to hospital management dashboards –
their strengths, weaknesses and what to consider before buying or building a
dashboard application.

You’ve seen them, you know what I’m talking about – they are
colourful, they have moving dials and flashing lights…… and they are often web
based – you can even look at them at home or from a conference – they are
‘dashboards’.

Approving expenditure to purchase or construct a management dashboard
application for a healthcare facility may seem a relatively simple task.
Particularly if all it needs to do is ‘hook up’ to existing ‘transactional
systems’ and start displaying all that wonderful data floating around, to
assist in managing a facility. ‘Lets get on with it!!’ you say. But is it
really that simple?

To answer this requires a closer look at the situation. Like any
major investment, there is a strong case for caveat emptor (let the buyer
beware)...and even if you follow that adage, in your reflective moments you may
still ask yourself ...’why on earth did we do this?’

So What is a Dashboard?

A dashboard, fundamentally, belongs to the broader category of
Executive Information Systems (EIS) which integrate data from a variety of
different sources and present it in an optimal format to decision makers.

The essential philosophy behind such systems is to help managers by
providing structured information necessary to quickly make decisions which
require informed judgement. They are designed to not only be user friendly; the
best, indeed, claim to have a near ‘intuitive’ element in -built for users.

Why do Hospital Managers Need Dashboards?

Hospital managers stand to benefit from dashboards for a number of
reasons, not the least being the environment in which they operate. Across the
world, hospital managers are being asked to track and act on more information,
in more dimensions then ever before.

An additional complexity is the lack of consistent evidence, or
even guidelines, about the best decision choices in any given situation, which
is partly a reflection of the complex permutations and combinations of
scenarios driven by different facility types and case mix, different funding
mechanisms and different regulatory environments in which hospitals sit.

Given this complex environment, it is worth underlining the following
comments in a recent Harvard Business Review article titled ‘The secrets to
successful strategy execution’. The article, reflecting the work of a major
consulting company, noted that: “The single most common attribute of…… (successful)
companies is that their employees are clear about which decisions and actions
they are responsible for. As a result, decisions are rarely second-guessed, and
accurate competitive information quickly finds its way up the hierarchy and
across organisational boundaries. Managers communicate the key drivers of
success, so frontline employees have the information they need to understand
the impact of their day-to-day actions.”

Dashboard and Information Dimensions to be
Considered

A number of dimensions need to be considered in contemplating what
a healthcare IT manager need from a dashboard. Often, these have fallen out of
organisational performance monitoring frameworks like the balanced scorecard. I
would argue that users can get more from a dashboard in the current technological
environment by not limiting themselves to overly restrictive frameworks.

For example, the temporal dimension is a newer one to consider; many
dashboards historically have used warehoused data which is out of date at the
time of usage – but nonetheless potentially helpful in understanding both the
recent and distant past. More and more systems are capable of delivering real
time data to users. Some systems (and their attached dashboards) are now
capable of delivering forecasts into the immediate or near future – for example
about issues such as hospital bed usage and occupancy.

Other considerations can also be thought of as breadth, depth and
complexity…… and may act as a useful guide when communicating with vendors or
in-house developers.

‘Breadth’ means the range of data items and content areas (eg: finance,
quality, access, HR) to be included in the dashboard.

Depth entails an ability to drill down on areas of concern. So for
example, at COO level, if hospital Emergency Department waits are lengthening
to unacceptable levels, what ability does a COO have to drill down by
interacting with a GUI to find which care unit or business areas are
contributing most to the delays? This takes a dashboard beyond a purely ‘data display’
system to one that can inform decision making and action in more practical
ways.

In relation to complexity, there is some broad guidance on what
constitutes useful dimensions (or quadrants) of data to incorporate into a
dashboard view for healthcare managers. A report by John King and Jeanne
Jenkins in the April 2008 issue of Healthcare Financial Management titled
‘Information management: why it’s vital to effective service line operation’ observes:
“It is essential for healthcare financial managers to understand and monitor
five areas of information critical for effective service line performance
tracking: Market share, Operational performance, Physician performance, Clinical
documentation and coding, Patient satisfaction.”

These areas, however, are hardly simple or straightforward when
extrapolated to a global context. A study by a University of Maryland team on a
web based dashboard in managing operating theatre performance notes: “The challenge
lies in aggregating and displaying these data in an easily accessible format
that provides useful, timely information on current operations.” [Surgical
Innovation (Vol. 15, No. 1, 2008)].

It is also known that individual and role-based preferences affect
what data is relevant and what are the relevant views to put into a dashboard.
In the healthcare area, a variety of studies have pointed out highly variable
stakeholder views about ‘report card’ elements to be considered in different
con texts and scenarios. As a result, reaching agreement about what information
content should be included in any kind of reporting mechanism (card or
dashboard) can be problematic, and an area of real concern in the development
of dashboards for hospital managers.

In addition, there always are new pieces of information which managers
are challenged to keep track of, ranging from the useful to the indispensable –
such as best practices. In real life, the trade off lies in the importance of
locally keeping track of such information versus the ease and cost of
implementing changes in the technical environment and business process to allow
that information to be measured, displayed and actioned.

The British National Health Service has seen work on using advanced
analytic techniques, specifically an SPC (Statistical Process Control)
technique called CUSUM (cumulative sum) charting in a web interface to compare
in-hospital mortality, length of stay and emergency readmission rate
acrosssites and organisations. This illustrates an example of where such
applications may head in the future.

Surely You Just Hook it Up and Go from There…

So if a hospital decides to proceed down the path of investing in
this technology, is there anything else managers need to think about…… ‘can’t
we just plug it in and get going’? One critical consideration is the business
context in which the technology will be hosted.

There have been attempts to provide advice in this area. Dr. John
Glaser, CIO of Partners Healthcare in the US, suggests five key steps in
setting up an appropriate environment in which to use business intelligence technologies
:

1. Establish business needs and value.

2. Obtain buy-in from managers.

3. Create an end-to-end vision.

4. Establish BI governance.

5. Implement specific roles for managing data quality.

Environmental and Contextual Considerations

One key consideration in purchasing or developing a dashboard are
the skill sets and knowledge bases of hospital managers in other words, the technologies they are (and
feel) comfortable with. There is evidence that healthcare managers have a poor
knowledge of predictive systems and techniques. Exacerbating this is a
considerable level of resistance to new technologies, among both hospital
employees and administrators.

A second factor concerns the specific healthcare organisational environment
(the kind of facility and the services provided) and the implications of these
for dashboard purchase or development. The metrics to be include in the
dashboard and the relevant time and drill down dimensions may vary enormously
depending on such questions.

Cost and obsoloscence should also be taken into account. The main
question here is how current and future data requirements relate to the needs
of funders, broader corporate requirements and those of regulators. There is
little point, for example, in having to run multiple dashboards, or a dashboard
to meet a range of separate and non-overlapping sets of needs around management
information.

Challenges and Constraints

Several challenges face both in-house developers of dashboards in
hospitals and purchasers of off-the-shelf systems. On a technical level, there
is the challenge of access to, conversion and integration of, disparate data
sources – ranging from historical paper-based collection systems and
operational databases, to warehoused data at the other end of the spectrum.
Another common challenge is to find a definition for optimal data quality – for
instance, real-time occupancy information versus the load this places on the
system.

From an organisational perspective, the issue of cost (and hence
delivery mechanism) is always a major consideration. Public healthcare in
particular is often cash strapped. As a result, ‘expensive’ solutions,
especially those impacting on end user devices or client side software, can be
problematic; web based applications in a thin client model are increasingly
seen as an answer to such challenges.

Resistance to change, and adequate levels of staff engagement are
other issues. The role of the clinician ‘leader’ or ‘champion’ is widely
recognized as a major factor in clinical systems projects. Management leaders
and early adopters can often only assist in the implementation of a developed or
purchased solution.

The framework and business processes attached to such initiatives
also remain crucial to take into consideration. ‘We have all this information
available at our fingertips…… what next‘?

This question is addressed below.

So You Have a Dashboard ….Now What……….?

In the process of deciding to purchase or build a dashboard a
quick checklist of questions which healthcare IT managers are advised to
consider is provided below:

Ó Does the dashboard relate to an organisation’s performance
framework – including financial, quality, access, clinical governance and other
dimensions?

Ó How does it relate to business processes – what actions
will be taken on the back of the data ?

Ó What ongoing quality processes are in place in
relation to the data displayed – and how does that flow on to the feeder
systems?

Ó Can the system be easily modified in the event of
new

board and executive priorities or changes in funding or regulatory
frameworks?

Ó How do prospective users evaluate a dashboard’s utility.
It is a tool…… is it working for you or not, and if it isn’t – what are you
going to do about it?

Buy Versus Build

Should a hospital IT manager buy or build a dash board ?

As with any MIS or EIS system, it is self-evident that executives be
closely involved in the choice/purchase or development of a dashboard. Given
the demands on their time and variable level of comfort with computer systems,
the attractions of a build scenario with in-house staff are often significant.

However, there is a major trade-off here: pre-existing solutions or
those requiring minimal local customisation already carry the benefit of being
‘tried and tested’ with a range of executives and managers from other
organisations, thereby minimising the chances of not meeting the users
(managers) needs.

There are a number of commercially available applications available
in this problem domain, with different emphases, pros and cons.

Other considerations in the buy versus build equation include:

Ó Cost – for instance large licensing fees.

Ó Potential skills transfer or development for
internal staff versus outsourcing and being vendor dependent on an ongoing
basis.

Ó Dashboards may be better if constructed by embedded staff
with the necessary technical, clinical and local organizational knowledge.

Ó Is the system easily maintained or improved moving forward
– in terms of functionality or content, depending on user preferences,
technological availability (e.g.: new end user devices such as tablets) or
organizational needs?

In the final analysis, dashboards can clearly provide an
opportunity to assist in hospital management, However, like any significant
investment or IT implementation, the key to success lies in making sure users
get what they really need, while not getting burnt along the way.