FEATURED: Real Time BI- Real Use Cases or Marketing Hype

Software vendors talk a lot about real time business intelligence.
So I am wondering whether members of Toolbox for IT think the talk is just vendor marketing hype or are there real uses cases?
If there are real uses cases is it done with traditional ETL, query, reporting, and analysis tools or does it require additional technology like CEP (complex event processing)?
---

This discussion is part of the Toolbox for IT Featured Discussion program.
If you would like to lead a future Featured Discussion, please email moderator at toolbox.com.

Hi there, I have been involved in a project where the use of ETL tools has been implemented to upload sale data at the end of each day to a Data Warehouse from a number of country based systems. Part of the justification for the implementation was the ability to track the changes in sales as new marketing campaigns were rolled out across the region. The traditional approach of looking at sales analysis at the end of the month was deemed to be too late in evaluating the effectiveness of marketing campaigns.

I think real time intelligence is CRITICAL to almost any business. I have led projects at Ford Motor Company (and Chrysler and others) which made visible, in real time, orders, product and part flows through manufacturing, and out both sides of the supply chain and transport to the dealer network. This real time visibility and the development of "Business Rules" to detect and prevent potential disruptions enabled phenomenal results:

* manufacturing throughput increases of up to 40%

* simultaneous inventory cost reductions of up to 33%

* more than doubling the ability to build a wider variety of product using the same people and equipment

We have now adapted our technology and methodologies to non - manufacturing intensive businesses and seen similar performance increases in their metrics.

The keys are real time visibility, throughout the enterprise and applying intelligence, both in the form of Business Rules and PEOPLE intelligence to detect and prevent potential problems / short falls.

Anything short of real time visibility is, frankly, "managing through the rear view mirror."

@Tral - Your response brings up another question, which is what constitutes real time. Is moving from monthly updates to daily, or is it updates as changes occur? I don't have an answer and I am not expecting one just wondering out loud.

@Hal - It sounds like you are using a combination of BI and CEP in the projects you work on. If so how intensive was building the event/rule models? How much work to integrate with traditional BI tools?

We integrated our Web Based (Entirely JAVA) SmartPlant tool with Legacy IT
systems and with RFID tracking and plant floor PLC machine controllers.
This was quite straightforward.

Initially, we simply provided real time visibility of the orders, product
flow, and part flows. People, seeing these flows, noted impending
disruptions and took actions to avoid or fix these disruptions. Then
Business Rules were elicited from these people where recurring analysis was
possible.

Real time Bi has plenty of very real use cases, especially in the call center and business process outsourcing space. Ideally on the floor of the call center, managers want metrics like average call handle time, average waiting time, etc displayed in large LCD monitors. They want to press additional troops if these metrics dip from the green zone to the red zone in the display.
Data can be collected every 15 min or so from automatic call distributor software powering the phone systems.

Another funny thing is that the definition of real time varies. The above use case is every 15 min or even shorter if you can get the data. Absenteeism is daily. Effectiveness metrics like customer satisfaction indices are about a week or so . Real time depends upon more on the metric!!

You can call it Thing-a-ma-jig instead of Real Time! There are metrics
that Call Centers use in real-time, are available to them, and metrics
that are not available to them in real-time. They are worried more about
getting work done with appropriate information than semantics!

Plus, "Real-time Operational Metrics" in many organizations deal with
Sales, Expenses, Production, Shipments, etc. They used to have all these
metrics at the end of the month, or week, or even sometimes quarters, a
couple of decades ago.

Now "Real-Time Operational Metrics" may mean daily metrics since that's
the periodicity that you can expect reasonably for the collection of all
this data, collation and meaningful transformation into some form of
intelligence.

Yes companies like Amazon.com may have hourly, even minute by minute
sales statistics from their online sales and even shipments.

We may disagree on whether these are all "real-time" or not. You may
wish to draw the line at milli-seconds but others may not. But in the
end it does not matter to the end users one way or another. What's our
goal? Making information as current as possible and as useful as
possible, or agree upon a precise definition of "real-time"?

- You appear to speaking of historical information, whether minutes ago
or hours or days ago - the operative word is "ago"
- I am talking about NOW -vs-ago, meaning I see it happening and I
intervene, changing the outcome

@Hal - In regards to your question how can there be variability in the definition of a phrase, it happens all the time. For example, people who talk about profitability often have different definitions. Some allocate SG&A costs across products, some don't.
Therefore I think it is important to have open and respectful discussions about what exactly we mean when we use certain phrases or metrics. You have made your definition of real-time clear. Given your definition can you accomplish real-time BI with traditional tools or do you need additional technology like CEP?

My opinion is that "the time it takes to take action" is not a decision quality issue, but an opportunity cost issue. For example, in retail if a sale item is not on the shelf when a customer comes in you have lost that sales. Here having CEP that monitors point of sale data coming from the registers and sends an alert to a stock clerk to restock the sale item first makes sense because of the opportunity costs.

However, if I am a marketing manager trying to determine what campaigns to run I don't need to be alerted in real-time everytime a lead comes in. Historical data and statistical models can help me understand the impact of products, demographics, marketing channels, promotions, fullfilment devices, etc.

In both cases (real-time and historical) quality decisions can be made.

Okay, I agree with your conclusion that in both cases, quality decisions can
be made. However, would you not agree that a decision made too late (or an
action arising from decision being too late) is NOT a quality decision, at
least by comparison?

@Nari - If I understand your post correctly you believe there are uses cases for real-time (as it happens updates), but other uses cases that do not require real-time.
If this is correct what do you think the split is (for example 10% real time, 90% other)?
Do you think traditional BI alone can handle real-time (as it happens)?
For example in the call center do you need specific integration into the telecom switches to get wait time metrics?
Not my area of expertise so please forgive me if I am not using the right terminology.
Regards,
Dan

Again, here is where we get into the issue of definition. My definition of quality would be that the decision would be effective at creating a desired result if acted upon.

Now I could make the decision too late to act or I could make the decision but fail to take action and the decision itself could still be a quality decision (happens to me all the time in trying to pick stocks :0)

All that being said I agree that time is an important factor in the business impact of decisions. It is just that the time factor is more important in some uses cases than others.
I was trying to get a sense of what those use cases are (I have some opinions of my own), and what kind of technology is required.

@ Katherine - You said "In the business where I work much of the trading is online. During busy patches the real time MI allows bottle necks to be identified quickly, before they become real issues."

Could you expand a little?
What kind of firm/trading (brokerage trading stocks, oil and gas trading futures, etc)?
What is MI?
Are you looking for bottlenecks in transaction processing, network bandwidth, something else?

A tool that can monitor call quality and agent performance, as it happens, could be very useful to Team Leaders / Supervisors / Ops Managers in a Call Center environment. If as an agent I am alerted during the call that the current call is below quality threshold, I may be in a position to take corrective steps. This would fall into "real time" category as per Hal's definition. Similarly, as Manager, if I am alerted when an agent's performance is below threshold during the shift, I may be able to take corrective steps. However, this may have some time delay - an alert can be raised only after the current call is over and performance upto this call has been measured. This would fulfill the decision quality aspects, although may not fulfill the time test.

There are tools that currently address the agent component to a large extent. I am not aware of any available solution to meet the Manager's needs.

Updated BI, as a transaction is completed, is definitely required for specific needs, but many a time I find that cost of delivering the information does not justify the value being created. As long as price-performance is positive, using CEP (where it is applicable) or specific point solutions are definitely gaining desired attention.

It absolutely is not a marketing hype!
I have seen several Projects where i have seen the real success of real time reporting through the set of tools and technology provided by IBM.
One such implementation was with IBM CDC(Change data Capture).
Capturing the changes has always been the evolving revolution in databases. These changes have found several uses in the industry and BI is one such domain.