First Look: Informatica CEP

I got caught up with Informatica recently, checking in with them on their Complex Event Processing (CEP) technologies. They have recently been talking about decision management, especially in healthcare where their new Informatica Proactive Healthcare Decision Management solution is based on their CEP engine.

Like me Informatica see a historical, summarization focus in business intelligence (BI) is insufficient to respond to the kind of threats and opportunities that are out there – responding more quickly to reduce decision latency. Informatica is trying to bring event-based data to bear on the problems that more static data has traditionally been used for, to make better decisions sooner by enabling what they call real-time operational intelligence.

Their CEP engine is designed with this in mind, building on Informatica’s strengths in data integration.. The CEP product is part of an environment that includes their data integration, event capture, master data management (MDM), information models, security, cloud and big data technologies and more– the Informatica Platform. Informatica talks about using this platform in a Sense (detect events from systems and people), Reason (apply rules and pattern detection, decide), Respond (determine an appropriate response, decide) and Visualize (results and performance against KPIs) sequence.

Back in 2009 Informatica acquired AgentLogic, a company focused on sense and respond systems in the public sector. The acquired products have been integrated and extended and are now packaged as Informatica RulePoint. RulePoint takes inputs from feeds or streams as well as through other sources such as web services, HTTP, databases, social media feeds, and an API, processes them concurrently against various rules, alerting users of different personas through multiple channels. As a response, RulePoint may also update databases, send alerts or messages, update dashboards etc. The engine handles event processing and is sequence and time aware. It allows you to specify rules and pattern matching logic to decide what to do about the event.

RulePoint is designed to make it possible for non-technical users to detect changes across dynamic information sources and correlate disparate data events. It processes data immediately as it is received from any source, avoiding the store/process delay of data warehouses/BI and reducing the detection to response time frame. Java-based, server-oriented and thin client-based RulePoint is both integrated with the Informatica stack and extensible in the areas of event detection, responses and analytics/formulas etc.

RulePoint runs rules against events, generates new events or alerts and enriches these responses with additional data. The web-based UI is role-based, multi user and multi tenant and supports various levels of skill in business users. IT users generally hook up data sources and actions. Business users can then write rules using a simple wizard based rules editor using fill-in-the-blanks templates, or re-use pre-built rules that are provided out-of the box as in the healthcare solution. The editor allows you to bring in functions like SQL, web services or geospatial functions as well as more straightforward conditions. More advanced users can write the rules in a freeform editor.

These rules can be converted to a template by parameterizing the settings. These templates can then be exposed to a user who can create rules based on the template by giving each parameter a value. These are real templates and users can control/rollback changes. When new parameters are added the user can specify a default value to ensure existing rules based on the template continue to run etc.

The user also configures the response to the rule and can include email, event transformations, service calls, http post, write to database etc. Rules can also trigger alerts that can be enriched with extra information to make deciding how to act easier. The system often sends multiple messages – one to email or mobile device, one to the user and one to database for logging for instance. More and more of these alerts are triggering automated processes not just alerting someone.

RulePoint also has support for profiling and scenario testing using the event history to see what happened and in general it is easy to replay events.

As noted, Informatica RulePoint fits into the overall Informatica stack. At the bottom of the stack are PowerCenter, PowerExchange (adaptors for databases, enterprise applications, and all other types of data sources), data replication (to event enable existing databases by listening to database logs without performance impact) and the Informatica B2B Data Exchange and B2B Data Transformation capabilities. These let you bring data in, transform and manage it. Layered on top of this is Informatica Ultra Messaging bus for event transport and delivery of this information in an event-based way. Informatica RulePoint then subscribes to the message bus and handles event processing. Informatica also has a Web Messaging infrastructure to manage push connections for web applications, user interfaces etc.

As noted earlier, Informatica has recently focused some of its CEP effort on Healthcare solutions, specifically healthcare delivery. Healthcare delivery in the US is often fragmented with lots of organizations involved. This can result in problems where patients are being re-admitted for the same problem, fail to make appointments for follow-up activities with other providers, and don’t use integrated pharmacy systems. Some customers found existing decision support systems too siloed and too passive to solve these problems. Using the Informatica stack they developed a system to listen to HL7 messages from existing systems, transform these messages, persist them and pass them on to RulePoint where the rules about what was supposed to be going on would execute, identifying exceptions and issues in real-time, that could be flagged and handled immediately by the relevant stake-holders.

Initially this was focused on visibility and on telling people that rules were being breached. The organizations using the solution continued to learn about the rules and situations, evolving how they handled the events. As they became more comfortable they have begun incorporating more historical information to enrich the data being used and to automate the response to what was being flagged. This helps reduce decision latency – moving from analysis to identify a preventable cause of death after the fact to identifying those patients most likely to be re-admitted to calling people who have not booked follow-up activities before those activities are actually overdue.

Besides healthcare, Informatica sees similar examples in data quality environments where proactive monitoring of data integration and quality problems is being used, in fraud and abuse for corporate credit card handling, and in alerting for security to flag needs for investigation and maritime alerts. Internally they are also applying the engine to monitor PowerCenter to apply best practices and governance in real-time and to use continuous correlation to see if processes are behaving as expected (potentially doing something about it if not).

Going forward Informatica is working with existing customers to find opportunities to reduce decision latency and response times, to develop pre-built solutions like the Proactive Monitoring for PowerCenter and the Healthcare solution for re-admissions. They are also looking at real-time analytics and operational intelligence solutions, augmenting existing BI environments with rules-based push subscriptions.