This is a blog describing some thoughts about issues related to event processing and thoughts related to my current role. It is written by Opher Etzion and reflects the author's own opinions

Saturday, June 28, 2008

On embedded intelligence within event processing application

In the previous post I have referred to the term "Intelligent Event Processing" - one question that I have asked - is this a new term ? how does it related to the other "X event processing" terms ? -- I am not sure if the term "intelligent event processing" will stick around, I would say that a better way to explain what it is may be - "embedded intelligence in event processing".

If we look at event processing architectures

- there are: producers who produce events, consumers who consume the processing results, and the EPN (Event processing Network) in the middle, which really does the processing. So where are intelligent techniques can help - here are some (real) examples:

1. In the producer - the producer has a video stream of all cars that pass below the camera, an intelligent process (using image processing techniques) isolates the license plate number of the car, and send it for further processing (security, traffic violation, billing etc..).

2. In the "meta-data" composition -- a "pattern detection" node typically looks at pre-defined patterns and attempts to detect them in run-time. In current applications the patterns are entered by the developers or users. In some cases the patterns are "moving target" like in - fraud detection -- if the patterns for fraud are discovered they are of little value, and thus in the other side of the law - people are constantly looking for new loopholes, thus, intelligent techniques, such as machine learning are used to refresh the patterns that are looked for in run-time. The run-time does not change - same pattern-detection mechanism, just different sources of where these patterns come from.

3. Intelligent nodes within the EPN -- in some cases the process of derivation of new events cannot be expressed as derivation expression and need some intelligent derivation process - e.g. a heuristic algorithm to determine the traffic light policies based on traffic events.There are many more examples - like creating predicted events and more -- but this was more to give some flavor. Is it useful -- yes, it is useful for a variety of applications. Does every CEP application need embedded intelligence -- not really. More - later.