Thursday, December 20, 2012

The title of this Blog is "event processing thinking", and much of the posts are my own thoughts about event processing, but today I would like to write about another topic: event oriented thinking.

An interesting observation from recent discussions is that in daily life we can very easily think in an event-driven way, when the phone is ringing we are either answering it or ignoring it, when we hear about traffic jam - we try to find another way, and so on... However - when it comes to thinking about computerized systems, many people are programmed to think in a request-driven way, which means: a person sends a request and the computer responses. Thus people are trying to use what they know even if they have scenarios that are event-driven. A very typical line of thinking is: event is a data, we should insert it in a database, and then ask query that reflects the situation that we wish to detect. The question is -- when are we going to ask this query. We can ask it on any update -- but if the event is just one component in a pattern we wish to detect (example: we wish to detect when a customer sent three requests within a single day - assuming that most customers don't send more than one request, then we can have 99.99% of the queries redundant), we can ask the query periodically - but then again, we can both ask redundant queries, and moreover, not able to react on time, since the granularity of our timing is wrong. There is also a notion of continuous query -- but this is not really a query, since it is not a result of a specific request.

The nature of event-driven scenarios are: we don't know when they are going to happen, we even don't know whether they are going to happen, but when they happen we want to do something - sometimes very fast (e.g. earthquake detection). Furthermore, the situation we would like to detect can be realized in a pattern of many events, and each individual event may trigger reaction, just increment some state, or even be ignored or filtered out.

The fact that people are trying to model and implement event-driven scenarios using the traditional request response is a thinking mismatch, and creates added complexity. I'll follow up with an example in one of the next posts. More - later.

Tuesday, December 18, 2012

This drawing is taken from the ACM SIGMOD Blog post by Thomas Haigh entitled "Fifty years of databases". It tells the story of IDS (Integrated Data Store) as the first database developed 50 years ago in GE. IDS was designed by Charlie Bachman, who received Turing award for his pioneering work on databases, and as Haigh remarks - Bachman was the first Turing award winner who did not have PhD, and actually spent his life in industry and not in academia. It is worth remarking that the two persons who followed Bachman by receiving Turing awards in the database area, Tedd Coddand Jim Gray were also from industry, in fact, the academic database community did not have until today any Turing award winner. Bachman's Turing award talk "programmer as navigator" was very insightful. Bachman compared himself to Copernicus who said that the earth is revolving around the sun and not vice-versa, and said that the computing world will revolve around data - where programming will be side effects of operations associated with data. We are not quite there, but for me it was an inspiring goal.IDS is far from the current databases we have today, but it laid the principles, and started from pure engineering position, theory came later. I am trying to compare the database area development to those of event processing, and think that in many respects we are still were databases have been 40 years ago, so the challenge is to advance it further... more on that -later.