Tags

RuleML Keynote

My colleague Eric Mazeran gave a keynote on ML, Optimization and Rules : time for agility and convergences at the Rule ML conference . I co authored the material with him, and the slides can be found here. It was well received as it gives a global view on how these three technologies can be used together. I'd like to comment on one slide of his presentation (slide 15 if you download the deck). Here is a slightly modified version of it:

It captures what I believe is the ideal data science project. It starts with data about some part of the world we are interested int (typically some business related data), and a business question we try to answer. I discussed the trap of starting data science projects without a business question in Start With A Question. Examples of relevant business questions include: which customers are likely to renew their yearly subscription? Which customers should I target with my marketing campaign? Which products should I recommend to which customers? What maintenance operations should I perform first? How should I replenish my inventory to best meet future demand?

Let's use the first question for the sake of clarity: which customers are likely to renew their yearly subscription? The first thing data scientists would do is to look at available data, and explore it to see if it can help answer the question. They will use various techniques ranging from data visualization to statistical and machine learning algorithms. Their goal is to find patterns that are correlated with customers subscription renewal. Finding these patterns is worth it. They can then be shared with decision makers. Data science projects can stop there, in which case they are Data Mining projects.

Other projects move to the next stage. The next stage is to use whatever patterns were discovered in the first stage to make predictions. For instance, if I have created a statistical or a machine learning model that predicts with good accuracy which customers will renew and which won't, then I can use that model regularly, say every month, to identify which customers are unlikely to renew. What the model ouptuts is a probability of customers renewing. This predicted probability is a very useful information for decision makers. For instance the marketing department may look at the customers that have a low predicted probability for renewal and target them with incentive actions. Projects can stop at this stage, in which case they are Machine Learning projects.

When projects stop after stage two, the eventual action (e.g. targeting some customers with incentives) is left to human decision makers. While this makes sense in many cases, in some other cases one wants to automate decisions. This is what a third stage is about. There are mainly two ways to automate the action piece.

The first one is to implement some business logic via a rule based system. For instance, a rule could be: if the predicted probability of renewal is less than 0.1 and the customer is a preferred customer, then send a coupon to the customer. This is the simplest way to transform machine learning output into actions. It is applicable when actions can be made one at a time, just by looking at some context and a machine learning prediction.

A second way is to be used when a number of actions have to be defined together. For instance, if we have a limited budget for coupons, we should not use a fixed probability threshold to select which customers should receive a coupon. We should rather focus on those most likely to respond to the coupon. A very common use case is when machine learning is used to forecast future demand (for instance future sales for each product and each store). Then we can use mathematical optimization to find the best replenishment plan for each store:

The last stage of the process is to monitor the consequences of the actions on the word we are interested in. By monitoring their effects we can determine if the machine learning predictions were accurate or not. That information can be analyzed by data scientists, which can lead to better models. As a matter of fact, we are executing the process as a continuous loop, where each iteration builds upon data produced by the previous iteration. This continuous loop is at the heart of learning machines.

There is much more content in Eric's keynote and I encourage readers to have a look at his deck. Let me conclude with a remark on how the above relate to analytics. Stage 1 corresponds to Descriptive Analytics. Second stage corresponds to Predictive Analytics. Third stage corresponds to Prescriptive Analytics. Fourth stage can be called learning from feedback, and has no real equivalent in traditional analytics. Interested readers can read more about the various analytics stages in the Analytics Landscape.