Here comes a discount example with the ruleengine including screenshots for easy following.

We presume we want to calculate a discount for people travelling from the US to an airport in the US or an airport in another region. Based on the age and/or the destination region of the traveller a discount is given.

For the processing of the data I use the Pentaho ETL tool (PDI - Pentaho Data Integration)

This is what the transformation looks like:

The transformation gets the sample data from the "Data Grid" step. A field "discount_percent" is added which will contain the results from the rule engine. Then the data flows through the "Rule Engine" step. As a result of the rules the rule engine determines the discount percentage. The stream of data then further continues to the "Calculator" step, where the calculation of the actual discount value is done and stored in the field "discount_value".The second stream going downwards from the "Rule Engine" step (an optional stream) will output the detailed results of all rules.

Here is what the sample data from the "Data Grid" step looks like, including the "discount_percent" field. This is the data that arrives at the "Rule Engine" step.

The business logic that determines the discount percentage is defined in rules and group of rules. Group of rules bundle rules together with "and" or "or" logic. Depending if the rule group passes or fails, one or multiple actions can be triggered. The action in this example would be to update the "discount_percent" field in the data stream.

To define the business logic a free web based application is available. The Business Rules Maintenance Tool. It allows to define complex business logic and actions that are executed based on the results. Because the rules and actions are defined independant fron your IT process (the ETL process in this case), the IT logic and the business logic is separated from each other. And thus if one of them changes, the other part does not have to be touched. A clear separation of responsibilities between the IT and the business.

Here is the business logic to calculate discount percentages:

All travellers under the age of three travel for free. 100% discount.

All travellers at the age of 50 or above years and travelling to Europe get a 25% discount.

In the screenprint below you can see two rule groups defined. These are the two definitions of the business logic listed above. Each rulegroup then contains the rules as they are described in the business logic.

Here is a screenprint displaying the rule for the first rulegroup to define the logic for children under the age of three. And at the bottom you can see the action that is executed.The rule is very easy. It is just one rule checking if the "age" field contains a value that is smaller than three and if so, the "discount_percent" field is set to "100".

The second rulegroup - shown below - contains the rules for the second business logic as listed further above. Here we have two rules: one checks if the "age" field contains a value that is greater or equal to "50". The other rule checks if the "destination_region" field contains the value of "Europe". The rules are connected with an "and" operator (you can see it in the subgroup on the left). So if both rules are true, then the rule group as such passes and then the action at the bottom is executed: the "discount_percent" field is set to "25".

Once all rulegroups, rules and actions are complete, you can export the project. A project is a container for all the parts of the business logic. The result of the export is a single Zip file. This file can then be referenced in the "Rule Engine" step in the ETL shown at the top of this post.

And now you can run the ETL. The rule engine will check the data and execute the actions as appropriate according to the data. Here is the screenprint of the results of the rule engine and after they passed the "Calculator" step:

Now the great thing is that when the business comes up and changes the business logic or comes up with additional requirements, then you can model them in the Business Rules Maintenance Tool and you won't have to touch the IT process - the ETL process. This enhances the quality of the overall process and also keeps the IT process cleaner: no mixing of IT and business logic. As business users are experts in their domain but not necessarily in the IT domain, it is more transparent ot them to manage the business logic in a tool made for them rather then looking through e.g. IT code or - in this case - an ETL process.

As always: I welcome all comments or questions. All tools - the rule engine, the web application and the transformation step are open source and free of use. There is documentation available and the tools are all already used in production environments.

When writing rules and group of rules that are connected by a somewhat complex logic, the user needs a way to test the rules.

While the Business Rule Maintenance Tool can not elegantly help in testing mass data to ensure that the important rule logic is complete and works as designed, it can help the designer of the rule logic by testing individual sets of data.

Recently I have implemented the concept of reference fields. It serves the purpose, that the user - a business user maybe - will not have to type in field names and field types. It also documents the interface to the target system where the rules will run.

As the next step I have now implemented a way for the user to run the rules in the web application and receive feedback of which rules and group of rules passed or failed and if the group as such failed or passed.

This will give the rule designer confidence that the rule logic works. But it does not mean that the rules should not be tested using a larger set of data. When using the ruleengine in a production environment to support the separation of responsibilities between IT and business users, for more cleanness of IT code and to generally enhance the quality of the system, then there should also be a a testing concept in place and that includes testing the rule logic.

The rule engine bundles all rules of a project into a single file. I have chosen files as the interface because they have several advantages over handling rules from say a database.

A project file is a version of the truth for a defined point in time

Files can easily be exchanged and backuped

Files can easily be used to import project back into the web application

Project files can be signed and checksums can be built to ensure integrity

So the rule logic designer would write the rules and actions and then test them with single data sets. Once the rule logic is complete, the relevant project is exported to a file. It is now ready to be run against a target system respectively against the data that should be tested.

The ruleengine can be run standalone or it can run on a server which expect connections and data from a client. Or it can be used in the Pentaho ETL tool. For this purpose a plugin is available, that can easily be installed with a click of the mouse button from within the ETL tool. Pentaho PDI is open source - so is all of my software, including the plugin - and thus it can be used freely to design ETL workflows.

Such a workflow can read from virtually any data source, route the data through the rule engine plugin and further route or filter the data based on the results of the ruleengine. As Pentaho scales, you can also scale the business rule engine if you have a lot of data. You may also run the ruleengine in server mode in say docker containers.

So when the rules are run against the appropraite data and all tests show that everything is working fine, the project file containing the rule logic can be moved to a production server.

Again - separate your IT logic from the business logic (rules); it will benefit both and establish a clear separation of responsibilities. Your code is cleaner and change to one part will not necessarily affect the other part.

The ruleengine was designed independant of a tool in mind and as such it integrates into any Java application or Java simmilar languages such as Groovy or Beanshell, just to name these two.

Go ahead and give it a try. Everything is available on Github https://github.com/uwegeercken and I welcome your feedback and ideas.

Yesterday I removed the functionality to attach a Pentaho PDI transformation to the web application.

That was a hard decision because I had spent a lot of time implementing this. Also research to get it working took me quite a while - many hours. But there are too many complications:

It creates dependencies to PDI/Kettle libraries that are difficult to maintain and test

It does not work (well/easy) in cases where users have installed third party plugins

It does not work (well/easy) with dynamic parameters or external files

The benefit of it was, that the user could select the fields and field types of the transformation, instead of typing them in. With the transformation attached, the user simply had to select from a dropdown box.

So I have simply removed it. But the good news is, that I have implemented something new: A way to add reference fields - that's what I call them - to a project. Alternatively these can be imported from a CSV file.

The effect for the user is the same as before: The user selects from a dropdown - no typing in of field names or field types. At the same time, I have a sort of documentation of the source fields that comes in very handy when writing rules or actions.

And it makes the tool - sorry Pentaho fans - independant of PDI. And that is what the rule engine and the Business Rules Maintenance tool always were. The rule engine can be used with any Java application to check, update or filter data based on business rules.

Give it a try. The tool and the rule engine are constantly growing and maturing.