Java and BPM Developer blog

Menu

Monthly Archives: June 2014

This means to be a sequel of my previous post, where we started discussing the benefits of writing rules. I intend to do a similar thing, but now centred on the concept of Business Processes. As with last time, this post is oriented to people just getting started with the concept of processes, who are still getting into it. Processes are a simpler concept to grasp if you compare it to rules, but I still think we should start with a basic description.

What is a process?

The simplest explanation of a process I can come up with is “a sequence of steps to achieve something”. Actually, I wasn’t able to come up with it, it is a dictionary definition. But it is as simple as that, just a sequence of actions that need to be taken, by either systems or people, to achieve a goal. If we talk about a business process, we just need to add a little more definition to it and say it is a sequence of steps on a specific domain, performed by systems or people, to achieve a specific business objective.

If you come from the developer end you might be thinking “so? any code I ever wrote is a sequence of steps towards something!”. And you’re right! Code itself follows a sequence. It also has a purpose. If it reads inputs from a specific source, it can interact with systems or people, depending if the source is a specific integration component or a user interface. There are, however, several advantages that process definitions provide over plain code. We’ll cover some of them in this post

Diagramming structure

The first and most user friendly feature is the possibility of seeing the process definition as a diagram. We won’t need to see the process as an obscure list of method invocations, but as an image like this:

If defined in the BPMN2 standard (the current most accepted standard for defining executable processes), all the diagramming information is available in the process definition itself, so the same information used to determine which is the next step to follow will also define the structure for visualizing the process. This is very helpful for business users who probably don’t know about a specific technology, but they have a very clear understanding of the what needs to be done at what time. Which leads to the next advantage.

Process definition as a collaborative tool

Once you have a diagram, non-technical people can understand each step of what is happening. Thanks to process editors provided by each implementation of BPM, people with no specific development background can also edit the content to match the real world scenario. This makes the process definition the centre of our development effort. It not only provides itself as a documentation thanks to the diagram, but it also works as a connection point between business analysts and technical people. Business Analysts need to worry only about the specifics of the process completion (that it should match the specific objectives, what order should the steps in the process be following, who should be in charge of what human task, etc), and technical people need to worry about integrating the process with any other existing service. Which leads to another advantage

Business users take care of business. Development takes care of integration

This leads to an eventual freedom from the development side, from having to worry about making each component to follow ever changing processes. Developers, in this paradigm, only need to worry about providing the necessary connectors to different systems and people. By connectors, we mean system invokers that can be called from the process, components that will be able to invoke or signal a process, and forms for people to be able to do human tasks. From a developer’s perspective, the process will be something like this:

Where you will only need to worry about connecting processes with other systems and providing a UI for new forms. In a way, it releases developers from having the complete responsibility of defining the structure of the system, allowing more people to collaborate in the definition of how things are done, and letting developers concentrate on connecting those definitions with other systems.

Scalability

Since development efforts on the process definition side will decrease over time, people in the development area can start working on what they really know to do best. Better connectors, and better environment configurations. Eventually, as process execution demand starts growing, you can start working on the scalability of your BPM systems. And since all activities move around the same structures (the process executions), you can concentrate on making ways for processes to take as little time executing steps as possible, managing signals in distributed ways, and managing responses from the connectors in a way that more than one node could be able to receive them.

This leads to not only better running systems, but also a more manageable scalability of them. As all core processes of the company start moving to BPM systems instead of legacy applications, high availability and scalability of systems starts becoming an ubiquitous feature.

A company as a process (composed of other processes)

The final stage of this endeavour is getting more and more processes of the company working inside a BPM system. Since processes can invoke other processes inside them (called sub-processes), you can make a hierarchy of components which can be as broad as desired, to cover as many operations of your company. An ideal point in this stage will be reaching the main process of the company, where usually client or service request are the input, and the objectives of the company are the output. Inside that process (which, beleive it or not, could be quite a simple diagram), you will invoke every other process you define for each specific case in a hierarchy of processes invocations.

Summing up

Processes become a great way to involve as many parts of the company in the definition of the steps needed to perform a task. Nobody knows better how to do a specific task than the operation teams, and nobody understands better the objectives of a company than the managers. Also, nobody understand better how to accomplish those tasks in an automated way than the developer teams. With process definitions, each one of them can collaborate on the same component, all working together the same goal: improving the objectives of a company.

Also, process engines should take care of two things: Coordinating tasks between other systems and people, and taking care of storing the state of current active processes. It is like this because it should let other systems with other objectives do what they do best. Web Services should be in charge of services. Rules should be in charge of complex decisions. In general, a good process engine should try to be something like the following description (which I like to read with De Niro’s voice in this scene of “Men of Honor” 🙂 ):

A process engine isn’t a monolithic system, it’s an integration platform. If it should be done by someone else, it invokes it. If it needs a person, it notifies him. If many tasks need to be done at the same time, it synchronizes them. If it is lucky, it will execute for a few milliseconds for a process that should run over a full year, for that is the closest it will ever get to being considered efficient.

And please stay tuned for the next post, where we will be discussing the power of putting rules and processes together!

I’ve always gone into a lot of detail regarding specific points of the rule engine, but I feel this is a topic which needs to be covered more. So this post is oriented to people who might just be getting started with the concept of rules and are still in doubt. I will try to explain some of the advantages of using business rules and rule engines in a way that, oddly enough, I haven’t found online yet.

What is a rule?

If you got as far as reaching this blog, you probably already have an idea of what a rule is. The most primal explanation we can give is “something that looks like this”:
when a condition is found to be true
then a consequence is executed

And that’s about it. Add as much syntax sugar as you want on top of it, but a rule will still have that basic structure underneath it all. I still remember the first question that rose to the top of my head when they explained what a rule was: How is this any better than writing code? It took me a while to figure it out, but I’ll try to make it easier for you

Let’s begin by explaining the reason of such a simple structure. It’s based on an Artificial Intelligence principle, where a specific structure of code can be represented as data, in a structure similar to the one shown in the following diagram:

Why would we want to represent our code as data? Because manipulating that data makes a system able to change what it is doing, and it is one of the pillars of machine learning. Having the execution code as data means that we can change the data and make the system behave in a different way without having to restart any component. But that is not even the beginning of why rules are something useful:

It is not what one rule can do, but what many together can do

If we think of an extremely simple case, where maybe we have one or two rules to define a scenario, we can definitely think of a simpler way of doing the same scenario using nothing but Java. The power of rules doesn’t reside in the possibility of explaining simple scenarios, but quite the contrary. It lies on explaining complex scenarios in a very simple and performing way. It divides a problem in the most atomic components: each individual decision we could take, together with the minimal information we need to take that decision. No rule should extend beyond that complexity.

Rules don’t know each other

This is a principle most rule engines abide: Rules are independent. Each rule doesn’t know if another rule even exists. All they share is the information about the state of the world. This information is shared, and some rule engines (drools included) allows for rules to modify this information, in order for other rules to be activated. This process of activating other rules by changes in the information of the state of the world (called working memory) is called inference.

Because of this mechanism, rules don’t need to be too complex to describe a complex scenario: a complex rule can be divided into simpler rules that infer parts of information of a complex scenario and feed it to the working memory, and other rules might depend on that data to get activated. Which leads to the next item regarding how to manage complex, ever-growing situations

Scenario becomes more complex? Just add more rules

This is how you should manage complex scenarios. You don’t have to worry about having a very complex list of commands to evaluate each condition on a complex decision. All you need is to keep creating atomic rules, until you have covered all the different cases. Rules become the epitome of decoupling software: Each rule should just worry about understanding a very small and basic concept from the information it can gather from the state of the world

Letting the rule engine do its job

Rules should remain that simple because it is the rule engine that will manage the complexity for us. Whenever we need to detect many similar conditions, the rule engine will create a performing way of evaluating those conditions as fast as possible, through the RETE (in drools 5) and PRHEAK (in drools 6) algorithms. They will do the heavy lifting. It’s something that we have seen before:

If you’re as old as I am, you might have been there when development teams started going from compiled code (like C or C++) to interpreted code (like Java). I didn’t like it that much at first: I needed more resources to run the programs I made, I had to learn new APIs, everything was a class now… it just wasn’t my cup of tea, at least until I got the hang of it. The thing was that, in order to write good Java code, I needed to follow very different standards from those to write good C code. I didn’t need to create fancy optimizations in our code because those optimizations were going to be done by the VM. And eventually, simpler code became more efficient than complex code hacks we brought with us from C, mainly because the VM optimizations were doing it so much better than us.

And in a sense, a rule engine does the same thing. It will take all our rules and transform them into something that can run in the most efficient way to find all the matches to our rules. In a system with many rules, the code to evaluate the conditions will be far quicker than the optimizations we might add to a batch processing on a code block.

And that is pretty similar when talking about rule engines and rules. Its like the VM and Java code all over again. Someone will be able to do a better job than you at optimizing simple scenarios, so we shouldn’t have to worry about optimizing the code so much as making it clearer.

Fitting more hands in a plate

There is a common saying in engineering: You can’t make 9 mother make one baby in a month. We use it as a clear example of how some tasks cannot be divided any more, and have specific dependencies between each other. The phrase is used to describe situations where you have coupled code, and even if we divide a problem into many different tasks, people will end up touching the same components (and experiencing the expected conflict) when trying to solve two different situations.

One of the greatest advantages of rules is how highly detached they are. This means more people can be editing the rules without stepping in each other toes. The simpler the rules, the less experience needed to edit them. And thanks to human language to rules mappings (like DSLs and Decision Tables), you can have non-technical people defining rules. This means more people can get involved in rule definition, and help extend a system that could be as complex as necessary, and do it in the simplest of ways.