Left. No right! Testing processes with scenarios.

18. June 2010,
by
Bernd Rücker

Test Scenarios

In a current project we develop an executable process. Hence, this process is a “piece of software”, which need to be tested. In software development this is common sense, so there are enough tools and best practices. Is that the same for technical process models? Hmm, not that easy. On the one hand we can use proved Java Test frameworks like JUnit for Open Source Process Engines like jBPM. On the other hand processes have some specialties, which aren’t surprising but somehow challenging…

For the process we devloped process scenarios, which indicate on which path a process instance flows through the process. The picture on the left shows a foto from the current project. By the way: The green mark shows: This scneario was successfull tested Sorry for the bad quaility of the picture, but otherwiese I couldn’t have posted it here.

To implement such a scenario we can write a unit test, which set up some test data, trigger the process and waits, till the process arrives in a defined state. As soon as the process waits for an external trigger (like a Human enter some data or a system sends an asychronous answer) the unit tests comes into action and “fakes” that answer. By doing so, the whole process can be walked through to check if the behaviour is as expected. Some sample code is shown at the very end of this blog post, I don’t want to frighten non IT folks too early

To define these kinds of scenarios is pretty typical by the way, we did that in quite some projects already. The downside is at the moment, that there isn’t much good tooling for that. In the area of Open Source Process Engines there exists basically nothing and the commercial tools tend to be too GUI centered, which normally isn’t developers friendly. I see the following requirements for such a tooling:

We want to organize these scenarios! We want to mark them in the real process model, save it and load it later on.

I want to compare the audit data from the really executed process with the defined scnerios (“Did the process really went that way?”). I want to recognize differences easily and graphically.

I want to create “smaller” scenarios for just parts of the process. In our example process we have different parts, e.g. data checking, customer creation, order provisioning, …. We want to test these parts independantly from each other (I want to spare the explanattion why we didn’t create subprocesses for them). So we want to start a scenario at an arbritrary node in the process, provide some input data and stop it at a defined node where we calidate the outgoing data.

Ideally Input and Output data are provided in a non technical format, like spreadsheets or Fitnesse.

Test Coverage

There is another interessting aspect of process testing: The so called Test coverage. The idea is easy: I want to make sure that every node and path in my processes was executed at least once. If you need every possible combination can be an endless discussion, but in my experience it would be already a big improvement for most projects if they had at least executed every path once. In our project I used a marker to indicate which parts we already tested as you can see indicated on the left. But this coverage report could be generated by the tooling itself, so to visualize this automatically would be another item on my wish list.

Currently we implement the mentioned use cases in camunda fox for JBoss jBPM 3.2.x and Signavio. The visualizations are done in the BPMN model in Signavio, the technical implementation and testing is done in jBPM. But we try to achieve that non technical folks can define, refine or at least read the test scenarios as well. Then we could make the tests and test coverage part of a process cockpit or process status as well.

Process Development Status Indication

By the way: Currently we use a printed BPMN diagram to indicate the status if the development, as you can see on the left. We want to link the nodes with JIRA-Issues, so we can create that indication automatically. Even if paper and whiteboards are good tools, that only works if the whole team is in the same room or at least building. Obviously it is nice to stand in front of the whiteboard for discusssions, bit not having it electronically has some disadvantages:

Not transportable: We could’t easily take the status and scenarios with us into meetings.

Changes: If there are process changes, we would have to reprint and redraw all scenarios.

Keep it up-to-date: After the project or in the next project we want to improve the scenarios or change them according to changed processes. The paper from the last project is normally not longer available at that time.

Distributed teams: As soon as your not together at the same physical location any more you need electronical help to get that running and the collaboration intact.

But this is enough content for an own blog post…

Okay, but I promised to give you some more technical details, so I present an example test case here (note that the asserts are hidden in the ProcessHelper);

public void scenario2() throws Exception {
testDataHelper.setCustomerAddressType(CustomerAddressType.VALID_ADDRESS);
... // set some more data for testcase
long orderId = startOrderProcess();
// provide a test stub for one service called on the way we wanted to "mock" out
processHelper.setTestStubResult(CheckAddressAdapter.class, createCheckAddressResultResult());
processHelper.waitForState("Fraud Check aufrufen");
// "fake" result of fraud service: Fraud OK
correlationFacade.callCorrelationService("Fraud", orderId, "0");
processHelper.waitForState("Bonität prüfen");
// "fake" result of boni service: Boni yellow
correlationFacade.callCorrelationService("Boni", orderId, "1");
// wait for the process to end
processHelper.waitForState("Ende", 70);
}

During the execution the ProcessHelper shows the flow of the process, we “just” need to add some functionality to validate this audit data aginst our expectation in the scenario:

About Bernd Rücker, CEO

Bernd Rücker is co-founder and CEO of camunda. But he is also software developer, trainer and consultant who was activley contributing to the Open Source Workflow engines jBPM 3 and Activiti before starting camunda BPM. He has profound experience in BPM projects and Java Enterprise. He worked with Process Engines and BPM for over 10 years in countless real-life projects. He is author of several books, numerous articles and regular speaker on conferences.