I have a workflow that interacts with some deeply asynchronous external systems. These systems send messages to my system and the receipt of each message is handled by its own thread. On receipt of a message, the thread needs to signal the process instance associated with the message to move it out of its wait state to the next state.

The issue is that the process instance may still be processing a previous message... and that multiple messagess may arrive and these should be used to signal the process instance in time arrival order.

Is there a way of "working with the grain" of jBpm to accomplish what is essentially queuing access to to a process instance, only allowing a thread to signal when the instance is not in use by another thread? ... or do I need to implement infrastructure external to jBpm based on something like JMS?

I had the same problematic. It's not a JBPM problem but a process state consistency. I presume that you're receiving msg form asynchronous systems via an MDB. In that case, you can change the PoolSize of that EJB to One, so that it will proceed messages one by one.

You exactly understand the problem... and you are right, it is not specifically a jBpm issue. There is a resource (the process instance) that needs to be "used/updated" by each thread in turn, not all at once.

The reason I did not want to use your suggestion is that I think that this would block all traffic... In 60 seconds I may have 1,000 incoming events/threads, destined for 950 different process instances. In the vast majority of cases there is no contention and i could, in theory, process 950 process instances in parallel... with only 50 blocking.

The only solution that I can think of is to either use the jBpm lock facility OR to have an external locking process.1) Using the jBpm lock facility

The problem with this is it is pretty heavyweight (in terms of database updates done) and also I have to write a fair chunk of code to deal with threads that die without releasing their lock etc etc etc.

2) Implementing a a lock facility using the database, for example mysql

SELECT GET_LOCK('lock1',10)

The problem with this is that I need to ensure that no thread attempts to acquire a process instance without first getting a lock.

Unfortunately the solution is multi node, so I am not able to do in memory locking easily.

I had hoped that jBpm might have a clever way of achieving this without me having to write any code. At an abstract level I believe that this problem is similar to a process instance reaching a task node whose task could be performed by multiple actors. Presumably in this case jBpm has some mechanism to prevent multiple actors fulfilling the task.

Guys.... please...... how does a random incomming message know how to signal the process? You clearly do not state that. You say just signal the process. Your example demonstrates that this is what you think you will do. What is the advantage of having a process if any incoming message for that process instance can signal the process independent of the state it is in....

I had hoped that jBpm might have a clever way of achieving this without me having to write any code.

It has, it's called the utopia feature.... now seriously... see my statement above. jBPM has no standard way to be able to service all different kinds of requirements.

What you have to build is a component that knows the relation between a state/task/node and an incomming message (e.g. via a task-name<->xsd mapping). An incomming message should trigger that code to look up a task with the corresponding name and signal that *task*, not the processinstance.

Presumably in this case jBpm has some mechanism to prevent multiple actors fulfilling the task.

It has... it's called assigning a task to an actor. Sure, you can circumvent this if you use a low level api. There will not be an access violation or anything if another actor does it, but that means *your* application provided a way to achieve this.

jse I understand your problem. What I have done is to add process variable in y process Instance to say of it's idle or not (in the wait state). If it's idle, i fire the process else I persist this msg to be proceeded by the process when previous message execution finish.So that, the execution is not blocked. It's monthread for each process Instance and not for all instances

I am fairly new to jBpm and am just looking for the appropriate "jBpm way" of solving the problem.

I think, perhaps, my first post was a bit too abstract to be clear. So... here is a more concrete example.

Lets envisage that jBpm (J) is responsible for escalating a problem with external system P. The means of escalation is email, using external emailing system E. All interaction with the external systems is via http. E(mail) will provide J(bpm) with delivery notification and read receipt information via http callbacks. These delivery notfifications and red receipts need to be sent to the P(roblem) system via http callback. So... as a sequence diagram we have:

The challenge here is that if Thread J2 gets blocked (perhaps because system P is a bit slow), and thread J3 receives the read receipt whilst thread J2 is still in progress, I need to ensure that thread J3 does not acquire the process instance until thread J2 has released it.

I think that you are saying that I could model this in a different way, using tokens to ensure that the two async events are handled correctly (i.e. thread J2 will signal the "delivery" token and thread J3 will signal the "read" token. For example:

I hope it is an internal emailing system where users have no control over whether or not they send de read notification.

think that you are saying that I could model this in a different way, using tokens to ensure that the two async events are handled correctly (i.e. thread J2 will signal the "delivery" token and thread J3 will signal the "read" token. For example:

btw, you seem to do things synchronous in a transaction like way from E to J to P. Is there a mechanism towards P if the 200 to E cannot be send? If not, why not just do it async? I think/hope btw this is a minimal version of the process to demonstrate your issue. Otherwise the added-value of jBPM is very little

Thanks for your answers. Am I right in thinking that two threads can, at the same time:1) load the same ProcessInstanceForUpdate2) Signal to their different tokens3) Close the context (and thus commit the changed process instance state back to the database)...without causing any clashes or hibernate exceptions, i.e. the different tokens provide true seperation between the two threads.

To your other point:

Is there a mechanism towards P if the 200 to E cannot be send? If not, why not just do it async?

The main reason for wanting to keep async to a minimum is that it is quite heavy weight, requiring additional database persistance etc