I will not review that, because that's not part of the jbpm codebase, if you decided to add your own TransactionManager, you need to make sure that it works correctly and that its not causing issues with the frameworks that use it.

Without making any changes, I was trying to understand this behavior. And I noticed that rollback only fails when I use debugger and try to see the execution by putting breakpoints and going line by line. If no breakpoints are used, rollback work correctly. First time I saw this problem, I was also using debugger.

But I still don't understand why this behavior when using debugger. I guess some kind of timeout is happening due to which rollback doesn't work. If this is a timeout issue, it has to be very small timeout. maybe couple of seconds. Because even when debugging I'm not waiting minutes but going line by line quickly.. may be just delay of 4-5 secs. Without understanding the issue, I'm not sure if it can happen in production too.

Good to know that you find the issue. and as far as I remember the default transaction time out is 5 seconds. Which IMO is ok. The Bitronix documentation mention 60 seconds here: http://docs.codehaus.org/display/BTM/Configuration, but I guess that the underlying DB is closing the connection after 5 seconds.

Can you share the stack trace? if everything goes ok inside the method invocation, it shoudn't take more than 5 secs at least not for the jBPM side.