I've been trying to get BDB JE to participate in JTA XA transactions using the Atomikos JTA implementation. However, when the current Transaction calls XAEnvronment#close() (as a prelude to committing the transaction), I get a TransactionException. Further investigation shows the exception to be thrown on line 142 of XAEnvironment.java. Just before that point in the code, a check is made to see if the transaction is currently suspended. If the transaction is not currently suspended the exception is thrown. I believe the exception should be thrown if the transaction IS currently suspended. Certainly, changing line 141 from

if (!isSuspended) {

to

if (isSuspended) {

and recompiling solves the issue for me, and BDB is able to correctly participate in the transaction.

Bungle wrote:
I've been trying to get BDB JE to participate in JTA XA transactions using the Atomikos JTA implementation. However, when the current Transaction calls XAEnvronment#close() (as a prelude to committing the transaction), I get a TransactionException. Further investigation shows the exception to be thrown on line 142 of XAEnvironment.java. Just before that point in the code, a check is made to see if the transaction is currently suspended. If the transaction is not currently suspended the exception is thrown. I believe the exception should be thrown if the transaction IS currently suspended. Certainly, changing line 141 from

if (!isSuspended) {

to

if (isSuspended) {

and recompiling solves the issue for me, and BDB is able to correctly participate in the transaction.

I'm not sure it's that easy, but that doesn't mean there isn't a bug in our code. Take a look at table 6-2 in http://pubs.opengroup.org/onlinepubs/009680699/toc.pdf

I'm passing in TMSUCCESS to the end() method. According to the table you directed me to, when this flag is passed in to xa_end, the only illegal state is if there is no transaction branch associated with the thread (state T0 in the table). If the transaction branch is in either state T1 (associated) or T2 (suspended), then xa_end(TMSUCCESS) should update the state of the transaction branch to T0 (not associated).

According to the table, a call to xa_end when the thread's transaction is suspended is only illegal when the TMSUSPEND flag is passed in.

I'm not sure exactly how the possible states of the Transaction and Txn class map to states T0, T1 and T2, but I would suggest that XAEnvironment#end() needs behave something like:

if (transaction == null) throw new XAException(XAException.XAER_NOTA); // Assuming a null transaction maps to state T0
if (tmSuspend) {
if (transaction.isSuspended()) throw new XAException(XAException.XAER_NOTA);
else transaction.setSuspended(true);
}
if (tmFail) new XAFailureException(txn);
if (tmSuccess) { /* Do nothing??? */ }

We've corrected the end() method, more or less as you've described. Our tests pass, but I'm wondering if you would be willing to test the jar before we commit the changes, as a double check. Please email me -- mark.hayes at o.com (o == oracle) -- and I'll send you the jar to try out.