Exception in thread "main" java.lang.IllegalArgumentException: out of field index :6
at org.datanucleus.test.AmountChanges.jdoProvideField(AmountChanges.java)
at org.datanucleus.state.AbstractStateManager.provideField(AbstractStateManager.java:1449)
at org.datanucleus.state.JDOStateManagerImpl.provideField(JDOStateManagerImpl.java:1803)
at org.datanucleus.store.mapped.mapping.PersistableMapping.postInsert(PersistableMapping.java:1030)
at org.datanucleus.store.mapped.mapping.EmbeddedPCMapping.postInsert(EmbeddedPCMapping.java:106)
at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:503)
at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:163)
at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:139)
at org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:2405)
at org.datanucleus.state.JDOStateManagerImpl.flush(JDOStateManagerImpl.java:3781)
at org.datanucleus.ObjectManagerImpl.flushInternalWithOrdering(ObjectManagerImpl.java:3430)
at org.datanucleus.ObjectManagerImpl.flushInternal(ObjectManagerImpl.java:3368)
at org.datanucleus.ObjectManagerImpl.flush(ObjectManagerImpl.java:3325)
at org.datanucleus.api.jpa.JPAEntityManager.flush(JPAEntityManager.java:666)

It seems that the issue is related to the presence of a many-to-one relation (i.e. a PersistableMapping)
since without that member there is no numbering problem

Description

An exception occurs in postInsert callback:
Exception in thread "main" java.lang.IllegalArgumentException: out of field index :6
at org.datanucleus.test.AmountChanges.jdoProvideField(AmountChanges.java)
at org.datanucleus.state.AbstractStateManager.provideField(AbstractStateManager.java:1449)
at org.datanucleus.state.JDOStateManagerImpl.provideField(JDOStateManagerImpl.java:1803)
at org.datanucleus.store.mapped.mapping.PersistableMapping.postInsert(PersistableMapping.java:1030)
at org.datanucleus.store.mapped.mapping.EmbeddedPCMapping.postInsert(EmbeddedPCMapping.java:106)
at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:503)
at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:163)
at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:139)
at org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:2405)
at org.datanucleus.state.JDOStateManagerImpl.flush(JDOStateManagerImpl.java:3781)
at org.datanucleus.ObjectManagerImpl.flushInternalWithOrdering(ObjectManagerImpl.java:3430)
at org.datanucleus.ObjectManagerImpl.flushInternal(ObjectManagerImpl.java:3368)
at org.datanucleus.ObjectManagerImpl.flush(ObjectManagerImpl.java:3325)
at org.datanucleus.api.jpa.JPAEntityManager.flush(JPAEntityManager.java:666)
It seems that the issue is related to the presence of a many-to-one relation (i.e. a PersistableMapping)
since without that member there is no numbering problem

Andy Jefferson added a comment - 13/Apr/11 03:24 PM Fix ? : debug it. Start by printing out what is ClassMetaData perhaps. Does the equivalent JDO case work ? then print out the ClassMetaData for that case and compare.
Workaround ? : debug it

Guido, I can't make a suggestion unless you let me know whether the ClassMetaData for JPA usage is "correct" (with the JDO case being 'correct'). Maybe the ClassMetaData isn't complete and so the "problem" would be in reading of annotations/XML and how that is mapped to internal ClassMetaData. If ClassMetaData is correct then the JDO equivalent case would not work either (and we have many JDO testcases none of which exhibit any problem). The only JIRA still open on embedded is related to allowing inheritance in embedded objects.

Andy Jefferson added a comment - 14/Apr/11 09:19 AM Guido, I can't make a suggestion unless you let me know whether the ClassMetaData for JPA usage is "correct" (with the JDO case being 'correct'). Maybe the ClassMetaData isn't complete and so the "problem" would be in reading of annotations/XML and how that is mapped to internal ClassMetaData. If ClassMetaData is correct then the JDO equivalent case would not work either (and we have many JDO testcases none of which exhibit any problem). The only JIRA still open on embedded is related to allowing inheritance in embedded objects.

The only thing to add is that api-jpa was (under some situation) setting "ownerField" to some value. With JPA it is impossible to have bidir relations with an embeddable, therefore ownerField should be null always, which is what SVN trunk does. Since you say "JDO version fails too" then this is obviously not your problem since JDO does allow bidir relations with an embedded field and that is the only situation where you would have ownerField as non-null (unless of course your JDO version was incorrectly mapped). You can easily verify whether that is the case by using SVN trunk of api.jpa

Andy Jefferson added a comment - 14/Apr/11 05:10 PM The only thing to add is that api-jpa was (under some situation) setting "ownerField" to some value. With JPA it is impossible to have bidir relations with an embeddable, therefore ownerField should be null always, which is what SVN trunk does. Since you say "JDO version fails too" then this is obviously not your problem since JDO does allow bidir relations with an embedded field and that is the only situation where you would have ownerField as non-null (unless of course your JDO version was incorrectly mapped). You can easily verify whether that is the case by using SVN trunk of api.jpa

Now if I remove zMember, everything runs fine.
The "bug" seems to be in the PersistableMapping.postInsert. The instance corresponds to zMember and is added
to AmountChanges that is embedded (only) into a MutableAmount.
The stack trace is:
Exception in thread "main" java.lang.IllegalArgumentException: out of field index :5
at org.datanucleus.test.AmountChanges.jdoProvideField(AmountChanges.java)
at org.datanucleus.state.AbstractStateManager.provideField(AbstractStateManager.java:1449)
at org.datanucleus.state.JDOStateManagerImpl.provideField(JDOStateManagerImpl.java:1803)
at org.datanucleus.store.mapped.mapping.PersistableMapping.postInsert(PersistableMapping.java:1030)
at org.datanucleus.store.mapped.mapping.EmbeddedPCMapping.postInsert(EmbeddedPCMapping.java:106)
at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:503)
at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:163)
at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:139)
at org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:2405)
at org.datanucleus.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:2381)
at org.datanucleus.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1608)
at org.datanucleus.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1435)
at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:724)
at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:749)
at org.datanucleus.test.MainJDO.main(MainJDO.java:30)

I think that the problem appears when a PC object is a member of an embedded class.
I have spent the whole day trying to understand where the problem might be.
I first thought the there was a problem in the numbering of field being -1 but they are -1 even without zMember and are fetched fine. Anyway, it should be a problem in the relative numbering but I really cannot understand
how numbering is managed.
I don't know if it is a case, but it works it AmountChange is embedded in MutableAmount superclass

Guido Anzuoni added a comment - 14/Apr/11 07:02 PM - edited I don't think it could be incorrectly mapped here is the relevant part:
<class
name="AmountChanges" embedded-only="true">
<field name="zMember" >
<column name="fk_a_def"/>
</field>
<field
name="totalPositive"
persistence-modifier="persistent">
<embedded>
<field name="euroCent" >
<column name="tot_var_pos"/>
</field>
</embedded>
</field>
<field
name="totalNegative"
persistence-modifier="persistent">
<embedded>
<field name="euroCent" >
<column name="tot_var_neg"/>
</field>
</embedded>
</field>
</class>
Now if I remove zMember, everything runs fine.
The "bug" seems to be in the PersistableMapping.postInsert. The instance corresponds to zMember and is added
to AmountChanges that is embedded (only) into a MutableAmount.
The stack trace is:
Exception in thread "main" java.lang.IllegalArgumentException: out of field index :5
at org.datanucleus.test.AmountChanges.jdoProvideField(AmountChanges.java)
at org.datanucleus.state.AbstractStateManager.provideField(AbstractStateManager.java:1449)
at org.datanucleus.state.JDOStateManagerImpl.provideField(JDOStateManagerImpl.java:1803)
at org.datanucleus.store.mapped.mapping.PersistableMapping.postInsert(PersistableMapping.java:1030)
at org.datanucleus.store.mapped.mapping.EmbeddedPCMapping.postInsert(EmbeddedPCMapping.java:106)
at org.datanucleus.store.rdbms.request.InsertRequest.execute(InsertRequest.java:503)
at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:163)
at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:139)
at org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:2405)
at org.datanucleus.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:2381)
at org.datanucleus.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1608)
at org.datanucleus.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1435)
at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:724)
at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:749)
at org.datanucleus.test.MainJDO.main(MainJDO.java:30)
I think that the problem appears when a PC object is a member of an embedded class.
I have spent the whole day trying to understand where the problem might be.
I first thought the there was a problem in the numbering of field being -1 but they are -1 even without zMember and are fetched fine. Anyway, it should be a problem in the relative numbering but I really cannot understand
how numbering is managed.
I don't know if it is a case, but it works it AmountChange is embedded in MutableAmount superclass
As you might guess, any help is greatly appreciated

So what you're really talking about is having a class with an embedded field, and with that having a FK to a non-embedded persistable? because your testcase is a long way short of "minimal" and not got time to dig through classes just to find that.

So what happens if you update AbstractMappingManager at line
241 and add 5 lines just after, so you get

This means that if the field is itself part of an embedded object BUT it relates to something non-embedded then we get the "real" metadata for the field.

The reason being, with "embedded fields" the AbstractMemberMetaData is an artificial object with only some of the information set (look at the ClassMetaData you posted above to see why). The EmbeddedXXXMapping classes all understand this IIRC, but if you then create a mapping back to a non-embedded class you need to supply the real AbstractMemberMetaData into the mapping so it will use that thereafter.

Left to see if that is what you're talking about, and test it. No DN tests use such a relationship from an embedded field - minority interest.

Andy Jefferson added a comment - 14/Apr/11 08:16 PM So what you're really talking about is having a class with an embedded field, and with that having a FK to a non-embedded persistable? because your testcase is a long way short of "minimal" and not got time to dig through classes just to find that.
So what happens if you update AbstractMappingManager at line
241 and add 5 lines just after, so you get
// Just get the basic mapping for the type
mc = getMappingClass(fmd.getType(), false, false, fmd.getFullFieldName());
if (fmd.getParent() instanceof EmbeddedMetaData)
{
AbstractClassMetaData cmdForFmd = datastoreContainer.getStoreManager().getMetaDataManager().getMetaDataForClass(fmd.getClassName(), clr);
fmd = cmdForFmd.getMetaDataForMember(fmd.getName());
}
This means that if the field is itself part of an embedded object BUT it relates to something non-embedded then we get the "real" metadata for the field.
The reason being, with "embedded fields" the AbstractMemberMetaData is an artificial object with only some of the information set (look at the ClassMetaData you posted above to see why). The EmbeddedXXXMapping classes all understand this IIRC, but if you then create a mapping back to a non-embedded class you need to supply the real AbstractMemberMetaData into the mapping so it will use that thereafter.
Left to see if that is what you're talking about, and test it. No DN tests use such a relationship from an embedded field - minority interest.

Andy Jefferson added a comment - 07/May/11 04:17 PM Further fix made to AbstractMappingManager to only apply that fix to relation fields. Likely will need refining further to particular relations, but no known problem case for now