Guys,
I totally understand how JPA persistence should work according to the spec. I also understand that EclipseLink is not obliged to preserve the order of the ops.
However I think it is a really useful feature and if not available by default than it should be at least available via a config option.
For now I have switched back to Hibernate in order to finish my project. However I'd like to use EclipseLink long term as it provides many great features helpful for
implementing legacy database designs in JPA. One of the reasons I really love using open source is that I could modify if when I need to and I could suggest my changes to the
community :)
After I deliver my current project I'll propose a patch providing the opportunity to change the default UnitOfWorkChangeSet implementation via an eclipselink property, as well as another implementation
of UnitOfWorkChangeSet which preserves the order of the operations in case they have not been reordered. I don't want to go into development details now, I just had a quick look at the code.
I think the leading principle when optimising things in general should be:
If you can improve it - do it. If you can do nothing to improve it at least don't mess it up, leave it the way it was ! :)
Thank you all for the quick responses !
Cheers,
Deyan
On Jan 17, 2013, at 16:27 , Christopher Delahunt <christopher.delahunt@xxxxxxxxxx> wrote:
JPA might not be the best tool for a raw data dump such as this, as it seems more geared to a straight JDBC insert statement type operation - since you are not bothering with the object level references.
That said, JPA did add in the flush method for this purpose. When you need to force statements to be executed before others, call flush. If this is a performance drain, try calling clear after flush as it will reduce the work the EM does on each iteration. This is the only real JPA solution if you cannot fix the object references.
Outside of JPA, you can use EclipseLink native methods to execute InsertObjectQuery instances to perform the insert.
If you care, the reason why you do not get the order you expect is that entities are cached, hashed on their IDs. When you call persist or merge, the object is put into the cache and later iterated over. Since there are no relationship dependencies here, the list received from the cache is inserted as it appears - random, since maps are strange. Unfortunately what you need is a cache that behaves both as a queue and a hashmap at the same time. Feel free to request the feature as allowing ordering on something other than the primary key might be something others find useful as well.
Best Regards,
Chris
On 17/01/2013 9:02 AM, Deyan Tsvetanov wrote:
> Ok,
> so long with eclipselink.
>
> thanks for the help anyway :)
>
> Best regards,
> Deyan
>
> On Jan 17, 2013, at 16:00 , Tom Ware<tom.ware@xxxxxxxxxx> wrote:
>
> I am saying that for a properly mapped and populated object graph, it shouldn't matter.
>
> On 17/01/2013 8:55 AM, Deyan Tsvetanov wrote:
>> So you don't see a problem that eclipselink randomises the order of the db operations when it can't determine it from the relationships ?
>>
>>
>>
>> On Jan 17, 2013, at 15:52 , Tom Ware<tom.ware@xxxxxxxxxx> wrote:
>>
>> As soon as you add the possibility of cascading (a highly used JPA feature), there is no choice but for the persistence provider to pick an order. With cascading, one persist or merge operation can result in a large set of insert, update and delete operations. A JPA provider(any JPA provider) needs to be able to execute these statements in an order that allows it to satisfy all the database constraints it is aware of.
>>
>> The key thing is that if you provide a JPA provider a properly mapped and populated object graph, it should be able to persist it without constraint issues regardless of the order your call your persist, merge and delete object in).
>>
>> -Tom
>>
>> On 17/01/2013 8:45 AM, Deyan Tsvetanov wrote:
>>> I see your point guys but it won't work for me.
>>>
>>> So EclipseLink tries to determine the best order of the database operations. That is OK.
>>> But if it can't determine an order it executes them in a RANDOM order instead of the NATURAL ( set by developer ) order ?
>>> That sounds spooky :)
>>>
>>>
>>>
>>> On Jan 17, 2013, at 15:43 , Christopher Delahunt<christopher.delahunt@xxxxxxxxxx> wrote:
>>>
>>> I believe Tom is stating that you are not setting the Role reference (parentRole is null when you insert). If you set that, then EclipseLink will see the dependencies and order the inserts appropriately.
>>>
>>> EclipseLink has a cache, so it should simply be a mater of calling newRole.setParentRole(em.find(Role.class, newRole.getParentId()));
>>>
>>> Best Regards,
>>> Chris
>>>
>>> On 17/01/2013 8:34 AM, Deyan Tsvetanov wrote:
>>>> Oops :)
>>>>
>>>> Wrong copy+paste. Please consider RoleType as Role. Role has a reference to another parent Role.
>>>>
>>>> I do have reasons indeed :)
>>>> But my mapping strategy is not the key point here.
>>>>
>>>> You are free to modify the example and make it your way. It will still fail for the same reason:)
>>>>
>>>> Regards,
>>>> Deyan
>>>>
>>>>
>>>> ------------------------------------------
>>>>
>>>> @Entity @Table(name="ROLE")
>>>> class Role {
>>>>
>>>> @Id @Column(name="ID", length=20, nullable=false)
>>>> String id
>>>>
>>>> @Column(name="PARENT_ID", length=20, nullable=true)
>>>> String parentId
>>>>
>>>> @Column(name="NAME", length=255, nullable=false)
>>>> String name
>>>>
>>>> @ManyToOne
>>>> @JoinColumn( name="PARENT_ID", referencedColumnName="ID", updatable=false, insertable=false)
>>>> Role parentRole
>>>>
>>>> }
>>>> On Jan 17, 2013, at 15:29 , Tom Ware<tom.ware@xxxxxxxxxx> wrote:
>>>>
>>>> How is RoleType mapped?
>>>>
>>>> Is there a particular reason you have made the parentId mapping writable and the parentRole mapping readOnly, or are these mappings just for your import tool? If I were to map these entities, I would consider the opposite... That would allow me to interact with the entities in a way that was much more Object-Relational, and I think in a way that would make using your mappings for other applications more practical. I'd write code more like this:
>>>>
>>>> beginTransaction()
>>>> for (XMLRoleWrapper xmlRoleWrapper xmlResults){
>>>> Role role = new Role();
>>>> em.persist(role);
>>>> role.setId(xmlRoleWrapper.id);
>>>> role.setName(xmlROleWrapper.name);
>>>> role.setParentRole(em.find(RoleType.class, xmlRoleWrapper.parentId));
>>>> }
>>>> commitTransaction();
>>>>
>>>> The problem with ordering operations exactly as the user writes them, is that it works well for very simple cases, but as soon as you add merging, cascading and the possibility of cycles it breaks down pretty quickly.
>>>>
>>>> -Tom
>>>>
>>>>
>>>> On 17/01/2013 8:12 AM, Deyan Tsvetanov wrote:
>>>>> Hi Tom,
>>>>>
>>>>> here is a more specific test case:
>>>>>
>>>>> @Entity @Table(name="ROLE")
>>>>> class Role {
>>>>>
>>>>> @Id @Column(name="ID", length=20, nullable=false)
>>>>> String id
>>>>>
>>>>> @Column(name="PARENT_ID", length=20, nullable=true)
>>>>> String parentId
>>>>>
>>>>> @Column(name="NAME", length=255, nullable=false)
>>>>> String name
>>>>>
>>>>> @ManyToOne
>>>>> @JoinColumn( name="PARENT_ID", referencedColumnName="ID", updatable=false, insertable=false)
>>>>> RoleType parentRole
>>>>>
>>>>> }
>>>>>
>>>>> I have created a tool that imports data from XML file. In this case the XML would look like:
>>>>>
>>>>> <data>
>>>>> <Role id="MAIN" name="Main Role" />
>>>>> <Role id="USER" name="User" parent_id="MAIN"/>
>>>>> <Role id="ADMINISTRATOR" name="Administrator" parent_id="MAIN" />
>>>>> </data>
>>>>>
>>>>> The MAIN role has NULL in PARENT_ID. The rest of the records have MAIN as a parent role.
>>>>> The order of the insert statements is vital for the successful import of the data.
>>>>>
>>>>> I have implemented a XML parser that creates new entities and persists them in the order they are coming from the XML file.
>>>>>
>>>>> HOWEVER
>>>>>
>>>>> Hibernate inserts the data in the correct order - e.g. MAIN, USER, ADMINISTRATOR.
>>>>>
>>>>> EclipseLink however would execute the insert statements in a random order every time, e.g. USER, MAIN, ... which causes a FK error.
>>>>>
>>>>> Flushing the session on each persist is not a solution.
>>>>>
>>>>> EclipseLink needs to execute the database statements in the same order as the persist() and merge() calls.
>>>>>
>>>>> Regards,
>>>>> Deyan
>>>>>
>>>>>
>>>>>
>>>>> On Jan 17, 2013, at 15:00 , Tom Ware<tom.ware@xxxxxxxxxx> wrote:
>>>>>
>>>>> Can you provide a specific test-case where the order actually causes a problem?
>>>>>
>>>>> The JPA specification allows a persistence provider to order statements in the way that is most efficient for it. This, in fact, is essential to solving the very problem you are worried about. i.e. A JPA provider has to be able to compute an order of statements that allow foreign key constraints to be properly satisfied. As long as your entities are configured so that JPA is made aware of all your constraints, the order of statements should not cause any problems of that type.
>>>>>
>>>>> The power of this functionality is that when you are writing your JPA code, you do not have to think about the database constraints, you just manipulate your objects and the order will be taken care of.
>>>>>
>>>>> -Tom
>>>>>
>>>>> On 17/01/2013 6:27 AM, Deyan Tsvetanov wrote:
>>>>>> Well, the order that EclipseLink chooses is RANDOM :)
>>>>>> It is a fairly simple example, only 1 entity with only 1 column.
>>>>>> I'd really expect that the INSERT statements are executed in same order as the
>>>>>> persist() calls.
>>>>>>
>>>>>> My real-life use case is importing data from an XML file into the database.
>>>>>> There are relations and FKs in my database and when exported and re-imported the
>>>>>> order of the XML entries, persist() calls and INSERT statements is critical.
>>>>>> EclipseLink basically inserts each XML entry randomly. Currently the only
>>>>>> workaround is to flush after each persist call. It could work for few hundred
>>>>>> calls,
>>>>>> but not for few thousand.
>>>>>>
>>>>>> JPA says nothing about the order of the database operations. When writing the
>>>>>> spec they probably have assumed that it would be logical to execute the database
>>>>>> operations in the
>>>>>> same order as the persist() or merge() calls.
>>>>>> This is not the case of mixed remove(), persist() and merge() calls, in our case
>>>>>> we have only persist() calls and the case is very simple.
>>>>>>
>>>>>> Best regards,
>>>>>> Deyan
>>>>>>
>>>>>>
>>>>>> On Jan 17, 2013, at 13:19 , Wim Bervoets<wbervoets@xxxxxxxxx
>>>>>> <mailto:wbervoets@xxxxxxxxx>> wrote:
>>>>>>
>>>>>> If you want to know the order in which the rows were inserted I use @OrderColumn
>>>>>> (eg. in combinantion with INDEX(..) function in a JPQL for example).
>>>>>>
>>>>>> I think that EclipseLink can choose the order in which it commits the entities
>>>>>> to the database... (I haven't read the JPA spec so this is an assumption)
>>>>>>
>>>>>> Wim
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 17, 2013 at 12:09 PM, Deyan Tsvetanov<deyan@xxxxxxxxxxxx
>>>>>> <mailto:deyan@xxxxxxxxxxxx>> wrote:
>>>>>>
>>>>>> Hibernate persists the entities in the correct order:
>>>>>>
>>>>>> 2:57:58,867 TRACE TypeFactory:72 - Scoping types to session factory
>>>>>> org.hibernate.internal.SessionFactoryImpl@395fa2b5
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,365 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST0
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,369 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST1
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,371 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST2
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,372 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST3
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,373 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST4
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,375 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST5
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,376 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST6
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,377 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST7
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,378 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST8
>>>>>> Hibernate: insert into ROLE (ID) values (?)
>>>>>> 12:57:59,380 TRACE BasicBinder:83 - binding parameter [1] as [VARCHAR] - TEST9
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jan 17, 2013, at 12:19 , Deyan Tsvetanov<deyan@xxxxxxxxxxxx
>>>>>> <mailto:deyan@xxxxxxxxxxxx>> wrote:
>>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>> I am experiencing a weird imho behaviour of Eclipselink and I'd really like
>>>>>> to hear some other opinions .
>>>>>>
>>>>>> I have a pretty simple entity with assigned IDs;
>>>>>>
>>>>>> @Entity@Table(name="ROLE")
>>>>>> public class Role implements Serializable {
>>>>>>
>>>>>> privatestaticfinallongserialVersionUID= 1L;
>>>>>>
>>>>>> @Id @Column(name="ID", length=20, nullable=false)
>>>>>> public String id;
>>>>>>
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> I am executing the following operations:
>>>>>>
>>>>>> public static void main(String[] args) {
>>>>>> EntityManagerFactory emf =
>>>>>> Persistence.createEntityManagerFactory("EclipseLinkJPATest");
>>>>>>
>>>>>> EntityManager em = emf.createEntityManager();
>>>>>>
>>>>>> em.getTransaction().begin();
>>>>>> for (int i = 0; i< 10; i++) {
>>>>>> Role r = new Role();
>>>>>> r.id = "TEST" + i;
>>>>>> em.persist(r);
>>>>>> }
>>>>>> em.getTransaction().commit();
>>>>>>
>>>>>> em.close();
>>>>>> emf.close();
>>>>>> }
>>>>>>
>>>>>> And I'd expect that the INSERT queries will be executed in the same order as
>>>>>> the persist() method is called:
>>>>>> TEST0, TEST1, TEST2 , etc.
>>>>>>
>>>>>> But in the real life the insert queries are in a random order every time:
>>>>>>
>>>>>> EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.11--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST1]
>>>>>> [EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.113--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST6]
>>>>>> [EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.114--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST2]
>>>>>> [EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.115--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST7]
>>>>>> [EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.117--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST4]
>>>>>> [EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.121--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST8]
>>>>>> [EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.123--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST3]
>>>>>> [EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.124--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST9]
>>>>>> [EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.126--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST5]
>>>>>> [EL Fine]: sql: 2013-01-17
>>>>>> 12:13:58.127--ClientSession(1694665796)--Connection(1795160456)--INSERT INTO
>>>>>> ROLE (ID) VALUES (?)
>>>>>> bind => [TEST0]
>>>>>>
>>>>>> As you can see the order if insert queries is:
>>>>>> TEST1, TEST6, TEST2, TEST7, TEST4, etc.
>>>>>>
>>>>>>
>>>>>> That is really weird and wrong ! :)
>>>>>> I dug a lot and could not find a solution.
>>>>>>
>>>>>> Please help :)
>>>>>>
>>>>>> Thanks in advance,
>>>>>> Deyan
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> eclipselink-users mailing list
>>>>>> eclipselink-users@xxxxxxxxxxx<mailto:eclipselink-users@xxxxxxxxxxx>
>>>>>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> eclipselink-users mailing list
>>>>>> eclipselink-users@xxxxxxxxxxx<mailto:eclipselink-users@xxxxxxxxxxx>
>>>>>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> eclipselink-users mailing list
>>>>>> eclipselink-users@xxxxxxxxxxx
>>>>>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>>>>>
>>>>> _______________________________________________
>>>>> eclipselink-users mailing list
>>>>> eclipselink-users@xxxxxxxxxxx
>>>>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>>>>
>>>>> _______________________________________________
>>>>> eclipselink-users mailing list
>>>>> eclipselink-users@xxxxxxxxxxx
>>>>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>>>>
>>>> _______________________________________________
>>>> eclipselink-users mailing list
>>>> eclipselink-users@xxxxxxxxxxx
>>>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>>>
>>>> _______________________________________________
>>>> eclipselink-users mailing list
>>>> eclipselink-users@xxxxxxxxxxx
>>>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>> _______________________________________________
>>> eclipselink-users mailing list
>>> eclipselink-users@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>>
>>> _______________________________________________
>>> eclipselink-users mailing list
>>> eclipselink-users@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>>
>> _______________________________________________
>> eclipselink-users mailing list
>> eclipselink-users@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>
>> _______________________________________________
>> eclipselink-users mailing list
>> eclipselink-users@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>>
> _______________________________________________
> eclipselink-users mailing list
> eclipselink-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
>
> _______________________________________________
> eclipselink-users mailing list
> eclipselink-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/eclipselink-users
_______________________________________________
eclipselink-users mailing list
eclipselink-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/eclipselink-users