a) if some lifecycle callbacks were defined, but no @HasLifecycleCallbacks annotation/mapping was set, warn the user
b) if some lifecycle callbacks were defined, but methods are not public, warn the user

In decoupled applications the model layer returns "data-transfer-objects" through the boundary into the controller/view layer. It would make sense to have Doctrine directly generate any data-transfer/value-object from native and dql queries.

SELECT s0_.id AS id0, s0_.a AS a1, s0_.b AS b2, s0_.c AS c3, s0_.d AS d4, s0_.e AS e5, s0_.f AS f6, s0_.g AS g7, s0_.h AS h8, s0_.i AS i9, s0_.j AS j10, s0_.k AS k11, s0_.l AS l12, s0_.m AS m13, s0_.n AS n14, s0_.o AS o15, 123456789 AS sclr16, s0_.p AS p17 FROM myEntity s0_ WHERE s0_.a = 1 AND sclr16 <= ? ORDER BY sclr16 ASC

SELECT s0_.id AS id0, s0_.a AS a1, s0_.b AS b2, s0_.c AS c3, s0_.d AS d4, s0_.e AS e5, s0_.f AS f6, s0_.g AS g7, s0_.h AS h8, s0_.i AS i9, s0_.j AS j10, s0_.k AS k11, s0_.l AS l12, s0_.m AS m13, s0_.n AS n14, s0_.o AS o15, 123456789 AS sclr16, s0_.p AS p17 FROM myEntity s0_ WHERE sclr16 <= 10

This is not possible unless you take advantage of Topological Sorting to map class dependencies like we do inside of UnitOfWork AFTER creating the ClassMetadata.

The necessity of having this is mandatory because we can never skip classes that have associations to other ones though FK.
You may try that, but it doesn't compensate the effort. I'd rather mark this bug as won't fix, but I'm leaving for you do that. =)

Hi there,
AbsractFileDriver is using the filename to know the managed class.

It's a cool feature because it's allow loading on-demand.
The problem is, that the filename must be the name of the Class.

It should be great to be able to manually map XML/YAML File description to a Class, like :
$drivers->addMappingFile ( array ( "filename" => "class", "filename2" => "class2") );

This feature is simple to implement, just add a new array inside AbsractFileDriver to know the mapping.
When using the current method with addPaths, parse the folder to get traditional XML/YAML file where filename corresponding to classname and add it to the mapping array.

Currently we have a set of Lifecycle events, but they seem to be misleading both in actual implementation and documentation.

One good example is prePersist and postPersist, which is only fired when you're creating new entities. It should be renamed to preInsert and postInsert.
As of preUpdate and postUpdate, they seem quite valid.

But if we rename prePersist and postPersist to (pre|post)Insert, we may have a situation where you wanna cover both insert and update.
For this, (pre|post)Persist should be reinstated, but acting differently from what it does currently.

1. The information isTransient() has to be moved to the ClassMetadataFactory and cached there.
2. The information getAllClassMetadataNames() can be cached
3. A debug/development mode should be introduced, leading to filemtime caching and checks so that you can use ApcCache and such in development.

Short: This issue is all about being able to use doctrine with naked domain objects without any use of doctrine classes.
I 'm not talking about PersistentCollection here, fully aware of that being tied into Doctrine, but those are injected, this is all about code dependency on ArrayCollection.

Seems like some of the UnitOfWork code is cable of handling other types of arrays, like:

// If $actualData[$name] is not a Collection then use an ArrayCollection.
if ( ! $actualData[$name] instanceof Collection) {
$actualData[$name] = new ArrayCollection($actualData[$name]);
}

But in __cascade* functions this is not the case in all but two:

if ($relatedEntities instanceof Collection) {
if ($relatedEntities instanceof PersistentCollection) {
// Unwrap so that foreach() does not initialize

2 however have:

if (($relatedEntities instanceof Collection || is_array($relatedEntities))) {
if ($relatedEntities instanceof PersistentCollection) {
// Unwrap so that foreach() does not initialize

Would it be an idea to do "instanceof Traversable" instead of " instanceof Collection"?

Note: If the fist code block is always performed before the last 2 blocks then there is no issue here, just a need to make it more clear in Doc that this is possible but that you should not rely custom implementation as PersistentCollection will be injected when loaded from db.

For usage with the foreign key as primary key features described in DDC-117 a derived id generator would be tons of useful. It is essentially a post generate id generator (sort of late pre insert though) assigned generator.

If you subscribe to loadClassMetadata you will usually modify the metadata for some classes.
The problem is, that that data has to be loaded from somewhere. But later down the chain you can't get to it. Now data specific to what you need in your loadClassMetadata would ideally reside in the same location.
If we take for example a file, than all data for a specific entity is in the same file.

My proposal would be to add function get(Original|Raw)MappingData into interface Doctrine\ORM\Mapping\Driver\Driver which would either return raw data or data in a object specific for that Driver or null if it doesn't make sense for that driver. Please note, that when loading from e.g XmlDriver we should return simplexmlnode or dom node as loadClassMetadata should be in its own namespace and not pollute the Doctrine one.

@Roman: The UnitOfWork (may) still be pushed as a listener into that entity, and still recieve noticies of update. Which may throw notices because the oid hashes are removed everywhere. Additionally you cant serialize the thing because you still got the UoW inside there.

I think that the UnitOfWork needs to maintain a map of spl_object_hash($newEntity)->$managedEntity for entities that were persisted via reachability during a merge. doMerge should then only call persistNew if the original entity has not already been persisted (if it has already been persisted it should merge the managed entity from the map). The map should be maintained until a flush() or until the UnitOfWork is cleared. The reasoning is as follows.

Obviously in this particular case we should use the return value from the first merge() as the parameter of the second merge which would give correct behaviour.

However, now imagine one Doctor has many Patients and many Patients have one Doctor, all the associations have cascade merge enabled, and further assume that $d1 (Doctor id=1) is already in the database. We now attempt to create two patients and assign them to the existing doctor:

This actually results in 4 rows being added to the 'patients' table instead of 2, I think because $p1 and $p2 are getting persisted both as the root objects and then again from the patient->doctor->patients array. Since the cascade merging happens internally we can't replace the array contents with the managed return values without walking through the object graph (in which case there is no point in using cascade merge in the first place). Maintaining a map in UnitOfWork will allow doMerge to ensure it doesn't persist the same entities twice.

I have tested this patch with my application and it fixes the problem in all my relevant test cases apart from one. The test case that's failing is one that persists a bi-directional many to many relationship, so the associations interweave with each other (if you know what I mean).

I wonder if perhaps doMerge need to continue cascading even if it finds an item in $this->mergedEntities

This is the Flextrine code that fails - it results in no entries in movie_artist. This might also be related to DDC-758?

testMultiMerge tests basic merging of two new entities, checking that only a single entity ends up in the database. This passes with Benjamin's patch.

testMultiCascadeMerge tests the more complex case of merging a OneToMany association. This also passes with Benjamin's patch.

testManyToManyPersistByReachability tests the ManyToMany case described above and this fails with Benjamin's patch, probably because doMerge doesn't cascade down entities that it has already merged and some ManyToMany associations are being ignored. Its a bit hard to be certain what is causing this as even without Benjamin's patch this test would fail due to DDC-758.

Added another failing test case - adding the same entity from different ends of a many to many bi-directional association to check that there isn't an integrity constraint violation caused by Doctrine trying to add the same row twice.

It fixes the two additional test cases - testManyToManyPersistByReachability and testManyToManyDuplicatePersistByReachability.

testManyToManyPersistByReachability was failing with your original patch because there are ManyToMany cases where an entity may have already been merged, but its still necessary to add it to an association and continue to cascade. Running the following with the original patch will miss out some of the associations.

Calling ->clear() and ->flush() after each merge is a workaround for the simple case, but unless I am misunderstanding I don't think its a solution for cases where the merging is happening automatically in cascadeMerge. I've actually encountered this issue in another project and scenario to do with creating REST APIs and merging JSON objects into entities, and applying the patch fixed it so a) I think this issue might be a more common that we first thought and b) the patch basically seems to work (plus it doesn't introduce any failing cases in the existing test suite). I can actually still find one edge case to do with cascading merging interlinked many to many associations that this doesn't fix, but I was planning to open that as a new ticket after this My feeling is that the current merge already has issues and this definitely improves it.

Just to summarize, the equivalent operation to having multiple merges and a single flush is to call merge followed by flush each time, with the whole thing surrounded by a transaction? Does this have a big impact on performance?

Ben - even given the decision not to implement this (and I do understand your thinking, as it is a major change), is there any reason not to implement the bit that ensures that the same entity isn't added to a collection twice during a merge? I can't think of a situation where this should be allowed, and I have a use case where I get 'DUPLICATE KEY' errors if this isn't there.

Setting such a query hint to TRUE should result in all entities being retrieved by that query to be read-only for the purposes of change-tracking. Note that the entities themselves need not necessarily be read-only in general.

This feature is a flush performance tweak that can be used to query for objects but not let the returned objects run through change-tracking on flush. Any other managed objects are tracked as usual so you can do a read-only query for 100 entities and persist a new entity in the same unit of work with optimal flushing performance.

We should find a way, using PHPUnit Data Providers or anything else, to check the serialize/unserialize of every property in the ClassMetadata instance, since errors here can be very subtle but dangerous.

Didnt find anything in the docs on this. Is D2 capable of doing an UPSERT [1] in case I am trying to persist an object that may or may not have been saved previously. Different RDBMS support different syntax for this case. Like MySQL has INSERT .. ON DUPLICATE KEY UPDATE (or even INSERT IGNORE) while the SQL standard defines a MERGE syntax which seems to be gaining support. Of course you can always fallback to a SELECT FOR UPDATE (or if you want to be hacky an INSERT which catches duplicate key violations .. but probably not a good idea since many RDBMS rollback on a failure inside a transaction).

In the MySQL Doctrine implementation, however, it is not the same as INSERT .. ON DUPLICATE KEY UPDATE. The replace() method implemented in Doctrine_Connection_Mysql uses the REPLACE INTO syntax, which is a DELETE and then INSERT when the key exists. This is fine, except for tables that use auto-increment fields. The delete-then-insert operation yields a new auto-incremented value, whereas INSERT .. ON DUPLICTATE KEY UPDATE would not.

The relevant code is in Doctrine/ORM/Mapping/Driver/DatabaseDriver.php only.

The idea is currently many-to-many tables are detected by checking that the table has foreign keys on all the primary key columns (no additional columns!)

Now with the 2.1 feature of foreign key/primary key entities this is not necessarily true anymore. You can have the primary keys being foreign keys BUT have additional columns that are not part of the primary key. This has to be detected.

If a foreign key-primary-key entity is found that has additional columns a ClassMetadata has to be created and the associations have to be created with the "id" => true flag in mapManyToOne().

Since the association from Customer to Cart can not be lazy, it would make sense to leave out the association in a query to avoid loading the carts like this:

select partial c.{id,name, ... anything except cart} from Customer c"

But this is ignored and the carts of all customers are fetched anyway. Query::HINT_FORCE_PARTIAL_LOAD is an alternative solution, however it has the disadvantage that it disables lazy-loading for all queried objects. If partial querying would honor associations this would allow more fine-grained control.

Has there been any work on a coding standards document yet?
I'm currently working on fixing documentation on this project, and it might be a good time to define a standard.
I've started compiling a few recommendations based on various feedbacks I've got in my pull requests, and I can post them here.
Please let me know if there have been previous attempts so far!

Why is it still missing in 2.3? I would require this for an extension that uses its own overridden entity persister and using a custom persister is the solution that you guys recomend for not overriding the entity manager.

Sometimes you want to save arbitrary information for an entity using a key -> value array-structure. JPA supports this by means of the @ElementCollection annotation with allows to specify HashMaps for example.

I propose a new AssocationMapping called "ElementMapping" / "ElementCollection" and annotations (options):

A problem when working with collection-valued associations is that almost all operations except add($obj) require the collection to become initialized in order for the operation to be performed properly. While this is all correct and beautiful OO-wise it may be problematic at times with regards to performance. Hence we might want to consider to provide some convenient methods along the lines of link/unlink (name suggestions?) which allow more direct, less OO collection manipulation. Such methods obviously would bypass the normal object lifecycle and the changes done through these methods will not be reflected in the in-memory objects and collections, unless the user keeps them in-synch himself.

In Doctrine 1.1 it is possible to skip the operation in the event handlers in Doctrine_Record_Listener using Doctrine_Event::skipOperation.

This no longer seems to be possible in Doctrine 2.0 Alpha 1, for example when handling a preRemove event to implement soft-delete behaviour. Perhaps a method could be added to \Doctrine\Common\EventArgs\LifecycleEventArgs to skip the operation, at least before the operation.

Without this implementing soft-delete would require the user to update deleted_at and deleted_by himself and then save the record. It could no longer be done automatically when removing a record because the record is then removed.

The problem is, full support for soft-delete throughout the system is not feasible and very fragile. Simple soft-delete through skipping the delete operation is the easiest part. Then you will probably want to modify all DQL queries so that they adhere to it automatically and then there will always be still queries that do NOT go through DQL, like even internal lazy-load queries or native queries or others, which would need to be modified also.

To sum it up, implementing soft-delete "inside" doctrine is absolutely not worth the effort and imho a bad idea and I'm certainly not willing to make lots of adjustments to the core that have a negative impact on performance just to make this soft-delete possible.

I really recommend handling "soft" deletes yourself, the normal way, by simply abstracting entity retrieval and persistence through a DAO/repository layer. As a nice side-effect you get less magic and it still works when you swap out doctrine for another persistence provider.

I am willing to add support for skipping deletes and maybe some other operations through events but I'm not willing to go any further, as explained above.