The current ClassMetadata API is based on a lot of array juggling (for performance reasons).

While that was understandable with PHP 5.3, all the array access operations are currently:

slowing things down

making the code very hard to read/understand

I suggest re-coding the ClassMetadata internals (public properties and such) so that well-described properties are defined.

Additionally, as a bonus, we'd get a performance boost by just moving all the class-alias and type resolution logic from the runtime into the ClassMetadataFactory (or similar) API, saving tons of performance at every run.

This would make metadata manipulation from events a bit messier (user needs to know which value to change during which event), but would allow using better constrained metadata structures in future, and that would disallow mistakes during event listeners execution as well (internal validation).

The Unidirectional ManyToOne association described in the docs does not lazy load correctly. The appropriate SQL will get executed, and the returned Proxy does pass type hinting for the correct class. However, the lazy loaded object always has the following properties:

_initializer_

_cloner_

_isInitialized_

lazyPropertiesDefaults

Any properties from the class definition do not show up. This is problematic when trying to get reflected properties and their values. Methods are correctly reflected.

Note: performance is one of our biggest requirements, and array hydration is meant to achieve that. As I stated above, plain SQL is probably better for this sort of operation, so the issue is more about deprecating the ArrayHydrator completely, providing utilities to manipulate SQL resultsets instead.

This PR fixes that, however it's a bit hacky. The SqlExpressionVisitor was made by type hinting for a BasicEntityPersister, preventing us from using us for a collection persister. Therefore I added a new interface to keep BC.

There is also a lot of code duplication (the whole "getSelectConditionSQL" was copied from the BasicEntityPersister), but without trait or BC, I have no idea about how to solve it.

When ordering by multiple fields, with the order by fields not contained in the original select statement the paginator outputs in an incorrect order.
I found the issue is in the LimitSubqueryOutputWalker in the preserveSqlOrdering function. This is the query it generates

[Doctrine\DBAL\DBALException]
An exception occurred while executing 'ALTER TABLE Unit CHANGE decimal decimal_numbers SMALLINT NOT NULL':
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'deci
mal decimal_numbers SMALLINT NOT NULL' at line 1

The wrong class is chosen when hydrating embeddables that are part of a nested structure. See `DDC3582Test` for a demonstration. The fix is to use `class` instead of `embeddedClass` to instantiate the embeddable in ReflectionEmbeddedProperty.

The test I removed from ReflectionEmbeddedPropertyTest was failing because you cannot instantiate an abstract class (and rightfully so). However, as this would not be possible in practice anyway (you always end up extending the abstract class), I think this test can be removed safely.

Consider the example where an entity contains an embeddable, that itself contains 2 different embeddables. So Entity -> Embeddable 1 -> (Embeddable 2, Embeddable 3). In this case, Embeddable1 will be instantiated as an Embeddable2 for some reason.

I looked at the code in the ReflectionEmbeddedProperty, and noticed that 'embeddedClass' was recently changed from 'class'. If I change it back, this test passes, but then the ReflectionEmbeddedPropertyTest fails on the scenario for abstract classes. @ocramius, I saw you were the author on that change, could you please take a look? Thanks!

The Paginator's CountOutputWalker leaves the ORDER BY clause in the subquery that's executed to get the total count.

The original (and current) version of that code included the following comment:

Note that the ORDER BY clause is not removed. Many SQL implementations (e.g. MySQL) are able to cache subqueries. By keeping the ORDER BY clause intact, the limitSubQuery that will most likely be executed next can be read from the native SQL cache.

My understanding is that MySQL does not, in fact, cache subqueries in the query cache, so cached results are not available between executions of different queries. The current MySQL manual still states that "Queries that are a subquery of an outer query" are not cached.

This pull request was merged in 2013 to address errors with mssql and paginating ordered queries. Midway through the discussion on that issue, it was determined that it would be better to just remove the ORDER BY clause for all platforms, not just MS SQL. However, that decision was reversed when it came time to merge the PR.

Does the comment on that method accurately describe the rationale for retaining the ORDER BY clause for the count query? If so, am I simply wrong about how MySQL operates, or does the comment refer to some other platform that benefits from this pattern?

I think this is getting conflated with the other Paginator-related ORDER BY changes and reversions that have happened breaking and fixing the actual sorting behavior. The pull you linked looks like its relating to the LimitSubqueryOutputWalker, not the CountOutputWalker.

My concern is that the rationale for keeping the ORDER BY intact for all the other platforms (only when running the count query, not the "real" one) is a performance-related one, and that it's a flawed rationale. (Or, at least, that the comment about it is.)

I've changed the test so that on Token association, there is no @Cache annotation.

The default behaviour would be to Doctrine set a proxy on non-cached association. Instead, it basically creates a "partial" object where any associations without the @Cache is set to null. I'm pretty sure it works the same ManyToOne association too.

During observe the metadata in a One-To-Many association I've noticed that, the inversedBy value is NULL but in the annotation its assigned to something like $images (look example). I think its wrong because the EntityGenerator dont get it and dont generate the add..., remove... and get...s Method stubs or the __constructor assignment to ArrayCollections.

I want to make two requests on an object and join a one to many relation using doctrine ORM. The first request has different condition on the joined object.
The first request should join one object, and the second one two, but both return one object when run with the same EntityManager context.

The attached file was automatically generated after upgrading doctrine/dbal onto version 2.5.1 .

Now i have DOWNgraded doctrine/dbal to version 2.4.4 .
Suddenly the doctrine:migrations:diff says "No changes detected in your mapping information.".

Before downgrading i had two identical databases. One locally installed and one in a vagrant box with debian as os.
On every database i did a "doctrine:migrations:diff" to generate an actual schema update file. What i was wondering ist, that i got two different files with different ALTER xxxxx statements regarding to FOREIGN KEY and INDEXES.

Looking at the previous SQL schema I found out that there is already e.g. the FK_12743CF7924C3225. This FK
will be dropped and newly added at the same time. It is actually not using the new FK name which doctrine would
like to see. However in the down sql it tried to drop the key which should be there.

In addition to that there is an issue with upper/lowercase Index names. This I got fixed by

return 'DROP INDEX ' . strtoupper($indexName) . ' ON ' . $table;

in MysqlPlatform:860 which is most likely not the right way. I am more concerned about the FK
issue, which I am unable to track down so far. Do you have any idea?

In version 2.5 added support sequence with the strategy of identity.
But I cannot get the correct name of the sequence with the strategy of identity. With the strategy sequence produces the correct name. It is taken from the configuration sequenceName. Strategy identity name sequence will not be taken from the configuration, and is always generated is automatically (which is not working properly in my case).

When I run php app/console doctrine:generate:entities AppBundle -v, I get the following exception:

[Symfony\Component\Debug\Exception\ContextErrorException]
Notice: Trying to get property of non-object

Exception trace:
() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataInfo.php:3251
Symfony\Component\Debug\ErrorHandler->handleError() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataInfo.php:3251
Doctrine\ORM\Mapping\ClassMetadataInfo->inlineEmbeddable() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:201
Doctrine\ORM\Mapping\ClassMetadataFactory->doLoadMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php:332
Doctrine\Common\Persistence\Mapping\AbstractClassMetadataFactory->loadMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:78
Doctrine\ORM\Mapping\ClassMetadataFactory->loadMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php:225
Doctrine\Common\Persistence\Mapping\AbstractClassMetadataFactory->getMetadataFor() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php:115
Doctrine\Common\Persistence\Mapping\AbstractClassMetadataFactory->getAllMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/doctrine-bundle/Mapping/DisconnectedMetadataFactory.php:201
Doctrine\Bundle\DoctrineBundle\Mapping\DisconnectedMetadataFactory->getAllMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/doctrine-bundle/Mapping/DisconnectedMetadataFactory.php:164
Doctrine\Bundle\DoctrineBundle\Mapping\DisconnectedMetadataFactory->getMetadataForNamespace() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/doctrine-bundle/Mapping/DisconnectedMetadataFactory.php:54
Doctrine\Bundle\DoctrineBundle\Mapping\DisconnectedMetadataFactory->getBundleMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/doctrine-bundle/Command/GenerateEntitiesDoctrineCommand.php:96
Doctrine\Bundle\DoctrineBundle\Command\GenerateEntitiesDoctrineCommand->execute() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:253
Symfony\Component\Console\Command\Command->run() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:882
Symfony\Component\Console\Application->doRunCommand() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:96
Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:126
Symfony\Component\Console\Application->run() at /Users/vladislav/Sites/doctrine2.5_tests/app/console:27

If I add columnPreffix to the @Embed annotation the exception is different:

[Symfony\Component\Debug\Exception\ContextErrorException]
Catchable Fatal Error: Argument 1 passed to Doctrine\ORM\Mapping\ReflectionEmbeddedProperty::__construct() must be an instance of ReflectionP
roperty, null given, called in /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataInfo.php on
line 952 and defined

Exception trace:
() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ReflectionEmbeddedProperty.php:61
Symfony\Component\Debug\ErrorHandler->handleError() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ReflectionEmbeddedProperty.php:61
Doctrine\ORM\Mapping\ReflectionEmbeddedProperty->__construct() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataInfo.php:952
Doctrine\ORM\Mapping\ClassMetadataInfo->wakeupReflection() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:721
Doctrine\ORM\Mapping\ClassMetadataFactory->wakeupReflection() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php:343
Doctrine\Common\Persistence\Mapping\AbstractClassMetadataFactory->loadMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:78
Doctrine\ORM\Mapping\ClassMetadataFactory->loadMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php:225
Doctrine\Common\Persistence\Mapping\AbstractClassMetadataFactory->getMetadataFor() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php:115
Doctrine\Common\Persistence\Mapping\AbstractClassMetadataFactory->getAllMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/doctrine-bundle/Mapping/DisconnectedMetadataFactory.php:201
Doctrine\Bundle\DoctrineBundle\Mapping\DisconnectedMetadataFactory->getAllMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/doctrine-bundle/Mapping/DisconnectedMetadataFactory.php:164
Doctrine\Bundle\DoctrineBundle\Mapping\DisconnectedMetadataFactory->getMetadataForNamespace() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/doctrine-bundle/Mapping/DisconnectedMetadataFactory.php:54
Doctrine\Bundle\DoctrineBundle\Mapping\DisconnectedMetadataFactory->getBundleMetadata() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/doctrine/doctrine-bundle/Command/GenerateEntitiesDoctrineCommand.php:96
Doctrine\Bundle\DoctrineBundle\Command\GenerateEntitiesDoctrineCommand->execute() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php:253
Symfony\Component\Console\Command\Command->run() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:882
Symfony\Component\Console\Application->doRunCommand() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php:96
Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /Users/vladislav/Sites/doctrine2.5_tests/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php:126
Symfony\Component\Console\Application->run() at /Users/vladislav/Sites/doctrine2.5_tests/app/console:27

doctrine:generate:entities [--path="..."][--no-backup] name

I don't think it's a Symfony specific issue. I tried using the built-in CLI tool and got the same results.

Yeah, this can't really work with code-gen, because embeddables require reflection to be initialized in order to operate, whereas the codegen cli-tools operate with a DisconnectedMetadataFactory, which skips reflection on purpose (it assumes that the class does not exist, therefore it does not start up reflection).

Now that the minimum PHP version is 5.4, it would be good to encourage the use of the short array syntax `[]`.
Converting the existing codebase to this syntax will promote this good practice and encourage everyone to follow it.

When I was trying to persist a new object, Doctrine reported a exception:

An exception occurred while executing 'SELECT version FROM wallet WHERE user_id = ?' with params [{}]:

Catchable Fatal Error: Object of class XXXBundle\Entity\User could not be converted to string in /Users/XXX/WebSite/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php line 91

From the stack trace information, I found the original exception is:

Catchable Fatal Error: Object of class XXXBundle\Entity\User could not be converted to string

The reason is that the entity Wallet has a OneToOne mapping to entity User, and user property is also marked as primary key. When the wallet object persists, Doctrine try to fetch the new version id and use user property to find object. But Doctrine doesn't handle the primary key is an object, not a basic type. So PDO can't use a object as a query parameter.

The idea was to have a simple (and clean) way to override the nullable property of embedded objects, so I've
added the ```nullable``` attribute on the ```@Embedded``` annotation and it have 3 possible values:

NULL: The nullable option that was defined on the attributes of the embeddable class won't be overriden;

TRUE: All attributes of the embeddable class will be marked as nullable and the embeddable instance only would be created when data is not NULL;

FALSE: All attributes of the embeddable class will be marked as non-nullable.

There's a lot of things to be improved (mostly on UnitOfWork), but it's fully working with basic tests as you can see on ValueObjectsTest::testCRUDOfNullableEmbedded() case.

All the upload screenshots which are used to explain what we are doing are taken from a real project developed with Mouf dependency injection framework (http://mouf-php.com/) that we use to configure the entity manager.

We wanted to create a FAQ where admin can add question/answer and use drag and drop to set the position of the question in the FAQ.

For this, at some point, in an entity, we used a protected keyword "order" in a class FAQ:

We would like to submit this new `EscapingQuoteStrategy` as we think it will be easier for developers to work with it, and it should not have a big performance impact (only an additional lookup in a table)

While Trying to Embed an object into a mapped superclass an exception is thrown:

PHP Fatal error: Uncaught exception 'Doctrine\ORM\Mapping\MappingException' with message 'Duplicate definition of column 'embed_someField' on entity 'Entity' in a field or discriminator column mapping.' in /home/dev/Workspace/symfony/perto/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/MappingException.php:555

Okay, I am sorry. I have main entity which includes Embeddable entity (Address), this address entity contains country attribute with ManyToOne relation, but this country is not being saved into the database table, other attributes like street (string) are being saved normally.

It will probably not implemented for now, as embeddables (in our vision) are fitting the use-case of ValueObjects. ValueObjects are (usually) supposed to be containing serializable data, and an entity reference is not serializable data.

Okay, understood. So it's not good idea to use embeddables for Address entity for example, with this country as relation to another table, right? It seemed to me like best case. Thank you for the support anyway.

It will probably not implemented for now, as embeddables (in our vision) are fitting the use-case of ValueObjects. ValueObjects are (usually) supposed to be containing serializable data, and an entity reference is not serializable data.

All tables should be created as MyISAM and "utf8_general_ci" collation instead of InnoDB and "utf8_unicode_ci". This works fine for all tables except for "user_groups" table because the options are not inherited. As a result the schema is not valid, because foreign keys cannot be created over 2 different collations.

To fix this, I modified class "\Doctrine\ORM\Tools\SchemaTool", function "gatherRelationsSql($class, $table, $schema, &$addedFks, &$blacklistedFks)" and added the following lines in the MANY_TO_MANY block:

Running doctrine:generate:entities from the command line gives entities that have instance variables arrays, but their names are singular. For example if there are many "tools" associated with a "toolbox" via a many to one or many to many realationship, the toolbox class should have an array variable named "tools". It has an array but the name of it is "tool" instead of "tools". The name of the variable should be singular for one to one relationships and plural for whenever its an array

When I run SchemaTool::getSchemaFromMetadata, it always adds foreign keys constraints for all association fields (in SchemaTool::gatherRelationJoinColumns). Since I've set the table's engine option to 'MyISAM', this is quite pointless, since they won't get created anyways.

When I run SchemaManager::createSchema, it does not detect any foreign keys, so the SchemaDiff from Comparator will contain a large number of (pointless) ADD CONSTRAINT FK_xxx FOREIGN KEY commands.

For the moment, I work around this by removing them in a ToolEvents::postGenerateSchemaTable listener, but shouldn't SchemaTool be able to detect this situation itself?

Embeddeds have 2 methods, as other types have: `addEmbedded` is a one-off adder with fluent behavior, and `createEmbedded` returns an instance of `EmbeddedBuilder`. The class is very thin right now, but it's a good way to lay ground for improvements on embedded creation in the future.

This setup works fine when persisting entities, the database schema is correct, etc. but when I fetch my `Location` entity afterwards the `bounds` property is wrong. It looks something like this. Calling `getBounds()` on `Location` returns something like:

Coords

latitude <float>

longitude <float>

northeast <Coords>

southwest <Coords>

As if the `Coords` embeddable has been transposed with the `Bounds` embeddable.

If I add another property to `Bounds` that isn't an embeddable (say a string) then it's hydrated correctly.

when i use class table inheritance with multiple levels in doctrine :
Account<=User<=Dealer
(Dealer inherits from User, User from Account)
Account has the discriminator column and mapping.
when i persist a new Dealer entity I get a foreign key error because there is no row in the User table.
So the order in which the insert statements are executed:
1) Insert into Account
2) Insert into Dealer (which causes the error)
3) insert into User
Can someone help me thanks in advance.

BasicEntityPersister#getOneToManyStatement() is building an array named $criteria. This array is built this way:

$criteria[$tableAlias . "." . $targetKeyColumn] = ...;

This means this array is indexed by keys looking like that:

t0.fieldName

But the function BasicEntityPersister#expandParameters() is used in this function, and this function is NOT able to handle SQL field name as keys, but PHP attributes, because it uses BasicEntityPersister#getType() which is doing this:

Same query produces different results. With apache module everything works like expected. With php-cli the join condition to i18n table is ignored and calling getCountries() returns all and not only that entity that is matched by join condition.

Looks like a caching issue. The amount of information provided is insufficient as it is. I'd suggest verifying if the generated SQL is the same, and checking that all caches were cleared both in CLI and in WEB sapis.

If the same results are produced on CLI and WEB APIs then I suggest trying to insulate the issue in a functional test to be run in both context. You probably have a different ORM bootstrap for CLI and WEB.

No, I didn't want to attach two times the same result. Wanted to attach it one time to show that those entitites that are wrong in the result doesn't appear in the sql result at all. I don't have different bootstraps. After a few short tests I think it is an error in th ArrayCache mechanism. The difference was that using Apache one initializer was not called. Calling this initializer in both cases leads now to wrong results in CLI and WEB API.

When the second query is not executed everything is fine. But when the second longer query runs and selects all i18n entities and after it the first query runs then the issue appears.

If you use indexBy="category", it does not work. Why are the object property names used for referencing in most of the mapping, but for indexBy you have to use the db field name? It is nowhere mentioned in the docs.
I didnt test this with non-association properties as indexBy.

I am adding and removing a non-persisted entity to a PersistentCollection before flushing. The collections field is an OneToMany relation with orphanRemoval. When the entity is removed it is automatically scheduled for orphanRemoval but is never checked if it is actually a managed entity before being scheduled which causes an exception. After it has been scheduled there is no way to unschedule it other than completely clearing the UnitOfWork.

There are some implementations that use the doctrine metadata to get the properties of a entity (DoctrineObject), and it currently uses the getFieldNames and getAssociationNames methods to retrieve and merge it.

Since the embeddables feature was added, that method will return more than one metadata for each embeddable property, and then the hydrator can never find the appropriate setter for that property.

This method allows some implementation to retrieve all mapped fields from an entity, something that can't be done using get_class_vars for example.

Of course this implementation could be done in the hydrator, but i don't think it is its responsibility to filter and merge data received from the ClassMetadata. Since you have methods to retrieve the metadata formatted as the database structure, you should also have methods to retrieve the information to the other side, in object structure.

ClassMetadataInfo is returning more than one result in getFieldNames() for the embeddables properties. This method is used by DoctrineModule\Stdlib\Hydrator\DoctrineObject in the extraction process.

Since its returning a number of results equal the number of properties of the embedded object, it will never find the correct setter for the extraction and that causes that property to be removed from the extracted array.

I was able to solve the issue by hacking into the getFieldNames() for testing and merging the duplicated entries, and then the object was successfully extracted.

By digging into the code, i found out that there is a mapEmbedded(), but instead of using that for embeddeds, its using the default mapField, which may be the root cause of the problem.

[x] Hack into the getFieldNames() method to see if the expected solution would work

[x] Remove multiple class declaration in the same file from the files i'll work with

My entities and database are in sync. If I use the default Doctrine connection configuration it shows "Nothing to update - your database is already in sync with the current entity metadata.". This is what I would expect and is correct.

But, the hydrateRowData executes another query to get the parent of each row (Doctrine DBAL Connection.php function executeQuery).
This function prepares a query to get the parent with a param (parent id) and i think the prepare function return the same statement (stmt) as the first query (the list of all the entity A).
So, the prepared query is executed but when the Doctrine ORM ObjectHydrator.php want to fetch the second row, he considers that there's no more rows to fetch and return the first rows.

I don't know if i am clear enough. Tell me if you want more details.

Oh. I forgot. If i try the same request with the same datas but in a mySQL database (pdo_mysql), i have a good result. I don't know if that helps.

The issue you described is very well known and it is a limitation of the ORM. What happens is following:

1. you have an inheritance A -> B | C | D (A being the root of the inheritance)
2. A has a self-referencing association to A (typically like a hierarchy of parent-child)
3. the ORM tries to load an A instance that has a reference to a parent of type A. In order to do so, the ORM has to resolve the actual type of the referenced parent, and therefore cannot lazily load it via a proxy (a query has to be executed in order to find out the value of the discriminator column)

This means that for performance reasons you should NEVER reference the root of an inheritance, but only leafs (where the ORM does not need to do this sort of resolution).

will fail if MyEntity is part of a join table inheritance hierarchy and hence the update is handled by the MultiTableUpdateExecutor. The insert statement for the temporary ID table causes the following PDOException:

SQLSTATE[HY093]: Invalid parameter number: number of bound variables does not match number of tokens

The problem is in counting the number of input parameters in the update part of the statement. Lines 127-131 of the MultiTableUpdateExecutor:

And when I try to execute it, leaving the first paramter null, I got this error:

An exception occurred while executing 'SELECT n0_.numero AS numero0, n0_.anno AS anno1, n0_.id AS id2, n0_.nombre AS nombre3, n0_.activo AS activo4, n0_.comite_tecnico_id AS comite_tecnico_id5 FROM nomencladores.norma n0_ WHERE n0_.anno LIKE '%34%' OR n0_.nombre LIKE '%sad%'':
SQLSTATE[42883]: Undefined function: 7 ERROR: operator does not exist: integer ~~ unknown
LINE 1: ...o_id5 FROM nomencladores.norma n0_ WHERE n0_.anno LIKE '%34%...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.

I think in somewhere Doctrine is failing or not, not so sure, if not what will be the solution around this issue I'm having?

Using a bidirectional one-to-many relation having an 'indexBy' clause on the owning side and a unique constraint over the indexed field and the 'mappedBy' field of the inverse side, replaced indexed fields being inserted before removed violated the unique constraint.

As stated in the code examples below, if done without the unique constraint, when replacing an indexed field, the row is replaced in the database (new one created, old one deleted).

So, the problem is when the one needs to use association key in composite identifier; they are added in the end of the identifier array, which is clearly not always suitable in regards to performance.
For example, following mapping:

Commit is linked with DDC-720 . Again, that gives me an impression that events should be cascaded:

In a nutshell, this change would mean that flush() can optionally accept an entity as an argument. When that happens, the resulting changeset will be limited to that entity and any entity reachable from it.

I'm using __wakeup to initialize a property with an empty ArrayCollection for a transient property. But when I try to "find" the entity and the uninitialized Proxy is already in the entity map, the proxy is simply set to "initialized" but my code to init the collection is never call (see https://github.com/doctrine/doctrine2/blob/master/lib/Doctrine/ORM/UnitOfWork.php#L2552). So neither the constructor of my entity nor the proxies' _load method are called.
Where should I initialize that collection?

I just added a postLoad event listener that seems to do the trick in this case, so I have three things to do
1. implement __wakeup to make proxies' load work
2. implement a constructor
and
3. add a postLoad Listener.

This is broken when the entity was loaded before via a relation and the proxy is not initialized. Doing a find in the same request finds the proxy in UoWs identityMap but __wakeup is not called then, so the property is not initialized.

MappingException: Invalid field override named 'emailCanonical' for class SiteCustomer.

I got the same error message, for usernameCanonical.
I need to override this field in order to remove the uniqness of the field
I use Doctrine v2.4.4 what is wrong with my annotation ? Why do I get this error when trying to upgrade my schema

[1] Doctrine\ORM\Mapping\MappingException: Invalid field override named 'emailCanonical' for class 'Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer'.
at n/a
in /home/jaybe/www/yProx/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/MappingException.php line 129
at Doctrine\ORM\Mapping\MappingException::invalidOverrideFieldName('Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer', 'emailCanonical')
in /home/jaybe/www/yProx/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataInfo.php line 2032
at Doctrine\ORM\Mapping\ClassMetadataInfo->setAttributeOverride('emailCanonical', array('fieldName' => 'emailCanonical', 'type' => 'string', 'scale' => '0', 'length' => '255', 'unique' => false, 'nullable' => false, 'precision' => '0', 'columnName' => 'email_canonical'))
in /home/jaybe/www/yProx/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/Driver/AnnotationDriver.php line 415
at Doctrine\ORM\Mapping\Driver\AnnotationDriver->loadMetadataForClass('Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer', object(ClassMetadata))
in /home/jaybe/www/yProx/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/Driver/MappingDriverChain.php line 103
at Doctrine\Common\Persistence\Mapping\Driver\MappingDriverChain->loadMetadataForClass('Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer', object(ClassMetadata))
in /home/jaybe/www/yProx/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php line 117
at Doctrine\ORM\Mapping\ClassMetadataFactory->doLoadMetadata(object(ClassMetadata), object(ClassMetadata), true, array('Ylly\CrmBundle\Entity\User'))
in /home/jaybe/www/yProx/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php line 318
at Doctrine\Common\Persistence\Mapping\AbstractClassMetadataFactory->loadMetadata('Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer')
in /home/jaybe/www/yProx/vendor/doctrine/common/lib/Doctrine/Common/Persistence/Mapping/AbstractClassMetadataFactory.php line 211
at Doctrine\Common\Persistence\Mapping\AbstractClassMetadataFactory->getMetadataFor('Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer')
in /home/jaybe/www/yProx/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php line 295
at Doctrine\ORM\EntityManager->getClassMetadata('Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer')
in /home/jaybe/www/yProx/vendor/doctrine/orm/lib/Doctrine/ORM/Repository/DefaultRepositoryFactory.php line 67
at Doctrine\ORM\Repository\DefaultRepositoryFactory->createRepository(object(EntityManager), 'Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer')
in /home/jaybe/www/yProx/vendor/doctrine/orm/lib/Doctrine/ORM/Repository/DefaultRepositoryFactory.php line 50
at Doctrine\ORM\Repository\DefaultRepositoryFactory->getRepository(object(EntityManager), 'Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer')
in /home/jaybe/www/yProx/vendor/doctrine/orm/lib/Doctrine/ORM/EntityManager.php line 759
at Doctrine\ORM\EntityManager->getRepository('Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer')
in /home/jaybe/www/yProx/src/Ylly/CmsBundle/Controller/CmsBaseController.php line 93
at Ylly\CmsBundle\Controller\CmsBaseController->getRepo('Ylly\Extension\SiteCustomerBundle\Entity\SiteCustomer')
in /home/jaybe/www/yProx/src/Ylly/Extension/SiteCustomerBundle/Controller/AdminController.php line 79
at Ylly\Extension\SiteCustomerBundle\Controller\AdminController->getSiteCustomerRepo()
in /home/jaybe/www/yProx/src/Ylly/Extension/SiteCustomerBundle/Controller/AdminController.php line 15
at Ylly\Extension\SiteCustomerBundle\Controller\AdminController->indexAction()
in line
at call_user_func_array(array(object(AdminController), 'indexAction'), array())
in /home/jaybe/www/yProx/apps/bootstrap.php.cache line 2952
at Symfony\Component\HttpKernel\HttpKernel->handleRaw(object(Request), '1')
in /home/jaybe/www/yProx/apps/bootstrap.php.cache line 2924
at Symfony\Component\HttpKernel\HttpKernel->handle(object(Request), '1', true)
in /home/jaybe/www/yProx/apps/bootstrap.php.cache line 3063
at Symfony\Component\HttpKernel\DependencyInjection\ContainerAwareHttpKernel->handle(object(Request), '1', true)
in /home/jaybe/www/yProx/apps/bootstrap.php.cache line 2331
at Symfony\Component\HttpKernel\Kernel->handle(object(Request))
in /home/jaybe/www/yProx/web/admin_dev.php line 14

Here is the stack trace of my error.
If this is not what you need, tell me what do you want and how to get it.
Thank you.

When the CountOutputWalker is used for pagination, it creates a count query of this type :

SELECT %s AS dctrn_count
FROM (
SELECT DISTINCT "here are the identifiers from the original query"
FROM (
"here is the original query"
) dctrn_result
) dctrn_table

The problem is that the original query inside the count query is executed without limiting the results number each time the pagination has to count the total number of rows. And when the total number of rows returned by the original query is large and/or with big selected rows, it can hurt badly the performance, even kill the server.
Maybe I don't understand something but in my opinion this count query does exactly what we try to avoid with pagination : load all data at one time !
So I don't understand why things are done this way. Again, I'm not a db specialist so I'm almost sure I'm missing something.

But the thing is I've met these performance issues with big select queries on a medium amount of rows (~35000) on MySQL (using knp-components Pager).
The ugly solution I found was to use the CountWalker instead of the CountOutputWalker except when the query has a "HAVING" clause. That was because the CountWalker produce a better count query for performance but cannot handle well "HAVING" clauses (as far as I know).
Finally I think the solution is to modify the CountWalker so it can take care of more complex queries and/or improve the CountOutputWalker to preserve performance.

Well, the issue is that the CountWalker cannot count stuff using a GROUP BY or HAVING clause by design. It is simply impossible to write the given SQL without ending up on what the CountOutputWalker is doing,(maybe a bit simpler in some cases thanks to a complete knowledge of what the query is doing, but not much and not in a general case).

The solution is indeed to tell the Paginator not to use the output walker when you know it is not necessary. This is precisely why there are 2 implementations of the pagination with a boolean flag to switch between them

And the output walker actually perform better than the tree walker in many platforms according to Benjamin Eberlei (not MySQL though), which is why it is hard to choose the best walker for queries being supported by both of them (a project can do it better than the core, as it knows which platform it uses)

Thank you for your very clear explanations !
I assumed I was missing something but I did not think it was just impossible to solve this problem "automatically" . That's a bit annoying but ok.
The solution for MySQL seems to use the CountWalker for queries without GROUP BY or HAVING clause and CountOutputWalker for other queries. A better solution would be to create a custom hand-made count query for queries with GROUP BY or HAVING clause. This is possible in "knp-components Pager" and, I suppose, in the other pagination libraries using doctrine-orm Tools.

But as "knp-components Pager" does not use the "doctrine-orm Tools Paginator" but only the walkers from doctrine-orm, it doesn't have the boolean flag to switch between CountWalker and CountOutputWalker. The CountOutputWalker is used if doctrine-orm version is 2.3.0 or more and that's it.
I've made a pull request to activate the use of CountWalker for queries without HAVING clause but I think now it's a bad idea. The solution should be to add the same boolean flag to the "knp-components Pager" library. I will talk to them about that.

Doctrine 2 fails to properly store data from serialize() into postgresql.
This happens because the postgresql pdo driver truncates text on NULL bytes when escaping values. This leads to truncated serialized objects being inserted into the database.

A ugly but working workaround for us is to call json_encode(serialize()) when saving to the database and unserialize(json_decode()) when reading the value back because json_encode properly serializes the NULL bytes of the serialize() output to readable text.

This is pretty ugly though and it would help if doctrine would provide a minimal encoding/decoding function for postgresql that converts all NULL bytes to something else to not break the object type on postgresql.

Agreed, we'll just go with base64_encode for now.
Maybe someone should update the documentation that using the object type on postgresql is not working as well as that using binary strings containing a NULL byte in any varchar/text column on postgresql is not working either.

SQLSTATE[42000]: [Microsoft][SQL Server Native Client 11.0][SQL Server]Column 'fund_assets.active' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.

It appears that Doctrine's GROUP BY clause does not work correctly in Microsoft SQL Server.

Benjamin Horn this looks more like a DBAL related issue. Can you please give more details how the broken SQL is created by you? Also please provide information about which DBAL and/or ORM version you are referring to as we have fixed a lot of quotation problems with DDL statements in current DBAL master over the last few months. So please also try using the latest master and see if you can reproduce the issue.
If the issue still persists it would be helpfult to know which database vendor and which version you are using (as different vendors and versions have different reserved keywords).

BUT and here comes the BUG finally:

my workaround

as a workaround, what i have to do is, to exectute 2 queries. 1st to get just Entity1.ids joined with Entity2.dates by using `getArrayResult()`
and second query to get Entity1 objeces by unique ids from 1st query.
and than manualy join those results in php.

Comparisons that use floating-point numbers (or values that are converted to floating-point numbers) are approximate because such numbers are inexact. This might lead to results that appear inconsistent:

I was affected by exact the same issue. Even when running git master at the beginning of last week, it was still broken.
Today I retestet. Luckily your latest commits (from beginning with: 445798ed46291f2639b3657142bd2f934d1be8a6) to the BasicEntityPersister seemed fixed it.

@revrev:
Could you please retest your example? This bug might be fixed. Thx.

I'm not sure if it's me, but I couldn't get "preRemove" callback/event working when "remove" action is triggered from related entity, but it works where the entity itself (the one with "preRemove" event) is removed directly.

Orphan removal can not be supported for Many-To-Many relations, because the feature makes the assumption that the related entity is "privately owned" by the entity defining the orphan removal feature. In a Many-To-Many relation a related entity can be related to multiple entities and therefore is not privately owned by the relation defining entity.

indexBy implementation on containsKey (and possibly other EXTRA LAZY implemented methods) of PersistentCollection in EXTRA LAZY mode uses property name as column name instead of column name for OneToMany collections

For example property = fieldName, column name field_name.
Set indexBy on OneToMany annotation to fieldName (as it should be ) and it tries to use fieldName in backend instead of field_name

Yes, and I've since then used the FQNS. I would get a class not found error when validating schema.
This is something that I know can't really be debugged by you guys, I was just hoping for a little direction. We are using memcache to store our metadata, and we are clearing and regenerating the metadata during every deploy (it is not autogenerated). I'm now using a commit hash as the namespace of the metadata, so we'll see if this fixes the issue.

When using class table inheritance with multiple levels i.e.:
Object -> SolidObject -> Building -> House
(House inherits from Building, Building from SolidObject, SolidObject from Object)

Object has the discriminator column and mapping. When persisting a new House entity I get a foreign key error because there is no row in the Building table.

I searched in the Code and found the problem. In the class "Doctrine\ORM\Persisters\JoinedSubclassPersister" in function "executeInserts()" around line 146 the array $subTableStmts is first declared and then filled with insert statements. The statement of the House-table is first added, and then the parent-tables (except the root-table "Object", since that one is executed first).

So the order in which the insert statements are executed is:
1 Insert into Object ...
2 Insert into House ... (which causes the error)
3 insert into Building ...
4 Insert into SolidObject

I edited the source to insert into the parent-tables (first SolidObject, then Building) before inserting into the table that belongs to the class that is persisted (House). And this works

It is highly recommended that in a production environment you cache the transformation of a DQL query to its SQL counterpart. It doesn’t make sense to do this parsing multiple times as it doesn’t change unless you alter the DQL query.

I have the same problem when is generated QueryCacheId It consider only the name of active filters and not the value of the filter
This is the code at line 646 of class \Doctrine\ORM\Query
protected function _getQueryCacheId()