I investigated, and found that some of the test cases for the SQL Server platform weren't actually correct SQL. Also, there were no test cases that covered what the paginator is doing, so I've written test cases for those. I will open a pull request for this issue.

The modifyLimitQuery method in SQLServerPlatform.php should be fixed to pass the fixed old tests and the new tests.

My concern is that that method is becoming too complex, but that's an issue for another day.

I traced the error, it seems database existence is either not checked or not functioning, and it tries to connect to the DB, resulting in an exception. In older versions the trace seemed like it checked if DB exists, and didn't try to connect.

I tested some different commit versions to assess where the bug was introduced - here is my brief list of working/non-working versions.

Marco Pivetta This command is precisely about creating the database when it does not exist yet. I think this failure is related to the guessing of the platform version in DBAL 2.5, which will require connecting to the DB early.

I tested with multiple commands, mainly drop and create were most common.

The difference is in the error that gets produced with the different commits applied - other one gives the "The database is not there" kind of error you would expect. The other version gives an "can't do the operation because I can't connect to the database" even when you try to create it, and gives a similar "can't drop the database because I can't connect to the database" kind of error, which still says it tries to do something weird. My screenshot could've been taken from the create-command too, the error was identical.

I should have a breaking composer.json file in my version control on another machine. I'll try to add that later to facilitate debugging.

Christophe Coevoet you are right, that DBAL now connects early if you request the database platform but that should not be an issue as you may not specify a non-existing database name when connecting to creating a new database, anyways. And as far as I can see the CreateDatabaseDoctrineCommand even explicitly unsets the database name in the connection params here: https://github.com/doctrine/DoctrineBundle/blob/master/Command/CreateDatabaseDoctrineCommand.php#L71-L80 for a "temporary" connection.
Dropping a database still should require the database to be existent so connecting to it before dropping should be fine or am I missing something here?

@Steve Muller - is it tries to create the connection and creates an error also when the database is not supposed to be there, for ex. in create action. Effectively breaking createDatabase command.

When I traced the code, it seemed to go in to the platform version checking, and seemed like it just didn't realize early enough that the DB isn't there (that was my guess - I don't know doctrine internals well enough to say if that's how it's planned to work or not).

---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~--
Dropping database schema...
./bin/doctrine-module orm:schema-tool:drop --force --verbose
---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~---~--
Dropping database schema...
[Doctrine\DBAL\Exception\DriverException]
An exception occurred while executing 'SELECT quote_ident(r.conname) as conname, pg_catalog.pg_get_constraintdef(r.oid, true) as condef
FROM pg_catalog.pg_constraint r
WHERE r.conrelid =
(
SELECT c.oid
FROM pg_catalog.pg_class c, pg_catalog.pg_namespace n
WHERE n.nspname NOT IN ('pg_catalog', 'information_schema', 'pg_toast') AND c.relname = 'state' AND n.nspname = ANY(string_to_array((select replace(replace(setting,'"$user"'
,user),' ','') from pg_catalog.pg_settings where name = 'search_path'),',')) AND n.oid = c.relnamespace
)
AND r.contype = 'f'':
SQLSTATE[21000]: Cardinality violation: 7 ERROR: more than one row returned by a subquery used as an expression
Exception trace:
() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php:82
Doctrine\DBAL\Driver\AbstractPostgreSQLDriver->convertException() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:116
Doctrine\DBAL\DBALException::driverExceptionDuringQuery() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:833
Doctrine\DBAL\Connection->executeQuery() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:761
Doctrine\DBAL\Connection->fetchAll() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:319
Doctrine\DBAL\Schema\AbstractSchemaManager->listTableForeignKeys() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:284
Doctrine\DBAL\Schema\AbstractSchemaManager->listTableDetails() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:268
Doctrine\DBAL\Schema\AbstractSchemaManager->listTables() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:1039
Doctrine\DBAL\Schema\AbstractSchemaManager->createSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:783
Doctrine\ORM\Tools\SchemaTool->getDropSchemaSQL() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:727
Doctrine\ORM\Tools\SchemaTool->dropSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/DropCommand.php:100
Doctrine\ORM\Tools\Console\Command\SchemaTool\DropCommand->executeSchemaCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/AbstractCommand.php:65
Doctrine\ORM\Tools\Console\Command\SchemaTool\AbstractCommand->execute() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
Symfony\Component\Console\Command\Command->run() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:891
Symfony\Component\Console\Application->doRunCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:126
Symfony\Component\Console\Application->run() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module.php:58
include() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module:4
[Doctrine\DBAL\Driver\PDOException]
SQLSTATE[21000]: Cardinality violation: 7 ERROR: more than one row returned by a subquery used as an expression
Exception trace:
() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:94
Doctrine\DBAL\Driver\PDOConnection->query() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:830
Doctrine\DBAL\Connection->executeQuery() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:761
Doctrine\DBAL\Connection->fetchAll() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:319
Doctrine\DBAL\Schema\AbstractSchemaManager->listTableForeignKeys() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:284
Doctrine\DBAL\Schema\AbstractSchemaManager->listTableDetails() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:268
Doctrine\DBAL\Schema\AbstractSchemaManager->listTables() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:1039
Doctrine\DBAL\Schema\AbstractSchemaManager->createSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:783
Doctrine\ORM\Tools\SchemaTool->getDropSchemaSQL() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:727
Doctrine\ORM\Tools\SchemaTool->dropSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/DropCommand.php:100
Doctrine\ORM\Tools\Console\Command\SchemaTool\DropCommand->executeSchemaCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/AbstractCommand.php:65
Doctrine\ORM\Tools\Console\Command\SchemaTool\AbstractCommand->execute() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
Symfony\Component\Console\Command\Command->run() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:891
Symfony\Component\Console\Application->doRunCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:126
Symfony\Component\Console\Application->run() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module.php:58
include() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module:4
[PDOException]
SQLSTATE[21000]: Cardinality violation: 7 ERROR: more than one row returned by a subquery used as an expression
Exception trace:
() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:92
PDO->query() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:92
Doctrine\DBAL\Driver\PDOConnection->query() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:830
Doctrine\DBAL\Connection->executeQuery() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:761
Doctrine\DBAL\Connection->fetchAll() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:319
Doctrine\DBAL\Schema\AbstractSchemaManager->listTableForeignKeys() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:284
Doctrine\DBAL\Schema\AbstractSchemaManager->listTableDetails() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:268
Doctrine\DBAL\Schema\AbstractSchemaManager->listTables() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/dbal/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php:1039
Doctrine\DBAL\Schema\AbstractSchemaManager->createSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:783
Doctrine\ORM\Tools\SchemaTool->getDropSchemaSQL() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/SchemaTool.php:727
Doctrine\ORM\Tools\SchemaTool->dropSchema() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/DropCommand.php:100
Doctrine\ORM\Tools\Console\Command\SchemaTool\DropCommand->executeSchemaCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/doctrine/orm/lib/Doctrine/ORM/Tools/Console/Command/SchemaTool/AbstractCommand.php:65
Doctrine\ORM\Tools\Console\Command\SchemaTool\AbstractCommand->execute() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
Symfony\Component\Console\Command\Command->run() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:891
Symfony\Component\Console\Application->doRunCommand() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at /Users/dominicwatson/Sites/flatscanner/api/vendor/symfony/console/Symfony/Component/Console/Application.php:126
Symfony\Component\Console\Application->run() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module.php:58
include() at /Users/dominicwatson/Sites/flatscanner/api/bin/doctrine-module:4
orm:schema-tool:drop [--dump-sql] [-f|--force] [--full-database]

In DBAL 2.5, many driver can rely on different versions of the platform. Unless the version is explicitly provided, the driver will guess it at instantiation time, killing the lazyness of the connection.
This is a critical issue in any context using DI as it means that injecting the connection into anything else will connect to the server.

Actually, the Connection class itself defers the guessing until the first time the platform is accessed. But many places in DBAL and in the ORM are retrieving the platform and storing it in a property of the class at instantiation time to avoid method calls when they need to access the platform. So this might be much harder to fix

Christophe Coevoet Can we analyze the use cases where retrieving the platform is necessary before actually connecting? I only know the custom type registering so far... Maybe we can defer that somehow?

Steve Müller The issue is that all those objects are calling $connection->getDatabasePlatform() in their own constructor to store a reference to it for faster access later (no more method calls). This means that *instantiating* the ORM (or some parts of DBAL) triggers the platform guessing, which connects to the DB. This breaks the lazyness and hurts DI contexts (maybe the ORM will not even be used in this process, but it was instantiated because of being a dependency in a complex object graph).

This is causing a lot of issues when using CI servers. Where it is very important that those things are lazy, since you do not have the database configured on most applications that build on a CI server.

This is also an issue for the project I am working as I am trying to use Symfony to to generate forms and so on in order to install the project. sForms and the main project's front controller trigger DI events and trigger this error. The workaround mentioned here (https://github.com/doctrine/DoctrineBundle/issues/351#issuecomment-65771528) fixes the problem but this cannot be a long term solution. I look forward to the fix for this issue.

I have the same issue as Craig. This poses a major hassle for applications that have an installer UI. We have a Symfony based application that gives the user options to configure the database such as driver, table, credentials, etc. Then the application will make a dynamic connection to check database, install data, etc. Eventually Symfony's cache is cleared to use the new credentials post install.

Because of this, the installer fails out of the box since it can't connect to the server.

We can't use the work around as mentioned above because of giving the option to choose what driver to use in the installer.

Craig Heydenburg this is expected behaviour. You can theoretically specify any value you want, DBAL will try to find the appropriate platform for you by the specified version via version compare.
See the example for PostgreSQL: https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/Driver/AbstractPostgreSQLDriver.php#L97-L119
As soon as the specified value/version does not match a specific platform version, DBAL will always fallback to the "default" platform (the one that was chosen by default prior 2.5).
Once the serverVersion parameter is set, auto detection of the proper platform is always bypassed.

Why do I have to quote any reserved word used in table or column name? Why Doctrine doesn't do this automatically for all table and column
names used in generated SQL queries?

Before you start to explain how complicated it is and what problems you will be faced with, try to look at excellent DIBI database layer - how it acts in this way - it's behaviour is very cool. Unfortunally at the moment the full documentation is in czech only, but here is a brief automatic google-translation to english - http://dibiphp.com/en/quick-start.

My suggestion to Doctrine 2 ORM/DBAL solution is:

1. Developer should never care about any escaping or avoiding any reserved words - it is not his business, the DBAL shoult solve it transparently and safely.

2. So there should be no need and even no possibility to add any quotation chars in @column or @table annotations as well as in DQL queries. ORM layer has nothing to do with escaping, it is all a business of the DBAL layer. Current possibility for manual escaping the names in mentioned annotations is totally wrong and should be discontinued.

3. DBAL should escape ALL table and column names transparently and automatically. There should be ne option to enable or disable the escaping, there is no reason for disabling it.

4. The escaping should be performed just in the final translation of DBAL queries to native SQL query, not earlier. This is the right place to do that.

My point of view (and the reason for the current implementation) is as follows:

Using reserved words is bad practice.

Quoting everything is like hitting all the SQL with a huge big hammer just to hit the 1% of reserved words (which are again, bad practice), thus overkill.

Quoting everything bloats the generated SQL (just to hit the 1% of reserved words which are bad practice to begin with)

Quoting everything automatically is like hiding the fact from developers that they use reserved words, thus hiding a bad practice and silently encouraging usage of reserved words in new database schemas. This is not desirable.

Quoting reserved words has more effects than simply making the database "accept" that identifier. It affects the case-sensitivity and that in a very inconsistent way across databases and operating systems (See http://www.alberton.info/dbms_identifiers_and_case_sensitivity.html , especially the conclusion). You say there is no reason for disabling it but in fact there are a lot of reasons to do so, so many that it is disabled by default in MDB2 and discouraged to enable it.

So, supporting selective quoting in the name of a (slightly) better interoperability with legacy schemas looked (and still looks) like the best solution for us. The support is limited, explicit, does not require much implementation or overhead and does not unnecessarily bloat the SQL.

There is only one solution for reserved words: not using them. Quoting is a workaround, not a solution and especially not a good one.

ps: I really wish quoting reserved words would not be available in SQL It's not available in most programming languages and noone cares, people just don't use reserved words, because they simply can't.

Hi Roman, thank you very much for your response! I storngly disagree with most of your points .

There is no doubt that using reserved words is bad practice - FROM THE VIEW OF DATABASE SYSTEM.

But we are discussing about ORM and DBAL. One of the biggest goals of ORM/DBAL is to provide transparent usage of the storage behind the scene. No matter if it is MySQL or PostgreSQL or even maybe something completely diferent.

The ORM/DBAL layer should prevent me from any specifics of particular storage as much as possible. I don't want to remember (and I never should to) that I cannot create entity Order because "order" is reserved word in some weird technology far away from me as ORM programmer.

It is strictly consistent with what you have written above in your PS - "It's not available in most programming languages and noone cares, people just don't use reserved words, because they simply can't" - just consider Doctrine 2 to be another programming language - and there is no real systematic reason in Doctrine 2 itself to prevent developers create entities named "Order".

Here is an analogy - It is the same as if you would say that you cannot use associative arrays in PHP because C-language or Assembler behind PHP doesn't support associative arrays. Yes, they don't support them but it is the responsibility of PHP to provide them. In the same way I don't want to respect this weird limitations of particular RDBMS behind Doctrine 2. This is Doctrine's responsibility to transparently cover the limitation.

Moreover, when list of registered keywords is different from one to the other RDBMS, so the naming of entities is strongly dependent on current database server.

Moreover, when I realize that I have used a registered keyword as lately as an error returns from database engine, not earlier.

I suppose here is probably no risk of SQL injection, but I feel the current Doctrine 2 acting to be "vulnerable" in very similar way, on principle. Simply - you are sending an unescaped piece of SQL query to the database without any warranty what it is. And sometimes it fails, sometimes not. From this view I don't consider overall escaping to be overkill at all, I consider it to be a necessity.

I am strongly convinced that developer working upon DBAL or even ORM layer should never think about such naming limitations and he even shouldn't know anything about reserved words in his particular DBMS.

Now to mentioned problems with case sensitivity. Resulting from the fact that Doctrine 2 entity names are case insensitive I belive that all table definitions and SQL queries comming from Doctrine 2 to database should act as case insensitive too. And that the only practicable way is to normalize (lowercase) all table and column names just on DBAL side before it is passed as SQL query to database.

There is actually a very good reason for not quoting. Oracle columns behave differently in their internal structure when escaped.

for example:

/**
* @column(type="integer")
*/
private $foo;

With quoting it would lead to a column "foo" being lower-cased IN the database and even returned so from resultsets. Without casing it would be a column "FOO". We would essentially need to implement lots of glue code just to get this annoying Oracle feature to work and i think Postgres has the same with lower-cased columns.

@"Hi Roman, thank you very much for your response! I storngly disagree with most of your points"

I guess we can agree to disagree then

@"But we are discussing about ORM and DBAL. One of the biggest goals of ORM/DBAL is to provide transparent usage of the storage behind the scene. No matter if it is MySQL or PostgreSQL or even maybe something completely diferent."

Actually, no, "hiding" the storage completely from the developer is not the goal just as it is not the goal to "hide" SQL. There is an object model on one side and a relational database on the other side. The goal is to provide a mapping between them which is not the same as "hiding" one from the other. In order to create good applications that use ORM technology you need to know both very well, OOP and relational databases. The goal is not to make relational database knowledge "unnecessary". This only results in inefficient use of the databases. The goal is to give people who know both sides equally well a tool to map between the two. Not even "portability" between different relational database vendors is a main goal of an ORM technology, it is just obvious to provide assistance with that as part of the mapping.

@"and there is no real systematic reason in Doctrine 2 itself to prevent developers create entities named "Order".

Noone prevents you from naming domain classes anything you want. Class naming is different from table naming. That the table name defaults to the class name is just that, a default, that can and should be changed if necessary.

@"Moreover, when list of registered keywords is different from one to the other RDBMS, so the naming of entities is strongly dependent on current database server."

Correct, and if you want to create a portable application that works, and will be deployed on, a different set of vendors, you need to have some knowledge of these databases and consider their characteristics. An ORM/DBAL technology does not give you any guarantee for complete and transparent portability between vendors and especially not that it will perform equally well on all of them. The ORM/DBAL technology helps you for the most part in a lot of cases with portability issues but it is no free ticket.

@"I suppose here is probably no risk of SQL injection, but I feel the current Doctrine 2 acting to be "vulnerable" in very similar way, on principle. Simply - you are sending an unescaped piece of SQL query to the database without any warranty what it is. And sometimes it fails, sometimes not. From this view I don't consider overall escaping to be overkill at all, I consider it to be a necessity."

Do not confuse identifier quoting with quoting/escaping of special characters as it is used for security reasons on input. Identifier quoting is absolutely not a necessity, it is a workaround for using otherwise reserved words as schema element names. Speaking of goals, it is neither a "goal" of ORM/DBAL technology to completely remove the possibilities of SQL injections. You can't. It'll always be possible with wrong usage.

@"I am strongly convinced that developer working upon DBAL or even ORM layer should never think about such naming limitations and he even shouldn't know anything about reserved words in his particular DBMS."

And I am strongly convinced that a developer working with a DBAL/ORM should know the underlying databases pretty well.

I think you're really not aware of all the consequences it has across different database vendors to quote every identifier. If not for developers using Doctrine, you cause at least any developer or application pain that does not access the database through Doctrine and is thus feels the full pain of case-sensitivity and mandatory quoting you enforced on the whole schema. Ubiquitious access to the data is actually a strong point of a relational database and it is far from uncommon that the same database is accessed by many parties.

I think the approach taken by DIBI is a bad idea and even worse if there is no way to turn this behavior off. Do they have Oracle or DB2 users? I'm wondering what the sysadmins behind these databases might think if they see this quoting nightmare since to my knowledge this is considered bad practice among them as well.

Yes, we're disagreeing on many points but if you really think identifier quoting is a good idea then you're ignoring a whole lot of prior experience (not only mine).

I was one of the lead developers of MDB2 and we just ran into tons of issues when we overly aggressively did identifier quoting by default. even the option caused lots of headaches. furthermore I agree that the ORM is not about turning an RDBMS into an Object Database, but instead to make a mapping possible. In this vain using reserved words or making all identifiers case sensitive will be a big pain for the people that do work one level lower aka the DBA's. heck even as a developer I frequently work on the DB's command line.

Now as for helping people prevent issues with reserved words. Back then I added some reserved word checking into MDB2_Schema. Obviously its hard to really keep track of all of the different reserved words for all RDBMS. Maybe its possible to work with this guy for this: http://www.petefreitag.com/item/290.cfm This way it could be possible to validate if the names chosen in the models will not cause issues with a certain list of RDBMS.

This helped me understand much about Doctrine 2 basic objectives - especially that it is designed mainly to "make a mapping possible" only, not to be as much as possible transparent layer between database and application. And even if I don't like this conception (because I personally think ORM should provide all such features - like automatic reserved keywords escaping - to make the particular database as transparent as possible), at the same time I fully understand all metioned arguments for doing things in such way. Thank you again.

I understand where there are difficulties in handling reserved words and backtick/quoting, and certainly one should always avoid the use of reserved words in their own schema designs. This is a given when one is able to exert control.

At present I am working on a project in which I am dealing with an outside database where I have no control over the schema, nor am I able to push the remote into making the most sensible changes to their schema. I must live with what they provide.

DBAL presents me with a set of invaluable tools that can not be used as-is, because it lacks the ability to handle quoting when generating schema sql. I'm sure there are some other places where I will find this lacking as well. This is disappointing.

Regardless of what we as developers should do when designing our own schema, we still need to be able to work and play with others who may not follow the same common sense conventions.

Edit:
My temporary quick solution to just "make it work", was to modify AbstractAsset::getQuotedName and force the use of $platform->quoteIdentifier. It did the trick for now until a more suitable solution presents itself.

Not every project is a startup. I tried to use doctrine2 on a customers database for a small web ui. Well I told them to rename their `iso3166-1` table and `alpha-2` field, then we had a good laugh. We made the mapping possible but i'll remember the one thing i learned: doctrine did not help, guide, prevent or cared at all. It did not even hesitate to spew invalid sql snippets when asked to dump. Its okay for me, but i've expected something more resilient from a DBAL.

What do you mean by "Quoting everything is like hitting all the SQL with a huge big hammer"? Is there a performance hit?

I have always quoted all names when working with PostGres. Not quoting them has always felt like not quoting strings in PHP (e.g. $foo[bar] instead of $foo['bar'] because unless the string is keyword or defined as a constant somewhere, you don't need to (although you will get a "Use of undefined constant" warning). In the early days of PHP, not quoting array keys was common example practise.

If you want quoting by default on everything we have a quoting strategy (in ORM) that you can use. I don't think quoting everything by default is a viable solution. Back in `Zend_Db` times this was eating up a lot of performance for no real reason. Users having a clean schema without horrors like columns called `order` or `group` should not be penalized because of users not using valid naming schemes.

Hello, if I understand correctly, the issue of quoting reserved keywords automatically is solved in https://github.com/doctrine/dbal/pull/302. Besides reserved keywords you can still decide quoting or not quoting identifier manually by passing quotes to the identifier or not.

1. Users should not have to worry about platform-specific quoting when using the query builder or helpers, the DBAL should do that for you.
2. Users should be able to explicitly quote using a standard quote (`), the quote would then be converted to the platform’s quote upon SQL generation, without any case change.
3. DBAL should not needlessly quote, it adds bloat and it has been said that there is a performance hit.
4. DBAL should not change the case without the user’s knowledge.
5. A connection configuration option (normalize_case) could be added:
• uppercase: always convert unquoted identifiers to uppercase
• lowercase: always convert unquoted identifiers to lowercase
• platform: will use the default value for specific platform. For the case of case-sensitive platform, even when unquoted (MySQL on UNIX), do nothing.
• null (default): no normalization
6. Future versions of DBAL could change the default value to platform, but this would greatly reduce the risk of causing BC breaks at the beginning, giving time to test everything.
7. When using Doctrine\DBAL\Connection::query directly, you must do the quoting yourself since the SQL is executed directly.

Sebastien, ad 3. that is incorrect. Read the ticket more closely, look at the PR, look inside schema tool and platform classes. There is already a lot of quoting+unquoting being performed in 2.* and a lot of assumptions. Having quoting enabled across the board might actually increase performance in some cases, because there will be less scanning for keywords (see platform classes) and possibly less quoting/unquoting across Schema*.

The problem is, the quoting right now works in some places and in some platforms and is being performed only when schema/schematool/dql needs it, but is being ignored in all other cases. This means that columns like "group" or table names like "platform" will fail randomly depending on platform/rdbms you actually use. It's a nightmare with cross-platform apps and a struggle for single-platform apps, where your tables are named according to domain-rules and happen to overlap with some rdbms.

Quoting identifiers being "a bloat" is similar to saying, that implicit quoting values is a bloat. Although from security standpoint the former is much rarer, it's the same for portability and stability of the DBAL across platforms.

The Postgres Schema code is very hard to read and inconsistent across the board. Some use pg_class, some pg_tables. Namespaces /Schema are not always properly checked for. There should be a really unified way on how to approach schema query issues.

Usually, one would not want to specify them and set the search_path instead. Or are you meaning the schema analysis queries used internally? (If so, yes, it kind of sucks in that case, but note that there are a bunch of *_is_visible() methods, e.g. pg_catalog.pg_table_is_visible(rel.oid) which will strip out invisible schemas directly. This may be simpler than injecting schema references all over the place in that it also processes permissions.)

Also, note that PG has a whole bunch of pg_catalog views, which are available in the information_schema.

I am not sure how to handle one firebird-specificity: Firebird does not preserve the case of identity-names and converts them all-upper, unless they are quoted. The behaviour looks quite similar to Oracle.

Currently I do not normalize names as the Oracle-driver does, which leads to the problem, that they are all-upper in the database anyway, but automatically quoted keywords are lowercase.

Which behaviour would you guys prefere:

Normalize them All-Upper if not quoted (including keywords)? This would allow case-insensitive references in queries, because Firebird handles unquoted names as all-upper.
Quote everything to preserve case: Unfortunately this would require manual quoting in every query.

Personally I do not really like this all-upper-pattern, but having to quote every identifier everywhere looks even more cumbersome, so it's propably the best to follow the behaviour of the oracle driver and assume uppercase if not quoted.

Does any platform have configuration-options? It might be a solution to let the user decide about the naming, e.g.

setNamingConvention(ALL_UPPER | PRESERVE_CASE)

BTW - Is the doctrine-dev mailinglist gone? I wanted to send this there, but it came back with an error.

As a long time firebird user I can say that for me it is common to use column names case insensitive. So if you normalize all SQL commands to upper case (except quoted names) is the natural usage in firebird for me.

Benjamin Eberlei this is an interesting approach and I like it. But I have some complaints about it.
1. I doubt users will be happy about forced default casing rules (ALL upper or ALL lower). Therefore we should think about adding a simple configuration option in DBAL allowing to override the default casing behaviour to the user's preference.
2. Using a consistent default casing means we ALWAYS have to quote identifiers as otherwise the underlying database could silently change the case again (don't know if this is an issue).
3. Introducing this approach in 2.x branch is a BC break as it breaks users' mixed-case identifier mappings.

For 2.x we should maybe at least make use of Identifier class throughout the platforms where necessary.

1. Users should not have to worry about platform-specific quoting when using the query builder or helpers, the DBAL should do that for you.
2. Users should be able to explicitly quote using a standard quote (`), the quote would then be converted to the platform’s quote upon SQL generation, without any case change.
3. DBAL should not needlessly quote, it adds bloat and it has been said in DBAL-40 that there is a performance hit.
4. DBAL should not change the case without the user’s knowledge.
5. A connection configuration option (normalize_case) could be added:
• uppercase: always convert unquoted identifiers to uppercase
• lowercase: always convert unquoted identifiers to lowercase
• platform: will use the default value for specific platform. For the case of case-sensitive platform, even when unquoted (MySQL on UNIX), do nothing.
• null (default): no normalization
6. Future versions of DBAL could change the default value to platform, but this would greatly reduce the risk of causing BC breaks at the beginning, giving time to test everything.
7. When using Doctrine\DBAL\Connection::query directly, you must do the quoting yourself since the SQL is executed directly.

Sebastien, ad 3. that is incorrect. Read the ticket more closely, look at the PR, look inside schema tool and platform classes. There is already a lot of quoting+unquoting being performed in 2.* and a lot of assumptions. Having quoting enabled across the board might actually increase performance in some cases, because there will be less scanning for keywords (see platform classes) and possibly less quoting/unquoting across Schema*.

The problem is, the quoting right now works in some places and in some platforms and is being performed only when schema/schematool/dql needs it, but is being ignored in all other cases. This means that columns like "group" or table names like "platform" will fail randomly depending on platform/rdbms you actually use. It's a nightmare with cross-platform apps and a struggle for single-platform apps, where your tables are named according to domain-rules and happen to overlap with some rdbms.

Quoting identifiers being "a bloat" is similar to saying, that implicit quoting values is a bloat. Although from security standpoint the former is much rarer, it's the same for portability and stability of the DBAL across platforms.

IMO the big problem is, that behaviour across the RDBMs may be completely different:

Some preserve case and are case-insenstive if not quoted (the nicest approach)
Some do not preserve case and normalize to lower or uppercase and are case-insensitive
And some do not preserve case and are case-sensitive (the worst)

The biggest issue arises if the DB needs to be used outside the Doctrine-Environment and all identifiers need to be quoted in statements.

SQL statements may contain question marks in strings, inline comments or comment blocks. The current implementation of the method "convertPositionalToNamedPlaceholders()" doesn't consider these implications and falsely replaces them by named bind variables.

Today exist many customers with Informix 11.50 and they will deveoplment with php+simphony, but doctrine will be an alternative inside the framework but it isn't supported.
My customers can't migration ( they are happy with informix) your application and he need a framework as simphony and an abstration of your environment as Doctrine.
There is a possibility to support Informix in Doctrine?
Thanks and regards

1. Add your driver shortcut to class-name `Doctrine\DBAL\DriverManager`.
2. Make a copy of tests/dbproperties.xml.dev and adjust the values to your driver shortcut and testdatabase.
3. Create three new classes implementing ``\Doctrine\DBAL\Driver\Driver``, ``\Doctrine\DBAL\Driver\Statement``
and ``Doctrine\DBAL\Driver``. You can take a look at the ``Doctrine\DBAL\Driver\OCI8`` driver.
4. You can run the testsuite of your new database driver by calling "cd tests/ && phpunit -c myconfig.xml Doctrine/Tess/AllTests.php"
5. Start implementing AbstractPlatform and AbstractSchemaManager. Other implementations should serve as good example.

Essentially Driver, Statement are the PDO equivalents of "PDO" and "PDOStatement", they even have the exact same API. OCI8 is a good example to see how it works. If Informix works with PDO then its even simpler, take a look at Doctrine\DBAL\Driver\PDO namespace, you don't need to implement the connection and statement then, just the driver taking care of the configuraiton.

The most complicated thing is to get the Platform and SchemaManager to work. Both are responsible for SQL generation for DDL statements for example. It makes sense to run the phpunit testsuite and fix the bugs when occuring until all tests pass.

Also looking at the platform and schema managers of other databases is a very good help

the static types should be removed and made to instances of types that can differ between platforms.

Following reasons:

1. Some vendors already convert values to PHP types, for example integers. Not converting again could save about 15% wall-time for integers for example.
2. Some vendors require different type code, this is currently handled by expensive if checks.
3. If you want to change a type for one vendor but also use another vendor in the same request, then you have a problem currently.
4. If types were platform aware then we could make registering custom types simpler.

AFAIK only MySQL supports changing column order after table creation. All other vendors need a table recreation for this. Why exactly do you need this? I cannot think of a use case where the order in which columns are stored in the table is of any importance. And as this is a MySQL only specific feature which needs some heavy calculations in the comparator, I doubt this will be implemented...

It's very convenient to use MySQL column order so that the columns show up in some sane order in DB management tools.
Also an application can show some tables in a sane way without explicit column order specification.

We are currently using it widely in our app. Our ORM is actually picking up the object specifications directly from the database and it's not specified in the application code.

Artem Goutsoul I get the point. But as I already stated only MySQL supports column reordering without requiring a table recreation AFAIK. How would you expect other platforms to behave in column reordering scenarios? I think it's weird if only MySQL platform can react to column reorderings and we have to adjust the comparator for a just a single platform. Benjamin Eberlei what would you say?

Mainly, we may have legacy database that we do know how Doctrine should operate. Under this circumstance, we may want to add a comment to the field defining the Doctrine DBAL type.
Even though after doing this, Doctrine still complains (in my situation, a SET column type) that it's unable to handle this column type.

Steve Tauber You have defined a custom data type. You need to override `Type::requiresSQLCommentHint()` to return true in your custom data type, otherwise Doctrine cannot infer the custom type mapping from the column comment and the comparator will always detect changes between Yaml and LONGTEXT. This is not directly related to this issue here I think.Guilherme Blanco I don't really get the issue you describe here. Is it maybe fixed already? Otherwise can you maybe try to explain again so that I can fix that?

there doesn't seem to be a way to correctly assign the type across multiple platforms. E.g. Postgres required "int" but mysql just requires "unsigned". Attempting to use "int" here with mysql will fail.

The method Doctrine\DBAL\Platforms method getIntegerTypeDeclarationSQL will return something like "INT unsigned" for sql, which also fails.

I have an installer script that uses ORM SchemaTool to create the entities in the DB and then populates with basic data using basic EM->persist calls via ORM.

Sequence objects are created, and when using the 10g Instant Client everything worked correctly, however, upon upgrade to latest version of the Instant Client Oracle seems to be expecting consistent case for these schema objects. It appears as if they are being created with a quoted name as they are created in lowercase. OraclePlatform::getSequenceNextValSQL, however, generates "SELECT entity_id_seq.nextval FROM DUAL" and this fails with error "General error: 2289 OCIStmtExecute: ORA-02289: sequence does not exist".

Executing "SELECT "entity_id_seq".nextval FROM DUAL" directly on the DB returns the correct value.

I cant seem to find the problem, in DBAL "lib/Doctrine/DBAL/Platforms/OraclePlatform.php" on line 171, the sequence statement is created with $sequence->getQuotedName($platform), but this only works if quoting is requrested for the sequence.

How do you actually create the sequence? Your entity doesnt have @GeneratedValue.

What does the create schema command say with "--dump-sql" flag? Is the SQL quoted?

Max Milaney Can you please test if this still exists in the current master branch? If so, can you please provide the information requested by Benjamin Eberlei ? Otherwise hunting this down is rather hard... Thank you!

I spent the time to develop a custom data type to workaround this bug: https://frickenate.com/2014/10/json-data-type-doctrine/ . I can't imagine anybody storing null expecting to receive back an empty array. That would be a fundamental break from the very purpose of the SQL NULL value.

the filterExpression on AbstractSchemaManager seems not to work for sequences.

This only happens under postgres.

It seems the way the sequences are handled are the culprit: It tries to get min_value etc of sequences without matching sequence names to the filter expression in advance.

If for example access to the sequences is denied, (Different schema without permissions for the current entity manager), any higher-level ORM operations like generating migration versions fail.

--------------------- UPDATE

the context is when using migrations. Positive regexp expressions do not limit the migration to a single schema. eg ^schemaname.$
Instead, all sequences on the current database are returned.
When trying to limit a migration to a single schema consecutively this doesn't work.
We are using a per-schema connection, so this results in a lot of hassle for us.

the context is when using migrations. Positive regexp expressions do not limit the migration to a single schema. eg ^schemaname.$
Instead, all sequences on the current database are returned.
When trying to limit a migration to a single schema consecutively this doesn't work.
We are using a per-schema connection, so this results in a lot of hassle for us.

I believe we are bumping into the same issue. Our webapp uses Migrations, but for our webapp we are limited to a certain schema within a bigger PostgreSQL database. We only have permissions on our own schema and public.
Now, listSequences in AbstractSchemaManager does filter the asset names correctly with the mentioned fix.

But the problem is with the step before, _getPortableSequencesList (see line 135 of AbstractSchemaManager):

This function goes out and does a _getPortableSequenceDefinition on every sequence in the unfiltered list. For every sequence, the _getPortableSequenceDefinition in PostgreSqlSchemaManager performs a SELECT:

I currently have a workaround running locally, which filters the sequences list before creating the PortableSequence's. This is probably hackish and a poor workaround, just posting here as a temporary solution and further illustration of the problem.

If those are reserved keywords, they should be added to the "PostgreSQLKeywords" class and they will be quoted by Doctrine. As far as I can see those keywords mentioned are not present in this class. Maybe there is something missing?

Obviously SQL Server doesn't support this. So what solution would you expect here? IMO such a scenario should be avoided as there is no reasonable way to deal with changing identity columns. I guess most people are not even aware of this. The solutions available are very risky (creating new schema and then migrating data for example) and can take very long to be finished and is extremly error prone. I don't know if we should support one of those workarounds.

The bug is about the SQL statement that does not work.
1. One solution could be to prevent generating this statement and issue a warning instead. (not preferred)

Two other solutions are mentioned in that Stackoverflow post I mentioned, namely:
2. Create a new table with identity & drop the existing table
3. Create a new column with identity & drop the existing column

That is just the way things have to be done in SQL Server (unfortunately), so it wouldn't be a workaround. I think this is a very reasonable way and in fact similar solutions to similar problems have already been implemented into Doctrine. See http://www.doctrine-project.org/blog/doctrine-2-4-released.html "ALTER TABLE support for SQLite (by hason) by creating new tables, moving all the data and then renaming."

Flip not many people actually use Sqlite in production, its more of a "play database" with Doctrine, wheras SQL Server carries much more weigh. I don't want to implement this kind of workaround, Steve Müller what do you think is safe here?

Really .. if it is ever implemented it would have to be in a few steps because SQL Server does not accept this one-line command like that. So basically when you don't want this solution it means that will never be implemented.

Talking about safe .. mind that the safety should also be ensured in other area's like 1. don't run --force on a production database 2. make backups. So when regarding the safety it's a very narrow area to take a look at. The operation could be setup in such a way that if the new table/column could not be created then the old one doesn't get deleted. That way you have always the safety like a database transaction.

I guess no point in making a Proof of Concept, because the general outline of the problem is actually not that complicated ..

Benjamin Eberlei we could of course use one of the approaches Flip mentioned, but the question is what to do in case something goes wrong during one of the steps required to alter an IDENTITY column. If it is acceptable that there exist "dead" tables/columns in case something goes wrong, we could try implementing that.

We are using prepared statements and parsing on top at the moment to implement things like array param quoting and such.

We should evaluate if it makes sense to introduce a strategy for preparing/parsing queries and make sprintf() usage a default in 3.0 instead of using prepared statements. At least for MySQL the PHP Driver guys are arguing for this kind of processing for years.

The default implementation provided by the AbstractPlatform class dispatches several events related to schema modifications (create table, change column and so on) and offers the ability to prevent the default behaviour by setting a flag on the dispatched event. However, it appears that some DDL operations aren't covered by this mechanism (foreign keys creation, for example, doesn't seem to be observable). Furthermore, the concrete platforms can override any of these methods, potentially leading to inconsistancies in components that expect those events to be dispatched.

I'm working on a large modular application build on top of Symfony, in which each bundle must have its own migration classes (i.e. migrations on a per bundle basis, not at the application level), possibly with references to other parts of the schema. I used to write those classes manually using the dbal api, but this is tiedous and error-prone, and I'd like to take advantage of the mapping-based schema generation. I found that using the event system and "filtering" ddl operations to keep only the operations related to a given bundle is a clean and easy way to achieve this goal, but the issues I mentionned prevent me to complete the implementation.

I'd be glad to hear your opinion on that topic and, providing you're interested in it, contribute in a way or another to improve the existing code.

The DBAL event system is not in its best shape and definately could use some improvement. Your easiest way however would be to hook the process of generating the whole schema, and then dropping tables from those two to only contain the ones you are interested in.

Schema should have support for triggers, views and stored procedures. Since this is not standardizable accross platforms easily, the idea is to allow arbitrary SQL DDL Statements to be registered on a Schema, maybe even qualified by platform.

I don't agree about removing the Migrations project in favor of using only the schema diff tool (which we already have as a command in the ORM btw). Migrating is not only about updating the schema. It also requires migrating data. Otherwise, it is not safe to use in production. This is why the

{doctrine:schema:update}

command displays a warning before running.
A good example is adding a new non-nullable unique field. Applying the schema update works on an empty DB but fails when the table already contains data.

Christophe Coevoet The idea is not to keep only ORM Schema-Tool, which is really only a Dev-Tool. We would rather add support for DbDeploy, Liquibase and Phinx into DBAL via some integration sub-component and using DBAL\Schema to create migration files for their formats.

If you declare a column of a table to be INTEGER PRIMARY KEY, then whenever you insert a NULL into that column of the table, the NULL is automatically converted into an integer which is one greater than the largest value of that column over all other rows in the table, or 1 if the table is empty. [...] Note that the integer key is one greater than the largest key that was in the table just prior to the insert. The new key will be unique over all keys currently in the table, but it might overlap with keys that have been previously deleted from the table.

So in other words, if you remove an entity and then create a new one, the new one will have the same id as the previously deleted one. If you do both operations on the same entitymanager, id references (in proxies f.x.) will suddenly get confused and point to something else (at least that's my current theory..)

The point is: SQLite doesn't act like MySQL as the documentation implies, and IMHO SQLite's current behaviour makes it useless for more complex scenarios. I've found some reference to this problem here:

Unfortunately, it doesn't mention any solution. The problem is that you can't override the columnDefinition, because it would have to be "INTEGER PRIMARY KEY AUTOINCREMENT", but then, you get an exception, because the Platform appends ", PRIMARY KEY(id)", so it's defined twice

As far as I understand there is a conflict in SQLite between having an autoincrement primary key and having a composite primary because you can only choose either way. The PR you referenced removed the autoincrement behaviour in favour of having the opportunity to define composite primary keys. So what would you expect to be the solution here?

Well, from a user's point of view, it would be nice if the SQLite Platform implementation would make full use of the possibilities of SQLite. That is to say: If someone uses composite primary, they get the behaviour Doctrine has right now, and those that use the simple case (which is recommended all over the documentation), get the behaviour previous to the pull request, i.e. autoincrement that works like in MySQL (which the documentation implies). As far as I understood the discussion in the pull request, the author was looking for a solution to implement this, but then the PR was merged before the issue was solved.

I guess what is bothering me is that the current behaviour breaks assumptions that I think many applications using Doctrine make. At least I know that in my own code, I never planned for the possibility of a database primary key being re-used for a different object, especially not during the same request. And like I wrote above, I suspect that Doctrine itself is not totally prepared for that situation either (also mentioned here: https://github.com/doctrine/dbal/pull/66#discussion_r173623). So IMHO the IDENTIY generator strategy for SQLite seems broken, or at least is behaving unexpectedly. The documentation says

AUTO (default): Tells Doctrine to pick the strategy that is preferred by the used database platform. The preferred strategies are IDENTITY for MySQL, SQLite and MsSQL and SEQUENCE for Oracle and PostgreSQL. This strategy provides full portability.

I don't really see how the current behaviour can be said to provide full portability (at least with MySQL, which supposedly uses the same preferred strategy)

Yeah I get your point. But it's always hard and error prone to work around the vendors lack of common features. A possibility COULD be to implement a trigger in columns declared as autoincrement which simulates the behaviour of an auotincrement column on inserts. Oracle uses a similar workaround with a trigger and a sequence to simulate autoincrement columns. But this is just an idea and has to evaluated for usability first.

Yes that's true. The documentation is misleading here. I guess that it was written before the issue came up and then was not updated. Unfortunately SQLite does not support native sequences which would make life a lot easier. But I will keep that in mind and investigate a solution for this.

Well, you cannot fix it for cases with multiple id columns (but the Doctrine documentation already suggests that they should be avoided where possible), but for single integer columns (which is the normal case, as suggested by documentation), SQLite provides all the necessary functionality, as long as you create the column with INTEGER PRIMARY KEY AUTOINCREMENT. So IMHO the best solution would be if support for this could be implemented somehow in the SQL Platform driver.

Granted, this is a very ugly workaround that only works because I know all my ID columns are actually called 'id' (and will never change), but I'm fairly sure that a more general solution could be built with reasonable effort.

Benjamin Eberlei I checked that lately and I came to the same conclusion. The reason why it was removed is to support composite primary keys which was not possible before somehow. We could add autoincrement if only a single column integer primary key is given I think...

every other positive value is an explicite specification of the cache size and results in `CACHE <size>` clause

Now there are two problems.

First one is that a value of `1` has a special meaning on some platforms, too. Oracle does not accept it and throws an error as the minimum allowed value is `2`. PostgreSQL allows a value of `1` but in fact interprets it as `NOCACHE`. SQL Server and SQL Anywhere accept a value of `1` but the behaviour is unclear compared to an explicite `NOCACHE` on these platforms. Now I am assuming that a cache size of `1` maybe also physically has the same effect as if specifying no cache at all and that this is more of a "compatibility" value and we maybe should not allow it at all to preserve portability throughout the vendors.

The second problem is the `null` value for `$cacheSize` in Sequences. This tells the vendor to use it's own specific default cache size which differs on all platforms. The comparator will always mark a sequence as changed then when `null` was initially defined for the sequence and the sequence is introspected from database again. This will lead to repeated `ALTER SEQUENCE` statements in the schema tool which is not good. Also we cannot determine in the platform itself if the sequence has really changed as there currently is no `SequenceDiff` class which could get passed to `AbstractPlatform::getAlterSequenceSQL()`. Instead we only get the changed sequence object. I have put together some [examples](https://gist.github.com/deeky666/6c4e47a275b1c9e7c068) to clarify on what I mean as it is hard to explain.

The way that the PDOSqlsrv driver fetches the identity value for a freshly inserted record will fail if there are triggers on a table that do a secondary insert on a table that also has an identity column.

This is the case when you set up merge replication in SQL Server. The replication creates a series of triggers on the tables to catch any of the changes made for the purposes of replication and inserts those changes elsewhere.

We have switched to using the native SQLSrv drivers instead to work around this since that uses "SELECT SCOPE_IDENTITY() AS LastInsertId" to fetch the value during the insert command.

I'm not quite sure whether we can fix this is in a reasonable way. See PDO's lastInsertId method in itself is documented to be very inconsistent and behaves differently throughout different drivers and even database versions.

This method may not return a meaningful or consistent result across different PDO drivers, because the underlying database may not even support the notion of auto-increment fields or sequences.

Because of this we do not have tests in our testsuite yet that cover the last insert id topic that work as expected on all drivers.
IIRC the fact that DBAL's implementation for the native sqlsrv driver uses "SELECT SCOPE_IDENTITY() AS LastInsertId" is a workaround to support this feature at all and come around the driver limitation IIRC. This as such is a dirty workaround and should IMO not be relied on for scenarios such as you describe.
I'm not quite sure what the expected behaviour of PDO drivers in general would be concerning triggers on PK columns that do other inserts regarding last insert IDs. IMO this is not a Doctrine bug but rather an unsupported use case or even a driver bug/limitation? Not sure on this.
The only thing we could actually do is implement the same workaround we have in the native sqlsrv driver for the PDO driver. But I would rather not do this for such an edge case scenarion.
But that's just my opinion

The main problem is the PDO driver using @@identity to get the last inserted id, which is pretty much the wrong way to do it in all but the simplest of cases... SCOPE_IDENTITY is there for a reason but won't work when called subsequently since it's run inside an sp_prepexec and will be out of scope anyway (it has to be tacked onto the end of the insert statement as it is in the sqlsrv driver)

There's another piece of code in the SQLSrvConnection that uses this method...

The schema-tools, used via the CLI, is not able to detect schema's changes when working on multiple schemas.

For instance, the ORM is configured with a "global" Oracle's user, having permissions on every schemas. To specify which entities belong to which schema, I've prefixed the table name with the corresponding schema.

When trying to do the following command:

orm:schema-tool:update --dump-sql

Doctrine returns me the following message:

Nothing to update - your database is already in sync with the current entity metadata.

If, instead of using the "global" user in the Doctrine's configuration, I set the user of the specific schema I'm trying to generate a table on based from an entity, it works.

Antoine Descamps Okay after having investigated on this in detail and thinking about the possibilities we have to find a solution for this, I came to the following conclusion:
ORM's schema tool and DBAL's schema introspection are designed to be bound to what Doctrine defines as "database" for each platform. This is the scope of the whole operation. In case of Oracle it is the "user". You cannot break out of this scope in any way as the current DBAL implementation is not designed for a "complete" schema introspection and therefore does not allow it at some points. Fully understanding your concern I am afraid we cannot find a reasonable solution for your use case at this point in development (2.x). Furthermore there is a good reason for limiting the schema introspection to a certain layer. If the schema tool would introspect the "complete" schema without regard to the database it is connected to, it would suggest a lot of schema changes from other databases that do not belong to the context of the entity manager / database. This behaviour would even cause a lot more annoyance to users as it is the default use case that users are only interested in one database per entity manager.
Also this problem is completely independant from your mapping definitions. So it doesn't matter whether you prefix your table names or not. When running the "update" operation on the schema tool, it is the online schema introspection part that is preventing the schema tool from behaving as you would wish.
Changing the schema introspection behaviour in DBAL would completely break BC and is at some places in the code not even possible without changing the API.
I am sorry that I have to disappoint you with this conclusion but we I am afraid we cannot do anything about your issue until we start developing 3.x. We might reevaluate your use case their and see what we can do.

Steve Müller I'm using Symfony2 and tying to do such things on MySQL (one entity manager, one user and multiple databases). I wrote my own realisation of schema:update based on code of original command. I have config parameter with list of schemas (databases) and then inside a command in the loop I reloading EM with selected database. Then, as usual, I'm getting diff of schema. At the end, a have diff (list of queries) for each database and can run it all at one. I know it's bad solution, but may be it can be done something similar and more elegant in the core? Like create white list of schemas parameter for EM. If thats can be done, there will not be a problem like:

If the schema tool would introspect the "complete" schema without regard to the database it is connected to, it would suggest a lot of schema changes from other databases that do not belong to the context of the entity manager / database.

At next, if in EM config are set more then one schema we always need use schema.table for tabe naming in the Comparator and other core module, and in generated queries. As profit, it must be not hard to allow using cross-database FK.

Currently, the main entry point Doctrine\DBAL\Connection is documented as a wrapper around Doctrine\DBAL\Driver\Connection.
This is very misleading and encourages other library to typehint against Doctrine\DBAL\Driver\Connection rather than Doctrine\DBAL\Connection. See https://github.com/symfony/symfony/pull/10720 for the original discussion.

Steve Müller The suggestion about Doctrine\DBAL\Connection and Doctrine\DBAL\Driver\Connection started around the misleading support of ping and how to consistently support it across different drivers.
The commit reference I pointed out is Benjamin Eberlei's resolution to remove misleading phpdoc around ping support (which is also highlighted in the ticket as "phpdoc should at least be changed").
If there's anything else that is missing, I'm probably not seeing. All I've done is followed dbal's discussion. =\

Your commit does not fix it at all. Benjamin Eberlei's comment was about the ping method only indeed. But he explained that Doctrine\DBAL\Connection should actually not be a Doctrine\DBAL\Driver\Connection except for legacy reasons, which is why makign it implement Doctrine\DBAL\Driver\PingableConnection was a bad idea even if it has a ping method.

Julia Smith can you please provide more details concerning your actual problem? I'm not sure I understand what the problem is here. I can't see how the SQL you pointed out changes a table's name in any way so I don't understand how that could change the table name's case.
Concerning identifier casing in general I would suggest you to read the following from the MySQL documentation:http://dev.mysql.com/doc/refman/5.5/en/identifier-case-sensitivity.html

I've created a custom type that stores its data in a VARCHAR field. As requiresSQLCommentHint() returns true the value returned by getName() is used as type hint. If I change the length of the field in the entity mapping the scheme won't be updated because getName() still returns the same string. Now I would like to add the length to the returned string. Because this requires access to the fieldDeclaration I suggest to pass this array as paramater to getName().

Dbal for db2 doesn't seem to be able to differentiate between a smallint and a boolean field on db2. If I make an entity which has a boolean field, it will create a table using type smallint. If I later try to update the schema, it will detect the column in the table as a small int, not a boolean. So, it will produce sql update statements to change the type from smallint to smallint, which is obviously unnecessary. I understand db2 doesn't have a boolean type, but it would be nice if dbal realized that a smallint is essentially already a dbal boolean.

Additionally, the update statement is invalid syntax and fails, although this is probably a separate issue.

I was thinking of giving this bug an attempt, but I want some advice on how to fix it properly from those who live and breathe this code base.

Additionally, I recently added a custom utcdatetime type , and now I'm having the same issue with that - doctrine reporting that I should alter all my date columns to modify them to a "TIMESTAMP(0)", even though they're already that type. So, I really think a proper solution would look at the actual mapping definitions being used to decide if a schema change should be suggested.

One idea I have is to modify Doctrine\DBAL\Schema\Comparator->diffColumn() so that it doesn't report this as a diff. However, I think this boolean to smallint mapping fact is platform dependent, so there would need to be some way for the Comparator to behave platform specific for this behavior. This sounds ugly to me.

Another idea would be to modify the specific platform objects themselves to take care of filtering out these frivolous differences in their getAlterTableSQL() methods. The platform object seems to know about type mappings, so maybe it would be appropriate to do something here.

I also see some event listeners that look like they might be able to be leveraged to pull this off here.

To move forward with the Transaction Object, here is an alternative proposal to #571.

This keeps the same basic idea, but now `createTransaction()` returns a `TransactionDefinition` object, which is configurable, and has a `begin()` method that starts the underlying transaction and returns the `Transaction` object:

Once the Transaction is created, its Definition is set and cannot be changed

No risk to forget to call `begin()`: if you do, you'll deal with a TransactionDefinition and just get a call to undefined method if you try to `commit()` it. Plus, your IDE will be able to warn you while coding.

I investigated, and found that some of the test cases for the SQL Server platform weren't actually correct SQL. Also, there were no test cases that covered what the paginator is doing, so I've written test cases for those. I will open a pull request for this issue.

The modifyLimitQuery method in SQLServerPlatform.php should be fixed to pass the fixed old tests and the new tests.

My concern is that that method is becoming too complex, but that's an issue for another day.

As a follow-up to DBAL-900 there are some platforms left that support partial indexes too and need the implementation for that feature.
Currently partial indexes are only implemented for PostgreSQL.
Additionally there should be a functional test covering creation and introspection of partial indexes if possible.

I wrote a detailed explanation here so it is hopefully easy to follow:

Background

With PostgreSQL there are three states for a column to receive a sequence generator: no generator, an internal shortcut towards generating an auto incrementing sequence (SERIAL), and a manually-created sequence. In its current state, DBAL accepts only a boolean "autoincrement" to its `getAlterTableSql()` method (via the passed-in `TableDiff`).

This results in the following scenario:

ORM maps a column to "AUTO" which is treated as a sequence (autoincrement = false)

Diff logic in `getAlterTableSql()` will always detect that autoincrement has changed, and that the requested value is false, so it will always issue a `DROP DEFAULT` statement

This is clearly a bug, and can be proved via a unit test; run the `AbstractPostgreSqlPlatformTestCase::testAlterSchemaSequenceToSequence` test that I have committed against the current DBAL code and you will see it fail (based on what is currently passed in via the ORM; see Q&A below for detailed explanation).

Solution

There already existed a few references to a not-yet-implemented `sequence` property of a column definition, which would store the name of the sequence being used on the column. This makes sense and allows us to support all three column states with regards to sequence, while also preserving backwards compatibility. The default is always null, so it will result in no changes to functionality on other platforms.

So, this PR:

Adds the `Column::_sequence` property

Correctly sets that property during the `PostgreSqlSchemaManager::_getPortableTableColumnDefinition` method (it actually was already there, just not being used)

Checks the value during the `Comparator::diffColumn` operation

Uses the value to more correctly determine when to create/drop a sequence during a schema alter

Adds the appropriate unit tests to verify the six different combinations here

Q&A

Is this just an ORM bug? Could the ORM be changed to just set autoincrement = true for the AUTO and SEQUENCE strategies?

This would solve the immediate problem, yes. However, it would leave no possible distinction between the three ORM mapping strategies of AUTO, SEQUENCE, and IDENTITY. Further, it would cause a break in `getAlterTableSql()` because setting autoincrement to true implies that we are using the database-generated identity sequence which has a specific name, thereby removing the ability for a user to define a sequence manually with a custom name and have it be used here. This strategy opens the door for addressing those cases later, and does not cause a BC break today.

This seems incomplete; for example, this still doesn't handle the case where a sequence is requested to change from AUTO to a specifically-named sequence.

That is intentional. I think these other cases could and probably should be handled, but they would require changes to both the DBAL and ORM. For example, we pass a `Sequence` object to the create/alter/drop sequence functions, but we do not pass one to the alter function. As a result, the actual sequence name is not available here, so we would absolutely need to make a more thought-out ORM change to solve this. I think that should be separate work, or it could just be something we do not support.

Won't this still require an ORM change to set the `sequence` property?

Yes, the following needs to be added to the `SchemaTool::gatherColumn` method:

This is an approach to make use of database vendors' bulk row insert query syntax.
As the nature of bulk inserts usually is to insert a LOT of rows into a table (primarily from an external source), the implementation tries to focus on good performance and low memory consumption. It is NOT intended for `INSERT INTO ... SELECT ...` statement queries.

TODO:

Add an "executor" class that encapsulates the `BulkInsertQuery` object, adds options like bulk size and internally automatically evaluates the underlying platform's limits for a single `INSERT` statement and splits queries accordingly.

An exception has been thrown during the rendering of a template ("An exception occurred while executing
'SELECT DISTINCT TOP 30 id_point_de_vente_reference0
FROM ( SELECT p0_.id_point_de_vente_reference AS id_point_de_vente_reference0,
p0_.reference AS reference1,
p0_.date_derniere_modification AS date_derniere_modification2,
p0_.blocage AS blocage3
FROM point_de_vente_reference p0_
WHERE p0_.id_partenaire_client = ?
ORDER BY p0_.id_point_de_vente_reference DESC ) dctrn_result
ORDER BY id_point_de_vente_reference0 DESC'
with params [2829]:SQLSTATE[42000]: [Microsoft][SQL Server Native Client 11.0][SQL Server]
The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions,
unless TOP, OFFSET or FOR XML is also specified.") in MainBundle:Default:store/list.html.twig at line 79.
I tried to change the class SQLServerPlatform with corrections found on the net, without success.

Altering a foreign key column column (for instance to make it not-nullable) or the index of a foreign key does not work on MySQL (to be exact, it does not work on some MySQL setups, but I haven't found the config setting impacting it yet). Making it work requires dropping the foreign key before altering the column/index and readding it after

There's an issue when using the default cache key generation for the result cache and using the same cache system across different connections as the generated key will be the same regardless of the connection used.

It is best practice to implement retry logic for transactions that are aborted because of deadlocks or timeouts. This makes such method available inside the DBAL and also adds detection for errors where retrying makes sense in the different database drivers.

Deadlocks and timeouts are caused by lock contention and you often can design your application to reduce the likeliness that such an error occurs. But it's impossible to guarantee that such error conditions will never occur. This is why implementing retrying logic for such errors is actually a must when you have to ensure the application does not fail in edge cases or high load.
Some references where something similar has already been implemented:

I chose the name `retryable` because it is consistent with `transactional`. I think the implementation is quite straight forward and fits very well with the DBAL design.

1. in the case of information present in both URL and "normal" parameters, I'm currently giving priority to the information from the URL; IMO this makes more sense than vice versa (base info would be defined in params, and each developer or environment has a URL that may or may not override that base info, such as charset)
1. the syntax for SQLite is `sqlite:///relativepath.db` or `sqlite://ignoredhost/relativepath.db` for relative, and `sqlite:////tmp/absolutepath.db` or `sqlite://ignoredhost//tmp/absolutepath.db` for absolute paths to the database file (I went back and forth on this, but this way is easier and more consistent; https://github.com/kennethreitz/dj-database-url does the same)
1. extra query params are simply "copied" into `$params` verbatim; makes sense IMO especially considering point number 1

By default, the AbstractPlatform creates datetimetz fields as datetime field for platforms not defining a specific creation for fields with timezones.
This can be an acceptable fallback if you know that your application always send the value as UTC even when the database can accept any timezone as input thanks to the datetimetz type (database not supporting timezone would still have consistent data if you always send the same timezone).
However it creates an issue for the SchemaTool: the field will obviously be reported as datetime when inspecting the DB, not as datetimetz (since it does not have a datetimetz). So for platforms not supporting datetimetz natively, the special comment should be used to avoid useless updates.
The special Doctrine command should however not be used for platforms supporting the type natively (PostgreSQL, SQLServer2008, Oracle and SqlAnywhere12).

I have an idea about a fix, so I may send a PR for this in the following days.

In Azure SQL, table ‘sys.extended_properties’ does not exist.
But SQLAzurePlatform inherits from class SQLServerPlatform, which uses it.
The code in question is here /Doctrine/DBAL/Platforms/SQLServerPlatform.php#L845.

Modifications to this portion of code were made to handle comments on columns differently in doctrine/dbal 2.5, but it used to work in 2.4.3.

Maciej Grajcarek looks like it is an issue with your SUM function implementation. If you change your DQL to use ORDER BY COUNT(ypk.value) instead of ORDER BY SUM(ypk.value), does it work then? If so, there is something wrong with your SUM function and therefore not an issue with DBAL.

SELECT * FROM (SELECT table_one.id, table_one.number, table_two.name, ROW_NUMBER() OVER (ORDER BY id DESC) AS doctrine_rownum FROM table_one LEFT JOIN table_two ON table_two.table_one_id = table_one.id) AS doctrine_tbl WHERE doctrine_rownum BETWEEN 1 AND 1 ORDER BY doctrine_rownum

it should be

SELECT * FROM (SELECT table_one.id, table_one.number, table_two.name, ROW_NUMBER() OVER (ORDER BY table_one.id DESC) AS doctrine_rownum FROM table_one LEFT JOIN table_two ON table_two.table_one_id = table_one.id) AS doctrine_tbl WHERE doctrine_rownum BETWEEN 1 AND 1 ORDER BY doctrine_rownum

We don't currently support this nor can support a change in the DateTime type in 2.x. If you want support for this, then I suggest using a custom type (as you are doing) or proposing something new for 3.x

When using the MasterSlaveConnection, the internal parameters array has a different structure than the usual Connection. This can cause the following methods to fail: getHost(), getPort(), getUsername() and getPassword().

I changed the MasterSlaveConnection so that it stores the parameters of the chosen connection in a private property. With this change, the methods that used to fail can now retrieve the configuration for the actual chosen connection, which is, in my opinion, clearly the expected behavior.

I chose to create a new private property in MasterSlaveConnection instead of inherit the $_params from Connection in order to not cause any confusion by having two different array structures in the same variable at different situations.

This is a fix for http://www.doctrine-project.org/jira/browse/DBAL-1057. It leaves the ability to set the platform and platform version from sha: 3176f51d1426022d305c6531a9b9bc93a868bddd, but does not try to connect if we're not already connected. This is consistent with the previous behavior (again see the linked sha) - before there was never a connection made to determine the platform.

The alternate solution is to connect, but surround this by a try-catch (`PDOException`, `DriverException`, `Exception`?) and return `null`. in case we have some situation where the database isn't created or the connection information is wrong.

I know this issue is causing a lot of problems around my world (I ran into myself yesterday), so thanks in advance for the attention.

This is not meant to be merged, at least not yet. The test added in #691 is flawed. It's failing NOT because there is an exception when trying to use a closed connection (in fact, if the connection is closed, it simply re-opens), but instead, the exception is:

(This was originally in #670 but am re-submitting due to a busted merge)

When dealing with legacy schema it would be nice to be able to map default values and not have the schema spit out ALTER statements each time. This works correctly today for basic integer and string columns, but does not handle columns with special types such as boolean or datetime.

In my MySQL table, the column is a varbinary(255) and the ORM entity was marked as type "binary".

How I fixed
--------------
I noticed that the `$type` in `SQLParserUtils::expandListParameters` was marking the type as 103. When this method was checking array types, it was not checking for binary strings. By simply adding the extra check for type 103 in this method, everything started working properly. Line comments below.

Extra
------
This worked to fix it for me and I haven't noticed any side effects of this, however I have not extensively tested all features. If there are any problems with this please let me know.

I want to use a new feature, specify the oracle IDENITY generator (http://www.doctrine-project.org/jira/browse/DDC-2875) . But after inserting a new row, doctrine causes the wrong sequence to find out the id of the inserted row. My problem is solved by adding processing the column name.

PostgreSQL allows the user to set the application_name is connecting
to database. It is useful for monitoring purposes.
Currently one could just manually add 'application_name=foo' to DSN,
but having a parameter eases setting it using DoctrineBundle.

Checking for the existence of the index to be dropped before trying to drop it.

The root of the problem, however, is that when the "from" schema is hydrated from schema-details via Table::__construct, a unique index is added for @JoinColumns that are flagged as unique. This also happens for joined table inheritance child tables.

Seems rather like a missing dependency in composer.json to me: we rely on PDO's APIs, and we're not really interested in polyfilling it when it's not available, as it's actually a lot of code to write for a little achievement :-\

Why provide non-PDO drivers then? In doctrine-dbal not a single method or object from PDO is used (aside of PDOConnection.php), it's all about accessing constants like PDO::PARAM_STR. This particular thing could be polyfilled very easily.

Sure they do, my point is that with minimal effort (that I offer to provide) those drivers could work without PDO as well. In fact that would make more sense - I can imagine that one of typical use cases for e.g. Mysqli driver is a situation where PDO cannot be used for some reason.

Correct me if I'm wrong, but I believe there is no real reason for doctrine-dbal to depend on PDO. Aside of accessing constants, PDO is only used by PDOConnection (which is only used by PDO-based drivers). PDO constants can be shimmed extremely easily.

I was originally having a problem where aliases were not working with order by/group by. I switched to 2.5.1 and this fixed the issue; however this led to another one. When applying a limit to a query using an alias in conjunction with order by, it generates the following error:

For example I have 2 entities (family, person) with ManyToOne(familyId) association in Person entity, and OneToMany(mappedBy="familyId") in Family Entity (Full class definitions are in the end of description)

We use doctrine/dbal in Integration-Tests and to prevent the unit-tests from connecting to a database, we specify `serverVersion` with something made-up (like `5.6`).

However, i'm not comfortable set `serverVersion` to a made-up server-version to prevent connections from being made, when there even isn't a server to connect to. With this PR merged, we could specify `serverVersion` to be `null` instead of something like `5.6` and still prevent connections from being made. `null` is a valid return value for `getDatabasePlatformVersion()`.

Hit by this too. The problem seems to be that the "public" namespace is not added to the table names by default and hence the diff between what postgres says (a "public" schema is created by default in the DB) and what our schema says.

I tried to solve this with a workaround by prepending "public." to all table names. It works for the first migration but then in the next migration will try to delete all tables without the "public." and create them again. So that's not working!

The solution is assuming that there's always a default 'public' namespace in the Schema.php class.

This method is vastly simpler and more reliable than the old method of parsing a whole lot of stuff.

A caveat: if a query is passed that contains a subquery with an ORDER BY clause, the inner ORDER BY clause will be dropped. SQL server doesn't support using ORDER BY inside subqueries. In the future I think we should consider throwing an exception when these are found.

For now, hold off on merging this. There is a PR open for the ORM Paginator that prevents it from sending queries with multiple ORDER BY clauses. The Paginator is the only place in the ORM where this occurs. Until that PR is merged, this one should probably stay open.

I'll be adding a set of functional tests for the Paginator later this week.

It generates automatically as is. I don't know why it tries add another index and drop mine. As I said - I add mapping on already existed table with index added by hand.
When I added a mapping - I need generate migration to sync my mapping and database. And it generates this migration.

The SchemaManager, when used with SQL Server, fails to quote incoming table names and column names that should be quoted. That means that when a table that exists in the database called 'quote-address' needs to be dropped, the drop table statement will fail because the identifer is not marked as quoted when the schema diff is created.

This patch adds a method isValidIdentifier to the AbstractPlatform API - the purpose of this is to check if an identifier is valid to be used in SQL on the platform.

This is used by a new protected method in AbstractSchemaManager, quoteIncomingIdentifier. This method checks if the passed string literal is a valid identifier using isValidIdentifier. If the identifier is not valid, it quotes it for the platform and returns it. If it is valid, or is already quoted, it returns the identifier unchanged.

When I remove the length field from the annotation, I would expect the field to then be interpreted as LONGTEXT as specified in the documentation. However, when I run orm:schematool:update, it says that there is nothing to update.

returns the columns sorted by name. This causes a problem in my application. Other platforms sort them on position. So when I build a model for a HTML table (placing datatype formatters for each column) the order of formatters gets messed up.

The equivalent method in other platforms like MySQL or PostgreSQL all make sure the order is preserved (the same as in the database).

I've implemented the driver for Firebird. It's currently tested with FB 2.5.

The driver uses the ibase-api. Originally I also had a PDO-based implementation. Despite it worked quite well in a real world application, it terribly failed in the Doctrine test suite because of strange transaction behaviour of the Firebird-PDO, thus I finally removed it.

The name of the driver is ibase_firebird, the platform name is firebird.

Because of the common ancestor Firebird an Interbase, it should be possible to use the implementation for Interbase, too. But this is not done nore tested, yet.

I figured they would caused tables to be created but I wouldn't expect defining a join to cause an exception of defining a table twice. Especially since it seems quite possible and logical to use @JoinTable more than once?

Another quite possible and logical example would be one big entity and then only wanting a subset of that data for a join so use a smaller entity to hold that data.

Not saying the use case provided is exactly sane, however I would expect it to work.

Even though the PDO SQLite driver is already available to interact with SQLite databases, PDO currently has a big limitation: it [does not allow to load SQLite extensions](https://bugs.php.net/bug.php?id=64810).

This functionality is provided by the `SQLite3` class, which becomes your only option if you need to use your PHP application with an SQLite extension such as [SpatiaLite](http://www.gaia-gis.it/gaia-sins/).

We are creating a Wrapper Class to allow the use of the Oracle Proxy User feature. For that I need to Wrapper a Class around DBAL\Connection.
Unfortunely, the wrapper class needs to be a subclass of DBAL\Connection which doesn't make sense, there should exist an Interface and the wrapper class should be forced to implement that interface.

That way I don't need to create methods to call all DBAL\Connection methods thru polymorphism.

When connecting to SQL Server without `dbname` parameter set, the underlying driver emits a `undefined index "dbname"` when trying to retrieve the database name connected to via `Doctrine\DBAL\Driver::getDatabase()`.
This PR implements the "live" retrieval of the current database as seen in other drivers already.

When connecting to SQL Anywhere without `dbname` parameter set, the underlying driver emits a `undefined index "dbname"` when trying to retrieve the database name connected to via `Doctrine\DBAL\Driver::getDatabase()`.
This PR implements the "live" retrieval of the current database as seen in other drivers already.

PostgreSQL fails to drop a database if there are active connections using that particular database.
This PR closes all active connections before dropping the database if dropping the database failed before.

Essentially MySQL's `uft8` character set is broken and does not support full UTF-8, and a better alternative, `utf8mb4` has existed for about five years now. Insofar as Doctrine has a MySQL default configuration, `utf8mb4` is a better choice.

This is an attempt to enable functional testing for SQL Anywhere 12 + 16 on Travis.
It uses precompiled `sqlanywhere` PHP extensions for 5.3, 5.4, 5.5 and 5.6. Server-wise it uses preinstalled, lightweight packages (around 15 MB). This allows for a fast environment setup.
The packages reside on my Google Drive account, the URLs for the server packages are secured by Travis encryption.
The only downside of using secured environment variables is, that builds won't work on forks because of different SSH keys but that is a fair trade-off IMO.