Django attempts to support as many features as possible on all database
backends. However, not all database backends are alike, and we’ve had to make
design decisions on which features to support and which assumptions we can make
safely.

This file describes some of the features that might be relevant to Django
usage. Of course, it is not intended as a replacement for server-specific
documentation or reference manuals.

Persistent connections avoid the overhead of re-establishing a connection to
the database in each request. They’re controlled by the
CONN_MAX_AGE parameter which defines the maximum lifetime of a
connection. It can be set independently for each database.

The default value is 0, preserving the historical behavior of closing the
database connection at the end of each request. To enable persistent
connections, set CONN_MAX_AGE to a positive number of seconds. For
unlimited persistent connections, set it to None.

Django opens a connection to the database when it first makes a database
query. It keeps this connection open and reuses it in subsequent requests.
Django closes the connection once it exceeds the maximum age defined by
CONN_MAX_AGE or when it isn’t usable any longer.

In detail, Django automatically opens a connection to the database whenever it
needs one and doesn’t have one already — either because this is the first
connection, or because the previous connection was closed.

At the beginning of each request, Django closes the connection if it has
reached its maximum age. If your database terminates idle connections after
some time, you should set CONN_MAX_AGE to a lower value, so that
Django doesn’t attempt to use a connection that has been terminated by the
database server. (This problem may only affect very low traffic sites.)

At the end of each request, Django closes the connection if it has reached its
maximum age or if it is in an unrecoverable error state. If any database
errors have occurred while processing the requests, Django checks whether the
connection still works, and closes it if it doesn’t. Thus, database errors
affect at most one request; if the connection becomes unusable, the next
request gets a fresh connection.

Since each thread maintains its own connection, your database must support at
least as many simultaneous connections as you have worker threads.

Sometimes a database won’t be accessed by the majority of your views, for
example because it’s the database of an external system, or thanks to caching.
In such cases, you should set CONN_MAX_AGE to a low value or even
0, because it doesn’t make sense to maintain a connection that’s unlikely
to be reused. This will help keep the number of simultaneous connections to
this database small.

The development server creates a new thread for each request it handles,
negating the effect of persistent connections. Don’t enable them during
development.

When Django establishes a connection to the database, it sets up appropriate
parameters, depending on the backend being used. If you enable persistent
connections, this setup is no longer repeated every request. If you modify
parameters such as the connection’s isolation level or time zone, you should
either restore Django’s defaults at the end of each request, force an
appropriate value at the beginning of each request, or disable persistent
connections.

Django assumes that all databases use UTF-8 encoding. Using other encodings may
result in unexpected behavior such as “value too long” errors from your
database for data that is valid in Django. See the database specific notes
below for information on how to set up your database correctly.

If these parameters already have the correct values, Django won’t set them for
every new connection, which improves performance slightly. You can configure
them directly in postgresql.conf or more conveniently per database
user with ALTER ROLE.

Django will work just fine without this optimization, but each new connection
will do some additional queries to set these parameters.

Like PostgreSQL itself, Django defaults to the READCOMMITTEDisolation
level. If you need a higher isolation level such as REPEATABLEREAD or
SERIALIZABLE, set it in the OPTIONS part of your database
configuration in DATABASES:

When specifying db_index=True on your model fields, Django typically
outputs a single CREATEINDEX statement. However, if the database type
for the field is either varchar or text (e.g., used by CharField,
FileField, and TextField), then Django will create
an additional index that uses an appropriate PostgreSQL operator class
for the column. The extra index is necessary to correctly perform
lookups that use the LIKE operator in their SQL, as is done with the
contains and startswith lookup types.

This is dangerous: it will make your database more susceptible to data loss
or corruption in the case of a server crash or power loss. Only use this on
a development machine where you can easily restore the entire contents of
all databases in the cluster.

Django expects the database to support Unicode (UTF-8 encoding) and delegates to
it the task of enforcing transactions and referential integrity. It is important
to be aware of the fact that the two latter ones aren’t actually enforced by
MySQL when using the MyISAM storage engine, see the next section.

MySQL has several storage engines. You can change the default storage engine
in the server configuration.

Until MySQL 5.5.4, the default engine was MyISAM[1]. The main drawbacks of
MyISAM are that it doesn’t support transactions or enforce foreign-key
constraints. On the plus side, it was the only engine that supported full-text
indexing and searching until MySQL 5.6.4.

Since MySQL 5.5.5, the default storage engine is InnoDB. This engine is fully
transactional and supports foreign key references. It’s probably the best
choice at this point. However, note that the InnoDB autoincrement counter
is lost on a MySQL restart because it does not remember the
AUTO_INCREMENT value, instead recreating it as “max(id)+1”. This may
result in an inadvertent reuse of AutoField values.

If you upgrade an existing project to MySQL 5.5.5 and subsequently add some
tables, ensure that your tables are using the same storage engine (i.e. MyISAM
vs. InnoDB). Specifically, if tables that have a ForeignKey between them
use different storage engines, you may see an error like the following when
running migrate:

The Python Database API is described in PEP 249. MySQL has three prominent
drivers that implement this API:

MySQLdb is a native driver that has been developed and supported for over
a decade by Andy Dustman.

mysqlclient is a fork of MySQLdb which notably supports Python 3 and
can be used as a drop-in replacement for MySQLdb. At the time of this writing,
this is the recommended choice for using MySQL with Django.

MySQL Connector/Python is a pure Python driver from Oracle that does not
require the MySQL client library or any Python modules outside the standard
library.

All these drivers are thread-safe and provide connection pooling. MySQLdb
is the only one not supporting Python 3 currently.

In addition to a DB API driver, Django needs an adapter to access the database
drivers from its ORM. Django provides an adapter for MySQLdb/mysqlclient while
MySQL Connector/Python includes its own.

At the time of writing, the latest release of MySQLdb (1.2.5) doesn’t support
Python 3. In order to use MySQLdb under Python 3, you’ll have to install
mysqlclient instead.

Note

There are known issues with the way MySQLdb converts date strings into
datetime objects. Specifically, date strings with value 0000-00-00 are
valid for MySQL but will be converted into None by MySQLdb.

This means you should be careful while using loaddata and
dumpdata with rows that may have 0000-00-00 values, as they
will be converted to None.

The collation setting for a column controls the order in which data is sorted
as well as what strings compare as equal. It can be set on a database-wide
level and also per-table and per-column. This is documented thoroughly in
the MySQL documentation. In all cases, you set the collation by directly
manipulating the database tables; Django doesn’t provide a way to set this on
the model definition.

By default, with a UTF-8 database, MySQL will use the
utf8_general_ci collation. This results in all string equality
comparisons being done in a case-insensitive manner. That is, "Fred" and
"freD" are considered equal at the database level. If you have a unique
constraint on a field, it would be illegal to try to insert both "aa" and
"AA" into the same column, since they compare as equal (and, hence,
non-unique) with the default collation.

In many cases, this default will not be a problem. However, if you really want
case-sensitive comparisons on a particular column or table, you would change
the column or table to use the utf8_bin collation. The main thing to be
aware of in this case is that if you are using MySQLdb 1.2.2, the database
backend in Django will then return bytestrings (instead of unicode strings) for
any character fields it receive from the database. This is a strong variation
from Django’s normal practice of always returning unicode strings. It is up
to you, the developer, to handle the fact that you will receive bytestrings if
you configure your table(s) to use utf8_bin collation. Django itself should
mostly work smoothly with such columns (except for the contrib.sessionsSession and contrib.adminLogEntry tables described below), but
your code must be prepared to call django.utils.encoding.smart_text() at
times if it really wants to work with consistent data – Django will not do
this for you (the database backend layer and the model population layer are
separated internally so the database layer doesn’t know it needs to make this
conversion in this one particular case).

If you’re using MySQLdb 1.2.1p2, Django’s standard
CharField class will return unicode strings even
with utf8_bin collation. However, TextField
fields will be returned as an array.array instance (from Python’s standard
array module). There isn’t a lot Django can do about that, since, again,
the information needed to make the necessary conversions isn’t available when
the data is read in from the database. This problem was fixed in MySQLdb
1.2.2, so if you want to use TextField with
utf8_bin collation, upgrading to version 1.2.2 and then dealing with the
bytestrings (which shouldn’t be too difficult) as described above is the
recommended solution.

Should you decide to use utf8_bin collation for some of your tables with
MySQLdb 1.2.1p2 or 1.2.2, you should still use utf8_general_ci
(the default) collation for the django.contrib.sessions.models.Session
table (usually called django_session) and the
django.contrib.admin.models.LogEntry table (usually called
django_admin_log). Those are the two standard tables that use
TextField internally.

Please note that according to MySQL Unicode Character Sets, comparisons for
the utf8_general_ci collation are faster, but slightly less correct, than
comparisons for utf8_unicode_ci. If this is acceptable for your application,
you should use utf8_general_ci because it is faster. If this is not acceptable
(for example, if you require German dictionary order), use utf8_unicode_ci
because it is more accurate.

Warning

Model formsets validate unique fields in a case-sensitive manner. Thus when
using a case-insensitive collation, a formset with unique field values that
differ only by case will pass validation, but upon calling save(), an
IntegrityError will be raised.

From MySQL 5.7 onwards and on fresh installs of MySQL 5.6, the default value of
the sql_mode option contains STRICT_TRANS_TABLES. That option escalates
warnings into errors when data are truncated upon insertion, so Django highly
recommends activating a strict mode for MySQL to prevent data loss (either
STRICT_TRANS_TABLES or STRICT_ALL_TABLES).

If you need to customize the SQL mode, you can set the sql_mode variable
like other MySQL options: either in a config file or with the entry
'init_command':"SETsql_mode='STRICT_TRANS_TABLES'" in the
OPTIONS part of your database configuration in DATABASES.

When Django generates the schema, it doesn’t specify a storage engine, so
tables will be created with whatever default storage engine your database
server is configured for. The easiest solution is to set your database server’s
default storage engine to the desired engine.

If you’re using a hosting service and can’t change your server’s default
storage engine, you have a couple of options.

After the tables are created, execute an ALTERTABLE statement to
convert a table to a new storage engine (such as InnoDB):

ALTERTABLE<tablename>ENGINE=INNODB;

This can be tedious if you have a lot of tables.

Another option is to use the init_command option for MySQLdb prior to
creating your tables:

'OPTIONS':{'init_command':'SET default_storage_engine=INNODB',}

This sets the default storage engine upon connecting to the database.
After your tables have been created, you should remove this option as it
adds a query that is only needed during table creation to each database
connection.

There are known issues in even the latest versions of MySQL that can cause the
case of a table name to be altered when certain SQL statements are executed
under certain conditions. It is recommended that you use lowercase table
names, if possible, to avoid any problems that might arise from this behavior.
Django uses lowercase table names when it auto-generates table names from
models, so this is mainly a consideration if you are overriding the table name
via the db_table parameter.

If you use the MyISAM storage engine please be aware of the fact that you will
receive database-generated errors if you try to use the savepoint-related
methods of the transactions API. The reason
for this is that detecting the storage engine of a MySQL database/table is an
expensive operation so it was decided it isn’t worth to dynamically convert
these methods in no-op’s based in the results of such detection.

MySQL can index only the first N chars of a BLOB or TEXT column. Since
TextField doesn’t have a defined length, you can’t mark it as
unique=True. MySQL will report: “BLOB/TEXT column ‘<db_column>’ used in key
specification without a key length”.

MySQL 5.6.4 and later can store fractional seconds, provided that the
column definition includes a fractional indication (e.g. DATETIME(6)).
Earlier versions do not support them at all. In addition, versions of MySQLdb
older than 1.2.5 have a bug that also prevents the use of fractional seconds
with MySQL.

Django will not upgrade existing columns to include fractional seconds if the
database server supports it. If you want to enable them on an existing database,
it’s up to you to either manually update the column on the target database, by
executing a command like:

Previously, Django truncated fractional seconds from datetime and
time values when using the MySQL backend. Now it lets the database
decide whether it should drop that part of the value or not. By default, new
DateTimeField or TimeField columns are now created with fractional
seconds support on MySQL 5.6.4 or later with either mysqlclient or
MySQLdb 1.2.5 or later.

If you are using a legacy database that contains TIMESTAMP columns, you must
set USE_TZ=False to avoid data corruption.
inspectdb maps these columns to
DateTimeField and if you enable timezone support,
both MySQL and Django will attempt to convert the values from UTC to local time.

When performing a query on a string type, but with an integer value, MySQL will
coerce the types of all values in the table to an integer before performing the
comparison. If your table contains the values 'abc', 'def' and you
query for WHEREmycolumn=0, both rows will match. Similarly, WHEREmycolumn=1
will match the value 'abc1'. Therefore, string type fields included in Django
will always cast the value to a string before using it in a query.

If you implement custom model fields that inherit from
Field directly, are overriding
get_prep_value(), or use
RawSQL,
extra(), or
raw(), you should ensure that you perform
appropriate typecasting.

SQLite provides an excellent development alternative for applications that
are predominantly read-only or require a smaller installation footprint. As
with all database servers, though, there are some differences that are
specific to SQLite that you should be aware of.

For all SQLite versions, there is some slightly counter-intuitive behavior when
attempting to match some types of strings. These are triggered when using the
iexact or contains filters in Querysets. The behavior
splits into two cases:

1. For substring matching, all matches are done case-insensitively. That is a
filter such as filter(name__contains="aa") will match a name of "Aabb".

2. For strings containing characters outside the ASCII range, all exact string
matches are performed case-sensitively, even when the case-insensitive options
are passed into the query. So the iexact filter will behave exactly
the same as the exact filter in these cases.

Some possible workarounds for this are documented at sqlite.org, but they
aren’t utilized by the default SQLite backend in Django, as incorporating them
would be fairly difficult to do robustly. Thus, Django exposes the default
SQLite behavior and you should be aware of this when doing case-insensitive or
substring filtering.

SQLite 3.6.23.1 and older contains a bug when handling query parameters in
a CASE expression that contains an ELSE and arithmetic.

SQLite 3.6.23.1 was released in March 2010, and most current binary
distributions for different platforms include a newer version of SQLite, with
the notable exception of the Python 2.7 installers for Windows.

As of this writing, the latest release for Windows - Python 2.7.10 - includes
SQLite 3.6.21. You can install pysqlite2 or replace sqlite3.dll (by
default installed in C:\Python27\DLLs) with a newer version from
https://www.sqlite.org/ to remedy this issue.

SQLite is meant to be a lightweight database, and thus can’t support a high
level of concurrency. OperationalError:databaseislocked errors indicate
that your application is experiencing more concurrency than sqlite can
handle in default configuration. This error means that one thread or process has
an exclusive lock on the database connection and another thread timed out
waiting for the lock the be released.

Python’s SQLite wrapper has
a default timeout value that determines how long the second thread is allowed to
wait on the lock before it times out and raises the OperationalError:databaseislocked error.

If you’re getting this error, you can solve it by:

Switching to another database backend. At a certain point SQLite becomes
too “lite” for real-world applications, and these sorts of concurrency
errors indicate you’ve reached that point.

Rewriting your code to reduce concurrency and ensure that database
transactions are short-lived.

For most backends, raw queries (Manager.raw() or cursor.execute())
can use the “pyformat” parameter style, where placeholders in the query
are given as '%(name)s' and the parameters are passed as a dictionary
rather than a list. SQLite does not support this.

Django supports Oracle Database Server versions 11.2 and higher. Versions
4.3.1 through 5.2.1 of the cx_Oracle Python driver are supported, although
5.1.3 or later is recommended as these versions support Python 3.

Note that due to a Unicode-corruption bug in cx_Oracle 5.0, that
version of the driver should not be used with Django;
cx_Oracle 5.0.1 resolved this issue, so if you’d like to use a
more recent cx_Oracle, use version 5.0.1.

cx_Oracle 5.0.1 or greater can optionally be compiled with the
WITH_UNICODE environment variable. This is recommended but not
required.

In order for the pythonmanage.pymigrate command to work, your Oracle
database user must have privileges to run the following commands:

CREATE TABLE

CREATE SEQUENCE

CREATE PROCEDURE

CREATE TRIGGER

To run a project’s test suite, the user usually needs these additional
privileges:

CREATE USER

ALTER USER

DROP USER

CREATE TABLESPACE

DROP TABLESPACE

CREATE SESSION WITH ADMIN OPTION

CREATE TABLE WITH ADMIN OPTION

CREATE SEQUENCE WITH ADMIN OPTION

CREATE PROCEDURE WITH ADMIN OPTION

CREATE TRIGGER WITH ADMIN OPTION

Note that, while the RESOURCE role has the required CREATE TABLE, CREATE
SEQUENCE, CREATE PROCEDURE and CREATE TRIGGER privileges, and a user
granted RESOURCE WITH ADMIN OPTION can grant RESOURCE, such a user cannot
grant the individual privileges (e.g. CREATE TABLE), and thus RESOURCE
WITH ADMIN OPTION is not usually sufficient for running tests.

Some test suites also create views; to run these, the user also needs
the CREATE VIEW WITH ADMIN OPTION privilege. In particular, this is needed
for Django’s own test suite.

Changed in Django 1.8:

Prior to Django 1.8, the test user was granted the CONNECT and RESOURCE
roles, so the extra privileges required for running the test suite were
different.

All of these privileges are included in the DBA role, which is appropriate
for use on a private developer’s database.

The Oracle database backend uses the SYS.DBMS_LOB and SYS.DBMS_RANDOM
packages, so your user will require execute permissions on it. It’s normally
accessible to all users by default, but in case it is not, you’ll need to grant
permissions like so:

In this case, you should leave both HOST and PORT empty.
However, if you don’t use a tnsnames.ora file or a similar naming method
and want to connect using the SID (“xe” in this example), then fill in both
HOST and PORT like so:

If you plan to run Django in a multithreaded environment (e.g. Apache using the
default MPM module on any modern operating system), then you must set
the threaded option of your Oracle database configuration to True:

By default, the Oracle backend uses a RETURNINGINTO clause to efficiently
retrieve the value of an AutoField when inserting new rows. This behavior
may result in a DatabaseError in certain unusual setups, such as when
inserting into a remote table, or into a view with an INSTEADOF trigger.
The RETURNINGINTO clause can be disabled by setting the
use_returning_into option of the database configuration to False:

'OPTIONS':{'use_returning_into':False,},

In this case, the Oracle backend will use a separate SELECT query to
retrieve AutoField values.

Oracle imposes a name length limit of 30 characters. To accommodate this, the
backend truncates database identifiers to fit, replacing the final four
characters of the truncated name with a repeatable MD5 hash value.
Additionally, the backend turns database identifiers to all-uppercase.

To prevent these transformations (this is usually required only when dealing
with legacy databases or accessing tables which belong to other users), use
a quoted name as the value for db_table:

Quoted names can also be used with Django’s other supported database
backends; except for Oracle, however, the quotes have no effect.

When running migrate, an ORA-06552 error may be encountered if
certain Oracle keywords are used as the name of a model field or the
value of a db_column option. Django quotes all identifiers used
in queries to prevent most such problems, but this error can still
occur when an Oracle datatype is used as a column name. In
particular, take care to avoid using the names date,
timestamp, number or float as a field name.

Django generally prefers to use the empty string (‘’) rather than
NULL, but Oracle treats both identically. To get around this, the
Oracle backend ignores an explicit null option on fields that
have the empty string as a possible value and generates DDL as if
null=True. When fetching from the database, it is assumed that
a NULL value in one of these fields really means the empty
string, and the data is silently converted to reflect this assumption.

The Oracle backend stores TextFields as NCLOB columns. Oracle imposes
some limitations on the usage of such LOB columns in general:

LOB columns may not be used as primary keys.

LOB columns may not be used in indexes.

LOB columns may not be used in a SELECTDISTINCT list. This means that
attempting to use the QuerySet.distinct method on a model that
includes TextField columns will result in an ORA-00932 error when
run against Oracle. As a workaround, use the QuerySet.defer method in
conjunction with distinct() to prevent TextField columns from being
included in the SELECTDISTINCT list.

The Django versions and ORM features supported by these unofficial backends
vary considerably. Queries regarding the specific capabilities of these
unofficial backends, along with any support queries, should be directed to
the support channels provided by each 3rd party project.