Datastore Functions

Note:
Developers building new applications are strongly encouraged to use the
NDB Client Library, which has several benefits
compared to this client library, such as automatic entity caching via the Memcache
API. If you are currently using the older DB Client Library, read the
DB to NDB Migration Guide

The functions described on this page are defined in the
google.appengine.ext.db package.

Functions

allocate_ids
(model,
count)

Allocates a batch of IDs in the Datastore for a Datastore kind and parent
combination.

IDs allocated in this manner will not be used by the Datastore's
automatic ID sequence generator and can be used in
entity keys
without conflict..

Arguments

model

The model key for which to allocate an ID batch. This is a regular
key,
but only the parent and kind of the key are necessary to determine which
ID sequence to use.

count

The number of IDs to allocate.

Returns a tuple of the first and last IDs that it allocates. For example,
if you allocated 10 IDs using this function you would get a return in the
format (1, 10), not a full list of created IDs.

Asynchronously allocates a batch of IDs in the Datastore for a Datastore
kind and parent combination.

This function is identical to
allocate_ids()
except that it returns an asynchronous object. You can call
get_result()
on the return value to block on the call and return the result.

Arguments

model

A
db.Model
instance,
key,
or string to serve as a template specifying the ID sequence in which to
allocate IDs. Returned ids should only be used in entities with the same
parent (if any) and kind as this key.

count

The number of IDs to allocate.

Returns a tuple of the first and last IDs that it allocates. For example,
if you allocated 10 IDs using this function you would get a return value in
the format (1, 10), not a full list of created IDs.

allocate_id_range
(model,
start,
end,
**kwargs)

Allocates a range of IDs with specific endpoints. Once these IDs have
been allocated, you can manually assign them to newly created entities.

The Datastore's automatic ID allocator never assigns a key that has
already been allocated (either through automatic ID allocation or through an
explicit `allocate_ids` call). As a result, entities written to the
given key range will never be overwritten. However, writing entities with
manually assigned keys in this range may overwrite existing entities (or new
entities written by a separate request), depending on the key range state
returned.

Use this function only if you have an existing numeric id range that you
want to reserve (for example, bulk loading entities that already have IDs).
If you don't care about which IDs you receive, use
allocate_ids()
instead.

Arguments

model

A
db.Model
instance,
key,
or string to serve as a template specifying the ID sequence in which to
allocate IDs. Returned ids should only be used in entities with the same
parent (if any) and kind as this key.

start

The first ID to allocate, a number.

end

The last ID to allocate, a number.

Returns one of (KEY_RANGE_EMPTY,
KEY_RANGE_CONTENTION, KEY_RANGE_COLLISION). If not
KEY_RANGE_EMPTY, this represents a potential issue with using
the allocated key range.

create_transaction_options
(**kwargs)

Creates a transaction options object (class
TransactionOptions) for controlling transaction execution. You
pass the resulting object as the first argument to the
run_in_transaction_options()
function.

Arguments

propagation

What to do if this transactional function is called from within another transaction:

ALLOWED

If already in a transaction, continue using it; if not, start one.

Note: If a function using this policy throws an exception, it is probably not safe to catch the exception and commit the outer transaction; the function may have left the outer transaction in a bad state.

MANDATORY

Continue in the existing transaction, if any; if not, throw a
BadRequestError
exception.

Note: If a function using this policy throws an exception, it is probably not safe to catch the exception and commit the outer transaction; the function may have left the outer transaction in a bad state.

INDEPENDENT

Create a new transaction, pausing any existing transaction.

Note: A function using this policy should not return any entities read in the new transaction, as the entities are not transactionally consistent with the outer transaction.

Note: The deprecated Master/Slave Datastore does not support XG transactions.

retries

Number of retries to attempt on failure of transaction commit.

deadline

Maximum time, in seconds, to wait for Datastore to return a result before aborting with an error. Accepts either an integer or a floating-point value. Cannot be set higher than the default value (60 seconds), but can be adjusted downward to ensure that a particular operation fails quickly (for instance, to return a faster response to the user, retry the operation, try a different operation, or add the operation to a task queue).

A model instance, an
entity key,
or a list (or other iterable) of model instances or entity keys to
delete.

deadline

Maximum time, in seconds, to wait for Datastore to return a result before aborting with an error. Accepts either an integer or a floating-point value. Cannot be set higher than the default value (60 seconds), but can be adjusted downward to ensure that a particular operation fails quickly (for instance, to return a faster response to the user, retry the operation, try a different operation, or add the operation to a task queue).

As with
put(),
if multiple keys are given, they may be in more than one
entity group.

An exception will always be raised if any error occurs during the
operation, even if some of the entities actually were deleted. If the call
returns without raising an exception, then all of the entities were deleted
successfully.

Caution: Deleting multiple entities in a
single operation does not guarantee that the deletions will take place
atomically unless the operation is performed inside a
transaction.
Other processes querying the Datastore may see inconsistent results even
when the query is performed with strong consistency.

delete_async
(models,
deadline=60)

Asynchronously deletes one or more model instances from the
Datastore.

This function is identical to
delete()
except that it returns an asynchronous object. You can call
get_result()
on the return value to block on the call.

Arguments

models

A model instance, an
entity key,
or a list (or other iterable) of model instances or entity keys to
delete.

deadline

Maximum time, in seconds, to wait for Datastore to return a result before aborting with an error. Accepts either an integer or a floating-point value. Cannot be set higher than the default value (60 seconds), but can be adjusted downward to ensure that a particular operation fails quickly (for instance, to return a faster response to the user, retry the operation, try a different operation, or add the operation to a task queue).

As with
put(),
if multiple keys are given, they may be in more than one
entity group.

This function returns an object that lets you block on the result of the
call.

An exception will always be raised if any error occurs during the
operation, even if some of the entities actually were deleted. If the call
returns without raising an exception, then all of the entities were deleted
successfully.

Caution: Deleting multiple entities in a
single operation does not guarantee that the deletions will take place
atomically unless the operation is performed inside a
transaction.
Other processes querying the Datastore may see inconsistent results even
when the query is performed with strong consistency.

get
(keys,
read_policy=STRONG_CONSISTENCY,
deadline=60)

Fetch the specific Model instance(s) with the given key(s) from the
Datastore.

Arguments

keys

Key
of entity to be retrieved, a string representation of the key, or a list of keys or their string representations.

read_policy

Read policy specifying desired level of data consistency:

STRONG_CONSISTENCY

Guarantees the freshest results, but limited to a single
entity group.

EVENTUAL_CONSISTENCY

Can span multiple entity groups, but may occasionally return
stale results. In general, eventually consistent queries run
faster than strongly consistent queries, but there is no guarantee.

Note: Global (non-ancestor) queries ignore this argument.

deadline

Maximum time, in seconds, to wait for Datastore to return a result before aborting with an error. Accepts either an integer or a floating-point value. Cannot be set higher than the default value (60 seconds), but can be adjusted downward to ensure that a particular operation fails quickly (for instance, to return a faster response to the user, retry the operation, try a different operation, or add the operation to a task queue).

If keys consists of a single key (or its string
representation), this function returns the model instance associated with
the key if the key exists in the Datastore, otherwise None. If
keys is a list, the return value is a corresponding list of
model instances, with None values where no entity exists for a
given key.

Asynchronously fetches the specified Model instance(s) from the
Datastore.

This function is identical to
get()
except that it returns an asynchronous object. You can call
get_result()
on the return value to block on the call and get the results.

Arguments

keys

Key
of entity to be retrieved, a string representation of the key, or a list of keys or their string representations.

read_policy

Read policy specifying desired level of data consistency:

STRONG_CONSISTENCY

Guarantees the freshest results, but limited to a single
entity group.

EVENTUAL_CONSISTENCY

Can span multiple entity groups, but may occasionally return
stale results. In general, eventually consistent queries run
faster than strongly consistent queries, but there is no guarantee.

Note: Global (non-ancestor) queries ignore this argument.

deadline

Maximum time, in seconds, to wait for Datastore to return a result before aborting with an error. Accepts either an integer or a floating-point value. Cannot be set higher than the default value (60 seconds), but can be adjusted downward to ensure that a particular operation fails quickly (for instance, to return a faster response to the user, retry the operation, try a different operation, or add the operation to a task queue).

If keys consists of a single key (or its string
representation), this function returns the model instance associated with
the key if the key exists in the Datastore, otherwise None. If
keys is a list, the return value is a corresponding list of
model instances, with None values where no entity exists for a
given key.

Asynchronously returns a list of composite indexes belonging to the
calling application.

is_in_transaction
()

Returns a boolean indicating whether the current scope is executing in a
transaction.

model_to_protobuf
(model_instance)

Creates the protocol buffer serialization of a Model
instance. A protocol buffer is Google's serialization format used for remote
procedure calls, and can be useful for serializing Datastore objects for
backup and restore purposes.

Returns an object of the appropriate kind class. If the kind class does
not exist, raises a
KindError
exception. If the object is not valid according to the model, raises a
BadValueError
exception.

You can save the new object to the Datastore just like any other
Model instance, such as by calling its
put()
method. The object retains the key it had when the protocol buffer was
created. If an object with that key already exists in the Datastore, saving
the deserialized object overwrites the existing object.

Caution: If the object's key uses a
system-assigned ID and that ID has not already been allocated for the given
path and kind, the save will succeed, but the ID is not reserved. An object
created in the future may be assigned that ID, and would overwrite the
earlier object. For safety, restore objects only in the same application
where they existed when they were serialized.

model_is_projection
(model_instance)

Returns True if the specified query
(model_instance) is a
projection query
instead of a query for a full entity.

Argument

model_instance

The query you are checking to determine whether it is a projection
query.

Returns True if the query is a projection query,
False if it is not.

put
(models,
deadline=60)

Writes one or more model instances to the Datastore.

Arguments

models

A model instance or a list of model instances to store.

deadline

Maximum time, in seconds, to wait for Datastore to return a result before aborting with an error. Accepts either an integer or a floating-point value. Cannot be set higher than the default value (60 seconds), but can be adjusted downward to ensure that a particular operation fails quickly (for instance, to return a faster response to the user, retry the operation, try a different operation, or add the operation to a task queue).

If multiple model instances are given, they may be in more than one
entity group.

An exception will always be raised if any error occurs during the
operation, even if some of the entities actually were written. If the call
returns without raising an exception, then all of the entities were written
successfully.

If models consists of a single model instance, this function
returns the corresponding
Key
object. If models is a list, the return value is a list of
corresponding
Key
objects.

Caution: Writing multiple entities in a
single operation does not guarantee that the writes will take place
atomically unless the operation is performed inside a
transaction.
Other processes querying the Datastore may see inconsistent results even
when the query is performed with strong consistency.

put_async
(models,
deadline=60)

Writes one or more model instances to the Datastore.

This function is identical to
put()
except that it returns an asynchronous object. You can call
get_result()
on the return value to block on the call and get the results.

Arguments

models

A model instance or a list of model instances to store.

deadline

Maximum time, in seconds, to wait for Datastore to return a result before aborting with an error. Accepts either an integer or a floating-point value. Cannot be set higher than the default value (60 seconds), but can be adjusted downward to ensure that a particular operation fails quickly (for instance, to return a faster response to the user, retry the operation, try a different operation, or add the operation to a task queue).

If multiple model instances are given, they may be in more than one
entity group.

An exception will always be raised if any error occurs during the
operation, even if some of the entities actually were written. If the call
returns without raising an exception, then all of the entities were written
successfully.

This function returns an asynchronous object on which
get_result()
can be called. The results returned are the same as for
put().

Caution: Writing multiple entities in a
single operation does not guarantee that the writes will take place
atomically unless the operation is performed inside a
transaction.
Other processes querying the Datastore may see inconsistent results even
when the query is performed with strong consistency.

query_descendants
(model_instance)

Returns a query for all the descendants of a model instance.

Argument

model_instance

The model instance whose descendants you want to find.

run_in_transaction
(function,
*args,
**kwargs)

Executes a function containing Datastore updates in a single
transaction.
If any code raises an exception during the transaction, all updates made in
the transaction are rolled back. Alternatively, you can use the
@db.transactional()
decorator.

Arguments

function

Function to execute.

args

Positional arguments to pass to the function.

kwargs

Keyword arguments to pass to the function.

If the function returns a value, run_in_transaction()
returns the value to the caller.

If the function raises an exception, the transaction is rolled back. If
the exception is a
Rollback
exception, it is not re-raised; any other exception is re-raised to the
caller.

The Datastore uses optimistic locking and retries for transactions. If
the transaction prepared by the function cannot be committed,
run_in_transaction() calls the function again, retrying the
transaction up to 3 times. (To use a different number of retries, use
run_in_transaction_custom_retries().)
Because the transaction function may be called more than once for a single
transaction, the function should not have side effects, including
modifications to arguments.

If the transaction cannot be committed, such as because of a high rate of
contention, a
TransactionFailedError
exception is raised.

Executes a function containing Datastore updates in a single
transaction,
retrying the transaction a specified number of times in the event of
contention. If any code raises an exception during the transaction, all
updates made in the transaction are rolled back.

Other than the ability to specify the number of retries, this function
behaves identically to
run_in_transaction().

Arguments

retries

Maximum number of times to call the function in the event of
contention within the entity group (more than one user attempting to
modify the group simultaneously).

function

Function to execute.

args

Positional arguments to pass to the function.

kwargs

Keyword arguments to pass to the function.

run_in_transaction_options
(options,
function,
*args,
**kwargs)

Executes a function containing Datastore updates in a single transaction
using the options specified in a
transaction options object.
If any code raises an exception during the transaction, all Datastore
updates made in the transaction are rolled back.

The transaction options object containing the settings used by this
transaction. To enable XG transactions, its xg parameter must be set to
True.

function

Function to execute.

args

Positional arguments to pass to the function.

kwargs

Keyword arguments to pass to the function.

If the function returns a value,
run_in_transaction_options() returns the value to the
caller.

If the function raises an exception, the transaction is rolled back. If
the exception is a
Rollback
exception, it is not re-raised; any other exception is re-raised to the
caller.

The Datastore uses optimistic locking and retries for transactions. If
the transaction prepared by the function cannot be committed,
run_in_transaction_options() calls the function again, retrying
the transaction up to the number of retries specified in the transaction
options object. Because the transaction function may be called more than
once for a single transaction, the function should not have side effects,
including modifications to arguments.

If the transaction cannot be committed, such as because of a high rate of
contention, a
TransactionFailedError
exception is raised.

Decorators

Makes a function run inside a db transaction. Thus, instead
of calling run_in_transaction(func), you can call
func().

Arguments

propagation

What to do if this transactional function is called from within another transaction:

ALLOWED

If already in a transaction, continue using it; if not, start one.

Note: If a function using this policy throws an exception, it is probably not safe to catch the exception and commit the outer transaction; the function may have left the outer transaction in a bad state.

MANDATORY

Continue in the existing transaction, if any; if not, throw a
BadRequestError
exception.

Note: If a function using this policy throws an exception, it is probably not safe to catch the exception and commit the outer transaction; the function may have left the outer transaction in a bad state.

INDEPENDENT

Create a new transaction, pausing any existing transaction.

Note: A function using this policy should not return any entities read in the new transaction, as the entities are not transactionally consistent with the outer transaction.

Note: The deprecated Master/Slave Datastore does not support XG transactions.

retries

Number of retries to attempt on failure of transaction commit.

deadline

Maximum time, in seconds, to wait for Datastore to return a result before aborting with an error. Accepts either an integer or a floating-point value. Cannot be set higher than the default value (60 seconds), but can be adjusted downward to ensure that a particular operation fails quickly (for instance, to return a faster response to the user, retry the operation, try a different operation, or add the operation to a task queue).

@db.non_transactional
(allow_existing=True)

Ensures that a function is run outside a db transaction,
even if called from within a transaction.

Argument

allow_existing

If True, allow function to be called from within an
existing transaction; if False, throw a
BadRequestError
exception instead.