Tag: datastore

This is an ultra-simplified example of how to delete old entities from the App Engine Datastore. The first 3 lines of code retrieves the current date, then subtracts 60 days from the current time (the multiplication converts days to milliseconds). DATE_PROPERTY_ON_ENTITY is the date property on the entity – when first writing the entity to the datastore, add the current date as a property. ENTITY_KIND is the entity kind we’re deleting.

Note that is a simplified function – it’s useful if you have a handful of entities that need deleting, but if you have more than a handful, you should convert to using datastore cursors and paging through entities to delete.

As the exception message suggests, this log indicates that the datastore encountered an internal error while handling the datastore operation. Unfortunately, there’s nothing that a developer can do to fix this error since it’s an internal App Engine issue.

Generally this type of exception fixes itself sooner or later; if it persists, file an issue at the App Engine bug tracker: https://code.google.com/p/googleappengine/issues/entry?template=Production%20issue

While using JDO or another datastore helper library, applications may encounter a value of ExQ when requesting a query cursor. This value means that there are no more results (entities) to display for that query, and consequently that there are no more query pages to return.

All applications using query cursors should check for an invalid or end of query results marker, and handle this in an application-specific way. A query cursor string that is null or empty (zero length) generally denotes the end of query results. The cursor value ExQ can be handled in the same way.

A word of warning: some third-party datastore helper libraries spawn an exception or error condition when applications attempt to access invalid cursors, or attempt to request a cursor when end of results has already been reached. When using these libraries, it’s important to fully read the documentation and understand how these libraries interact with the App Engine datastore.

A quick note for today: some developers are experiencing the below error while attempting to vacuum an application’s indexes:

Error Details:
Line 0, column 10: Unable to find property 'application' on class:
com.google.apphosting.utils.config.IndexYamlReader
.
com.google.appengine.tools.admin.AdminException: Unable to perform vacuum_indexes
at com.google.appengine.tools.admin.AppAdminImpl.vacuumIndexes(AppAdminImpl.java:346)
at com.google.appengine.tools.admin.AppCfg$VacuumIndexesAction.execute(AppCfg.java:1605)
at com.google.appengine.tools.admin.AppCfg.executeAction(AppCfg.java:327)
at com.google.appengine.tools.admin.AppCfg.<init>(AppCfg.java:210)
at com.google.appengine.tools.admin.AppCfg.<init>(AppCfg.java:121)
at com.google.appengine.tools.admin.AppCfg.main(AppCfg.java:117)

Google seems to have changed the index backend during the 1.8.6 App Engine release, and this is causing older SDK versions to break when they attempt to issue the vacuum indexes command. To fix this, update your SDK version to the latest available.

Here’s a Java code example to search the datastore for all entities within a kind older than a given date.

The variable kind is the entity kind being searched, add_date is a property on each entity that is set to the date the entity was created, and entities is a java.util.List object containing the returned entities. The variable time_point represents a point in time; we query the datastore for all entities with a date less than that.

In this example we filter on PropertyOne, requiring it to be equal to true. You can also set other inequalities such as greater than ( > ). Kind is the kind of the entities to query, and PropertyTwodemonstrates ordering by descending order. CustomStruct is the struct that was used to create the entity. Remember to put your entity processing code just before the last brace ( } ).

Occasionally you may see the error Oops! We couldn’t retrieve your list of kinds from the datastore viewer screen of the App Engine admin console:

Generally this is a transient error: it essentially means that the App Engine admin console is currently too busy to show a view of your datastore’s contents. Wait a few minutes and refresh the page, your datastore’s information should appear.

Seeing this error can also mean that the datastore is empty; for example, if it’s a just-created application.

Here is a code snippet demonstrating how to retrieve a datastore entity by a known kind and integer ID. This code also retrieves the entity ID from a HTTP parameter.

R represents a http.Request reference. The entity ID to retrieve is accessed from the user’s submitted form under the name entity_id. Kind represents a string containing the name of the kind to retrieve from the datastore. The retrieved entity is stored into the variable entity, which is the struct CustomStruct.

In short, this code pulls out the Entity ID from the HTTP request, converts it into an integer, manufactures the entity’s key using the kind and entity id, and then retrieves the corresponding entity. The code then translates the retrieved entity into the defined CustomStruct variable entity.