google datastore transactions and scalability

I've been studying google's datastore that comes with their apps engine, and there's no discussion of it here yet. Their design has a lot of potential getting away from the scalability issues of regular databases where you end up having to deal with distribution, clustering and replication.

The key to transactions in their datastore lies in "entity groups." An entity is like a flexible row or record. Upon adding an entity to the database you have to associate it with a group if you ever plan to combine an action or query on that entity with another in the group in one transaction. This is not to be confused with having them in the same table. The "kind" of entity works like a table in that in GQL you select from entities of a particular kind.

Anyway, this means that if I am tracking modifications of Item entities using Revision entities, I will need to put the Revisions for an Item in an entity group with that Item. It also suggests that I cannot have a Revision entity that involves 2 Items unless I can do it without combining the Items modification into one transaction with the creation of the Revision entity.

On the face of it I think designing an application with this in mind has great benefits. I wonder if there are any significant disadvantages.

I must have missed that thread the first time. It was interesting. I completely agree with your not liking fixed column widths. We use a (somewhat off-the-beaten-track) database product that has variable length everything, and for the most part, I can't understand what the point is of doing otherwise.

Greg
Wednesday, April 09, 2008

Deleting …Approving …

Google datastore is fine for simple read/write webs (like blogs, wikis or online shops) but not for websites required to do massive calculations. I appreciate this product for its simplicity to get things started but eventually you will feel constrained by their framework. I still think the best longterm bet is to configure your own stack having virtual hardware supplied on-demand by vendors like Amazon. I wouldn't start anything big on Google and not only because it's not portable.

lubos
Monday, April 14, 2008

Deleting …Approving …

Thanks lubos. The reading I've been doing is giving me the same impression.