Optimistic Locking with MongoDB

At Highgroove, we build database-backed web applications. These days, there are many options when choosing a database backend. We normally start with relational database systems (e.g., PostgreSQL and MySQL) because they are very mature and feature niceties like ACID transactions and locks.

On the other hand, we also work with newer NoSQL database systems that often trade features like transactions, locks, and joins for higher performance and scalability. One popular option is a document-based store called MongoDB. By design, MongoDB does not support ACID transactions (though many operations are atomic) or traditional locks.

In many cases MongoDB, paired with an object-relational mapper like mongo_mapper is a great solution for Ruby and Rails applications. And after the break I explore a solution that allows a developer to “lock” a MongoDB document while still maintaining the high performance and other features MongoDB is known for.

mongo_mapper loads documents from MongoDB into memory as a Ruby object. Typically, a program then changes a few values and saves the record. Notably, saving the record actually resets every field in the document to the values in the Ruby object.

Unfortunately, data and work can be lost if multiple processes (e.g., different web server processes/threads or background workers) are interacting with the same documents at the same time.

For instance, in the code below if two processes execute post = BlogPost.find("4e13cda850b86112c9000001") at roughly the same time, one will end up overriding the other because the values in memory will be stale.

====

This problem is not unique to MongoDB or mongo_mapper, but the potential solutions are different from relational databases because MongoDB documents cannot be exclusively locked.

One way to solve the problem is to only interact with MongoDB through its atomic operations, but many of the conveniences of an object-relational mapper are lost for a problem that may only occur sporadically.

Another way is to use optimistic locking, a technique that reliably determines if a record has been updated by another process between the time it was initially loaded into memory and the time a save is attempted. If the record has been modified by another process, an error is raised and the object must be reloaded. If objects are not in high contention and retrying the operations is easily accomplished, optimistic locking is a good solution that keeps performance high in the average case.

If using optimistic locking with a background job, a reasonable response to a stale document error would be to reload the record, retry the operation, and attempt a save again. On the other hand, if two end users are in contention, the stale document could be presented to one of the users for manual conflict resolution.

We have written a small gem that implements optimistic locking with mongo_mapper called mm-optimistic_locking. It is very straightforward to use and we think it does its job well, but we definitely welcome feedback.