Jan Amoyo

When updating documents, MongoDB has a useful feature to insert a new document when no document matches the query criteria. This feature is called an upsert. Sadly, as of this writing, DynamoDB misses on this feature out of the box.

Thankfully, there's a way to achieve this. The idea is to do it in 3 steps: (1) Get the previous copy of the item. (2) If a previous copy exists, update it. (3) If it does not exist, insert the item ensuring that concurrent requests do not overwrite each other. Here's a snippet written for Node.js:

The trick is to declare a Condition Expression in the put step to ensure that an item only gets inserted if a previous copy does not exist (Line 46). This ensures that when handling concurrent put requests, only the 1st request succeeds and the others fail with a ConditionalCheckFailedException error. We then check for this error type to determine if any of the failed requests should be retried as update requests.

Below is a simple JavaScript utility for publishing messages across browser windows of the same domain. This implementation uses the browser's localStorage and the storage event to simulate the behavior of an inter-window topic.

Lines 5 and 6 adds a source and timestamp property to the message to ensure uniqueness
Lines 7 and 8 converts the message to JSON and saves it to the localStorage
Lines 12 and 13 uses the event.key to filter which message should be processed by the callback
Line 14 converts the JSON value to a message objects and passes it as an argument to the callback
Lines 18-20 returns a function that when called, removes the subscriber from the topic

This works because every time an item is stored in the localStorage, all browser windows sharing the same localStorage will receive a storage event detailing what has changed (except for the window that wrote to the localStorage).

However, the problem with the above implementation is that it doesn't scale if the number of subscribers increases. Every time a storage event is fired, the JavaScript engine will have to iterate through each listener regardless whether the listener is interested in the event or not.

To address this problem, we can use a map to index the callbacks against the event.key. Below is the updated version of the above code:

Lines 5-14 registers a single storage event listener that uses a map to look-up callbacks identified by the event.key
Lines 22-27 saves the callback into the listeners map, identified by the derived key

Line 5 compiles the expression passed to the directive.
Line 6 listens for scroll events on the directive element (requires jQuery).
Line 10 checks if the scrollbar has reached the bottom.
Line 12 applies the compiled expression to the scope.

Lines 4 and 5 loads the HTML templates into Karma. Because they will be pre-processed by karma-ng-html2js-preprocessor, they will eventually get loaded as JavaScript files. Note that even if Angular will be included as part of the Browserify bundle, it is important to load it explicitly. Otherwise, the templates cannot be made available to Angular.

Line 10 tells karma-browserifast to include all unit tests as a Browserify bundle. This makes it possible for unit tests to execute JavaScript modules using the "require" keyword.

Line 21 tells karma-ng-html2js-preprocessor to load the JavaScript templates as an Angular module named "karma.templates". This will later be used in unit tests to allow testing of directives that use templates.

Unlike traditional databases, MongoDB does not support transactions. So suppose you have multiple documents and want to perform a "save all or nothing" operation, you'd have to simulate a transaction from within your application.

Implementing such a transaction in NodeJS can be tricky because IO operations are generally asynchronous. This can lead to nested callbacks as demonstrated by the code below:

Despite not having a dependency on other documents, inserting each document must be performed in series. This is because we have no way of finding out when all other documents will finish saving. This approach is problematic because it leads to deeper nesting as the number of documents increase. Also, in the above example, the callback functions modify the docs variable - this is a side effect and breaks functional principles. With the help of Async, both issues can be addressed.

The parallel() function of Async allows you run multiple functions in parallel. Each function signals completion by invoking a callback to which either the result of the operation or an error is passed. Once all functions are completed, an optional callback function is invoked to handle the results or errors.

The above code can be improved by implementing document insertions as functions of parallel(). If an insert succeeds, the inserted document will be passed to the callback; otherwise, the error will be passed (these can later be used when a rollback is required). Once all the parallel functions complete, the parallel callback can perform a rollback if any of the functions failed to save.

Lines 8, 11, and 14 inserts a new document to MongoDB then passes either the saved document or an error to the callback.

Lines 18-21 checks if any of the parallel functions threw an error, and calls the rollback() function on each document passed to callback of parallel().

Lines 28-33 performs the rollback by deleting the inserted documents using their IDs.

Edit: As pointed-out in the comments, the rollback itself can fail. In such scenarios, it is best to retry the rollback until it succeeds. Therefore, the rollback logic must be an idempotent operation.