Open Source Replication and Offline Sync for Node.js and Devices

Today, StrongLoop is excited to announce support for replication and offline synchronization in our LoopBack API framework. What makes this unique is that it is completely open source and powered by Node.js. Loopback is an open source API framework written in Node.js used to connect mobile devices to enterprise datasources. This new functionality is available for Oracle, SQL Server, MongoDB, PostgreSQL, MySQL and any LoopBack connector which supports create, read, update and delete (CRUD) operations.

Enterprise mobile apps require offline sync capabilities

The ability to work offline has emerged as a requirement for almost all enterprise mobile applications that are data-driven. Up until now, developers first had to figure out how to locally store a subset of the application’s data. Second, they had to implement a mechanism that could keep the data synchronized on both the client and server. The previous generation of synchronization and replication technologies that tried to address these challenges were low level and inflexible, with little choice or variety in the data sources they supported. Fine grained controls on the behavior for common use cases like change detection and conflict resolution were also lacking.

Replication and offline support in Loopback

What this means for developers creating isomorphic JavaScript apps is, that for the first time they’ll be able to easily synchronize to and from various databases without requiring constant network connectivity. LoopBack’s replication also handles the complexity of moving data between mobile devices, mobile device to server and server to server. This means developers can focus on the the front end versus the mechanics of how to replicate data between disparate databases whether they be in cloud or in the datacenter.

Why replication?

Replication allows you to easily sync data in various databases from Node.js or the browser. This takes care of moving data between clients and servers and lets you focus on what makes your app useful. Below are some use cases that I find particularly interesting:

In the browser, attach your model to a local storage database and sync with the server when you have connectivity.

In a turn based multiplayer game sync game objects (models) with an in memory database on the server and client.

In an enterprise application, sync your Oracle and Mongo databases, allowing you to use features from Mongo while using Oracle for persistence.

Replicate data from a slower database into Redis for better read performance.

Check out this video with Al Tsang – StrongLoop CTO, for a complete walk through of how isomorphic JavaScript and LoopBack’s replication, offline sync and conflict resolution work.

A simple replicaton example

Below is a user story that describes a simple todo list application.

As a busy person on the go, I should be able to read and write to my todo list. I need access to my todo list on my phone while I walk to the office. While at work, on my laptop, I should be able to view the tasks I’ve entered on my phone and vice-versa.

This app represents the typical “sync” problem. You have data on several client devices that must be kept in-sync. Below is a basic example of LoopBack in the browser that replicates a todo list to and from a LoopBack server.

// --- browser.js ---// create the client browser appvarclient=loopback();varmemory=loopback.createDataSource({// a simple in-memory databaseconnector:loopback.Memory});// create a LoopBack server datasource// allowing us to call methods on the loopback servervarremote=loopback.createDataSource({url:'http://localhost:3000/api',connector:loopback.Remote});// use the same model on the server and the clientvarRemoteTodo=require('./todo');// create a local model to keep data on the devicevarLocalTodo=RemoteTodo.extend('LocalTodo');// attach the models to the correct data sourceRemoteTodo.attachTo(remote);LocalTodo.attachTo(memory);// sync the data every 2 secondssetInterval(function(){LocalTodo.replicate(RemoteTodo);RemoteTodo.replicate(LocalTodo);},2000);

Below is the server side of the basic todo application. It attaches the server’s Todo model to both MongoDB and the LoopBack REST API. You could swap MongoDB out for MySQL, Postgres, Oracle, or any other database that LoopBack supports. To learn more about how to work with models in LoopBack, check out the documentation.