SPARKSEE High Scalability

by Sparsity Technologies

Architecture

Design

SPARKSEEHA has been thought to minimize developers' work to go from a single node installation to a multiple node HA-enabled installation. In fact, it does not require any change in the user application because it is simply a question of configuration.

To achieve this, several SPARKSEE slave databases work as replicas of a single SPARKSEE master database, as seen in the figure below. Thus, read operations can be performed locally on each node and write operations are replicated and synchronized through the master.

Figure 1.1: SPARKSEEHA Architecture

Figure 1.1 shows all components in a basic SPARKSEEHA installation:

SPARKSEE master

This is responsible for receiving write requests from a slave and redirecting them to the other slave instances. At the same time the master itself also plays the role of a slave.

Only a single node of the cluster can be configured to be the master. The election of the master is automatically done by the coordinator service when the system starts.

The master is in charge of the synchronization of write operations with the slaves. To do this task it manages a history log where all writes are serialized. The size of this log is limited, and it can be configured by the user.

SPARKSEE slave

Slaves are exact replicas of the master database; they can therefore locally perform read operations on their own without requiring synchronization.

However, for write operations the synchronization with the master in order to preserve data consistency is a must. These writes are eventually propagated from the master to other slaves. Therefore, the result of a write operation is not immediately visible in all slaves. These synchronizations are made by default during the next write operation; however there is an optional polling to force synchronization that can be configured by the user.

It is not mandatory to have a slave in the architecture, as the master can work as a standalone.

Coordinator service: Apache ZooKeeper

A ZooKeeper cluster is required to perform the coordination tasks, such as the election of the master when the system starts.

The size of the ZooKeeper cluster depends on the number of SPARKSEE instances. In every case, the size of the ZooKeeper cluster must be an odd number.

SPARKSEE v4.7 works with Apache ZooKeeper v3.4. All our tests have been performed using v3.4.3.

User application

As SPARKSEE is an embedded graph database, a user application is required for each instance. As it has already been mentioned, moving to SPARKSEEHA mode does not require any update in the user application.

Note that the user application can be developed for all the platforms and languages supported by SPARKSEE. For the current version will be running on Windows, Linux or MacOSX and using Java, .NET or C++

Load balancer

The load balancer redirects the requests to each of the running applications (instances).

The load balancer is not part of the SPARKSEE technology, therefore it must be provided by the user.

In order to achieve the horizontal scalability, this redistribution of the application requests must be done efficiently. A round-robin approach would be a good starting solution but depending on the application requirements smarter solutions may be required. In fact, using existing third-party solutions is advisable.

More information about load balancing strategies & available solutions in this article.

How it works

Now that the pieces of the architecture are clear, let's see how SPARKSEEHA works in different scenarios or acts in typical operations using these components. Below is an explanation of how the system acts in the described situations.

Master election

The first time a SPARKSEE instance goes up, it registers itself into the coordinator service. The first instance registered which becomes the master. If a master already exists, it becomes a slave.

Reads

As all SPARKSEE slave databases are replicas of the SPARKSEE master database, slaves can answer read operations by performing the operation locally. They do not need to synchronize with the master.

Writes

In order to preserve data consistency, write operations require slaves to be synchronized with the master. A write operation is as follows:

A slave wishes to perform a write operation and sends it to the master.

The master serializes the operation in the history log, performs the write, and replies to the slave when it has been successfully achieved.

From the master the slave receives a fully updated list of write operations, which are extracted from the history log, and records them in addition to its original write. This operation preserves the consistency of the database.

If two slaves perform a write operation on the same object at the same time, it may result in a lost update in the same way as may happen in a SPARKSEE single instance installation if two different sessions want to write the same object at the same time.

Slave goes down

A failure in a slave during a regular situation does not affect the rest of the system. However if it goes down in the middle of a write operation the behavior of the rest of the system will depend on the use of transactions:

If we are in an auto-committed mode (the user does not explicitly start/end transactions), the system remains operational when the slave fails.

If the write operation is enclosed within a transaction, the master will not be able to rollback the operation and will therefore be blocked for any more write operations. This behavior can be explained by the fact that the master keeps waiting to finish the transaction, which will never be received as the slave has crashed.

Slave goes up

When a SPARKSEE instance goes up, it registers itself with the coordinator. The instance will become a slave if there is already a master in the cluster.

If polling is enabled for the slave, it will immediately synchronize with the master to receive all pending writes. On the other hand, if polling is disabled, the slave will synchronize when a write is requested (as explained previously).

Future work

This is a first version of SPARKSEEHA, so although it is fully operational some important functionality is not available which will assure a complete high-availability of the system. Subsequent versions will focus on the following features:

Master goes down

A failure in the master leaves the system non-operational. In future versions this scenario will be correctly handled automatically converting one of the slaves into a master.

Fault tolerance

A failure during the synchronization of a write operation between a master and a slave leaves the system non-operational. For instance, a slave could fail during the performance of a write operation enclosed in a transaction, or there could be a general network error.

This scenario requires that the master should be able to abort (rollback) a transaction. As SPARKSEE does not offer that functionality, these scenarios cannot currently be solved. SPARKSEEHA will be able to react when SPARKSEE implements the required functionality.