GemFire XD Compared to Other Data Management Systems

GemFire XD and traditional relational databases

Most J2EE applications use a JDBC-compliant database to store data. GemFire XD has little administrative requirements and no scaling limitations, and offers an alternative to the traditional, centralized database architecture. GemFire XD data can be partitioned and/or replicated in memory across a cluster of machines. In some respects, this is similar to embedding a database like Derby, except that the data can be managed primarily in memory, replicated for high availability, and partitioned for scale. Data can be managed either in an embedded fashion (collocated in the application server JVM) or managed in standalone JVMs.

Unlike traditional databases, not all operations with GemFire XD are constrained by ACID properties. Instead, GemFire XD provides this control to the control to application designers, who can choose ACID constraints based on performance, availability, and consistency requirements. Although data consistency and data integrity are key element of the GemFire XD design, the design assumes access to abundant memory and relaxes some of the transactional properties.

Some popular databases support a richer SQL syntax with custom extensions and query engines that produce highly-optimized query plans for complex SQL statements. GemFire XD supports many common SQL features, but it is primarily optimized for key-based access. This version of GemFire XD supports joins only on colocated data. See Designing GemFire XD Databases.

GemFire XD and Pivotal GemFire

Although it uses Pivotal GemFire components, GemFire XD incorporates a more sophisticated SQL query engine that compiles a query plan into byte code. GemFire XD also has a much more sophisticated cost-based optimizer. The configuration and deployment model for GemFire XD is simpler and is designed to be intuitive to anyone having experience with relational database systems.

GemFire XD is based on standards such as SQL, JDBC, and ODBC making it very straightforward to adopt in existing applications that use relational databases. The product can be used in a large eco-system of other products and frameworks - object relational mapping tools (Hibernate, NHibernate, etc), schema editing and database management tools (SquirrelSQL), DB replication products, CDC (Change data capture from external relational databases), Spring JDBC, and so on. Applications that use the standard SQL syntax supported by GemFire XD can easily migrate to and from other relational databases.

GemFire XD and other in-memory databases

Many in-memory database offerings were designed to keep all data in memory in a single process space. Although many support replication for failover, they are similar to relational databases where all updates are routed through a single primary process. The design of such systems does not support partitioning data sets across a servers, and does not allow the cluster size to grow or shrink at runtime. Databases of this type are best suited when the data set size is small and the concurrent access requirements are modest.

In contrast, GemFire XD data and processing can be distributed to as many servers as are required to handle the volume and load at a given time.

GemFire XD and other cloud or NoSQL databases

NoSQL and cloud databases provide scalability and high availability properties for Web applications, but they typically do so at the expense of data consistency. Products such as Amazon's Dynamo use an eventually consistent transaction model, while others allow only a single entry update as part of a transaction.

GemFire XD's design, though driven by horizontal scalability and availability, imposes fewer restrictions. NoSQL databases provide only key-based access, or use a proprietary syntax for queries. Those products that do support queries do not support joins.

The premise behind GemFire XD is to capitalize on the power of SQL as an expressive, flexible, and very well-understood query language. GemFire XD alters only the design underpinnings in common databases to provide scalability and high performance. By providing ubiquitous interfaces like JDBC and ODBC GemFire XD is significantly easier to adopt and integrates into existing ecosystems. GemFire XD is highly optimized for key-based access, but it does not limit applications from doing complex queries with joins, functions, aggregations and grouping.