Spark Boosts IBM Event Store

Posted by, jgp on 2017/06/22

A simple store, selling purses and artefacts… Well, it could be Events

IBM just announced Event Store, a hybrid datastore to store events. The originality? Events can be streamed in and it is based on Apache Spark. IBM claims to be able to store 1m events per second per node.

As it is based on Spark, it easily integrates with Java, Scala, Kafka, JDBC (Java DataBase Connectivity), IBM Streams, and more on the development/integration front. It saves its data in Parquet in S3, HDFS (Hadoop Distributed File System), or object storage like Cleversafe (now an IBM company), GlusterFS (hopefully my friends at Scality are in the loop as well).

IBM Event Store Architecture | Source: IBM

Under the Hood

Its design allows ingestion and analytics at the same time. IBM Event Store is also thought with HA (High-Availability) in mind, so an engine can disappear, but the cluster stays. This is very certainly achieved by leveraging Spark’s cluster functionalities.

IBM Event Store, understanding the components | Source: IBM

Watch Event Store in Action

IBM just released a video on YouTube, in less than 4 minutes, get a good sense of what it does. Thanks Phil for sharing.