Terror plot spy project helps Kinetica crank up database speed

Ingest billions of records per minute with no indigestion

HPC blog Kinetica presented its in-memory database solution at the recent OpenPOWER European Summit in Barcelona, and served up some benchmark results. You can see the entire presentation in the video below, but here are a few highlights.

First, for those of you unfamiliar with Kinetica, the company came out of a US military research project that aimed to identify terrorist types before they could strike. The problem was that it could take 90 minutes or more to comb through a mass of data and provide actionable intelligence on a specific target individual.

The founder of what eventually became Kinetica solved this problem by rewriting the database to run in memory and to use the latest accelerators. The end result is a blazingly fast database that can deliver intelligence in near real time.

While the solution runs on x86, it runs particularly well on the OpenPOWER platform, due to its new NVLink GPU interface. So how fast are we talking? Here are a couple of examples:

On a BoundingBox for one billion records, Kinetica was 561 times faster than MemSQL & NoSQL

On a Min/Max with one billion records, Kinetica was 712x faster than MemSQL & NoSQL

For SQL queries using 30 Kinetica OpenPOWER nodes versus 100+ x86 nodes running SAP HANA with 150 billion rows, Kinetica turned in near sub-second results compared with 13 seconds for HANA on a SELECT, 45 seconds for a HANA JOIN, and 35 seconds for a GROUP BY query. That’s pretty sporty.