IBM: We have placed the Big Data moon on this stick

IBM Research has come up with a new package it calls BLU Acceleration, intended to speed up (big) data analysis and reporting of data held in DB2 and Informix databases, amongst others.

The BLU Acceleration package includes these handy features:

Data skipping - an ability to skip data that doesn't need analysing, such as duplicate info

Parallel analysis across different processors

Avoiding data modelling layer creation, which leads to a greater ability to analyse data transparently to the application

Actionable compression in which compressed data can be analysed without having to be decompressed first

IBM claims these features collectively speed analytics and reporting by up to 25 times, saying:

BLU Acceleration extends the capabilities of traditional in-memory systems - which allow data to be loaded into Random Access Memory instead of hard disks for faster performance - by providing in-memory performance even when data sets exceed the size of the memory.

They don't explain how they managed this miracle.

IBM customer Kent Collins, a Database Solutions Architect at the BNSF Railway Company, said BLU Acceleration worked for him:

"Using DB2 with BLU Acceleration … we're performing tasks more quickly than ever before, for example, one of the queries improved over 100-fold, and our storage consumption went down by about 10 times."

There is an ongoing DB2 Technology Preview and if you can get access to this you can probably find out more about the BLU bundle of acceleration techniques. The schematic El Reg has seen talks about 7 big ideas in the preview: