Spark meets HAL: Apache's cluster master goes deep

IBM and co: Data for the masses – up the workers! As a phrase, “democratisation of data” is rather glib – but it does have a serious purpose. The thinking: making use of company data should not be the preserve of just “professionals”. This was certainly a theme at the recent European Spark summit in Brussels. Spark, the open-source framework used for building clusters run by the Apache Foundation, is finding its way into a variety of organisations – from startups to major enterprises – and is very much associated with data analysis and predictive applications. Spark dates from 2009, originating among researchers at the University of California, but arguably hit the big time when IBM flung its corporate weight behind the open-source cluster framework last year. IBM committed 3,500…