Apache Spark is breaking down the barriers between data scientists and engineers, making machine learning easier and is out growing Hadoop as an open source framework for cloud computing developments, a new report claims.

IBM said it will throw its weight behind Apache Spark, an open source community developing a processing engine for large-scale datasets, putting thousands of internal developers to work on Spark-related projects and contributing its machine learning technology to the code ecosystem.

Databricks, a company started by the founders of Apache Spark, an open source processing engine for large-scale datasets, has secured $33m in funding. The news comes as the firm pushes its first commercial Spark-based offering live.