In the Apache Foundation’s own words, Spark is called “A fast and general engine for large scale data processing.” Parsing this we get: Fast – Spark relies on an in-memory processing model which drastically reduces IO operations, and can be up to 100x faster than Hadoop on similar tasks General Purpose – Batch processing, interactive […]