SPARK (Application)

Introduction

Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala and Python, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine learning, GraphX for graph processing, and Spark Streaming.

Documentation

Mailing List

To sign up for email notices about pending version updates, removals and other
important announcements for this software package,
sign in.

Announcements

Mar 6, 2018: Redfin has been shutdown to be reinstalled with the Compute Canada software stack, therefore the sharcnet spark modules are no longer available on it. To compensate, the new sharcnet spark 2.2.1 and 2.3.0 modules have been installed on wobbie which has four 128GB memory 24core nodes and two 512GB memory 24core nodes.