Month: May 2015

For some reason when I’m trying to use the Spark from CDH it doesn’t work with PredictionIO 0.9.3,
So I use spark 1.3.1 binary with hadoop 2.6 support and I extracted mine to: SPARK_HOME=$PIO_HOME/vendors/spark-1.3.1-bin-hadoop2.6

From CDH part I only use the HBase part as the event server storage.

I use Elasticsearch as metadata storage.

I use LocalFS as model storage.

I installed spark standalone server manually (not from cdh) (spark 1.3.1 with hadoop 2.6 support)
– For this test case I’m using a spark master with 4 workers node and let say I installed at spark://my.remote.sparkhost:7077
– If you don’t know how to install a stand alone spark server, please read the spark manual.