Install Hadoop on the Orange Pi

Install Java on Orange Pi

Let’s put the yellow elephant on the Orange Pi! Hadoop is a framework for distributed data storage and processing used in big data applications. We will set up a single node Hadoop cluster on the Orange Pi. This may or may not be practical in a production environment, but it’s a good way to learn to configure Hadoop, load a file on the hdfs and run a MapReduce job.

The first thing to do is to install Oracle Java. That’s because the version of Open Java Development Kit that the Orange Pi comes with is not compatible with Hadoop. Also, Oracle Java will generally run faster than Open JDK on the Orange Pi. At first I upgraded the OpenJDK to the suitable version, then I installed Oracle Java and I noticed a significant increase in performance.

To install Java, visit the Oracle JDK download page. From here select the Linux ARM 32 version, which is the one to use for the Orange Pi Plus 2e. Following the steps from here, unpack the archive:

sudo tar zxvf jdk-8u101-linux-arm32-vfp-hflt.tar.gz -C /opt

Next, run the following commands and select the newly installed Java version:

Now lets put a file on the distributed file system. I chose to upload the small text file in this Raspberry Pi Hadoop cluster post, so I can compare the execution times. Put the file in you home directory. Then:

cd
hdfs dfs -copyFromLocal smallfile.txt /smallfile.txt

This will copy the file to the hdfs. Let’s run the classic word count example on this file:

You can check the status of your Hadoop jobs at http://localhost:8088/cluster

The execution time (mm:ss) for the small text file was 01:25, which is an improvement from the 2:17 on one Raspberry Pi 1 and even from the 1:41 it took on a cluster of 3 Raspberry Pi’s 2, as seen here. Furthermore when running the word count on the Gutenberg books text file from ‘Because we can geek’, it took about 02:03 as compared to 03:25. Overall the increase in performance makes sense, because the higher rated storage technology, RAM and CPU of the Orange Pi Plus 2e.