Rumblings by a Java guy on Java, Clojure, Scala and Groovy

ScalaTest a MapReduce using Akka

I was trying to learn Scala and I wanted to kill several birds in one shot. Let me tell you, I am not disappointed, I feel comfortable working with Scala. If you are coming from Java world, Scala is comparatively more complex, but once you get past initial hurdle you will like it. I wanted to learn

In this example, I am taking advantage of Akka‘s Actor supports for breaking chunk of tasks and processing in parallel and aggregate the final results.

For a starter, SBT is a build tool similar to Maven, extensively used for Scala development. Refer project/plugins.sbt, this has integration with Eclipse IDE. Once you get the code from github, run the below command, you notice 2 files .project and .classpath got generated.

sbt eclipse

Now import the project in Eclipse as Import => “Existing project into workspace”. Once the project is imported into Eclipse, we can take advantage of IntelliSense and other IDE features and develop the application in a easy way compared to writing the scala code in TextPad.

As always, I will start writing a test, the lowest level test is the aggregation test, which takes a map of words and the number of times it has occurred and aggregates it. I used WordSpec for this ScalaTest as below,

Write a Map unit test to take a line and create a Result object. If you notice carefully, the map and reduce object implements Akka’s Roundrobin Routers where the line is processed by multiple threads in a roundrobbin way.