Mapreduce

At Vidya we currently offer two courses, Software Engineering in Java and Agile Software Project Management with Scrum. In response to popular demand…OK, like eight or nine people…we are currently working on a third course to be ready by Summer 2015 tentatively called Analytics with Apache Spark.
As “Big Data” becomes more and more of a thing, there just aren’t enough software engineers who know the tools and techniques for doing meaningful, performant, cloud-scale analytics.

Please take a look at my latest column for Government Computing News where I gently introduce you to the basics of Hadoop and describe cool technologies that build on it. Just to give you an idea, here is the unedited introduction.
It’s a funny word. You have only a vague notion of what it is. You’ve heard that it takes a lot of work but is potentially really beneficial. Maybe if you learned more about it you too could enjoy its benefits.

There is an episode of Family Guy where Peter Griffin gets a segment on the Quahog 5 News called “What Really Grinds My Gears.” During the segment, he opines on random things that bother him in his own hilarious and offensive way.
When it comes to software projects and technology, I have my own list of pet peeves that continues to grow. While not quite as hilarious though potentially just as offensive to some, here are a few.

Although most developers and users are still feeling their way through Hadoop and (more specifically MapReduce), the truth is Google wrote that paper in 2004. That’s ten years ago! Million Dollar Baby won Best Picture that year. Yeah! by Usher, Lil John and Ludacris topped the charts in the United States. And Facebook had only just started to kill work productivity and violate your privacy.
As long ago as that feels, it is an eternity in technology.

Let me first say I love Java. There is a reason it’s the most popular programming language in the world. For me personally, I made a career out of building systems in Java, and I even teach a course in Java.
But when it comes to Big Data, Java simply doesn’t cut it.
Everybody knows functional languages have enjoyed a renaissance as Big Data has become a thing. And for good reason: immutable (or mostly immutable) state, lazy evaluation, the natural fit with recursion, and so on.