Strata + Hadoop World 2013: My Perspective

November 7, 2013

Nitin Motgi is Founder and CTO of Cask, where he is responsible for developing the company's long-term technology, driving company engineering initiatives and collaboration. Prior to Cask, Nitin was at Yahoo! working on a large-scale content optimization system externally known as C.O.R.E.

Please note: Continuuity is now known as Cask, and Continuuity Reactor is now known as the Cask Data Application Platform (CDAP).

“Tech Geeks. Your chariot awaits: SFO->NYC”. As I drove past this billboard on the 101, I felt very excited. The entire team had been working very hard for the past 2 months. This was the moment of truth. We were going to announce thegeneral availability of Continuuity Reactor 2.0 and our strategic relationship with Rackspace. The team would get to meet actual customers and developers, and see Reactor 2.0 in action in the real world.

How has Hadoop World evolved?

I distinctly remember my first Hadoop World in 2011. There were very few sessions and exhibitors, and less than a thousand attendees. This year, there were eight session tracks and more than 2000 attendees. The themes have evolved from research orientation to business orientation. The topics ranged from waste management and machine learning to retrospective analysis and case studies. It felt great to see the community evolving from solving hard technology problems to solving real business problems.

News from Hadoop World

There were a lot of interesting announcements at Strata + Hadoop World 2013. My favorite was the general availability of YARN (a.k.a. MR-279, NextGen-MR). Lot of effort and hard work from many people in the Hadoop community to make YARN GA – special kudos to Vinod K and Arun Murthy for their leadership. From the Continuuity perspective, this was extremely important as Reactor 2.0 is built completely on YARN. All of the Reactor runtime subsystems over Hadoop run within YARN containers supported by our open source project, Weave.

News from Continuuity

As I mentioned earlier, Continuuity came out of public beta and released Continuuity Reactor 2.0. Now, you can build and deploy production Big Data apps using our platform. We added a collection of new features and developers can now run an entire end-to-end application in the Reactor with an easy-to-use, integrated web server. I will leave you the gory details as you can read all about it in our earlier blog post, or at GigaOM, Forbes and Wired.

Celebration

Our motto is “work hard, play harder”. Hmm! Shall we reverse it? We didn’t leave anyone behind – most of our team traveled to New York City. We all worked hard, so we deserved some downtime. TheIrish Pub was our favorite hang out place as we watched Red Sox (yay! – this is for Vikram) win the World Series. We also hosted a party for our customers, partners and collaborators on Wed night. The discussions ranged from YARN, Big Data applications to the most delicious Ramen place in NYC. We had so much fun.

I am thrilled about the future. This is just the beginning. Lot of good things to come. I can’t wait for Strata + Hadoop World 2014. Till then, always be shipping!

P.S. Don’t get left behind, download our SDK and let us know how we can make it better.