Wednesday, March 13, 2013

Big Data goes to the Big Apple

Next week is the NYC* Big Data Tech Day. It looks even bigger and more badass than last year's Cassandra Summit. There is a good blend of use cases from people that already have Cassandra in production as well as cutting edge development that hasn't yet gone mainstream.

I'm really looking forward to John McCann's talk. He is going to present on Comcast's use of Cassandra as the backend of their DVR system. Netflix has been fairly open about their use, but guiding a ship as big as Comcast into the new era of NoSQL is a feat of shepherd-ship I'd love to hear about.

Likewise, on the more cutting-edge side of things, I'm looking forward to Thomas Pinkney's talk on graph-databases. That may be the next ingredient in our architecture. We were targeting Neo4j, but if a Cassandra-based graph database is mature enough we would love to use it. That would allow us to centralize on a single storage mechanism that scales. (FTW!)

Finally, Ed Capriolo's talk promises to be a good one. As the veteran's know, once you've got your CRUD operations down. There is a whole world of potential out there in data processing. On our second try, we decided to go with Storm for our data processing layer, but I believe Ed has an innovative perspective on things. (Hopefully, we'll see mention of intravert-ug, which is what happens when smart people like Nate McCall, Ed Anuff and Ed Capriolo have a baby.)

Also, I should mention that Taylor Goetz and I will be presenting on our Big Data journey, which has culminated in a Big Data platform that we are extremely happy with, where we've combined Storm, Kafka, Elastic Search and Cassandra into a slick/fast/scalable/flexible data processing machine.

I believe there is still room if you want to sign up. If it is anything like last year, not only will the talks be informative, but the collaboration sessions before, in-between and after are worth their weight in gold.