Category Archives: Development

241 properties have been deprecated in Hadoop 3.0. Is that enough change for you? Dozens of sites have dedicated space to discovering and explaining all the new goodness in Hadoop 3.0. I haven’t read any that discuss the problems that … Continue reading →

We talk a lot about hiring T shaped people at my current gig and I think it’s a misnomer for a couple of reasons. First, it implies a ratio of depth to width that is askew. Developers and Admins in … Continue reading →

Graph databases are a really neat concept. We’ve started playing with Neo here as we attempt to link customers with visits and actions based on those visits. It seems like a really good fit at first glance. Our challenge is … Continue reading →

The second definition of nudge, according to Webster, is to “prod lightly: urge into action.” We use that concept in our data environments for various long running processes; for things that we want to happen frequently, but with an unknown … Continue reading →

AWS Redshift has been popular lately around my current gig. We’ve got a couple of clusters in use and a few more in POC mode. The in-use clusters are easy to justify pre-paid instances. A few thousand dollars and you … Continue reading →

R is the latest Hadoop darling. It is an open source language that “is widely used among statisticians and data miners for developing statistical software and data analysis. Polls and surveys of data miners are showing R’s popularity has increased … Continue reading →

We are usually left to bash for scripting Hadoop functions. It’s the default in Linux and it’s usually good enough. There are enough “bash-isms” that will cause your Java/pig/database people serious heart ache. If you’re new to Hadoop, go ahead … Continue reading →

An adventure is only an inconvenience rightly considered. An inconvenience is an adventure wrongly considered. -G.K. Chesterton Sometimes our ego gets the best of us. This seems to occur more often in Hadoop than anywhere else I’ve worked. I’m not … Continue reading →

I thought we would start a weekly series on some lessons we’ve learned. Many of the topics we’ve learned the hard way so we thought it might be helpful for those a few steps behind us. YMMV, but we wish … Continue reading →