The Hardest Part about Big Data

"Big Data" is a term that has been tossed around rather loosely over the past year or so. There are many moving parts and various strategies in the space. There are a few leaders in data management strategy but there are many more that are lagging behind. As Jer Thorp notes in his blog post, "Big Data" is not a resource that has been hiding deep underground somewhere and we just happened to stumble across it in the recent years. Data is created every millisecond, second, minute, hour, dayâ¦ you get the point. And we're struggling to keep up with it all. Most of it is even completely unrelated to our actual immediate goals! How can we even imagine utilizing this data for forecasting and predictive analytics if we can't even handle the data already created?

Even though there is literally tons of data as there is out in the world, data management can still be an entirely unique process. When we talk about the basics of data usage strategy, we need to talk about WHY. One can spend their entire career designing an epic data management tool, complete with master data management, cloud computing, predicting analytics, visualization, the whole kit and caboodle. But it won't matter if there is no potential for application. Sure, there are data scientists out there who simply love collecting data, running analytics and visualizing the results. For B2C retailers, that's just not enough.

For a huge corporation like Wal-Mart or Amazon, data analytics can be used for targeted marketing, inventory management, etc. For smaller to midsize retailers, the driving focus behind data analytics has been better understanding the overall customer experience. This is how each of the overwhelming number of e-tailers differentiates themselves from the online crowd. Take a company like Zappos, who, before acquired by Amazon in 2009, prided themselves on an optimal customer experience for all, no matter the cost (they still do, mind you, it's just a little more difficult after growing at such an alarming rate). In order to do this, they used sentiment analysis on their customers (and non-customers) and adjusted their business practices accordingly. To learn more about Zappos' strategy, check out this video of Alex Soria, Supervisor of Statistical Analytics at Zappos, speaking about the company's customer service approach using analytics. For even more information, you can attend Big Data World Canada, where Patrick Martin will be speaking in detail about data management strategy at Zappos.

The bottom line: there are SO MANY different uses for data. Data is such a general term to describe something so expansive that we have no way of really understanding just yet. Just like you wouldn't let your packed attic alone until you figured out the unknown way to purge and organize all your excessive belongings, we need to start hacking away at the data now. To start – you have to understand what your unique needs are and you can attack the data from there. If you want to manage your inventory more effectively, you'll need more data on orders, including frequency, size, product, delivery location, warehouse location, etc. For increased order conversion, data will consist of website hits, clicks, click trails, e-basket items, conversation rates, etc. I think you understand my point. Of course, we can use data across all business practices, but not all at once. That is exactly how tech teams, marketing teams and corporate governance become overwhelmed and frustrated when they don't see results. The hardest part about Big Data is to truly understand what your priorities are within your business and choosing data to best inform those decisions. Then, maybe we can start talking all that other dataâ¦