Concentrate: Big Data Starts with Focus

These kinds of talks are always interesting, for one reason or another. Whether it is because the presenter is introducing something useful, or because the talk generates some buzz afterwards, there is always something to discuss when they are over. And recently, talks on Big Data have been appearing on TED and TED-X channels, along with business-focused YouTube channels and other business and tech themed venues. Many of them appear with the express purpose of extolling the virtues of Big Data.

And I have to say, this video is very similar in that regard, insofar as it is focused on Big Data and its application to something like health care. It is only a ten minute video, but Kaskade presents some interesting snippets of information, the most prominent being that Big Data is useless without focus.

He is not the first person to bring this up. This article presents an almost scathing review of current industry-wide confusion regarding Big Data, especially as it relates to things like user-reported information:

With what you have read so far, by using hundreds or thousands of data points (big data), we should be able to predict the habits and decisions of one or many people. However, people are actually unpredictable. Sometimes people even lie! In fact according to Grant, the biggest lie people make is about their income or weight.

Another article here talks about the failure of Big Data in the business space, and the author echoes Kaskade’s earlier comment about the high failure rate of Big Data projects:

Given the wide variety of new and complex solutions, however, it’s no surprise that a recent survey of IT professionals showed that more than 55% of Big Data projects fail to achieve their goals. The most significant challenge cited was a lack of understanding of and the ability to pilot the range of technologies on the market.

On some level, none of this is news. When new idea like Big Data is presented to the public, it quickly takes on a mind of its own, depending on who is doing the thinking. We all know people love to throw around words like “disruptive” and “paradigm-shifting,” and there is no shortage of industry gurus prophesying that the Next Big Thing is going Revolutionize The Way We Do XYZ. I would imagine that in reality, these kinds of transformations occur on a much slower time scale — evolution, not revolution.

But the reason this is all so interesting right now is because we get to see the emergence of Big Data as a discipline, as a new way of looking at information, and not just as some business gimmick. If Kaskade is right, Big Data may eventually be able to help folks with cancer, if only because it can place valuable information into the hands of people that need it, and fast.

In this arena, Kaskade’s exhortations to focus, to build the application first, seems to strike a nerve. The reason that half of the Big Data initiatives are failing is probably covered under a multitude of reasons, but chief among them has to be a lack of focus, and also a lack of understanding. This may be inevitable in some cases, as people are more in love with buzzwords than ever before. It is easy to catch Big Data fever, especially if you are never required to explain what Big Data is or why it is important. But that doesn’t meant the core of the project is doomed; indeed, these may simply be growing pains.