The Secret to Big Data Answers: Ask Better Questions

With all of the data flooding across the Internet these days, you’d think it would be easy to get useful insights for your business. It’s not; what’s usually missing in the search for big data answers are the right questions that’ll yield them.

The key is to develop a solid business case, and here are 5 areas you’ll want to address:

First, and perhaps most importantly, when starting your big data analytics project you must have a clear idea about what you want to explore. “Improve customer retention” and “cut supply chain costs” just don’t qualify, because they’re so broad. Instead, take time to distill those goals into the constituent parts, or influences, that impact them. Big data insights emerge from comparing and contrasting data, so maybe you want to explore the relationship between pricing and customer purchases, or supplier deliveries and weather (for example). The more time you spend on this first step, the better your answers will be at the end.

Second, inform your ideas with a scope for the project. This step defines what data you want to look at (and can look at, in terms of availability, usability, and reliability), and therefore limits, or focuses, your inquiry to what’s achievable. The old adage “garbage in/garbage out” applies here; not only do you want to understand your scope of data, but also the organizational components of analyzing and applying your results. A data science expert can help you on the former, but the latter is a business-level activity. History is rich with stories of companies that “knew” something had to be done, but couldn’t muster the organizational wherewithal to do it. Ask questions about who’ll be vital to operationalizing what you might learn from your project…and how you’ll gain their involvement and support.

Third, your business case should include a clear section about the format of the output you’re looking for. Are you hoping to build some real-time indicators for the movement or interactions of certain types of data, or a one-time static benchmark comparison? Do nuanced details matter more than starkly obvious relationships? What about directional insights, such as giving you or your team some forward-looking indicators based on past/present data? It might be helpful to literally ask yourself the question, “I want a user of the project’s output to be able to BLANK,” and then use your answer for “blank” to drive how you envision the format(s) of your project’s output. When you’re talking to a data scientist, the answers to these questions can form the basis of use cases, which help them configure the guts of your project.

Fourth, the issue of how you’ll manage the project, or governance, is a tricky one, because it involves not only how it’ll be run, but also how its output will impact other projects/departments in your organization. You’ll want to question who’ll be empowered to make management decisions for it (and who needs to be involved, from approvals to need-to-know), and what authority they’ll have over sources of data; for instance, will she or he have the ability to insist on quicker response time from other employees, not to mention decisions about budget? Is the project going to be run like any other project in the company and, if so, will it be hampered by lack of speed, flexibility, etc. (a big data project isn’t the same process as building a widget)? How will conflicts be resolved so, for instance, if the project’s output suggests a change in another group’s activities, how is the go/no-go decision going to be made? The last thing you want hampering your implementation of your big data project’s answers is running up against a lot of unanswered questions about how.

Fifth, ask questions about what you expect from your big data vendor. If you’ve properly posed and at least tried to answer the questions raised in this blog post, you’ll be far more likely to choose the right vendor…but it’s still going to be tough, considering the variety of firms that claim to possess the skills to do generic big data projects (ranging from marketing firms to analytics agencies, and many independent contractors between). The core question you want to answer before you solicit proposals is to clearly delineate what you want them to do vs. what you’ve done/will do internally. There are going to be many areas in which you’ll want to rely on their expertise, such as constructing the queries, managing the data, and implementing the required privacy and security conventions. But your questions about what you want to accomplish, what you have to work with, how you’ll constrain the project so it has focus, and the ways you want to implement it will liberate any vendor you choose to give you the answers you truly need.

The success of your big data project will be directly dependent on your success at asking and, when you can, answering lots of questions.

For more insight, check out our three new lessons onBig Data, where we examine its origins and the technologies that underpin and enable big data projects, as well as exploring how different departments, such as HR, operations, accounting and marketing are already employing big data to their advantage. Check out a clip from the lesson below.