How to extract big knowledge from big data

By Jay Liebowitz, Orkand Endowed Chair of Management and Technology

Most everyone now has heard one of the hottest terms today—big data. Big data is a big deal, especially in such data-intensive industries as cybersecurity, finance, healthcare, marketing, transportation, energy, and others. And, many of us are already familiar with the 3V’s of big data—volume, velocity, and variety of data. But, the key question is, “How do we extract big knowledge from big data?”

The answer to this question is partly through analytics, which is also a growing field within various sectors. Some people look at data analytics, in terms of educating future “data scientists,” and still others are exploring business analytics through educating a new kind of “business analyst.” The new breed of analytics specialists need to have a combination of skills including statistical techniques, applied mathematical methods, advanced machine learning algorithms, data visualization, and business and communications skills.

The new breed of analytics specialists need to have a combination of skills including statistical techniques, applied mathematical methods, advanced machine learning algorithms, data visualization, and business and communications skills.

Often times, we can find analysts who have the technical savvy, but they lack the business and communications skills to explain to managers and executives how results from analytics can inform organizational decision making. Also, the new breed of analysts must be able to possess some underlying instinct through experiential learning to know whether the analytics results seem reasonable.

Part of the challenge for these new analysts, whether you call them data scientists or business analysts, is having that “gut feeling” for knowing whether the analytics have gone awry or whether they seem to have produced some valuable results. Recent MIT conferences on big data have talked about this value of intuition and making smarter decisions.

The Partnership for Public Service and IBM’s October 2012 joint report titled “From Data to Decisions II” talks about the importance of tapping a mix of people with different backgrounds and strengths. The report highlights the importance of an analytics team approach which takes into account background and experience. This approach should lead to a credible set of analytics results that focus on the goals and objectives for that enterprise. Having a multidisciplinary perspective should provide great value to the analytics team.

Universities and colleges are offering new degrees and programs in analytics. The Master of Science in Analytics program at North Carolina State University has been a model for most of these analytics programs and has recently doubled in size due to its popularity. Other programs include:

The University of Maryland University College (UMUC) online Masters in Analytics (plans are underway to start this summer)

Stevens Institute of Technology Masters in Business Intelligence and Analytics

New York University Masters in Business Analytics

IBM and the College of Business Administration at Ohio State University formed a new Advanced Analytics Center, as of November 2012.

It has already been shown that business analytics can inform decision making. But how can we ensure that the next generation of analysts is prepared to extract the “big knowledge” from the “big data”? Certainly, having some background in knowledge discovery techniques as part of the analytics team may provide some helpful way to look for hidden patterns and relationships in the data and text. But, how do we know that these relationships are worthy of further analysis and exploration? How does “instinct” factor into the results?

It’s a Catch-22: you want analysts with technical and organizational experience, but new analysts may typically have just the technical prowess as they haven’t had enough years in the organizational setting to best understand the context and business setting of the organization. Thus, the team approach with a mix of background and experience will help address this concern in terms of building “instinct” into the analysis.

Other ways to instill instinct into organizational decision making through analytics are:

Provide the analysts with rotational assignments within the organization so that they can better understand the functional components of the enterprise to be well-grounded in what the organization does

Assign senior-junior analyst mentoring so that the younger analysts can learn from the experiences of others

Don’t take an answer for granted—bounce the analytics results off of others in the organization to check the validity and reasonableness of the results.

Dr. Jay Liebowitz is the Orkand Endowed Chair in Management and Technology at University of Maryland University College. His newest book is Big Data and Business Analytics (Taylor & Francis, May 2013).