It's essential to adopt a broader scope about data and analytics, create a very flexible and agile IT framework, and build a strong foundation for data science.

It's no secret that digital technologies generate enormous volumes of data. According to various industry estimates, data volumes double about every two years. At the same time, however, only about half a percent of data is currently being analyzed.

"The challenge with big data is not the data," says Dorman Bazzell, vice president and America's practice leader for Insight & Analytics at Hitachi Consulting. "There is plenty of data. The challenge with big data is developing a set of meaningful use cases that address key business challenges."

It's both an opportunity and a challenge that keeps business and IT leaders up at night. "There have been enormous changes in this space over the last few years," states Luc Ducrocq, director of the Insight and Analytics Practice at Clear Peak, a Greenwood Village, Colo., management and data consulting firm. These encompass areas as diverse as artificial intelligence (AI), blockchain, the internet of things (IoT), speech processing, visualization tools, data as a service, and a growing self-service focus on analytics.

According to a 2016 report from Capgemini, "Big & Fast Data: The Rise of Insight-Driven Business," 65 percent of business and IT leaders agree that they risk becoming irrelevant and/or uncompetitive if they do not embrace big data, while 64 percent say that big data is changing traditional boundaries and 24 percent are already experiencing ingress of competitors from adjacent sectors.

To be sure, understanding how to use big data effectively is increasingly at the center of digital success. Broader and deeper insights can fuel a level of innovation and disruption that would have been unimaginable only a few years ago. But, in order to achieve superior results, it's essential to adopt a broader scope about data and analytics, create a highly flexible and agile IT framework, and build a strong foundation for data science.

Going Beyond the Database: The Year of Intelligence

It's clear that all digital roads lead to data. Ducrocq describes 2017 as the "year of intelligence." Yet, the ability to put analytics and predictive analytics to work to deliver bottom-line results is a growing challenge. It's important to understand how to use big data effectively.

For one thing, Hitachi Consulting's Bazzell says, "Users and organizations are demanding greater access to broader data sets." But the challenges don't stop there. There's a need to build out an IT framework and compute capacity to collect, manage and process all the data. This can ripple into partnerships and APIs.

There's also a need to plug in machine learning and AI to automate processes and learn dynamically, and to develop a foundation for data science and analytics within an enterprise. Bazzell notes that this is not "a one and done process. It requires multiple iterations of data machination to identify the most relevant data, manage data quality and gain understanding of the data."

Clear Peak's Ducrocq believes that business and IT leaders must focus on three key areas: infrastructure, the IT framework to support data and the individuals in the organization who ultimately drives the initiative.

89 Degrees is one company that is using big data effectively. The integrated marketing and analytics services firm has worked with industry heavyweights ranging from Hyundai to IKEA. It hosts a variety of analytics solutions—including SAS Visual Analytics, SAS Marketing Automation and SAS Hadoop Connector—to take analytics to a broader and deeper level.

The firm plucks data from hundreds of sources and points—including point-of-sale transactions, purchase locations information, web data, email data, billing information and various logs—to create highly targeted marketing campaigns. "We build probabilistic models that translate to marketing approaches and methods," says Rosie Poultney, vice president of analytics at 89 Degrees. She adds that the firm has had to adapt as big data became "really big data."

89 Degrees relies on a Hadoop data storage framework, clouds, machine learning algorithms, APIs and a group of data scientists to push insight into new and sometimes unfamiliar territory. "The ability to process large volumes of data quickly is critical," Poultney says. "But we also require a great deal of flexibility so we can add new parameters and data points on the fly." In some cases, the company is dealing with daily data sets as large as 20 gigabytes for a single client, she notes.

Big data is more than a business initiative, and it's more than a collection of tools and technology.

"In today's business environment, it's necessary to implement and activate systems that support fast data," Poultney points out. "Big data isn't just a buzzword or an interesting concept. It's a way to unlock value. But to succeed, you have to identify the business problem you want to solve and determine how big data can achieve that."

Using Big Data Analytics to Gain Deep Insights

How can business and IT executives navigate the evolving big data landscape? How can they best put analytics and predictive analytics to use?

Clear Peak's Ducrocq says that it's important to recognize that IT is rapidly becoming a commodity, and organizations must focus on business value. This means that the entire organization must work together to spot value points. It also means that every organization must build a unique infrastructure that supports the value proposition and then continually evaluate the situation.

Hitachi Consulting's Bazzell advises organizations to think broadly and creatively about a data framework. This includes plugging in conventional data sources, as well as tapping into the internet of things and connected machines and sensors to produce both human and machine data analytics.

In fact, he points out that analytics now transcends human thinking. "If you look at the IoT more broadly as anything that produces data, receives feedback and adjusts behavior, then humans become 'sensors' as well," Bazzell points out.

Indeed, this emerging paradigm requires a new way of thinking about data and analytics that extends through the entire organization. At the same time, "Open the flood gates of data to data scientists, and let them do what they do best: explore and correlate seemingly unrelated data points," Bazzell recommends.

This doesn't mean giving analysts and data scientists carte blanche to chase unicorns and rainbows. It's crucial to "hold data scientists 'feet to the fire' so that they have the flexibility to explore, but they are accountable to the goals, timing and budget of the use case," Bazzell explains.

Make no mistake, a new era has emerged, and business and IT leaders must adjust and adapt.

"If a data initiative is driven by IT and the business side is not involved, you might as well shut it down because it isn't going to succeed," concludes Clear Peak' Ducrocq. "You have to understand the business problem or challenge you are attempting to solve, who cares and why they care, and what is the cost of doing nothing.

"IT must help the business understand whether the data challenge can be solved and, if that's possible, how to achieve maximum results."