Is Big Data the New Black Gold?

Image: verifex/Flickr

One of the top challenges facing organizations today is a lack of business insights. Without the integrity and reliability of vital information, an organization is ill prepared to make sound decisions anywhere and anytime — at the pace required by today’s competitive business landscape.

Big Data represents a tremendous opportunity to drill down and tap into these critical insights. In fact, the powerful potential to mine and refine this vital, valuable resource points to a direct comparison to a similarly vital resource in the modern economy: crude oil.

The comparison has strong merit. The current data bonanza harkens back to the early days of the oil boom. Big Data has become a vast, largely untapped resource capable of powering progress, and it requires a certain expertise to extract and refine to maximize its utility.

But unlike a fossil fuel, data isn’t necessarily spent once used; its value can continue growing as new pieces of information are added, provided that it is managed effectively. In this way, data may be more like wind or solar energy, which can also be used in multiple forms to serve more than one purpose.

However, with the right technology and skill, it can be refined into a powerful, renewable force, gaining strength and value as part of a growing stream of information. And, much like raw crude cannot be used to fuel a car, raw data in and of itself is not insightful, but with the proper refinement and context placement, it can yield tremendous insight.

In order to maximize the value of this renewable resource, enterprises must find a way to capture and manage it at every phase of its lifecycle, from creation to archiving, to ensure that data is appropriately interpreted and converted into insight. But, an effective data strategy must go beyond merely visualizing and analyzing data on a dashboard, although these are important components of the process.

To truly maximize the value of data, organizations must rethink how they create, edit and store it. They must analyze the architecture and standards, quality, governance and management processes at every phase. Once controlled appropriately, data can yield the insights that make it more valuable than black gold.

Gartner projects that the volume of information is growing at an annual rate of 59 percent worldwide and Google already process 600 petabytes of data monthly. The number of organizations storing 1 petabyte of data — that’s a million gigabytes (or should we say a thousand terabytes?? Or both?? Not sure which one has more impact)– has increased from 16 to 29 percent in the past three years. Few companies are prepared for this onslaught: the more data stored, the more complex the systems required to manage it.

New data can make enterprises smarter, but it can also disrupt business for enterprises unable to effectively tap into and analyze it with the right tools. IDC projects that, while organizations will increase spending on Big Data from $3.2 billion in 2010 to $16.9 billion in 2015, more than 85 percent of Fortune 500 organizations will fail to exploit Big Data for competitive advantage.

How can your company be one that takes advantage of the opportunity?

Working with a partner that specializes in the fine art of data refinement is critical to harnessing Big Data’s potential and creating a strategic roadmap to reach your goals. Incorporating technology platforms and solutions, organizational governance and change management through a unique Enterprise Data Lifecycle Management (EDLM) framework is vital for extracting the most intrinsic value from raw data.

Too often, Big Data projects focus on the technology and not the business outcome. To ensure success, the data you seek must be tightly aligned to your overall business strategy. Ask yourself and your business what the purpose is.

Is it…

Search – finding that needle of insight in a haystack of seemingly unrelated events, and a deluge of non-standard content?

Big Picture – understanding the whole by analyzing and linking the behavior of the parts?

Track and Predict – detecting patterns over time and predicting trends?

Reporting – diving into very large data sets to make business decisions in real or near real-time?

Understanding your goals is paramount to developing a successful data strategy, no matter which approach you choose. And, whether Big Data is compared to oil, wind or solar energy, one thing is clear: It requires multidisciplinary expertise to extract, refine, and manage at every phase of its lifecycle. By doing so, companies can effectively tap the value of the 21st century’s most promising source of actionable business intelligence, Big Data – the new black gold.

Arvind Singh is a serial entrepreneur with experience in building global businesses for the last 25 years.