How to Find the Needle in the Big Data Haystack

The term "Big Data" refers to increasingly robust sets of data that serve as the "feedstock" for the Internet of Things (IoT). These very large datasets – collected via networked sensors and processed with advanced software – can help refiners, drilling contractors and others in the oil and gas industry detect problems in physical machinery with unprecedented speed and detail. However, the vast scale and scope of information now available to operating companies can create another problem: what to do with it all.

"The sheer volume and diversity of data that is available is overwhelming," Jeff Ladner, vice president of environmental performance with the operational excellence software and information services provider Sphera, told Rigzone. "Embedded in that narrative are more substantial challenges, though."

One such additional challenge entails correlating data across disparate systems – operations, maintenance, environmental, safety, production and other inputs – and transforming it into a single, logical, usable, quality data set, noted Ladner.

"All of the information is managed in different systems, often by different business functions with localized objectives," he explained. "The result is a data puzzle that can be a limiting factor when looking at how to use new tools and techniques to derive insight from vast information resources."

Read on for Ladner's perspective on how oil and gas companies can find the proverbial needle in the growing "haystack" of voluminous data available to them.

Rigzone: What are the actual and potential effects of having more data than you can handle?

Ladner: The most common effects of having too much data are organizational paralysis and focusing on low-value analytics. Organizational paralysis stems from simply not knowing where to start and the uncertainty regarding the value that an analytics project may yield. As a result, projects are pursued based on individual expertise, perceptions of value, ease of execution and availability.

As companies look to address more complex challenges, the quality of the available data is found to be lacking in material ways that limit its usefulness. This translates into latent costs in terms of time and investment covering data collection, data cleansing, master data governance, standardization and data quality to ensure a reliable, high-quality stream of data. This is particularly true when companies strive to address behaviorally driven outcomes, such as personnel safety, which is my background vs. addressing engineering-driven outcomes, such as a well production or sulfur recovery unit performance.

Rigzone: How do companies currently prioritize which data to process and analyze? What are the drawbacks of the status quo?

Ladner: We see a significant number of projects focused on providing visibility to operational performance through dashboards and reporting. Metrics incorporated in these project are commonly simple descriptive metrics used by management internally, prescribed by industry groups or required by third parties such as regulators. The drawbacks of this approach is that descriptive analytics in and of themselves do not help you reliably determine what steps to take to improve performance. When the metrics are based on people performing activities, they are also subject to gaming in which case reporting takes precedence over actually completing the work. The result is the analytics do not yield the intended outcomes. Predictive analytics are required. Analytics focused on statistically correlating actions and outcomes are necessary to provide a basis for evaluating the effectiveness of steps taken and making continuous improvements. Progressing to this stage is a more intensive process.

Ladner: Companies need to be intentional. It's critical to align domain expertise with statistical expertise with enabling technology. Technology alone or statistics alone are not going to yield breakthrough insights. One of the challenges is that Big Data and analytics are widely used terms and often used in very general ways. They are not a silver bullet that will solve intractable problems. They are simply tools that when applied intentionally can yield new insights.

Domain expertise in terms of industry knowledge, business process understanding, and insight into the data and what it means are all critical. Valuable insights are most readily derived when all of the capabilities are brought to bear on a specific target outcome. In the oil and gas industry, safety is paramount. We work with many companies on helping them enable disciplined execution of their safety programs and drive continuous improvements. Our breakthroughs are not coming as the result of helping companies better report on injury rates in support of their safety programs and regulatory reporting. We're leveraging in-depth, expertise-driven analysis of safety business processes focused on identifying, assessing and mitigating risk. Insights are the result of looking at how people-driven processes are performing and identifying specific actions that can optimize process design and execution. It's the difference between superficial analytics on the time to close for action items in an incident investigation and a more in-depth analysis of front-line supervisor engagement in the companies' safety culture.

Rigzone: Are there any industries that are providing leadership in how to make better use of all this data?

Ladner: The promise of Big Data and analytics is their ability to drive revolutionary insights into complex business problems. Some of the most complex problems in oil and gas are those that intrinsically rely on people, not just equipment. One that we regularly encounter is: "How do we prevent the next fatality?" That being the case, industries that are focused on understanding and shaping people's behaviors, such as retail and service industries, have the potential to provide new perspectives and ways of approaching these people problems that are not necessarily the straightforward engineering-based approaches you would otherwise take if seeking to optimize aspects of a refining operation.

Matthew V. Veazey has written about the oil and gas industry since 2000. Email Matthew at mveazey@rigzone.com

WHAT DO YOU THINK?

Click on the button below to add a comment.
Generated by readers, the comments included herein do not reflect the views and opinions of Rigzone. All comments are subject to editorial review. Off-topic, inappropriate or insulting comments will be removed.

Your comments have been received and will be posted after being reviewed. If you have any questions, please feel free to contact us.