The Rise of Data-Driven Intelligence

An emerging concept called data-driven intelligence is a model for a new perspective on gathering insight from a vast pool of data. I’ll contrast data-driven intelligence with more familiar application-driven intelligence models.

One of the earliest names for information technology was "data processing," which encompassed the need for both data and processing power. However, the glamour of IT for many years was in application development, where a processing- or computing-centric focus ruled the roost. From birth (creation) to death (deletion), most data remained within the control of applications. Of course, applications that analyze data after it has been created have long existed (such as business intelligence and seismic processing), but these applications were a small fraction of practical IT uses. Not any more.

Application-Driven Vs. Data-Driven Intelligence

In his book "Reinventing Discovery" (which I recommend, by the way), the author Michael Nielsen discusses data-driven intelligence and contrasts it with artificial intelligence and human intelligence. He defines data-driven intelligence as the ability of computers to extract meaning from data. He differentiates it from artificial intelligence, which he says takes tasks that humans are good at and aims to mimic or improve human performance (such as chess playing) and human intelligence (such as our ability to process visual information). According to Nielsen, data-driven intelligence complements human intelligence by solving different kinds of problems. (Big data, anyone?)

Let's examine what it means from an IT perspective. Application-driven intelligence tends to create, read, update and delete data to fulfill an initial purpose, such as a workflow process to manage order processing, shipping and payment collection. By contrast, data-driven intelligence takes existing data (human- or machine-generated) and uses it for a secondary or additional purpose, such as performing e-discovery on email files or a big data analysis that uses external information gleaned from the Web for upselling or cross-selling customers. Sensory information (such as meter reading) or machine/computer-generated information (such as logs) are created first and then analyzed by a downstream process (which may be in real-time) as appropriate.

From an IT perspective, the application development methodologies (as well as the skill sets of the developers) may be different. From an operational perspective, the service level agreements (SLAs), such as for performance and recoverability of the data, may have to be planned differently. The resources (servers, networks, storage) have to be planned differently as well. IT is familiar with application-driven intelligence-based applications, but has to learn more how to deal with data-driven intelligence applications, such as big data.

Application-Driven Vs. Data-Driven Intelligence

Application-Driven Intelligence

Data-Driven Intelligence

Primary Goal

Substitute application intelligence for human intelligence in managing a process

Extract meaning and knowledge from data

Description

Data is created and managed to fit the needs of the application; typically, the creation of data is part of a process using the application.

The application is created and managed to fit the needs of the data, which may be (and likely are) created independent of the application

Example

Supply chain management

Customer relationship management

Content management

Online transaction processing

Big data

Data warehousing

Search engine

Sensor-based analysis

IBM's Watson or Apple's Siri

Source: the Mesabi Group, November 2012

There is nothing new under the sun. Data-driven intelligence (such as statistical analysis using techniques like regression analysis, linear programming and simulation modeling) have been around for a long time. More recently, new concepts have emerged, including data warehousing, online analytical processing and data mining. The problem is that terms such as advanced analytics, business intelligence and big data are regarded as valuable by businesses, but existed as isolated IT islands. However, viewing these siloed (or at best overlapping) efforts and thinking of them in terms of data-driven intelligence provides a way of bringing them together to emphasize the importance of a data-centric focus.

Yes, there are hybrids. Data-driven intelligence can be inserted in an operational system, such as retail sale to check a credit card to see if it is fraudulent, or at points within a supply chain.

Data-driven intelligence is an additive view that broadens our understanding and does not replace application-driven intelligence. Let software intelligence continue to multiply and add to our understanding and the value that we derive from IT.

Next page: Thinking About Data-Driven Intelligence Applications

David Hill is principal of Mesabi Group LLC, which focuses on helping organizations make complex IT infrastructure decisions simpler and easier to understand. He is the author of the book "Data Protection: Governance, Risk Management, and Compliance." View Full Bio

If you thought consumerization killed UC, think again: 70% of our 488 respondents have or plan to put systems in place. Of those, 34% will roll UC out to 76% or more of their user base. And there’s some good news for UCaaS providers.