Big Data evolution depends on asking the right questions

As we develop experience with Big Data, more organizations will opt to ask their own questions for what they hope will yield greater returns. Here's an example how this can unfold.

You can purchase a "best practice" package of Big Data Analytics with prefab reports, or you can start asking your own questions to derive unique approaches for your business. Initially, organizations could opt for the former, because it allows them to get their toes into the big data waters-but as we develop experience with Big Data, more organizations will opt to ask their own questions for what they hope will yield greater returns.

This approach will be evolutionary, and the best way to demonstrate how it might unfold is by example.

Let's take the case of a customer information system used by a bank. There is nothing new about systems like this. They have been around since the 1980s, and bank marketing departments actively use them for their marketing campaigns. These systems segment customers into various demographic groups that are delineated by age and location. Data is spooled to data warehouses from customer records in bank transaction systems. The data enables marketing to promote a new mortgage loan product by targeting the group most likely to be interested (customers between the ages of 25 and 55).

But let's say that marketing wants to do more-like incenting customers to use their credit and debit cards.

A data warehouse that contains only "static" information about customers (like what their age is and where they live), cannot tell marketing who the biggest credit and debit card users are. This is an area where IT likely will need to work with the card company to obtain transaction history data and then integrate it with the static customer information already in the marketing data warehouse. The process requires system integration-and also followup operations such as an agreement with the card provider on how often data will be updated, and an update to IT internal procedures that addresses how the update will be done in the data center.

This works well for awhile-but then marketing comes back to IT for more. Now it wants to look at customer mobility patterns-because it wants to establish half a dozen self-service kiosks in downtown buildings in a major metro area. Naturally, marketing wants to know the traffic patterns of its customers in the city to determine the "busiest" spots, because each kiosk must turn a certain number of transactions per day to be profitable.

Customer traffic information can't be obtained from the existing data warehouse, so IT goes out to an outside "tracking" company that monitors the GPS sensors on personal cell phones. The right set of data extractions, preparation and integration can "marry" this big data to the customer transaction-derived data in the bank's marketing data warehouse. This enables marketing to ask the question: where are the "hot spots" in the city that our customers gather around?

By now, IT is wiping its brows and hoping there won't be more-but there is.

The data warehouse is working so well that marketing now wants to launch a new customer acquisition campaign! Naturally, it plans to incent customers to bring in new customers by putting $25 for each new customer into their accounts like it always has-but it doesn't want to stop there. Marketing approaches the CIO and asks, "Can we also see who our customers know in the social media and reach out to them?"

The next step is to again return to a third-party Internet intelligence provider-this time for big data harvested from social media sites that shows who bank customers know. This data is prepped and again integrated into the marketing data warehouse.

The result is an evolution from a traditional data warehouse originally built from transactional information to a hybrid data warehouse that now contains both transactional and big data. Can this evolution work for most organizations?

Yes, it can-for these reasons:

The growth in into hybrid data warehouses will be painful, but it will be gradual-and that will help.

As long as that evolution is driven by "good" questions that drive good business, people on all sides of the activity (marketing, IT, C-level executives in our earlier example) are going to "get" why it should happen.

Data warehouse evolution will not only integrate data from multiple sources-it will also integrate work efforts between departments (like marketing and IT) that have only sparingly worked together in the past. And that's good for business.

About Mary Shacklett

Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President o...

Full Bio

Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President of Product Research and Software Development for Summit Information Systems, a computer software company; and Vice President of Strategic Planning and Technology at FSI International, a multinational manufacturing company in the semiconductor industry. Mary is a keynote speaker and has more than 1,000 articles, research studies, and technology publications in print.