Big Data Dreams: A Wake-up Call

Following on the heels of “develop our social media strategy” and “develop our model for social media ROI”, the latest critical “to-do” item for digital marketing teams is “utilize big data”.

Like the two prior objectives mentioned above, digital marketing managers are feeling pressured to take action on an approach that has become prevalent in business and mainstream media, and that has resultantly reached the general attention of executive management, prompting them to whip-up a frenzy of activity to ensure that they’re keeping their company up with the latest digital trends.

All three of these digital marketing activities have true potential to drive more effective marketing. Unfortunately, for the first two, the sense of urgency and intensity surrounding their introduction in most organizations in combination with a persistent general oversimplification of the effort required to achieve them and a risk aversion that has spawned experiments rather than true investments has resulted in unrealistic expectations, disappointment at results, and a failure to fund the subsequent effort that would actually ultimately deliver the expected returns.

The Next Big Thing
The current interest in building a “Big Data” approach to marketing runs this same risk. The topic of “Big Data” is everywhere, with promises to revolutionize business. Unfortunately, the understanding of what “Big Data” really is and what harnessing it for a business will take are not quite as common. That gap between expectation and reality holds the risk for another set of disappointments following under-funded corporate experiments in the area.

To help organizations avoid the same half-started initiatives they’re now nursing under social media strategy and qualitatively informed digital communications planning, it is important that conversations about “big data” actually move away from the common terminology and the hype to address what's really exciting about them.

The Root of it All
What most managers are talking about when they say “big data” has nothing to do with the actual size of the datasets they are processing, nor does it have anything to do with the possibilities for real-time response (in marketing, operations and logistics) inherent in true big data systems. Most organizations do not have Petabytes of data to process, nor are they structured to respond in real time to what the analysis of any amount of data is telling them. Instead, most organizations have sales reports, and hundreds or thousands of Gigabytes of customer data for a CRM they haven’t yet utilized to full capacity or functionality in the 5 – 10 years they’ve had it. They typically take weeks, months or quarters to consider changes in strategy and tactics indicated by quantitative analysis of sales, operations or the market.

What these typical organizations are excited by when they discuss “Big Data” is the promise of better data-driven decision making. However, as the point about CRM above indicates, achieving this isn’t about getting more data and installing more technology to analyze that data. It is first about a change in organizational thinking and structure to more effectively apply the broad array of data sources already available for quicker and more flexible management decision making.

Getting Real
The benefit of the “big data” discussion is that executives and managers are once again turning their attention to all of those information systems they have lying about and asking how they can be utilized to more effectively and competitively drive the business. The threat is that without organizational change these same executives and managers will rush out to install even more systems incompletely instead of first optimizing the data that they already have at their disposal.

First, they will not recognize that any sort of data-driven management, especially “Big Data”, means fundamental change in the way the business operates, from strategic planning to management decision making to the minutiae of operations. If they don’t recognize this, they can gather and analyze as much data as they like without seeing any business results. (See most “social media listening” initiatives as an example). If they do realize this, they may be threatened about where they fit into this new organization. Sr. Managers’ fear of personal obsolescence via innovation is the most common cause of (the slow and quiet) death of corporate initiatives. This will be the primary topic of the eMetrics presentation.

Second, organizations will frame effective “Big Data” initiative activity in terms of new technology implemented and the size of certain primary data sets as opposed to looking for the more effective integration of the “Broad Data” already available across their organization. They will also over-value achieving big data insights on their own, and will miss out on opportunities to find unique insights from shared data sources. This will be the topic of a SxSWi panel I will be moderating with the Executive Director of the Spiegel Digital & Database Research Center at Northwestern University, the Sr. Manager of Customer Engagement at United Airlines, the Sr. Analytics Manager at Dell, who will give insights into how to start with a “broad data” approach to build to “big data” dreams.

3 Responses to “Big Data Dreams: A Wake-up Call”

"...achieving this isn’t about getting more data and installing more technology to analyze that data. It is first about a change in organizational thinking and structure to more effectively apply the broad array of data sources already available for quicker and more flexible management decision making."

No doubt this is true, but reading the post, I was hoping for a hint or two as to what "change in organizational thinking and structure" might mean, concretely. It's not hard to believe that managers aren't using the data they have optimally. What, specifically, has to be different? And, if this gives away your presentation, at least a hint would be nice.

Hi Judith. Thanks for the comment. To answer your question, the most critical change in thinking and structure must be around what type of management is valued and rewarded. Traditionally, the move to a management position in an organization comes as a reward for skill or competency in some area, i.e. being "the best" at something the organization needs done. Since competency in a skill is a common path to management, maintaining the need for that skill in the organization becomes the manager's preoccupation. This is why newcomers to organizations often observe archaic and inefficient practices in their organization - some manager is making sure things continue to be done the way they've always been done, aka the way that continues to make their management position necessary.

So the change in thinking and structure that is necessary is a system of promotion and reward for the best thinkers, not necessarily the best doers. In a state of constant evolution and change, what we do today is probably not what we need to be doing next year. What's needed to respond to change (and the patterns that emerge from big data) is not competency in action, but competency in analysis and interpretation. Managers need to be rewarded for their ability to bring about change, and organizations need to avoid a structure that allows managers to build little empires that hold resources in stasis and actually obstruct change.

In talking with industry analysts, a comment I've heard continuously is that the organizational scheme that served us in the Industrial Age doesn't serve the needs of business with high volume, velocity and variety of data.