Coping With Metrics Overload

Not a day goes by when we aren’t confronted with one of the latest buzzwords: big data, data analytics, data visualization or dashboard metrics. But is access to all of this data really living up to the hype? Sometimes it seems that the cost of these data outweigh the benefits. Instead of providing useful insights, oftentimes the data contributes to information overload.

One reason for this is that the technology we’ve been developing in this space has focused on collecting, storing and processing more and more data. Not enough efforts have been directed toward developing reliable metrics and models that can consistently link the data to company performance.

Recently, The Walt Disney Co. started piloting wristbands that allow resort guests to unlock their hotel rooms, pay for meals, enter the theme parks and obtain passes to attractions and rides. You can imagine how much data this technology would generate. You can also imagine the potential insights you can extract from the data. But for those who have had the privilege of managing data analytics efforts, you can also imagine drowning in pages of reports and metrics. How much money do guests spend on an 85-degree day versus a 60-degree day? How frequently do guests return to their rooms during their stay and when? Is gift shop purchasing correlated with what rides and attractions visitors get passes for?

Disney has the sophistication and resources to effectively use this data for their marketing and operations efforts, but we can’t all be Disney. So how do we handle all of the data and metrics generated from our technologies, ranging from sales metrics and social media metrics to market research and customer tracking data?

1. Stay focused. The first step is to start with strategic goals, and structure the data collection and analysis around those goals. This is common knowledge in offline marketing research but somehow it has been lost in the world of big data analytics, where analysts are seduced by interesting metrics and data visualization. Instead of asking, “I wonder whether gift shop spending is correlated with passes for rides and attractions?” we should be asking, “Who are our heavy spenders and how can we identify them?” The first question results in an interesting correlation metric that may inform a strategic decision. The second question stimulates a suite of metrics that would clearly inform segmentation and targeting strategies.

How does Disney do data for marketing and operations?

2. Measure what matters. Too often, we build metrics based on what is easy to measure instead of measuring what matters. Thus, once we identify our strategic goals and questions, we need to hone in on the metrics we would need to answer those questions. We want our strategic objectives to guide our analysis and data collection rather than letting the data structure define the metrics and analysis. When we let the data drive the process, we often end up with an abundance of metrics that we don’t really know what to do with.

3. Establish a baseline and set benchmarks. While it might be great to know how much the average customer spent in the gift shop after riding Space Mountain, it’s not clear if that is good or bad, or if any of our marketing efforts around the Space Mountain experience worked. What we need are baselines and benchmarks to measure success. Before launching any new promotion, we need to establish a baseline to tell us what our performance metrics normally look like. We also want to set a target for the metric that will tell us if our promotion was successful, profitable and worth the investment.

4. Monitor past success. In the world of metrics overload, it is important to monitor success by tracking the metrics linked to the organization’s strategic objectives and comparing them to baseline and benchmark metrics after the campaign. Better yet, smart organizations should maintain a library of baseline, benchmark and outcome metrics to improve service. This would allow the organization to identify elements of their past activities that worked and those that fell short, and learn how to optimally design their activities moving forward.