This is the blog of David M. Raab, marketing technology consultant and analyst. Mr. Raab is Principal at Raab Associates Inc. The blog is named for the Customer Experience Matrix, a tool to visualize marketing and operational interactions between a company and its customers.

Wednesday, May 23, 2012

Ask most marketers how they measure performance, and they’ll tell you they look at results: incremental revenue or return on investment if they’re available, or response rates if they're not. Industry experts take a similar approach, focusing largely on the need for better revenue measures. The situation – and barely concealed frustration – is captured perfectly in the headline from a recent Forrester Consulting study sponsored by Silverpop: “Response Metrics Are Used To Evaluate Success, Leaving Customer Or Business Impact Metrics Largely Ignored”.

I agree that revenue is important, but humbly suggest that there’s more to marketing measurement than ROI. Marketers need several types of information – and shouldn’t let the quest for performance measures prevent them from meeting other requirements.

Here are six non-value measures that marketers should build into their reporting systems.

Benchmarks. Sure, marketing’s job is to generate revenue, but is generating $1 million good or bad? The only way to know is by placing the number in context, which could be this year’s marketing plan or last year’s actual results. Even if marketers can’t measure revenue, they can set benchmarks for metrics like number of leads generated, funnel conversion rates, or cost per order. In fact, measuring components that contribute to results gives better insights into marketing performance than reporting on the results themselves. For this reason, marketers who can’t directly measure marketing-generated revenue should think twice before creating complex indirect estimates that are hard to understand and have limited credibility. The money would probably be better spent on reports that provide a clearer picture of what’s actually happening.

Projections. Past results are interesting but the future is more important. Again, the real need is to understand the factors that determine future results, such as response rates and funnel velocity. Changes in these can give early warning of risks and opportunities. The trick is to distinguish real trends from random variations, so marketers react quickly without chasing too many false alarms.

Operations. It’s easy to make mistakes in setting up a marketing program, especially one with multiple stages, lead scoring models, and decision rules. Even careful testing can’t always capture all program steps or contingencies. Marketers need reports on the number of people in each program stage and receiving each message, and they need a model that lets them know whether those numbers are reasonable. Reports should show both program-to-date and weekly or daily results: cumulative data show major errors such as bad program logic, and short-term results capture small problems, such as a missed processing step, that could get lost in a program-to-date aggregate.

Exceptions. Projections and benchmarks put data in context, but marketers don't have time to comb through every figure. They need exception reports to highlight the most important variations, both positive and negative. Marketers also need tools to drill into the exceptions so they can understand what happened and identify new opportunities.

Testing. Formal tests are the most certain way to understand the impact of marketing projects, but they often require special reporting tools such as ways to compare results for different customer groups over time. Incremental revenue is the ultimate measure for test evaluation, but often other metrics such as response rates or velocity are easier to capture and more directly relevant. Reaping the full benefit of tests also requires systems to distribute results and catalog findings for future reference.

Strategic Goals. Marketing plans should be based on corporate strategy, but long-term goals fade into the background once marketers start making tactical choices based on day-to-day results. The reporting system should provide direct measures of strategic objectives – things like penetration of new market segments and exploration of new channels – so marketers can see the cumulative impact of deviations from the original plans. Many strategic goals, such as process change, staff training, and systems deployment, are not measured in revenue at all.

The types of measures I’ve just described don’t replace revenue and ROI reporting. Rather, they meet needs that revenue reports alone cannot. Ideally, all these types of information will be combined in a marketing dashboard that provides a quick overview of critical information and allows drilling into details when necessary. Marketers should realize that the contents of this dashboard will change over time as their focus shifts to different programs and strategic goals. They should also recognize that good reporting will generate new questions as it uncovers risks and opportunities that would otherwise have gone undetected. The system should make those questions easier to answer, but marketers shouldn't expect their total work to decrease. What they can expect is that better reporting will increase the value created by their efforts: yet another new metric, Return on Reporting, should go up.