Search

(This post is a re-purposing of a speech I gave to the FPRA/PRSA-Orlando on July 23, 2009. You can download the slides here.)

Public relations measurement is at a crossroads. Old techniques are no longer sufficient. Old metrics are no longer applicable. Old thinking must be replaced by new. The need for accountability, and to prove the value of PR and social media programs, has never been greater.

As we look to the next year, here are five things to forget and five things to learn about public relations measurement in 2010.

Things to Forget in 2010

1. Media Relations Focus

A focus on media relations fails to capture several important aspects of PR – brand, reputation, crisis, employee communication and DTC to name a few. Also, the importance of traditional media is declining. Numerous studies have shown people don’t trust what they read in the media, they trust each other. I believe it was Hauser and Katz who coined the term ‘you are what you measure’ in 1998. If measurement is focused on media relations that is how the public relations function will be judged.

__________________________________________________________

2. Outputs

The need to put PR results in a business context has never been greater. We need to be able to address the question – what are we doing to help drive the business? If you are focused on output metrics like impressions or message delivery, you will always have a hard time explaining business impact. Instead, we need to focus on outcomes and answer the question – what happened as a result of our program or coverage? Understanding outputs has primary benefit as a diagnostic tool rather than a ‘scorecard’.

__________________________________________________________

3. Impressions (and Multipliers!)

The most common PR metric today is Impressions. While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek. Impressions also (greatly) overstate actual relevant audience. Generally only a fraction of any particular magazine or newspaper’s circulation meets your target audience demographics. And impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content. To compound the problems, many PR practitioners use a multiplier on impression numbers to account for pass-along readership or a mythical credibility advantage PR has over other communication tools. The simple fact is there is no factual basis (e.g. research proof) that multipliers should be used in any case.

__________________________________________________________

4. Ad Equivalency (AVEs)
There are many reasons why using ad equivalency as a proxy for PR value is not advisable. Here are five good reasons they should be avoided:

AVE calculations vary and there are no standards. Tonality, article length, competitive mentions and other factors are handled differently.

AVE results can be misleading. AVEs may be trending up while metrics like message communication, share of favorable positioning and share of positive press are falling.

AVEs reduce PR to just the media dimension by only assigning a value in this area.

AVEs only apply to traditional media. What is the AVE of a positive conversation about your company on a leading blog?

How much is it worth for a troubled company to not appear in the Wall Street Journal? AVEs cannot address this.

__________________________________________________________

5. Return on (Engagement/Influence/etc.)
Not a day goes by on Twitter without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks. Wrong and wrong. Most of these folks either don’t understand ROI or don’t know how to obtain the data necessary to calculate it. There is also a lot of confusion between creating value and ROI. Generating awareness creates value, for example, but may not immediately result in demonstrable ROI.

__________________________________________________________

Things to Learn in 2010

1. Total Value of PR

The majority of current PR measurement efforts focus on marketing/sales and output metrics. The Total Value Cube is a way to visualize and think about all the potential value your PR and social media efforts deliver. Beyond marketing to include brand and reputation, beyond outputs to include engagement, influence and action, and beyond revenue generation to include cost savings and cost avoidance.

__________________________________________________________

2. A New Model for Measurement
Many public relations practitioners regularly get their Outputs confused with their Outtakes or Outcomes. Outtakes is not often used in the U.S. – it seems much more prevalent in Europe. The overall terminology is confusing and is defined in different ways by different practitioners. Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them. In short, the terms are too much ‘inside baseball’.

What we need is a metrics taxonomy that is easier to understand and explain. I like this one.

Exposure – to what degree have we created exposure to content and message?

Engagement – who, how and where are people interacting/engaging with our content?

Influence – the degree to which exposure and engagement have influenced perceptions and attitudes

Action – as a result of the PR/social media effort, what actions if any has the target taken?”

__________________________________________________________

3. Three Zones of Measurement

From the left, companies or brands control, own or manage websites – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with. This zone is measured primarily by web analytics. In the middle are the actual social networks and conversations between individuals. In this zone we are interested in data sets that cannot be gathered solely using web analytics packages. How often is the brand being mentioned in conversation? What is the sentiment of the comments? How often is the brand being recommended and by whom? Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone. The third zone represents all the real-world, offline transactions that may be of interest. Did someone visit the store or attend or event? Did they buy a product? Did they recommend the brand or product to a friend over coffee? Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.

Your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones. The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa.

__________________________________________________________

4. New Metrics

_________________________________________________________

5. The Difference Between Impact/Value and ROI
ROI is a form of value/impact, but not all value takes form of ROI. ROI is a financial metric – percentage of dollars returned for a given investment/cost. The dollars may be revenue generated, dollars saved or spending avoided. ROI is transactional. ROI lives on the income statement in business terms.

Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product. Value creation occurs over time, not at a point in time. Value creation is process-oriented. Value lives on the balance sheet.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

In yesterday’s New York Times, you may have read the article, Spinning the Web: P.R. in Silicon Valley, an interesting although not overly insightful piece. From a social media measurement perspective, two items caught my eye. The first, referring to Brian Solis, Principal of FutureWorks, about how he calculates social media audience figures:

“Instead of calculating the impressions an article gets by estimating a publication’s circulation and pass-along rate, Mr. Solis counts the number of people who tweeted about a company and their combined following, the number of retweets or clicks on links, as well as traffic from Facebook and other social networks.”

Toward the end of the article, we learn:

“By 6:30 p.m. on the day Wordnik went live, Brew’s staff calculated that 1.43 million people had seen tweets about it.”

Setting aside for a moment that the article and these sorts of audience metrics take a broadcast-oriented view of Twitter (Mr. Solis discusses the shortcomings of the NYT viewpoint here), the emerging view of audience measures for Twitter is to calculate the Followers of each person tweeting about the subject of interest, and then adding Follower numbers for each person retweeting the subject and so on. The issue here, much as it is in traditional public relations, is that the audience figure that results from these sorts of calculations grossly overstates, by one or two orders of magnitude or more, the actual “audience” for these tweets. It is a hypothetical number that assumes everyone that possibly could see a tweet has in fact seen it, and everyone who sees it is relevant to you/your brand. This is fantasy of course.

On the issue of relevant audience, here is a quick example. At the time I pulled these figures, the audited circulation of the New York Times was 4,974,000. Most PR practitioners getting a ‘hit’ in the NYT would claim this as their audience. However:

If you were only trying to reach a C-Suite audience with your message, the actual audience reached would be 598,000 or 12% of the total circulation

If you were trying to reach Women, your audience would be 1,937,000 or 39% of the total

If you were trying to reach 25 – 54 year old Men, your potential audience would have been 2,930,000, or 59% of the total number.

There is a large difference between how many people theoretically can see a tweet, versus how many actually saw it/read it, versus how many of those seeing the tweet find it relevant to them, versus how many engaged with it by hitting a link or retweeting. Part of my issue with this is the language we use to report the figures. For the Brew staff to use these numbers to estimate 1.43 million people “had seen tweets about it” is wrong. If they had said 1.43 million people had an opportunity to see the tweet, it would have been more realistic, although still greatly overstating actual relevant audience.

This problem of audience inflation has already been institutionalized in public relations. The use of Impressions as an output metric does not mean a true impression in the branding sense, but rather an opportunity to see the content. To make matters worse, many PR practitioners believe Impressions should be factored by either dubious pass-along readership figures and/or use of a multiplier to account for the mythical credibility advantage PR enjoys over impressions generated from advertising. The simple fact is there is no research-supported, fact-based argument for using any adder or multiplier in public relations when calculating potential audience (here’s an IPR white paper on this subject I co-authored).

For Twitter and other social networks we lack demographics and data about tweet readership averages (i.e. what is the probability that any one tweet is actually read) that would allow for more precise audience estimates. In the absence of data, believable assumptions should be used:

Out of all the opportunities to see, how many actually read the tweet? 10%?

Of those reading the tweet, how many find it relevant to them (or from the other perspective, how many of the readers are in your intended target audience)? Maybe 10% again?

You can see how our audience estimate has already been reduced by a factor of 100. This may well still be overstating the actual, relevant audience. The issue here is that unrealistic and overstated audience figures have the potential to hurt credibility and call into question other data and metrics that may be more grounded in fact. Actually the more meaningful metrics pertain to engagement or outcomes rather than exposure/outputs. It is more meaningful that 40,000 visited the Wordnik website as a result of the campaign discussed in the NYT article than the overstated 1.43 million who were estimated to have seen the tweets. 40,000 is real. 1.43 million is fantasy.

Look back five years and the PR measurement field was full of challenges:

– Emphasis on media relations at the exclusion of other high-value PR activities, almost always
– Oriented toward outputs and not outcomes, consisting
– Primarily of media content analysis, with
– Little primary audience research, and
– No codified thinking on how to approach ROI determination.

Now add social media. Old metrics like Impressions lose meaning. It’s about engagement and not eyeballs. Consumers have broad platforms to voice opinions about your brand. Conversations are more effective than messages. So needless to say, social media measurement is a highly fluid, and rapidly evolving field. Lots of opinions, not much consensus. Here is where I believe we are at a high level.

Early efforts to measure digital and social media focused almost exclusively on web analytics. I would say the majority (80%?) of social media measurement in 2009 still focuses on web analytics, although many other forms of data and research are being used by leading organizations and practitioners.

Today, the frontier in social media measurement is evolving toward measuring the conversations and behavior patterns occurring within social networks. The third area of interest is in tracking and connecting online and offline behavior and actions. Here is a simple graphic (you may have a much better way of showing this) that shows these three primary interest areas, or zones, for social media measurement.

From the left, companies or brands control, own or manage websites – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with. This zone is measured primarily by web analytics. In the middle are the actual social networks and conversations between individuals. In this zone we are interested in data sets that cannot be gathered solely using web analytics packages. How often is the brand being mentioned in conversation? What is the sentiment of the comments? How often is the brand being recommended and by whom? Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone. The third zone represents all the real-world, offline transactions that may be of interest. Did someone visit the store or attend or event? Did they buy a product? Did they recommend the brand or product to a friend over coffee? Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.

Although I have attempted to define three distinct zones of measurement necessary to address the full spectrum of social media impact and ROI, your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones. The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa. It won’t take five years to get there.

MetricsMan is the personal blog of Don Bartholomew, SVP Digital and Social Media Research at Ketchum. Everything posted on this blog is his personal opinion and does not necessarily represent the views of Ketchum or its clients.