Good PR Measurement and Delivering Bad News

05 Oct 2008

I’ve just finished preparing a lecture for my upcoming class this week and have been culling some wonderful information from Katie Delahaye Paine’s book, “Measuring Public Relationships.”

Among the absolutely useful and easy-to-understand advice Katie offers are nuggets like:

AVEs – Advertising Value Equivalents are a bad measurement of value because, among about 50 other reasons, you can’t compare apples to oranges: As Katie says, “..there is no scientific evidence to demonstrate that a six-column inch ad has the same impact as a six-inch story in the same publication.” Amen.

There are indeed other valuable, albeit not perfect, ways to measure the impact of, well, impacting relationship with stakeholders like CPMs (cost per thousand impressions – and maybe someone can explain to my mathematically-challenged self who the genius was who thought to throw a “1,000” in the formula, and CPMCs – Cost Per Messages Communicated (better) that is based upon message impressions, rather than article impressions.

Measurement is wonderful, and in the field of public relations (NOT ADVERTISING, NOT MARKETING) something that I consider to be an evolving area. But here’s the rub:

Too often than not, I have seen fastidious and excellent research carried out (usually internally and not paid for through a vendor) that absolutely contradicts the thinking of a senior executive or company leader. And I have died a little internally when I have seen this wonderful research get treated like CIA secret documents headed for the burn bag.

What to do then? Katie mentions, importantly, to run the internal traps before planning a research program, but I have often seen that senior executives are fascinated with research — until it goes against their thinking.

Mark

Share this:

Like this:

Related

Comments

Can you not do research that supports anything? Everyday we see new studies on everything from Global warning, to exit polls to Wall Street. Some very convincing, some not very convincing. Some scientific. Some not so scientific. If I lead a team and very clearly understand my boss’s philosophy or my clients philosophy, can’t I do research or present the findings of my research in a way that 1) Satisfies the “boss” and 2) Pushes for larger, more expensive, more profitable campaigns for my firm. How can a client trust the research? As a PR professional, I clearly understand you need the research to guide messaging, objectives and tactics, but whether those meeting objectives will require $150,000 or $1.5 million are really up for debate when dealing with major clients, right? Just curious where the ethical line is drawn? I assume it’s push for larger and let client decide where to stop, that’s just good business …

I loved one of the last comments “but I have often seen that senior executives are fascinated with research — until it goes against their thinking”
How true could that be in some situations.

I think its interesting at the same time to find people who say “our clients are this type of person, from this background, etc. etc.” yet if you were to say “how do you know?”…they wouldn’t be able to give you a factual answer.
Research helps me daily in deciding where to place ads, to know what is effective, etc. It is hard to have data sitting on my desk telling me what works and who our clientele is – but then because of “what we’ve always done” we don’t always listen to the research and use our resources 100% effectively.
Does anyone else have a department that has a hard time trying something new after research tells them they should lean another way?

Data-driven approaches to measuring PR are new concepts for me. As a PR director for a small non-profit (many years ago), we set goals and employed strategies to meet those goals. If the goals were met, we believed we were successful. If not, well…”if at first you don’t succeed, try, try again.” I think we would have been much more effective and efficient if we’d utilized measurement strategies (if only we’d known). However, while the strategies are quite detailed, they are not always exact–which, from what I’ve gathered, needs to be taken into consideration. One thing K.D. Paine mentioned in her book about AVE calculations served as a “heads up” to that: “On average, 10 to 20 percent of earned media coverage is negative, yet most companies include this coverage in their AVE calculations, even though it would not be the sort of coverage one would want to purchase or would appear in an ad.” In this instance, should I be looking at the “outcome” vs. the “outtake?” Does the same hold true with CPMs?
So much to learn.

Since I have read Katie Paine’s book “Measuring Public Relationship”, I do realize about how to measure the goals if I can’t measure them. One thing, I like in Paine’s book is that she mentions about output, outtake, and outcome in every capture which I thing it is the most important for measuring strategy.I agree with katie that “a six-column inch ad has the same impact as a six-inch story in the same publication”. I believe that they have to go together in other to measure the value of goals.

Measurment is wonderful when you really know how to measure!! I worked for a pr firm that would tweak the results a lot to make it seem like it was a huge hit when in reality it wasn’t. We would receive media clips about our client and as long as they mentioned our client that was enough. We would then use the AVE formula on each clip and exaggerate the ad value. In this case we were just looking for quick results and not looking to build trust or maintain a relationship with the public.

On the point of research I think it shouldn’t matter whether it contradicts the thinking of a senior executive or company leader because if it is good research it should be kept. Good research is not based on the opinions of executives it is based on facts and only facts can speak the truth to the public.

I believe that research and measurement is a great tool to be used before, during, and after a campaign. It can be used to your advantage whether it supports or denounces your plans and opinions. It can be used to improve the public relations efforts, and possibly make it successful. A senior executive or company leader will always have profits in mind, so anything that goes against it is bad in their books. This is where persuasion is important, and learning to use the research and measurement to your advantage, to explain and persuade the senior executive or company leader on what course of action is better, even if it means higher costs because with measurement data, the long-term impact (and hopefully increased profits and success) can be better supported.

After reading this book, I had a couple of epiphanies about implementing some of her ideas with my staff but in a training aspect. What really stood out to me was the comment about not being able to measure something after the fact since you have nothing to compare it to to see if your results were up or down. I see that happen way too many times, especially in training. Great instructors will get the assessment of the student BEFORE they start the course rather than afterward. It only helps the instuctor know what areas they need to focus more on and the same holds true with public relations and its impact. Another great point that mentioned how companies will pay hundreds and thousands of dollars for advertising and a measurement program only could cost them less than $10k. This book really focuses on how we need to start with basic understanding of measurement in public relations and how it can pay for itself in the end.

Not only do the senior executives put this information in the burn pile, but they find reasons and excuses for the results. I have been in situations where the research/researcher has been criticized because the results were not what they were seeking. How do you deal with this? Are we supposed to aim for the research results to please these people or educate them?

In Measuring Public Relationships, Katie cites Thomas Paine: “It is error only and not truth, that shrinks from inquiry”. So an excellent research, based of facts and driven by common goals should pass the test of contradicting the executive’s opinion. Just as some of my colleagues, I believe that persuasion plays an important role in defending the results of your research against the senior executive or company leader that always thinks in terms of profit. In the end, one of the goals of your research should be about finding a path to increase the profits – one has to persuade the boss that even if the research is expensive and it may not provide immediate profits, on the long run, researches have the means to increase profits because they are continuous improvement processes.

How on time and on point was this book as it relates to what I do at my 9-5!! I love the way Katie broke down the different ways and means of measuring data.

Just a week ago my boss asked me to prepare a ‘final report’ showing the outcome of this summer’s internship program through my firm. When I took on the challenge of reaching out to students throughout the Metropolitan area to pique their interest about the CityCenterDC project, I took the more traditional approach (fliers, web postings, career fairs and presentations to schools), as well I should have.

I knew the response I received was considerable compared to last year’s results (based on the applicants)only I wasn’t sure how to ‘measure’ my findings other than creating a visual graph (cheesy and elementary I know). Completely ignorant to the proven methodologies of measure as Katie breaks it down in her book I realize I have a lot more to learn than I imagined; I’ve barely scratched the surface.

Who knew there were thorough, specific and more effective ways of measuring data, especially in PR!? This is information I can definitely use to my advantage in various ways.

Sorry I digress…but I can certainly understand Mark’s point about senior execs being fascinated with the research until it goes against their own thinking.

I’ve seen this display in many meetings.

I’ve seen (in my firm) the need and desperation for good communal relationships but the “higher-up’s” can’t bring themselves to be completely honest with those in which they seek those good communal relationships.

As long as they receive feed back, be it good or bad, they’re satisfied; as long as a buzz is generated and people are talking they’ve essentially accomplished their goal. I’m curious to see how that tactic will fair in the end when the project is complete and ‘support’ from the community is sought yet again…

Lets be honest. In todays job market, people want themselves to look good. I think the reason why most PR practitioners are afraid of measurement is not because of the extra work, but because a good program will likely reveal the TRUTH! For good senior executives, this will be worthy information which could help decide whether to alter or improve their PR practices. But to the PR practitioner, measurement results could expose their strategy flaws. For small agencies or departments, this is not good because the unsavory results may be blamed on one individual. But overall I love the book. It does get a little “math-y” at times, but the step-by-step process is great info!

To begin with the usual boss’s reaction, yes, it’s really frustrating the fact that a hard-worked research may end up in a trash can just because it doesn’t satisfy the expectations of the boss. To me it sounds like avoiding the truth.

As far as the measurement process is concerned, it’s clear that AVE is not a quite satisfactory method according to the fundamental differences between Public Relations and Advertisement we learned, but the other measurement methods seem over-simplified as well.. Well, maybe the CPMC method can give us some actual results regarding the amounts we pay related to the amounts of messages we communicate, but those results will be strictly quantitative. The quality of the results caused by our communication (that means the impressions in recipients’ minds), at least from my point of view, cannot be estimated that way, it’s almost impossible to capture public opinion. I believe that this could only be achieved through interpersonal communication thus, inevitably, in a small scale sample.

I completely agree with Joe. Overall, you can disput the results you receive or what measurement service you want to use, but in the end, it is all about the results and how it will help your PR plan. You have to figure out what message you want and who your target audience and then it is imperative to implement a way to measure your message. If you receive positive results, then good job – your message is in line with your goals. If you receive negative results, you have modify your plan.

The key also is to have everyone from the senior executives to the administrators know your goals and the message you want to portray. It is important to have everyone on the same page so you know what to do with the results once you receive them. It is a learning experience but you have to come up with a way to measure success of campaigns.

I have read, underlined, re-read, thought about examples, and so on in regards to Katie Paine’s book. I kept trying to connect what she explained to my old job at a small non-profit. I looked at the web links she included and in the end, had difficulty trying to imagine my old boss allowing me to do most of this. Yes, she would want me to research and follow up on articles printed, but I could practically hear her yelling, “I don’t want you to waste time on ______…. but do _____.”

Katie’s explanations are perfectly valid, but I think that a communications practitioner would better understand (or maybe I should write, “listen to”) her than an executive who wants to see his/her name in the media and see rising profits.

What did help me to understand most of what Katie discussed was using wal-Mart as an example due to its 1,200-1,400 media impressions per day. At times I found it difficult to apply what she discussed to small non-profits, something that I was most accustomed to.

I enjoyed reading the earlier chapters (1-4) rather than the latter ones and felt I could understand them more easily. In fact, I have an interview tomorrow for a potential job and plan on asking most of the questions Katie listed in, “Ten Questions Every Communications Professional Must be able to Answer” in order to gain a better understanding of the organization and its communications department.

On a different note, I guess those who chose communications as a profession to stay far away from math will eventually have to face numbers and equations again.

I agree that it’s better to measure your efforts and results in PR field, just like most of the business do. No matter how mavericks some PR professionals may be, I think this field is too broad for them to make judgements entirely based on “feelings”. I guess while many of the companies do agree on the measurement idea, what’s holding their backs are the “extra” efforts they have to put in; and honestly, research and measurement did require a lot of efforts, budgets and time. But I appreciate what Katie’s trying to clear that, it’s not something you consider to be extra works; instead, PR measurement should be considered a required foundation of your PR business. I believe that it’s also one of the solid building blocks for a successful long-lived PR firm.
And I definitely think researches should be carried out based on facts rather than on the favorability of the C-suites. Of course, like Katie emphasized, the measurement should be conducted before knowing the goal of the company. But it doesn’t have to be something that the leaders love to see. Because then it totally kills the original purpose of a research: to help company grow and improve. I think the trend nowadays is to challenge the head person (in respectful way of course) rather than catering to them. And the solid proof of researches with those clear numbers are definitely good supports for PR professionals to persuade the leaders. Those leaders should also respect the value of good researches and recognize the salience to take those researches into account.

In the non-profit organization where I intern, research is essential in planning our conference and seeing the demographics of the people who attended the previous years so we know where we need to spend our time advertising. We also use the research in order to see who where our donors and sponsors that we may be able to count on in the future.

I don’t feel that the research should go to waste; I feel that there could be a way where one could sway the boss and show them why it is important.

I love this quote by PR consultant Claire O’Sullivan: “Using an AVE to measure how effective PR has been is like using a thermometer to measure speed – it’s the wrong tool.” I think this sums it up perfectly. Advertisements and articles are perceived differently by readers – one having made its way on merit and the other by paying its way in. They perform different functions and are not comparable.

Katie Paine hits the nail on the head when she says, “..there is no scientific evidence to demonstrate that a six-column inch ad has the same impact as a six-inch story in the same publication.”

Many PR agencies rely on AVE, perhaps, for its simplicity and relatively low monetary value. It may create an impressive figure to show clients, but is it actually indicative of its success?

Relying heavily on AVEs as a tool for measuring effectiveness may not be in the best interest of a PR firm – at least not in the long run. I don’t know enough about CPMs to comment on its effectiveness. But I don’t think there is any one single evaluation tool that can measure the effectiveness and success of a campaign – especially in today’s rapidly evolving new media scene.

While some agencies suffer from distorted perceptions of success, others do not even consider evaluating and measuring the effectiveness of their campaign – like the small non-profit I once worked for. I think the biggest obstacle here is that measurement and evaluation are often an afterthought, an option – not a necessity.

One of the most interesting things within PR is measurement and its importance now days. As Barbara mentioned during our last class, clients are becoming savvier and they want to measure efforts and results. But how much of this valuable information really gets used on future campaigns? We keep seeing how big corporations are doing more and more research just to keep it and said they did the homework. Maybe we don’t longer measure using the pile of media clippings, but the clients approach to measurement is exactly the same as it was: quantity versus quality.