Thursday, May 7, 2009

IAB's Lost Opportunity for Innovative Thinking on Social Media Metrics

Yesterday, the Interactive Advertising Bureau (IAB) released a report entitled, "Social Media Ad Metrics Definitions." I appreciate the organization's desire to "stimulate growth by making the reporting of metrics more consistent," but the document left me wanting far more. I felt there were a number of potential missteps and omissions, ranging from the merely annoying to the consequential.

The report doesn't so much outline revolutionary new ways of thinking about Social marketing analytics as it does apply existing advertising models to Social Media. With so many seeking answers to the challenges of Social Media measurement and ROI, I believe the IAB missed an opportunity to take a leadership position with innovative thinking on this topic.

I considered if I was being too critical of the report; after all, it mentions "Ad Metrics" right in the title. The IAB didn't label the report as being about "Word of Mouth," "Buzz," or any of the other labels we apply to Social Media. No, it advertised itself as being about "Ad Metrics," and had the report stuck to that topic, then the approach and information presented may have been right on target.

The problem is that the IAB report doesn't just focus on advertising in Social Media; it invites readers to expect more than just advertising thinking. The report opens with a smart and succinct declaration of how Social Media differs from the online media of the past:

"Social Media has added a participatory element where an individual not only receives information but has the ability to take part in the creation and distribution of content. Furthermore, social media tools have enabled a dialogue and discovery around this content. It is the combination of these unique and appealing aspects that defines the true value of social media."

After reading this, I expected interesting and insightful thinking about how to measure "the true value of social media," but I didn't really see it in this document.

The report starts with a list of General Social Media Metrics, and if the IAB recognizes Web 2.0 as a bold new world, you wouldn't really know it from the metrics cited. The list is heavy with familiar metrics such as Unique Visitors, Cost per Unique Visitor, Page Views, Visits, Return Visits, and Time Spent. The future apparently looks an awful lot like the past.

Among these metrics, I was disturbed to see "pop-under ads" listed as a type of content that may be included in Unique Visitor statistics for Social Media sites. Why include this hated vestige of old-fashioned, one-way, annoyance advertising in a document that should set a bold new direction in participatory marketing? After all, it was the IAB itself that in March reported that pop-unders were the least clicked form of online advertising. Had the IAB wanted to demonstrate that it is setting a new direction by embracing the underlying precepts of Social Media--such as transparency, authenticity, and two-way dialog--then Pop-Under Ads should have been mentioned only to encourage their extinction.

Since the report set the stage with its discussion of "participatory elements," consumer "creation and distribution of content," and "dialogue," it was surprising to see how few of the General Social Media Metrics dealt with anything other than presence at touchpoints. Visitors, Page Views, and even Time on Site cannot reveal anything about the "true value of Social Media." That value instead comes from metrics the report buries in a long bulleted list of "Relevant actions taken," including Messages sent, Invites sent, Newsfeed items posted, and Comments posted.

Having suggested these ways of measuring Social Media dialog, the report punts when it comes to furnishing guidance on how to gather that data. While providing definitions of decade-old Web metrics such as Unique Visitors and Time Spent, the report makes no attempt to describe how to measure those "Relevant actions taken." Instead of focusing on the sort of information one might find in the "Help" section of an Omniture or WebTrends Dashboard, this report could have blazed trails by delving into the more challenging, less understood, and more important questions about how to measure relevant actions that occur across a vast variety of Social Media tools and sites.

Perhaps one of the most concerning aspects of this report is that a reader might get the idea that every action is equal. The report suggests that "Comments posted" are worth measuring, but it says nothing whatsoever about sentiment within those comments. In the entire report, the following words do not appear even a single time: "Sentiment," "Attitude," "Rating," "Positive," and "Net Promoter Score." Apparently the IAB thinks that all comments should be tabulated in aggregate, regardless of whether they are disparaging or complimentary.

The report's neglect for the vital aspects of sentiment in Social Media is also evident in the way the concept of "influence" is discussed. "Influence" is introduced not as the capacity to affect desired change in the actions and attitudes of a specific audience, but instead is defined as "Average number of friends among users who have installed application." The size of one's network is important but hardly is a measure of influence. Think of it this way--if you wanted to change the attitudes of technology decision makers, would you focus on actor Ashton Kutcher or Michael Arrington, founder of TechCrunch? According to the ridiculously narrow definition suggested by the IAB report, Ashton's 1.6 million followers give him more influence on technology (and every other topic) than Arrington, with his meager 550,000 followers.

The report contains other dubious contentions. For example, it is suggested that a blog author's credibility can be ascertained by evaluating attributes such as the length of time an author has been posting on a topic and the number of links he or she has to sites with "conversation phrases from the client’s RFP or IO." To me, these metrics suggest an author's pertinence to the desired topic but say nothing of his or her credibility. Credibility may be ascertained by tabulating the quantity of readers within the defined target audience, number of Diggs and inbound links from high-quality sites, and ratio of posts that are unsponsored versus paid advertising, as well by more qualitative means.

I came to the IAB report with a great deal of interest and high expectations for the knowledge and direction it would provide on the vital topic of Social Media metrics. The fact it fails to advance inventive and farsighted thoughts on the difficult subject of Social Media metrics is a tremendous lost opportunity for the IAB, marketers, publishers, and Social Media sites. I surely did not expect them to have all the answer since the world of Social Media is still young and rapidly changing, but this report did not come close to outlining the kind of thinking around which "all players in the Social Media space will coalesce."

2 comments:

I agree that they missed an opportunity, but it's understandable. Their perspective is advertising. Heck, it's their middle name. And although the power of social media is weak, it is more akin to direct response than it is to advertising.

How weak is it? We're only able to do some triangulation, not direct measurement. But our guesses can be informed by research being done with how memes spread through a network.

For what it's worth, I posted a blog entry about some of this research just an hour ago. It explores what network research can do to help us attribute lifetime value to customers who actively endorse a brand on a social network.

About Experience: The Blog

The world is changing rapidly, both for consumers and brands. Consumers are more empowered than ever before and traditional business models are under attack.

In an increasingly social, mobile and real-time world, brands are created not by the messages they broadcast but by the experiences they offer--ones that create empathy, build trust, earn loyalty, spur Word of Mouth, encourage collaboration, and provide ever greater value to customers in innovative ways. On this blog, we explore how brands are built and business improved via Customer Experience Management, purposeful corporate culture, social and mobile business strategy and collaborative economy models.

You are welcome to participate, criticize, praise, critique, expand, or correct the information and opinions found on this blog. Spam, off-topic, or crude comments will be deleted, but all others are welcome.

About the Author

I am Augie Ray, Research Director covering customer experience at Gartner. I conduct and publish research and advise Fortune 500 clients on the value, process, measurement and tools of customer experience. This includes topics such as Voice of the Customer (VoC), personas, customer journey maps, CX governance, and customer experiences metrics that are leading metrics of brand success.

Previously, I was Director of Global Voice of Customer Strategy for a Fortune 100 financial service company. My background includes more than 20 years of experience in digital, brand, customer experience and social media.

In the past, I led social business at USAA, a firm recognized for its innovative use of communities and social customer care within the financial service industry. I also consulted and published analysis as a Forrester analyst covering digital marketing and social media. In addition, I led a diverse $9 million agency team with specialties in digital development, digital experiential marketing and community strategy.

The future will bring a great deal of innovation that offers opportunities to organizations that are agile and willing to cannibalize their own business models (but it will severely challenge those organizations that cannot.)

The views expressed on this website/blog are mine alone and do not necessarily reflect the views of my employer.