Jason Falls blog, Social Media Explorer, is right near the top of the list of my favorite social media blogs. Jason had a post last week about social media measurement that led to a pretty lively discussion in the comments, on twitter, and on a number of blog posts that linked to the original post. For some reason, I can’t seem to get this one out of my head…there was a lot I agree with in the discussion, but also a number of things that just don’t ring true to me. In no particular order, here are some of my thoughts:

Like it or not, ROI matters

Based on what I saw in the conversation that followed Jason’s post, people’s views about ROI measurement seem to fall along a continuum. On one end is the group that argues that measuring ROI might be hard, but it’s not impossible, and, given that the end goal of social media participation is to grow the business, social media marketers are either going to figure it out or they’re going to get screwed. Dan Thornton makes the argument pretty effectively in his comment when he says that engagement metrics are important, “but you still need to figure out where engagement sits for the rest of the business, and how it’s integrated into other areas. If it’s contributing to natural search results, for example, then without any measurement of other outcomes, the results are all attributed to SEO work, and engagement is disregarded.”

At the other end of the spectrum is a group that, at best, is ambivalent about the idea of trying to measure social media ROI in terms of financial metrics. Shannon Paul’s comment is indicative of this:

“I understand that businesses make decisions based on the bottom line, but isn’t social media engagement all about humanizing organizations? Ultimately, businesses are made of human beings and most human beings I know are motivated by a number of things in different measure — profit is only one such motivator.”

Shannon is another one who’s writing some really good stuff on her blog, but I’m with Dan on this one (just in case the title of the post didn’t tip you off). Peter Kim summed up my feelings about this best when he said “I have and will always believe that the purpose of marketing is to sell stuff, whether direct response or 30-year sales cycle. Marketers who don’t believe that their job is to ultimately sell something should become receptionists instead, if all you want to do is talk.”

Yes, metrics that indicate the quality of conversations are important and, yes, people should get more out social media participation than financial gain. Frankly, I don’t think you can be successful without these things. But at the end of the day, if participation isn’t going to result in revenue for the business, the initiatives aren’t going to get funded and it’s all for naught. Marketers can get away with this now because we’re still deep in the early adopter phase, but this won’t last long. Particularly in this economic environment, companies are going to move quickly from exclusively measuring things that indicate level of participation to measurements that tie to revenue.

You can’t determine the right metric without first identifying the goals

Katie Paine talked about letting your objectives drive your choice of metrics in the video interview that accompanied Jason’s post (watch this video…the ROI on the 11 minutes you’ll invest to watch it is very high 🙂 ). Paraphrasing her comment, “in order to determine ROI, you need to know what the R is.” In other words, you need to decide what you’re trying to achieve. I think this is exactly right and it a point that gets missed often. The metrics for measuring word-of-mouth effectiveness, for example, are going to be very different than the metrics for brand loyalty.

Incidentally, as part of this discussion, I’ve seen a number of tweets/comments saying that social media isn’t for attracting new customers, it’s for building relationships with existing customers. Maybe I’m just misunderstanding what people are saying because, at face value, this doesn’t make sense to me and there are plenty of word-of-mouth case studies that refute it. Some examples: 1) NetQoS’ viral video campaign – two of the primary goals were to increase traffic and drive leads. The net result of their campaign was that it added $500,000 into the pipeline, 2) Caminito Steakhouse, where they’ve seen a 30% increase in sales concurrent with a significant improvement in search engine rankings on key terms…they haven’t drawn a clear line that shows the link between participation, improved PageRank and increased revenue, but I guarantee you that it wouldn’t be hard to build a model that shows clear correlation, and 3) Martell Home Builders – take a look at the comment from Pierre Martell (the owner of the business). Lead gen is a key part of their strategy and according to Pierre, “from an ROI point of view, because of the real estate fees were saving, it didn’t take many sales to justify this approach from a pure dollars and cents point of view.”

Traffic still matters

Katie argues that traffic doesn’t matter. I disagree. By itself it doesn’t, but in conjunction with other metrics, I think it’s still valuable. Give me two blogs that are equally relevant to my customer, have the same average number of comments, have the same PageRank, etc. Are you telling me that, even if one of these blogs has twice the traffic as the other, it’s no more valuable for word-of-mouth than the other?

Small and mid-size businesses have different measurement capabilities and needs than big businesses

Note the mention of ROI in the NetQoS, Martell Homes and Caminito Steakhouse word-of-mouth case studies. For all three of these SMBs it just isn’t that difficult to measure ROI because the marketing mix isn’t that complex and the customer touchpoints are easier to track. Very different than a big company, where measuring ROI requires fairly complex modeling since there are so many more possible drivers of revenue. Do the metrics that the small business uses encapsulate all of the benefit provided by social media participation? Definitely not, but it doesn’t matter. Despite the fact that some of the value generated doesn’t get captured by the metrics (e.g., for NetQoS, prospects that become aware of the company as a result of the video campaign, but visit the site through organic search), there’s still enough measurable value to clearly justify the investment.

Measurement on the front-end is very different for the small business as well. Big companies might think it’s important to conduct detailed analysis to determine influence, but small companies have neither the time or money for this. While traditional metrics have problems, they’re simple and, when combined with engagement metrics, they’re good enough for small businesses.

So what does your social media dashboard look like? What are your social media goals and what are the metrics that you track most closely to determine your success?

15 comments

@Britton – thanks for joining in. I can see how bundling it into a broader package of activities might reduce some of the pressure, but I’m not sure it solves the core problem that many people are facing in their dealings with large companies. Specifically, large companies look at it as an issue of opportunity cost. I.e., they’re trying to decide whether the return is worth the required effort, given that they’re weighing it against traditional marketing tactics. Given this, the options for the social media marketer are to either face the ROI question head on or try to move the C-level audience toward a different set of metrics (which I think is going to be a tough row to hoe).

There’s a really good comment discussion about this in the Peter Kim post that I linked to above.

Bravo! You’ve generated quite a vibrant discussion. I’m with you on the necessity of measuring ROI. I just wonder whether social media interactions should be the locus of our metrics or we should simply bundle our activities into larger endeavors that lend themselves more easily to measurement. I guess that’s not so helpful for a “pure play” social media service provider, but I’m thinking it’s best — in many cases — if social media activities are an embedded input in, say, a lead generation program (justified by the production of sales-ready leads) or a more conventional PR program (with conventional PR awareness metrics) rather than a vulnerable line item that stands alone and must be actively defended.

@Paul May, in response to your Q about retention vs. acquisition…I’ve used Social Media for retention measurement in terms of customer service in the following ways: reduced inbound calls to a Help Desk, reduced email costs and reduction in bums in seats at inbound centres. For SMB’s they can use Social Media tools to move the customer service to the customers (loyal customers answering questions via Blogs or Wiki’s as an example.)

I think we look at “marketing” too narrowly. In it’s truer sense (as per Drucker) marketing is every outward facing activity of the “business” as a whole. Every point of contact with prospective employees, media, customers and suppliers (all the stakeholders) is “marketing” since it lends to building the Whole Business. This is a more holistic approach.

For the first time ever a business can truly market itself in all aspects. If a business is not marketing itself, it is not a business.

@Webconomist –
Great addition to the conversation. I’d be interested in understanding why you view retention as the ROI sweet spot, rather than acquisition. This seems to be a pretty common meme, so it would be good to better understand what you mean. First, what types of companies are you referring to – established brands or SMBs? For this group, what is it about retention that makes it easier to demonstrate ROI than acquisition?

@Marc Meyer –
Interesting to view the distinction as one of ROE versus ROI. Captures the distinction between putting a dollar in and expecting X dollars out as opposed to expecting a return from equity built over time.

I still think the importance of this distinction really depends on the goals (per Peter’s comments) and the size of the company. The smaller the company and the more the goals look like traditional lead gen or direct marketing goals, it seems to me that the less problematic measurement is (even though social media is not transactional).

@Tom – On my way to take a look at your analysis and post.

@Jeanne – glad you found us and thanks for your comment. We’ll try to keep it coming!

Fantastic blog-> Found you through Twitter- content rich and well written! I’ll look forward to reading your updates to follow as our Real Estate Team in Columbus, OH is keenly interested in all aspects of the social media world. Also, just a quick note to say that we DO use social media venues to keep up with old/current friends and clients and make new friends on the web. Why else would we invest so much time and energy?
Best~Jeanne

ROI matters, but the problem is, social media right now is not transactional. Eventually we could see some type of premium put on everyone’s conversations or the ability to participate IN a conversation but right now social media is not a black and white e-commerce function. It’s all about ROE.

True you have some social media business models that exist that swhirl around “more access” or more tools, but for the most part, it is still Ad driven with a a focus on data mining. I think most companies are wary of, or are of the mindset of… “been there done that” when it comes to strappin their wagon to that type of business model.

Totally agree with the comment on marketing is about selling. And Social Media is a marketing tool, and it should contribute to revenues.

We’ve found success in measuring in terms of generating leads (and measuring them) and also in “loyalty” by engaging customers to reduce the cost of service and improve recurring purchases – and this is where I think Social Media can hit the sweet spot on the bottom line – retention more than acquisition.

Social Media can keep the conversation going between the campaigns, building an active customer base and driving referrals and repeat sales…great post!

@Ben and @Allen –
Glad your enjoying the content and thanks for stopping by. I appreciate the kind words.

@Shannon and @Peter –
Ah, that definitely clarifies things. Shannon, it sounds like for both you and Jason it’s really an issue of practicality and expectations, rather than one of “guiding principals.” Thanks for clearing this up.

I wonder how much of the unfair expectations are a result of the newness of social media marketing versus the fact that there are more measurement opportunities/tools in the digital world. I.e., does the fact that digital provides so much more visibility into visitors, traffic sources, etc result in an unrealistic expectation when it comes to measurement)?

On a different note, it seems to me that the SMB versus big brand distinction that I mentioned in the post is even more relevant than I’d thought. The differences that I’m seeing:
– SMBs typically have a much less complex set of objectives…primary focus is lead generation, with product management being a close second…there’s very little focus on brand building,
– Lead gen lends itself to measurement much more easily than brand marketing
– less focus on precise measurement because of time/money constraints and because there are no real competitors to social media marketing for the SMB…i.e., SMBs don’t have to worry about the social media scalability problem, and nothing can compete with the cost effectiveness of social media.

This is quite a good summary of much of the discussion regarding the ROI of social media lately.

Let me please just go on the record as saying that I am nowhere near what I would consider “ambivalent” toward ROI measurement as it pertains to social media marketing. I work every day to make sure that my efforts benefit the bottom line of my employer. I just think that the traditional methods of ROI when it comes to marketing, and especially PR, have always been nebulous and vague. Social media didn’t start this problem, yet proponents of social media are always pressured to solve it.

I think measurement is and has always been more complicated than simply equating activities in terms of dollars earned. Sometimes it’s evaluating cost savings and expanding share of voice. Yes, profits and increased market share should always be measured in correlation to any social media objectives, but simple 1-to-1 metrics hardly ever tell the whole story.

I met Jason last week and we discussing the ROI issue – it strikes me that social media suffer from the same precision double-standard as digital media in general. That is, while traditional mass media are allowed to use assumptions and approximations to gauge effectiveness, emerging media must provide iron-clad proof of success. Not that knowing a precise answer is a bad thing, though.

If the world of marketing measurement breaks down into brand vs. direct, I’d say digital tactics fall pretty cleanly along those same lines.