Nate Elliott's Blog

Our recent report on why Facebook is failing marketers has caused quite a bit of conversation — with some supporting our findings and others disputing them — and we think that’s healthy. We fully stand behind our data and our conclusions, and we welcome the chance to further discuss what’s working and what’s not working in social media. Conversations like these can only push the industry forward and help all social marketers and sites become more successful.

In particular, we wanted to address a few common questions people are asking about our research:

Facebook’s score didn’t look that low. Are they really failing marketers? Facebook offers marketers access to the largest audience in media history and it knows a remarkable amount about each of its users and their affinities. By all rights, Facebook should be driving significantly more value for marketers than other sites and channels — but according to our survey, they’re not. Forrester’s Data Center of Excellence has looked at this data many different times, through many different lenses, and every view of the data supports this conclusion.

Who exactly are these 395 marketers and eBusiness executives you surveyed? Put simply, they’re the buyers Facebook cares most about. They all work at the manager level or higher in interactive marketing or ecommerce roles at companies with more than 500 employees and more than $500 million in annual revenue. They all have knowledge of their company’s marketing budgets, and they all reported that their companies spend at least $500,000 per year on digital marketing or on eCommerce technologies and services.

Why did you only ask about marketers’ “satisfaction” with Facebook, not actual ROI? Because not every marketer uses the same definition of ROI. Some Facebook marketers tell us they’re trying to build greater brand awareness or preference; others tell us they’re trying to drive direct leads and sales; and still others say their goal is to increase customer satisfaction and reduce churn. With marketers targeting such a broad range of objectives, asking them about their satisfaction with the business value Facebook provided was the best way to represent all their responses on a single scale.

Comments

Thanks for clarifying in (2) that the marketers in your survey are the buyers that Facebook cares most about. The critiques your piece has received from within the marketing community have come (inter alia) from people expressing their personal or professional opinion but having access to a good deal of data that informs it (such as I have, or the folks at Nanigans do) but often they're constrained in being able to call out client-specific examples so it's important for our industry that we can all understand the methodology behind these looks at the market - we don't all have to agree with the conclusions or the process of course, the discourse is valuable.

I've seen now first-hand the tremendous effort and care that Forrester analysts put into their research, like the Waves, and I know too from experience how easy it is for people to Monday-morning quarterback the hard work you guys put in.

That being said - it sounds like you want to be more transparent about your methodology, which I applaud. Thus I'd request the following:

1. Can you confirm where the "79% of companies market on Facebook" stat comes from? Is it the unduplicated % total of the answers to the question "Which of the following Facebook marketing tactics does your company use", in Figure 1? The top response there is 72% saying they "Post updates to our branded page", so I presume that it's mostly that but it would be great to just confirm this figure, also since this was the lead-off to the PR push that you guys did for this report.

2. This is the most important one since this is the data that your argument seems to hinge on. Figure 2 ("Facebook Creates Less Business Value Than Any Other Digital Marketing Opportunity") - can you provide the full breakout of the data including the n's for the different responses. I'm assuming not everyone could do every single thing, so there would be a number of unchecked boxes, or "N/A"s or "don't apply" for some of them - we're all aware not all companies offer online ratings or reviews on their websites, or use branded communities (which I'd guess is low in penetration?). This would be important to be able to determine statistical significance - though of course also being able to know how many people filled in how many of the 13 different responses would be helpful too (since presumably you may be more interested in companies doing multiple things and comparing them to one another, though the data may be thin for that kind of analysis). I think we can set aside the fact that the categories are not mutually exclusive and collectively exhaustive (e.g. mobile marketing & word of mouth marketing are confusing relative to Facebook/Twitter/LinkedIn).

3. Can you provide details on the incentives that were offered by e-Rewards / Research Now for participation in this PARTICULAR survey, as well as the total compensation that these people receive per year from surveys? I'm sure this might be difficult to come by but it looks like Research Now does provide some of that at an aggregate level.

4. Are the folks in your sample at enterprises, or agencies that operate on behalf of enterprises? Does your survey partner validate that these people work for those companies? Any more information on the screening process I think would go a long way towards assuring people on the quality of the sample here. Also perhaps a description of any other 'filtering' that was done on your end since it doesn't seem like this is intended to be a random sample, which is fine, but then I think we all need to understand a bit better what's behind it.

Beyond that, the level of detail about that we've provided about our samples, data and methodology in this report is consistent with what we provide for our other published research. As you yourself note, we take tremendous effort and care with our research. This effort and care has been the foundation of Forrester's business for 30 years, and means we're satisfied with our data and stand behind this research and these findings.

Justin - I actually think marketer surveys and opinions can be valuable too, because it is not just about performance but also about how the marketer feels about the medium, does it make them feel like they are doing a good job for their customer/client.

I know this sounds odd - and ideally the marketing channel works well AND makes the marketer happy - but for a period of time (could be a long time actually though not indefinitely) "feeling good" about a channel is actually an important metric.

Jason Keath said it well elsewhere that it is better that people complain about a medium than ignore it!