Surprise! Your Facebook Videos Really Weren’t Watched That Long

It could happen to any of us. A small error in maths that makes us think we have more money, more time, more space or more anything than we really do. But most of the time, those errors only affect us.

When it happens at Facebook, millions of advertisers are impacted. According to reports in the Wall Street Journal (WSJ), a key viewing statistic was “inflated” as much as 80%. What’s more, some are implying it wasn’t just a simple maths error.

Facebook’s push for video

Facebook has been building its video platform for the last several years. It purchased QuickFire Networks in 2015 to help bring their video capabilities up to speed. Earlier this year they launched live streaming globally, as well as a Twitch-like video game streaming channel.

Facebook earlier this year reported that videos on its site were viewed 8 billion times on an “average day”. That made the online service appear to be a prime target for advertisers using video. After all, if they get just a portion of those 8 billion, then video ads could be very profitable.

The analytics

Here’s where it all goes a bit sideways.

There is no industry agreed-upon definition of what counts as a “view”. As far as Facebook is concerned, it’s anything over 3 seconds. As a point of comparison, YouTube is estimated to use 30 seconds as a cut-off.

The fact that they use 3 seconds is known, but what wasn’t known was how that impacted their other analytics. Specifically, how they calculated “average duration of video viewed”.

It turns out that for the last two years, that average didn’t include videos watched for less than 3 seconds. That might not sound too important, but it can add up to big numbers for advertisers.

That’s because the metric only reflected the time spent by “views”, which by their definition is 3 or more seconds. But there may have been many more watching the video for a shorter period of time.

For example, let’s say you ran a 30 second ad on Facebook. Their metrics showed 100,000 views (to keep the maths easy). And your average duration time was 15 seconds. You’d correctly interpret that as meaning those 100,000 viewers watched about half your ad – on average.

The trouble is, maybe 500,000 people watched your ad. Only 400,000 of them watched for less than 3 seconds. You would never know about those 400,000 based on the data Facebook gave you. But if you did, you’d realise your average duration time was much less than 15 seconds. In fact, four-fifths of the people who saw your ad, didn’t want to watch it for even 3 seconds let alone 15.

Here’s what Facebook said on its business blog,

“The metric should have reflected the total time spent watching a video divided by the total number of people who played the video. But it didn’t – it reflected the total time spent watching a video divided by only the number of “views” of a video (that is, when the video was watched for three or more seconds). And so the miscalculation overstated this metric.”

After the Facebook announcement about the error, Publicis, a major ad buying agency, did an in-depth review of the data. They estimated that the metric had been overestimated between 60-80%.

While they acknowledged their mistake, Facebook was quick to point out no other metrics were affected. And that they are releasing a new metric which will accurately reflect the data.

The impact

The admission has caused many advertisers to review their metrics and buying patterns. They may have made different decisions about where to spend their money if they had accurate data from Facebook. And that could have added up to hundreds of millions in less revenue for the company.

The WSJ reported that Publicis sent a memo to their clients and implied that the new metrics were intended to “obfuscate” the earlier issues.

“In an effort to distance themselves from the incorrect metrics, Facebook is deprecating [the old metrics] and introducing ‘new’ metrics in September. Essentially, they’re coming up with new names for what they were meant to measure in the first place,” the WSJ quoted from the Publicis memo.

Honestly, I don’t know what else Facebook could do besides fix the problem. Perhaps they could have kept the old metric name and simply changed the calculation. But undoubtedly that would lead to confusion later about whether it had been fixed or not.

The bigger issue though, and perhaps the long-term impact of this might be an opening up of their data. I can hear many of you laughing at the idea. Like most big tech companies, Facebook guards its data closely. It’s arguably their most precious resource.

But at the same time, there’s no ability for a business that pays for their advertising to verify the data they receive from Facebook. Publicis’ memo also called for 3rd party verification of the data and processing. “Two years of reporting inflated performance numbers is unacceptable”, the memo reportedly said.

Will Facebook ever actually submit to a validation process, or open up their API to external analytic companies (beyond the few they currently have partnerships with)? They certainly aren’t the only tech company with this type of issue relating to their data – and the trust people put into it.

For now, I’ll bet nothing much changes. If the ROI of video marketing efforts on Facebook ever begins to go down though, the big ad buying companies may decide to spend their money elsewhere.