Facebook’s fake news algorithm seems to be working

Publishers are doing... fine?

Facebook’s January 12 announcement that it would begin to deprioritize news in users’ News Feed left publishers shaking in their boots. “[B]y making these changes, I expect the time people spend on Facebook and some measures of engagement will go down,” admitted Mark Zuckerberg, much to the horror of every major media outlet, most of which relied heavily on the traffic generated from the site. And for a while, it truly did look like the apocalypse was nigh: The Outline’s investigation from early March showed that traffic for most conservative publishers and nearly all publishers of viral and needlessly polarizing content experienced a significant drop in the month following the News Feed change.

In the wake of Newswhip’s recent analysis of top publishers’ Facebook engagement data over March and April, many have come to the similar conclusions of partisan bias (though the winners and losers often switch, depending on who’s talking). However, new information that takes into account the last four months as a whole — rather than merely looking at month-to-month trends — tells a much different story.

According to data The Outline obtained from research tool CrowdTangle, a subsidiary of Facebook, Facebook’s January news feed algorithm change has had little to no effect on mainstream conservative and liberal publishers in the long run, with most actually experiencing increased interaction rates following February. However, publishers of clickbait, purposefully polarizing content, and/or blatantly fake news have experienced a significant sustained drop in interaction in the months following Facebook’s January News Feed deprioritization announcement.

The Outline came to these conclusions after analyzing the Facebook interaction rates of 20 publishers from November 1, 2017 to April 20, 2018. CrowdTangle calculates a particular Facebook page’s interaction rate by dividing the average number of interactions (i.e. likes, comments, shares, etc) in a given time period by the size of the account.

Gateway Pundit

Gateway Pundit interaction rates, Nov. 2017 - April 2018.
Crowdtangle

Breitbart

Breitbart's interaction rates, Nov. 2017 - April 2018.
Crowdtangle

PragerU

PragerU interaction rates, Nov. 2017 - April 2018.
Crowdtangle

Pages known for publishing inflammatory ultra-right-wing content like Gateway Pundit, Breitbart, and PragerU all experienced considerable sustained drops in interaction following the News Feed change. Jim Hoft’s right wing blog, Gateway Pundit — known for its love of blatantly inaccurate content and completely unfounded conspiracies — experienced a 69 percent decrease in user interaction in the months after Facebook’s announcement. Similarly, the popular right-wing online “university,” PragerU, experienced a roughly 42 percent decrease in interactions over the four month period, and Breitbart’s interaction rate dropped by 30 percent.

Fox News

Fox News interaction rates, Nov. 2017 - April 2018.
Crowdtangle

CNN

CNN interaction rates, Nov. 2017 - April 2018.
Crowdtangle

New York Times

New York Times interaction rates, Nov. 2017 - April 2018.
Crowdtangle

Washington Post

Curiously enough, mainstream publishers from both sides of the aisle did not appear to experience similar interaction rate decreases in the months after Facebook’s news feed deprioritization announcement. Though a few companies had particularly low rates for the month of February, most recovered relatively quickly and went on to have sustained growth throughout the following two months.

Though Fox News experienced a dip in interaction during the month of February, it returned to a normal rate within four weeks and continued to rise throughout April. CNN’s interaction rate stayed at a steady 0.03 percent throughout the four month period. Similarly, the New York Times’ interaction rate remained at 0.02 percent from January to April. Though the Washington Post experienced a brief spike in March, its interaction rate returned to 0.04 percent the following month.

It’s worth noting that Facebook does not make information like reach and clicks available to the public, so interaction rates do not contain that data. The interaction rates cited here were generated from share, like, and comment data provided by CrowdTangle. The term “engagement” is commonly used in reference to shares, likes, comments, and link clicks.

“We have found that, in general, the more folks interact with a post, the more likely it is that they are also clicking through to it,” CrowdTangle explains in an FAQ on the subject. “However, that's not always the case (sometimes people like a post that they don't click through, and sometimes people click through to a post they don’t like). In general, shares are probably the best proxy. The social network APIs currently don’t give public access to clicks or reach, so they will not be displayed within the dashboard.”