Caught in the Net: The Impact of ‘Extremist’ Speech Regulations on Human Rights Content

San Francisco – Social media companies have long struggled with what to do about extremist content that advocates for or celebrates terrorism and violence. But the dominant current approach, which features overbroad and vague policies and practices for removing content, is already decimating human rights content online, according to a new report from Electronic Frontier Foundation (EFF), Syrian Archive, and WITNESS. The report confirms that the reality of faulty content moderation must be addressed in ongoing efforts to address extremist content.

The pressure on platforms like Facebook, Twitter, and YouTube to moderate extremist content only increased after the mosque shootings in Christchurch, New Zealand earlier this year. In the wake of the Christchurch Call to Action Summit held last month, EFF teamed up with Syrian Archive and WITNESS to show how faulty moderation inadvertently captures and censors vital content, including activism, counter-speech, satire, and even evidence of war crimes.

“It’s hard to tell criticism of extremism from extremism itself when you are moderating thousands of pieces of content a day,” said EFF Director for International Freedom of Expression Jillian York. “Automated tools often make everything worse, since context is critical when making these decisions. Marginalized people speaking out on tricky political and human rights issues are too often the ones who are silenced.”

The examples cited in the report include a Facebook group advocating for the independence of the Chechen Republic of Iskeria that was mistakenly removed in its entirety for “terrorist activity or organized criminal activity.” Groups advocating for an independent Kurdistan are also often a target of overbroad content moderation, even though only one such group is considered a terrorist organization by governments. In another example of political content being wrongly censored, Facebook removed an image of a leader of Hezbollah with a rainbow Pride flag overlaid on it. The image was intended as satire, yet the mere fact that it included a face of a leader of Hezbollah led to its removal.

Social media is often used to as a vital lifeline to publicize on-the-ground political conflict and social unrest. In Syria, human rights defenders use this tactic as many as 50 times a day, and there are now more hours of social media content about the Syrian conflict than there have been hours in the conflict itself. Yet, YouTube used machine-learning-powered automated flagging to terminate thousands of Syrian YouTube channels that published videos of human rights violations, endangering the ability of those defenders to create a public record of those violations.

“In the frenzied rush to delete so-called extremist content, YouTube is erasing the history of the conflict in Syria almost as quickly as human rights defenders can hit ‘post,’” said Dia Kayyali, Program Manager for Tech and Accountability at Witness. “While ‘just taking it down’ might seem like a simple way to deal with extremist content online, we know current practices not only hurt freedom of expression and the right to access information, they are also harmful to real efforts to fight extremism.”

Related Updates

Special thanks to legal intern Maria Bacha who was the lead author of this post. EFF, Student Press Law Center (SPLC), Pennsylvania Center for the First Amendment (PaCFA), and Brechner Center for Freedom of Information filed an amicus brief in B.L. v. Mahanoy Area School District urging the U.S...

We pause this week to celebrate our longtime friend Gene Crick, a digital rights activist and former president of EFF-Austin, who passed away August 14 in Texas at age 73. Gene was a tireless advocate for free speech, a free, open, and inclusive Internet, and digital rights for everyone in...

Media outlets reported this week that an international student at Harvard University was deported back to Lebanon after border agents in Boston searched his electronic devices and confronted him about his friends’ social media posts. These allegations raise serious concerns about whether the government is following its own policies regarding...

The U.S. Court of Appeals for the Second Circuit last week became the first federal appellate court to rule that Section 230 bars civil terrorism claims against a social media company. The plaintiffs, who were victims of Hamas terrorist attacks in Israel, argued that Facebook should be liable...

The U.S. Department of Homeland Security (DHS) and one of its component agencies, U.S. Customs and Border Protection (CBP), released a Privacy Impact Assessment [.pdf] on CBP’s practice of monitoring social media to enhance the agency’s “situational awareness.” As we’ve argued in relation to other government social media surveillance...

The Senate Judiciary Committee recently held a hearing on “Protecting Digital Innocence.” The hearing covered a range of problems facing young people on the Internet today, with a focus on harmful content and privacy-invasive data practices by tech companies. While children do face problems online, some committee members seemed...

In a long-awaited ruling, the Second Circuit has found that the replies section on President Trump’s Twitter @realDonaldTrump is a public forum and that the President cannot block his critics from reading his tweets or participating in the forum merely because he dislikes the views they express. This ruling...

EFF is representing People for the Ethical Treatment of Animals, challenging on First Amendment grounds the practice of Texas A&M University of automatically and manually blocking PETA and its supporters from posting comments on the university's official Facebook page, a forum that is otherwise open for public comments. In response...

This month, in many parts of the world, the LGBTQ+ community is celebrating Pride and, both online and off, the tech industry has paid lip service to supporting the community. Many social media companies participate in Pride parades or offer photo filters or other digital swag for users to show...