Trump in the Rearview Mirror: How to Better Regulate Violence-Inciting Content Online

It was a watershed when Donald Trump lost his social media megaphone on Facebook, Twitter, and other platforms right after the January 6 attack on the US Capitol, since none of these companies had ever dumped a head of state. But this moment also represents a failure: The companies barred him for inciting violence only after he had done it successfully. They identified incitement to violence in the rearview mirror, by waiting for violence to happen.

The failure of social media companies to recognize and act on dangerous speech—rhetoric that can inspire intergroup violence—is not limited to Trump. Facebook, Twitter, and other platforms must find more effective ways to handle the accounts of other world leaders and influential figures whose words can move people to violence.

Facebook runs the largest system of censorship the world has ever known, governing more human communication than any government,1 and other platforms such as YouTube and Twitter also regulate speech on a huge scale. Though Facebook CEO Mark Zuckerberg and other industry leaders have sometimes conceded that they should not be wielding this power on their own, they have just barely begun to let outsiders put a finger (not so much as a hand) on the wheel.

Social media companies need external standards and oversight for all of their decisions. At a minimum, they should establish a clear, authoritative source of parameters for rules—and of guidance on how to apply them, including disclosing and explaining the rules to the public.

Some prominent advocates have proposed that companies follow international human rights law.2 It contains by far the world’s best-known set of rules for governing speech and could improve how companies regulate content. This would also allow for better transparency and oversight on behalf of billions of people who use social media. But, as I argued recently (and in greater detail) in the Yale Journal on Regulation,3 international human rights law was made for governments, not private companies. Since Facebook and Twitter are not countries, the law would need to be adapted for their use: interpreted to clarify how (and whether) each of its provisions are suited to this new purpose.

Without thoughtful interpretation, applying international law could make corporate content moderation more confused and ineffective, not less so. Here, I illustrate the need for such interpretation by focusing chiefly on freedom of expression and the principal human rights treaty on that freedom, the International Covenant on Civil and Political Rights (ICCPR), which has been ratified by nearly 90 percent of the countries in the world.

International speech law

Companies will not find out from international human rights law what content or accounts to ban, since it’s a guide for how to regulate, not a set of substantive rules. For example, international human rights law does not actually define hate speech, although that is a major category of expression that most platforms restrict. Some national laws, like Germany’s Network Enforcement Act,4 also attempt to prohibit hate speech from social media without ever defining it, underscoring the need for a shared, transparent interpretation of the idea. Another way in which human rights law needs adapting is that it instructs governments that they may curb freedom of expression for reasons of national security. Tech companies must not make decisions regarding national security on behalf of governments, and they have no national security needs of their own.

The ICCPR’s Articles 19 and 20 set a floor and a ceiling for restrictions on speech and should be the core of a regulatory framework for content moderation by companies. Article 19 protects the right to seek and receive information of all kinds, regardless of frontiers, and through any media.5 Article 19(3) sets conditions for all restrictions on speech. These must be provided by law, necessary (and proportionate), and legitimate. By “legitimate” the treaty means that speech may be restricted only to protect one or more of five interests: national security, public order, public health, morals, and the rights and reputations of others.6 Also, no restriction may violate any other provision in the ICCPR, such as its ban on discrimination.7 In my article, I set out the relevant details of these three prongs, with my proposed application to social media companies.

Article 20 describes two forms of content that countries must prohibit, adhering always to the terms of Article 19: propaganda for war and incitement to discrimination, hostility, or violence. Both are insufficiently clear for regulation by companies (or by states, for that matter). It might take courage for a social media company to declare content “propaganda for war” and restrict or remove it on the basis of Article 20—especially if its author is powerful, like a head of state—but the potential benefit is enormous. Companies have more leverage in dealing with some governments than they believe and than they have used, for example, when governments are attacking their own citizens.

Article 20(2) of the ICCPR requires countries to prohibit by law “[a]ny advocacy of national, racial, or religious hatred that constitutes incitement to discrimination, hostility, or violence.” This unusual construction, with its use of both the terms “hatred” and “hostility” and also both the terms “advocacy” and “incitement,” has inspired much confusion and debate. Further, Article 20(2) is also outdated in naming only three bases for identity—nationality, race, and religion—not others that are found in some national laws and also in the rules of social media companies, such as age, gender, sexual orientation, refugee status, or caste. If this article is to be used as the basis for speech regulation, the list needs updating.

To clarify Article 20(2), then-UN High Commissioner for Human Rights Navanethem Pillay in 2012 launched a project that led to a UN document known as the Rabat Plan of Action.8 It indicates that “‘hatred’ and ‘hostility’ refer to intense and irrational emotions of opprobrium, enmity and detestation toward the target group; the term ‘advocacy’ is to be understood as requiring an intention to promote hatred publicly toward the target group; and the term ‘incitement’ refers to statements about national, racial or religious groups which create an imminent risk of discrimination, hostility or violence against persons belonging to those groups.”9This presents several questions, including how social media companies, or anyone else, are to gauge when there is an imminent risk.

The Rabat Plan suggests a six-part threshold test for speech to be criminalized: context, speaker, intent, content and form of the speech, extent of the speech act, and likelihood of violence, including imminence.10 However, imminence is a poor standard for online content moderation. If companies wait to respond to dangerous content until violence is imminent, it is usually too late to prevent it. For example, if social media companies had banned Trump on January 5 or 6, when hundreds of his supporters had already travelled to Washington, DC, with weapons and plans to attack the Capitol, it would have been too late to prevent the riot.

The threshold test also indicates that there must be intent to incite.11 Intent can be very difficult to discern, especially online, and is often variable: Frequently, the person who originates inflammatory content intends to incite violence, but people who share it do not or vice versa. Rabat bears promise but needs to be adapted for use in online content moderation. Another test that may be useful is the analytical framework for “dangerous speech,” which I have defined as any form of human communication that can increase the risk that its audience will condone or take part in violence against another group.

Only one other international human rights treaty, the International Convention on the Elimination of all Forms of Racial Discrimination (ICERD),12 is directly relevant to online content regulation. However, it requires a far broader ban on “all ideas…based on racial superiority or hatred” that conflicts with the ICCPR. This apparent conflict has not been resolved or explained by the relevant authorities, though the Committee on the Elimination of Racial Discrimination—the UN body charged with interpreting that treaty—seems to have deferred to the ICCPR by directing that when speech is criminalized under Article 4 of the ICERD, “the application of criminal sanctions should be governed by principles of legality, proportionality, and necessity.”13

Clearly, there is plenty of work to be done. In that light, here are a few of many outstanding questions about how such companies might use—and comply with—the law.

• How should international human rights law contend with platforms that are end-to-end encrypted so that companies do not have access to content, as in the case of WhatsApp? Does this call for a different regulatory standard?
• What does “least restrictive means” mean in the context of platform policies, taking into account such features as encryption and objectives like user privacy? How is compliance with Article 19’s legitimacy prong to be evaluated where companies regulate for several reasons, including some that are, and others that are not, specified in the treaty?
• States are permitted to enter reservations to a treaty as part of ratification. Should there be some analogous process for companies if they are otherwise unwilling to subscribe to the law?

If and when it is adequately interpreted, international human rights law might serve as a useful guide for companies to regulate speech—and for outsiders to hold the companies accountable to an external standard. This should improve social media companies’ rules, since the law requires that speech restrictions be necessary, legitimate, and provided by law. Users would be better able to understand the rules, and to hold companies accountable for errors.