Yesterday and today, Mark Zuckerberg finally testified before the Senate and House, facing Congress for the first time to discuss data privacy in the wake of the Cambridge Analytica scandal. As we predicted, Congress didn’t stick to Cambridge Analytica. Congress also grilled Zuckerberg on content moderation—i.e., private censorship—and it’s clear from his answers that Facebook is putting all of its eggs in the “Artificial Intelligence” basket.

But automated content filtering inevitably results in over-censorship. If we’re not careful, the recent outrage over Facebook could result in automated censors that make the Internet less free, not more.

Facebook Has an “AI Tool” For Everything—But Says Nothing about Transparency or Accountability

Zuckerberg’s most common response to any question about content moderation was an appeal to magical “AI tools” that his team would deploy to solve any and all problems facing the platform. These AI tools would be used to identify troll accounts, election interference, fake news, terrorism content, hate speech, and racist advertisements—things Facebook and other content platforms already have a lot of trouble reliably flagging today, with thousands of human content moderators. Although Zuckerberg mentioned hiring thousands more content reviewers in the short term, there is uncertainty whether human review will continue in the long term to have an integral role in Facebook’s content moderation system.

Most sizable automated moderation systems in use today rely on some form of keyword tagging, followed by human moderators. Our most advanced automated systems are far from being able to perform the functions of a human moderator accurately, efficiently, or at scale. Even the research isn’t there yet—especially not with regard to nuances of human communication like sarcasm and irony. Beyond AI tools’ immaturity, an effective system would have to adapt to regional linguistic slang and differing cultural norms as well as local regulations. In his testimony, Zuckerberg admitted Facebook still needs to hire more Burmese language speakers to moderate the type of hate speech that may have played a role in promoting genocide in Myanmar. “Hate speech is very language-specific,” Zuckerberg admits. “It's hard to do [moderation] without people who speak the local language.”

An adequate automated content moderation system would have to adapt with time as our social norms evolve and change, and as the definition of “offensive content” changes with them. This means processing and understanding social and cultural context, how they evolve over time, and how they vary between geographies. AI research has yet to produce meaningful datasets and evaluation metrics for this kind of nuanced contextual understanding.

But beyond the practical difficulties associated with automated content tagging, automatic decision-making also brings up numerous ethical issues. Decision-making software tends to reflect the prejudices of its creators, and of course, the biases embedded in its data. Released just a couple months ago, Google’s state-of-the-art Perspective API for ranking comment toxicity originally gave the sentence “I am a black woman” an absurd 85% chance of being perceived as “toxic”.

Given the fact that they are likely to make mistakes, how can we hold Facebook’s algorithms accountable for their decisions? As research in natural language processing shifts towards deep learning and the training and usage of neural networks, algorithmic transparency in this field becomes increasingly difficult—yet it also becomes increasingly important and paramount. These issues of algorithmic transparency, accountability, data bias, and creator bias are particularly critical for Facebook, a massive global company whose employees speak only a fraction of the languages that its user base does.

Zuckerberg doesn’t have any good answers for us. He referred Congress to an “AI ethics” team at Facebook but didn’t disclose any processes or details. As with most of Congress’s difficult and substantive questions, he’ll have his team follow up.

"Policing the Ecosystem”

Zuckerberg promised Congress that Facebook would take “a more active view in policing the ecosystem,” but he failed to make meaningful commitments regarding the transparency or accountability of new content moderation policies. He also failed to address the problems that come hand-in-hand with overbroad content moderation, including one of the most significant problems: how it creates a new lever for online censorship that will impact marginalized communities, journalists who report on sensitive topics, and dissidents in countries with oppressive regimes.

Let’s look at some examples of overzealous censorship on Facebook. In the past year, high-profile journalists in Palestine, Vietnam, and Egypt have encountered a significant rise in content takedowns and account suspensions, with little explanation offered outside a generic “Community Standards” letter. Civil discourse about racism and harassment is often tagged as “hate speech” and censored. Reports of human rights violations in Syria and against Rohingya Muslims in Myanmar, for example, were taken down—despite the fact that this is essential journalist content about matters of significant global public concern.

These examples are just the tip of the iceberg: high-profile journalists, human-rights activists, and other legitimate content creators are regularly censored—sometimes at the request of governments—as a result of aggressive content moderation policies.

Congress’ focus on online content moderation follows a global trend of regulators and governments around the world putting tremendous pressure on platforms like Facebook to somehow police their content, without entirely understanding that the detection of “unwanted” content, even with “AI tools,” is a massively difficult technical challenge and an open research question.

Current regulation on copyrighted content already pushes platforms like YouTube to employ over-eager filtering in order to avoid liability. Further content regulations on things that are even more nuanced and harder to detect than copyrighted content—like hate speech and fake news—would be disastrous for free speech on the internet. This has already started with the recent passing of bills like SESTA and FOSTA.

We need more transparency.

Existing content moderation policies and processes are almost entirely opaque. How do platform content reviewers decide what is or is not acceptable speech, offensive content, falsified information, or relevant news? Who sets, controls, and provides modifications to these guidelines?

As Facebook is pressured to scale up its policing and push more work onto statistical algorithms, we need to make sure we have more visibility into how these potentially problematic decisions are made, and the sources of data collected to train these powerful algorithms.

We can’t hide from the inevitable fact that offensive content is posted on Facebook without being immediately flagged and taken down. That’s just the way the Internet works. There’s no way to reduce that time to zero—not with AI, not with human moderators—without drastically over censoring free speech on the Internet.