Earlier this month, OpenAI revealed an impressive language model that can generate paragraphs of believable text. It declined to fully release their research “due to concerns about malicious applications of the technology.” OpenAI released a much smaller model and technical paper, but not the fully-trained model, training code, or full dataset, citing concerns that bad actors could use the model to fuel turbocharged disinformation campaigns.

Whether or not OpenAI’s decision to withhold most of their model was correct, their “release strategy” could have been much better.

The risks and dangers of models that can automate the production of convincing, low-cost, realistic text is an important debate to bring forward. But the risks attached to hinting about dangers without backing them up with detailed analysis and while refusing public or academic access, need to be considered also. OpenAI has appeared to consider one set of risks, without fully considering or justifying the risks they have taken in the opposite direction. Here are the concerns we have, and how OpenAI and other institutions should handle similar situations in the future.

Some Background: What Is Language Modeling and Why Is It Important?

OpenAI’s new language model does surprisingly well at a number of tasks, most notably generating text from a couple of seed sentences. The information they have released shows that the research could be a leap forward in language modeling.

Language modeling is an area of contemporary machine learning research where a statistical model is trained to assign probabilities to sentences or paragraphs of text. This model can serve as a building block for a variety of language-related machine learning research tasks. Beyond generating coherent paragraphs of text, language models can also help answer questions, summarize passages, and translate text. Recent work from the past year has massively improved the state-of-the-art for language modeling, most of which has been fully open-sourced.

Researchers Should Encourage a More Nuanced Dialogue Around AI

OpenAI gave access to the the full model to a small number of journalists before releasing their research. On release, the media published articles with titles like, “Brace for the robot apocalypse.”

The amount of misinformation now spreading about the current capabilities of state-of-the-art language modeling are reminiscent of past media hype around Facebook’s AI research. This points to the necessity for deliberate education around AI capabilities, not existential fear-mongering about vague threats.

Unfortunately, thanks to OpenAI’s decision to give model access to journalists instead of sharing the model with the public—or even merely hosting a discussion between journalists and other experts both inside and outside the AI research community—it’s hard to say just how advanced or scary OpenAI’s model really is. Despite the conversations they may have had internally, all OpenAI published was a bullet-point list of general “risks” associated with its model, failing to provide any semblance of a rigorous risk assessment. This blocks independent researchers from studying the risks identified and trying to identify ways to mitigate those risks. This is particularly problematic because the risks OpenAI pointed to are all possible for powerful actors to recreate.

Release the Full Model to Academics

Since releasing trained language models bolsters academic work on other downstream machine learning tasks, like the ones mentioned above, most work in this field is heavily open-sourced. BERT, Google’s language model and a predecessor to OpenAI’s GPT-2, was published and fully open-sourced less than half a year ago. Since then, it has already generated (and continues to generate) massive waves of downstream language understanding research. OpenAI broke this trend of defaulting to openness by questioning the societal repercussions of releasing fully-trained language models.

And when an otherwise respected research entity like OpenAI make a unilateral decision to go against the trend of full release, it endangers the open publication norms that currently prevail in language understanding research.

In the AI space, open publication norms are all the more important, given that research capabilities are already so highly centralized. Many frontiers of AI research require massive amounts of computing power and data to train and test, and OpenAI’s GPT-2 language model is no different. In this sort of ecosystem, a lot of groundbreaking AI research comes from private research labs as well as publicly funded sources. Private institutions are already disincentivized from releasing large chunks of their research, like datasets and code that may contain proprietary information. An open research culture, however, provides a social incentive for private entities to publish as much as possible.

OpenAI’s decision threatens that open publication culture. That means large privatized AI research centers may start thinking this is an acceptable thing to do. We could start to see fewer publications—and a world that resembles an arms race between corporate giants rather than a collaborative research forum.

To minimize this sort of impact, OpenAI should at least offer full model access to academic researchers and continue to encourage a culture of peer-to-peer knowledge sharing in the AI research community.

Stop Using “Responsible Disclosure” Analogies to Justify Withholding Research

In defending its decision to withhold its fully-trained model, training code, and full dataset, OpenAI characterized its decision as an “experiment in responsible disclosure.” “Responsible disclosure” refers to a process in the information security industry where security researchers withhold vulnerabilities from the public until they are patched. But responsible (or coordinated) disclosure means eventual public disclosure: the goal is always for the knowledge to become public in a reasonable amount of time.

OpenAI’s decision here has nothing to do with “responsible disclosure”—this is a misplaced analogy and misunderstands the purpose of the term of art. Rather, they use the term here in order to justify withholding research from the public, with no date or plan for final release.

The analogy is broken in so many ways as to make it fundamentally useless. In the case of generating “believable fake news,” there is no “vendor” that OpenAI can approach to mitigate the problem. The “vulnerability” and risk is societal. It is us, and society as a whole, who must be informed and take steps to develop ways to detect or otherwise manage the consequences of convincing computer-generated text. Even if this research were as dangerous as OpenAI suggests, there is no finite period of time for which failing to disclosure would lessen the risks; in fact, the risks would only increase as powerful institutions begin to reproduce their research, and widespread understanding of the risks would be stymied.

Create a Discussion Outside OpenAI Around Release Decisions

This incident points to the need for consensus-building among independent researchers—from both public institutions and private corporations and ranging in expertise—before such decisions are made.

From this demonstration, we’re not convinced that a single research organization is capable of performing objective and comprehensive evaluations of the ethical and policy impacts of its own work. OpenAI’s post indicates they were hesitant to take on that responsibility, and they defaulted to locking their model down, rather than defaulting to openness. Until the AI research community can come to a consensus on process for such decisions in the future, we hope that OpenAI and other research organizations will step back—and consult a broader quorum of policy experts and researchers—before making such dramatic “releases” in the future.