This AI Tool Is So Terrifying, Its Creators Don’t Want To Release It

Artificially intelligent tech can be phenomenally (and hilariously) terrible at completing the tasks it sets out to accomplish.

Think of the InspiroBot, an algorithm that infamously creates inspirational posters with questionable nuggets of wisdom along the lines of "before inspiration, comes the slaughter" and "if you want to get somewhere in life you have to try to be dead". Or the AI-written Harry Potter novel (Harry Potter and the Portrait of What Looked Like a Large Pile of Ash), in which Ron performs "a kind of frenzied tap dance" before eating Hermione's family.

But now, researchers at San Francisco-based OpenAI have announced the development of a text generating algorithm that is so terrifyingly, mind-bendingly brilliant at its job that they've decided to keep the technology under wraps, concerned about the potential ramifications it could have in terms of fake news.

The purpose of the tech (GPT2) is to create complete articles on any subject from a human-written prompt. The result is a finished piece that sounds perfectly plausible – but is, in actual fact, complete baloney. The team described their research in an as yet non-peer reviewed paper online.

Here's an example piece, human prompt in bold:

In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.

The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science.

Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.

Dr. Jorge Pérez, an evolutionary biologist from the University of La Paz, and several companions, were exploring the Andes Mountains when they found a small valley, with no other animals or humans. Pérez noticed that the valley had what appeared to be a natural fountain, surrounded by two peaks of rock and silver snow.

The algorithm was trained using a bank of some 8 million web pages posted to Reddit, all of which had a "karma" score of three or more. This meant that three or more people had rated the content valuable, either because it was informative or entertaining.

The end result, created word-by-word, is very often believable but complete nonsense, filled with false quotes and misattributions.

It doesn't always get things right. Sometimes it repeats text and other times it switches topic randomly and nonsensically. The team also note occasional "world modeling failures" – for example, fires taking place underwater (a fact that would be physically impossible). Generally, they say, the algorithm performs better on topics, like politics and popular culture, that are highly represented among the 8 million web pages compared to, say, highly specialized technical subjects.

Still, it's enough to draw concern about the potential implications the algorithm could have when it comes to online scams and fake news. In a prompt submitted by the Guardian, for example, it generated an entirely believable fake story on Brexit, complete with quotes from opposition leader Jeremy Corbyn and references to the Irish Border – one of the most controversial out of a series of controversial issues surrounding Brexit.

And so, at least for now, the team have decided they will not make the algorithm publically available. But they do hope it will spark a debate about how to use and control AI technology, like DeepFakes.

"These findings, combined with earlier results on synthetic imagery, audio, and video, imply that technologies are reducing the cost of generating fake content and waging disinformation campaigns," the research team explain on the company blog.

"The public at large will need to become more skeptical of text they find online, just as the ”deep fakes" phenomenon calls for more skepticism about images."