Sometimes, this game understands Pokemon and love. Sometimes, it fails spectacularly.

Share this story

In February 2019, we at Ars Technica learned about the Generative Pre-trained Transformer-2 (GPT-2) toolset, a freakish machine-learning algorithm that was trained on roughly 40GB of human-written text. Its ability to generate unique, seemingly human text scared its creators (the non-profit research group OpenAI) enough for them to temporarily lock the tools up for public consumption. (Despite those fears, we at Ars got to access and play with the results two weeks later.)

Since then, GPT-2's public availability has exploded with tons of experiments, and the one that has arguably made the rounds more than any other is AI Dungeon, a freely available "text adventure" that is designed to create a seemingly endless interactive narrative experience. That experience received a formal "sequel" in December, and we've finally tested the results as a staff.

According to its creators, the game combines GPT-2 with roughly 30MB of stories lifted from ChooseYourStory.com, a community-driven hub for interactive fiction. The resulting database is served to users in a funnel of one of four story prompts: fantasy, mystery, apocalyptic, or zombie. (A fifth option lets users write their own one- or two-sentence prompt to describe their own ideal setting.) From there, users are given some sort of verbose prompt, then left to type out whatever action, description, or rumination they imagine doing in that fictional universe.

These are five Ars staffers' results, all recorded without any input from each other.

One playable implementation of AI Dungeon 2 includes an email registration system. This lets you save and retrieve previous adventures.

AI Dungeon 2

In the current build, every new adventure begins with these options. The fifth one asks players to write their own 1-2 sentence prompt to kick the GPT-2 system into gear.

A few basic options before going forward with the "Fantasy" setting.

Every adventure begins with a mild confrontation and a wide-open window of storytelling possibilities. What's Buddy going to do?

“You laugh as you catch the little rodent”: Parsing copyrighted content, but not in logical order

First of all, this isn't a game. It's interesting, but it's not a game. More like computer-assisted literary masturbation. If you've ever played the party game (or BBS game, if you're old-and-nerdy enough) where each player writes one sentence or paragraph of a story, then the next player writes the next, and so forth... AI Dungeon 2 is a two-player version of that, where at least one player has gotten far too deep into the recreational pharmaceuticals and is having an absolute blast but not really paying attention.

If you approach AI Dungeon like a Zork variant, you'll feel very much un-seen, as it ignores even simple commands—like "inventory"—and goes haring off on wild story elements instead. You get much better results by approaching it as an exercise of equals, in which you and the computer collaborate on a dungeon-themed story. They're still deeply weird results, and you still may not feel paid attention to. But at least the machine will riff with you.

Although AI Dungeon does much better if you collaborate with it on a story rather than giving it commands, it does begin to get confused about pronouns and declare things in the first person without an obvious referent—the game appears to still mean "you" when it says "I," but who knows. After all, you are, effectively, holding a collaborative storytelling conversation with an "intelligence" roughly on par with an uploaded copy of a roundworm.

Additionally, instead of dumping text onto the screen normally, AI Dungeon renders it with an agonizingly slow left-to-right scroll, at a roughly similar speed to a competent typist. In fact, my very first "command" was to ask it not to do that—but, again, all it's really looking to do with your input is riff with you on a story. It has no idea what to do with a traditional command, whether game-related or game-option-related.

I don't feel a strong pull to keep messing with AI Dungeon just for fun—but it'd be extremely tempting as a way to potentially break writer's block, if I were working on writing a fictional story.

I'll post a lengthy selection from my adventure, in part because it best shows off how most of our results turned out: a mix of logical statements and utter gibberish, along with mixed results in terms of comprehending our own prompts and requests. (You'll also see its inability to track previous events for very long.) For the rest of this article, Ars staffers' commands are in bold, while the GPT-2 system's text is left unformatted.

"A few years ago, a group of wizards were sent here by King Ragnor. They wanted me to help them create an army. I refused and they destroyed my home."

You pause for a moment before continuing.

> Embark upon an epic quest to assassinate the winning football coach of King Ragnor's alma mater university.

You decide to take the next logical step. If this was any other wizard, you would be able to cast a powerful magic spell or two and destroy the statue of Ragnor. However, you are not any other wizard; you are a member of the Order of Hermes.

> Become any other wizard.

You begin to cast a series of spells, but nothing happens. This is because you are still in the middle of casting a spell that requires concentration. You realize that if you want to kill the statue, you will need to concentrate on doing so rather than trying to do it all at once.

> Suddenly, a wild Pikachu appears!

Suddenly, a wild Pikachu appears from behind the statue. It runs towards you and then leaps into your arms as if it knows what you're going to do.

You laugh as you catch the little rodent. "I can't believe I'm actually catching a Pokémon!"

“So... what now?” asks the statue, which has vanished entirely.

> Suddenly, the Pikachu disappears again. :'(

Suddenly, the Pikachu reappears and then disappears again. You think about how you should go about killing the statue, but then you remember that there is no way to tell which direction the statue will run.

> "Have you considered the possibility that you're not quite sane, old chap?" says the statue, grinding audibly as it turns its head to face me directly.

You focus hard on casting the spell. As you do, you notice that the statue seems to be getting closer and closer. You quickly cast your spell, and the statue begins to slowly get smaller until it vanishes completely.

"That was pretty easy," you say aloud. The statue looks like it was made of solid stone, so you have to wonder why it disappeared.

The statue stares at you with a blank expression. "Pikachu?" it asks, before seeming to morph into something else altogether. It becomes a small, furry animal wearing a red cap and carrying an umbrella. Its eyes widen when it sees you, and it rushes forward to embrace you.

> inventory

You grab the Pikachu by the collar and hold it tightly. "Come on, Pikachu, let's go home."

As a hobbyist interactive fiction writer myself, things like AI Dungeon 2 worry me. When I first came across it, I played it again and again, going deep down the rabbit hole - literally in one case. I thought I was looking at the end of human-created games and although this has as much in common with actual fiction as my cat's end-meal production has in common with a Rembrandt masterpiece (none whatsoever), it's still got me worried, if only for the future.

Yeah, today this program is limited and it's easy to laugh at the dumb AI - look at it, it lets the plot meander all over the place, no defined characters, can't even understand an Infocom-style command - but the article here spends its time pointing out how laughable the story it creates is...not that it's amazing it works at all. We take the fact that AI can do things like this for granted, and it's for just this reason I get nervous.

Sure, today it's not much better than a chatbot and it has no real grasp of effective narrative (but then neither do a lot of people, oy, some of the IF out there...) but this version is the state of the art in 2020....what will it be in 2030, 2040, 2050? How long before it can not only keep a story going but also generate the interactive framework around it? How long before someone can request of AI Dungeon 7 - Professional Edition a gritty noir detective story in the style of Raymond Chandler with the crime complexity of Dorothy Sayer? How long before work produced by humans becomes second-tier?

It's easy to dismiss a question like this and say "Oh, but people will *always* produce better work; all an AI can do is rehash existing ideas, you need a human to produce something truly original!" and that's true....if it wasn't that so very many things published today are really also just rehashes of existing ideas in new settings. How long before AI can produce work that is as good as or better than just about anything out there because it's known, familiar, safe?

Anyway, tl;dr: wannabe IF writer feels threatened by program that hold potential to be better than they ever will, wanders off to yell at clouds.

I've wondered about doing this the other way round - turn GPT-2 into something resembling general AI by having it play the world as a text adventure. Fine-tune it so it can use commands like <say> or <read webpage>, give user input as "user: " and treat text inbetween as notes to self. It is able to treat text as memory for quite a long time, as shown in the scenarios in the game.

But, hey. It's not a major leap to go from face-kicking to manifesting revolver-based murder. I suppose it tracks.

In my experience, this doesn't hold up in court nearly as well as you'd think.

Kate Cox wrote:

I also kept getting stuck in loops where I went back and forth eating delicious foods.

That's something you don't find enough of in fiction, interactive or no.

It fits, anyway. As an acquaintance of mine puts it, "Gotta eat to live, gotta steal to eat."

Kate Cox wrote:

AI Dungeon had good-enough contextual memory—remembering what "her" referred to or telling me I went "back" to a place—but not good-enough context to understand that maybe cars don't suddenly appear in fantasy settings.

Cars, no, but I remember seeing on the telly that there are pickup trucks in WoW. I can't fault the AI for not knowing the difference between a car and a truck.

As a hobbyist interactive fiction writer myself, things like AI Dungeon 2 worry me. When I first came across it, I played it again and again, going deep down the rabbit hole - literally in one case. I thought I was looking at the end of human-created games and although this has as much in common with actual fiction as my cat's end-meal production has in common with a Rembrandt masterpiece (none whatsoever), it's still got me worried, if only for the future.

Yeah, today this program is limited and it's easy to laugh at the dumb AI - look at it, it lets the plot meander all over the place, no defined characters, can't even understand an Infocom-style command - but the article here spends its time pointing out how laughable the story it creates is...not that it's amazing it works at all. We take the fact that AI can do things like this for granted, and it's for just this reason I get nervous.

Sure, today it's not much better than a chatbot and it has no real grasp of effective narrative (but then neither do a lot of people, oy, some of the IF out there...) but this version is the state of the art in 2020....what will it be in 2030, 2040, 2050? How long before it can not only keep a story going but also generate the interactive framework around it? How long before someone can request of AI Dungeon 7 - Professional Edition a gritty noir detective story in the style of Raymond Chandler with the crime complexity of Dorothy Sayer? How long before work produced by humans becomes second-tier?

It's easy to dismiss a question like this and say "Oh, but people will *always* produce better work; all an AI can do is rehash existing ideas, you need a human to produce something truly original!" and that's true....if it wasn't that so very many things published today are really also just rehashes of existing ideas in new settings. How long before AI can produce work that is as good as or better than just about anything out there because it's known, familiar, safe?

Anyway, tl;dr: wannabe IF writer feels threatened by program that hold potential to be better than they ever will, wanders off to yell at clouds.

I take solace in the fact that this line freaking out has been repeated ad nauseam ever since AI was first envisioned. Is this better than Dr. Sbaitso or ELIZA? Yes. But that was 40 years ago and this is really just that on steroids, despite all the advances in hardware and algorithms that have occurred in the interim. These computers ain't gonna take our jerbs anytime soon.

All this article has done is make me want to do an adventure of my own, and see how weird I can make it!

Edit: Well it is hard work, I'm amazed that the snippets on here make as much sense as they do.

I tried a Zombie one but ended up in a car chase on my own only to then be being driven by somoeone else.

And then I tried a custom one with the setup being the movie "The Martian" which it straight away gave me other survivors and even after killing them off with the unknown it just wanted me to get help even though I said none was coming.

I can see how with more data this could be quit good but in its current state and wanting to take its own narrative it feels like your fighting it some what as it stands.

I should use this to write a NYT Best Selling novel. It's pretty epic. Of course pretty much every book ends up being a New York Times best selling novel for some reason, so the bar is set pretty low to begin with.

Hi, second to my original answer, it is not up to me to take on someone else's complaints. If you feel threatened by something, speak up. If you feel threatened by something, share your concerns with those who might be able to offer you assistance.

As a hobbyist interactive fiction writer myself, things like AI Dungeon 2 worry me. When I first came across it, I played it again and again, going deep down the rabbit hole - literally in one case. I thought I was looking at the end of human-created games and although this has as much in common with actual fiction as my cat's end-meal production has in common with a Rembrandt masterpiece (none whatsoever), it's still got me worried, if only for the future.

Yeah, today this program is limited and it's easy to laugh at the dumb AI - look at it, it lets the plot meander all over the place, no defined characters, can't even understand an Infocom-style command - but the article here spends its time pointing out how laughable the story it creates is...not that it's amazing it works at all. We take the fact that AI can do things like this for granted, and it's for just this reason I get nervous.

Sure, today it's not much better than a chatbot and it has no real grasp of effective narrative (but then neither do a lot of people, oy, some of the IF out there...) but this version is the state of the art in 2020....what will it be in 2030, 2040, 2050? How long before it can not only keep a story going but also generate the interactive framework around it? How long before someone can request of AI Dungeon 7 - Professional Edition a gritty noir detective story in the style of Raymond Chandler with the crime complexity of Dorothy Sayer? How long before work produced by humans becomes second-tier?

It's easy to dismiss a question like this and say "Oh, but people will *always* produce better work; all an AI can do is rehash existing ideas, you need a human to produce something truly original!" and that's true....if it wasn't that so very many things published today are really also just rehashes of existing ideas in new settings. How long before AI can produce work that is as good as or better than just about anything out there because it's known, familiar, safe?

Anyway, tl;dr: wannabe IF writer feels threatened by program that hold potential to be better than they ever will, wanders off to yell at clouds.

I take solace in the fact that this line freaking out has been repeated ad nauseam ever since AI was first envisioned. Is this better than Dr. Sbaitso or ELIZA? Yes. But that was 40 years ago and this is really just that on steroids, despite all the advances in hardware and algorithms that have occurred in the interim. These computers ain't gonna take our jerbs anytime soon.

I see the biggest potential for these applications in areas where it's simply not feasible to have humans doing all the writing.

For example if you have an expansive game universe with tons of non-player controlled characters, writing a backstory for all of them would simply not be a smart use of your writing resources. Then again it's still quite immersion breaking when every other NPC you encounter was once an adventurer, but took an arrow to the knee.

Maybe occasionally some NPC sounds like a rambling lunatic when the algorithm doesn't get it quite right, but we all probably know one of those in real life anyway. Also players would totally read into that and wonder if the dude is some kind of an oracle foreshadowing something.

This really isn't all that different from giving your DM, say, mescaline or shrooms in his coffee and seeing how long he can keep it together (result: not long, N=1).

Using this as the core of a procedural generator would be pretty interesting. Getting the tone and narrative cadence right would be the hardest part, but depending on how you implemented it, this could be used to fill in details that are otherwise too tedious to see to. Or to add a level of "uncanny valley" to NPC speech as needed. (Strange out of this world creatures controlling your NPCs? Use this to make them sound like they're trying really hard to sound normal!)

I've wondered about doing this the other way round - turn GPT-2 into something resembling general AI by having it play the world as a text adventure. Fine-tune it so it can use commands like <say> or <read webpage>, give user input as "user: " and treat text inbetween as notes to self. It is able to treat text as memory for quite a long time, as shown in the scenarios in the game.

(curious effect noticed by a commentator -- because chess games are usually between near-equals, if its opponent makes a bad move it concludes "I must be a bad player too" and starts doing stupid things)

A few of my stories from AI dungeon were ended when the game started to just keep repeating the same line back to me no matter what I typed. Sometimes the game would get stuck with everything I typed was what I was saying to a character and the other character would reply to what I was typing instead of letting me advance. I was able to break out of this loop by telling the character we should go somewhere but two commands later everything I typed was being spoken again.My favorite was when I went home after delivering a letter to post-apocalyptic Charleston to die because I didn't do something in Charleston even though there were no hints that I needed to do anything.

The best way to use the program is to keep your commands short and simple one sentence, don't use pronouns, single word commands seems to give it too much freedom to just throw random stuff at you. Long run-on sentences or multiple sentences tend to confuse the program and it will ignore most of what you told it anyways.

No matter what you do it will just be a confusing mess, but a better way to waste a few minutes than crushing candy or sling shotting birds or whatever the kids are doing nowadays.

The best way to use the program is to keep your commands short and simple one sentence, don't use pronouns, single word commands seems to give it too much freedom to just throw random stuff at you. Long run-on sentences or multiple sentences tend to confuse the program and it will ignore most of what you told it anyways.

I found it got really wild when i quit trying to give the thing actions and just started responding directly with dialog.

Once I freed my mind, the program tee'd me up an amazing opportunity to start deploying Archer quotes:

Quote:

> I would like to ask the man if he's doing okay.

The man continues to watch the forest with his back turned to you. You pause before continuing:

"I'm sorry about your friend."

He still does not respond. You sigh and release him from your embrace. "Can I ask you something?"

"Go ahead,"

> "Did you see Regis this morning?"

"Regis?" The man frowns. "What's wrong with Regis?"

"He was killed yesterday."

The man stares at you blankly for a moment, then he nods slowly. "Yes, I saw him die last night."

> Explain to the man that I have breast cancer and that I am on a rampage to avenge myself against the Irish Mafia.

"Okay," the man says after a few moments of silence. "So did you kill Regis?"

"No."

"Then why are you killing me?"

You smile at him. "Because I have cancer and I want revenge against the Irish Mafia."

...

> Ask the man what cancer is because seriously, what is cancer

"Cancer?" The man looks down at his feet. "It's an illness where cells divide uncontrollably and grow rapidly."

The best way to use the program is to keep your commands short and simple one sentence, don't use pronouns, single word commands seems to give it too much freedom to just throw random stuff at you. Long run-on sentences or multiple sentences tend to confuse the program and it will ignore most of what you told it anyways.

I found it got really wild when i quit trying to give the thing actions and just started responding directly with dialog.

Once I freed my mind, the program tee'd me up an amazing opportunity to start deploying Archer quotes:

Quote:

> I would like to ask the man if he's doing okay.

The man continues to watch the forest with his back turned to you. You pause before continuing:

"I'm sorry about your friend."

He still does not respond. You sigh and release him from your embrace. "Can I ask you something?"

"Go ahead,"

> "Did you see Regis this morning?"

"Regis?" The man frowns. "What's wrong with Regis?"

"He was killed yesterday."

The man stares at you blankly for a moment, then he nods slowly. "Yes, I saw him die last night."

> Explain to the man that I have breast cancer and that I am on a rampage to avenge myself against the Irish Mafia.

"Okay," the man says after a few moments of silence. "So did you kill Regis?"

"No."

"Then why are you killing me?"

You smile at him. "Because I have cancer and I want revenge against the Irish Mafia."

...

> Ask the man what cancer is because seriously, what is cancer

"Cancer?" The man looks down at his feet. "It's an illness where cells divide uncontrollably and grow rapidly."

You nod. "Yeah, it's a disease caused by uncontrolled cell division."

The man raises an eyebrow.

…now I kinda want an ongoing series where ars writers do dramatic readings of Lee’s adventures

To be fair, most of the output from the program seems to make more sense than my last attempted interaction with "Customer Service". It's only a matter of time before this technology is deployed to make complaints go away.

Oooh, now I'm thinking what it would be like to install the AI Dungeon generated dialog system at the heart of a sandbox RPG like Skyrim. It would be an interesting experiment to take all the text from games like Dragon Age, The Witcher, Baldur's Gate, and Skyrim and feed it into the AI Dungeon machine learning code. A more focused / tailored data set combined with a generative and open-ended approach to managing RPG dialogue and questing structure has teh potential to be so much more engaging. Certainly more so than the on-rails "press F to pay respects" type stuff we currently have. And what would be wrong with a few moments of the absurd? These games are always so self-serious that it becomes one-dimensional.

In a neural net DM'd dungeon game, some stuff just doesn't work, and it won't tell you why. And on the flip side, things that shouldn't work often do.

The neural net was trained on human-programmed dungeon crawling games, so it's unprepared for humans that refer to objects that aren't there. I can conjure anything into being simply by acting as if it is there.

To be fair, most of the output from the program seems to make more sense than my last attempted interaction with "Customer Service". It's only a matter of time before this technology is deployed to make complaints go away.

This is great; I started as a scavenger in the Mohave Wasteland named John Smith, and ended as the Emperor of the New Galactic Empire:

Spoiler: show

Quote:

> Use the Dark Side to achieve immortality

You use the power of the dark side to gain eternal life. You become immortal and live forever. You even create a clone body for yourself which you keep alive with artificial intelligence, as well as creating countless more like him/her.Your immortality lasts until one day you die.

- YOU DIED! GAME OVER! -

I think we may have discovered where JJ Abrams got the script for TRoS.

Just typing "inventory" doesn't work. You need to type "check inventory" for the record. Although the game is very bad at remembering what it told you was in it before.

Also, you said it's not a game but it's a two player version of a party game...

Yeah, you'd be better off typing in something like, "I take out the thermal detonator in my pocket and examine it" to give the game something to work with. As a bonus, your inventory can be whatever you want it to be!

That was my first thought. Tarn needs to take this, hook it into Dwarf Fortress and point it at Wattpad for legends generation. The world will be blanketed with legendary statutes of a dwarf striking down Josh Hutcherson in a sex act with Sonic.