Creating Unbeatable Videogame AI

Super Smash Bros. Melee is a multiplayer fighting game released for the Nintendo GameCube in 2001. For the last decade and a half, it has become one of the premier fighting game eSports, and it is the reason Nintendo still makes a GameCube controller for the Wii U. Smash Melee has an intense following, and for years the idea of an AI that could beat top-tier players at Melee was inconceivable – the game was just far too complex, the strategies too demanding, and the tactics too hard.

[Dan] a.k.a. [AltF4] wasn’t satisfied that a computer couldn’t beat players at Melee, and a few years ago started work on the first Melee AI that could beat any human player. He just released Smashbot at this year’s DEF CON, and while the AI is limited, no human can beat this AI.

The architecture of Smashbot

Smashbot isn’t a device with a camera you point at a TV that also plugs into a GameCube – the state of GameCube emulation has advanced far enough that the Dolphin emulator is a perfectly reasonable substitute for a cute little cube. Architecturally, this AI reads the game state from the Dolphin emulator – effectively looking at everything, and only everything, a human player would see. The AI then does its thing, sending button presses to the emulator just like a human pressing buttons on a USB gamepad. There’s nothing in Smashbot that a human couldn’t do, it’s just that Smashbot has superhuman performance.

Although Smashbot will beat any human player, there are a few limitations. Each character in Melee has different capabilities, strategies, and techniques. So far, Smashbot can only play as Fox. That’s not to say Smashbot can’t eventually play as any other character, it’s just that allowing this would require twenty times the amount of work. For similar reasons, and because the stages in Melee are interactive, the only stage Smashbot can play is Final Destination, or the frozen version of Pokemon Stadium. Additionally, because the strategies between characters are a very complex version of rock, paper, scissors, the AI can only play against Marth. It should be noted that in Melee competitive circles, a Fox v. Marth match nearly always means Fox loses. This is the worst possible case scenario for an AI, but if it can win this match, it can do anything else.

The obvious question — and first one asked at the Q&A after the talk — was, ‘what would happen if you make the bot play itself?’. There are two answers to this. Right now, Smashbot is written so it will always pick player two. The short, glib answer is ‘nothing. It can’t do that yet.’

The significantly better answer to the question is much, much more interesting. In short, each AI would choose the best, fastest attack for their character, in this case Fox, and the one-frame Reflector attack chained into ‘multishines’ (it looks like Fox is punching the ground). Since both bots are chaining one-frame attacks together against each other, but no attack will hit, this means the game timer will eventually run out. When that happens, the game goes into Sudden Death, and the stage starts collapsing. If a piece falls on a player, that player takes damage and loses. This damage is ‘random’, but effectively controlled by a random seed in the game, and Smashbot will be able to figure all of this out before the match has begun and try to position itself so it can block all attacks against the other bot, multishine for eight minutes, and wait for the stage to collapse.

That’s not exactly an answer to what would happen if the bot could play against another bot. It would either be the most boring competitive match of Melee ever or the most exciting ever. We just won’t know until [Dan] finishes up Smashbot.

You can check out a video [AltF4] made earlier showing off Smashbot below.

True, the random number part is cheating, but not necessary for ordinary gameplay. It’s peeking into the machine’s RAM, AIUI, is just to save the bother of having to do machine vision and comprehend everything that’s going on on the screen. I think that’s fair.

To an extent, if it’s only reading stuff it can’t really be cheating. It’s only way to affect the game is through the joypad, and it can’t read the other player’s mind. It’s a fair achievement, although I suppose that depends how well balanced the game is. By now I suppose it’s pretty well balanced, spam-proof.

[Bechoff] would have put ‘random’ in inverted commas because it’s not ‘random’.

A computer cannot generate a random number so it creates a ‘pseudo-random’ sequence of numbers (from a seed) that *appears* random to a human. It is NOT however in any way random and the sequence of pseudo-random numbers can be reverse engineered by simple math to find the seed and every ‘pseudo-random’ number in the continuing sequence can be calculated in advance. This is the opposite of the meaning of random.

Putting aside semantic arguments such as whether this qualifies as cheating or ai, it’s a very simple bot. It doesn’t have many states and actually makes a risky move in attempting a grab during Marth’s invincibility frames because it isn’t aware of them.

If nothing else, it’s an effective but prohibitively specialized program. It’ll need to be abstracted quite a bit if support is to be added for arbitrary characters or multiple opponents.

“There’s nothing in Smashbot that a human couldn’t do, it’s just that Smashbot has superhuman performance.”

Well, no. Smashbot gets the game state immediately as it’s generated, and sends commands immediately. Humans can’t do that, because they have to view the display (which takes time to generate) and they have to push buttons (which takes time to send).

It’d be interesting to add a “perception lag” and a “control lag” to the AI to see how far you need to go before people can beat it.

I’ll go a step further: Without any sort of perception/response lag, I consider the AI to be cheating.

Because it takes time for the video signal to go from the Gamecube through the TV to the viewer’s eyeballs, the AI is effectively responding instantly to information the player hasn’t even received yet. Depending on the TV, that display lag can be anywhere from 10 to 200 milliseconds! That’s a huge advantage for even a human player, let alone an AI with near-instant reaction times.

Players can compensate by trial and error. It’s how they beat 3 of the highest level ai’s in the game on their highest level in team. We can’t compensate for it being able to perform moves faster than our console can receive button presses.

I’m not sure what you mean by “compensate” – compensate for the perception lag (the time it takes to display the image)?

People compensate for it by anticipating, but that’s exactly the point – the AI here *isn’t* anticipating. It just knows. Anticipating and learning about your opponent requires *way* more processing and learning than is being done here.

“In short, each AI would choose the best, fastest attack for their character, in this case Fox, and the one-frame Reflector attack chained into ‘multishines’ (it looks like Fox is punching the ground). ”

I am not familiar with Super Smash Bros. Melee so maybe I’m missing something but how is that considered an AI?

If this AI consists of merely selecting the most powerful attack available and precisely timing it over and over again, there is not much “intelligence” involved. It’s simply capturing game state and sending a command. Maybe this is a limitation of the game (not having much diversity in attacks or lacking counter measures for them) but I’d expect a real AI to analyze opponent moves, selecting best attack for given situation, finding weaknesses in their defenses, learning opponent’s tactics and countering them, etc.

Not trying to underestimate the effort and technical skill that are required to create such a bot but this doesn’t sound anything like “unbeatable AI”, but rather taking an advantage of computer’s ability to beat player instincts by being able to issue a command faster.

“If this AI consists of merely selecting the most powerful attack available and precisely timing it over and over again”

No, that’s not what it does. That’s only what it would do first – with a blank slate, at the start of a match – because, well… why wouldn’t you? Your opponent hasn’t done anything yet, you’re trying to win the game, so… strongest attack first.

After that it has strategies, tactics, and goals based on the situation in the game, which certainly makes it look like an AI.

But then, why would two bots fighting each other issue the same command over and over again to no avail until the time runs out? Wouldn’t you expect from a any basic AI to try something else if current actions are fruitless and the opponent does the exact same thing?

The decision tree here is simple: if I do my super-powerful attack, something good might happen, but nothing bad will happen. If I do anything else, something good might happen, but something bad might happen, too.

I thought the multishines were just the bot showing off when the opponent was definitely going to die. I didn’t think it was selecting it as the “best” move. Correct me if I’m wrong though, since I haven’t exactly read through the code or anything.

I suspect that a Fox on Fox Smashbot battle would depend heavily on how the bot is programmed to compete against a Fox player. Judging from how the bot plays against Marth, I’m thinking the match would probably be a dodge-fest or a back and forth grapple contest.

Back in ’95 I became addicted to Samurai Shodown 2. At Level 4 setting it was incomprehensible for me to win vs CPU. That said 2 years before GodHood at Street Fighter 2. With Ryu and Hadoken and Shakunetsu I defeated all HS friends (or cheapo Bison and zip, zip, zip across non stop). Except when Mortal Kombat came out and a friend had the inside track on timing and combo presses… :( It got worse. Tekken was alien to me. Never won. Don’t know didn’t feel right. The Mojo came back with Soul Caliber – Mitsurugi. SWAG! I gave up on fighting games to be honest but there is one gem that trumped all. Samurai Shodown 3 – Genjuro – Bust mode. Non-stop ULT weapon breaker combo it was basically a twist on both the Hadoken and Shakunetsu… Basically invincible during movement, action, and recovery. Because the recovery time of opponent lasted as long as making the resonance still. So yeah. Take the most OP move simplify or automate it and you’d never lose.

Conversely the first time in my life the AI impressed me was Half-Life and then F.E.A.R. “How do these bots know to squad support one another?!?!” I gave up when I finally play Warcraft 3. The comp didn’t have fog of war and went in for your weakest areas.