The problem with AI in games

The problem with AI in games

One of the most common complaints I see about strategy games is that the AI doesn’t play well. That is, in fact, a reality and there are a lot of reasons for it which I want to talk about in this blog.

The biggest factory today against good AI is the bell curve. If you can make the AI “good enough” to beat 90% of the players on a reasonably high difficulty, it gets very tough to justify the extra expense in making it better.

As much as people complain about AI in say, Civilization V, it is probably good enough to beat most people on Prince or higher. But that is little consolidation to those who find it easy to beat. The same is true on Fallen Enchantress and its expansion, Legendary Heroes. Once you make it good enough to beat “most people” it gets hard to justify spending money on more time on it.

A friend of mine in Firaxis used to beat Civilization V (or IV, I forget which) because he had figured out patterns in the random number generator. That’s a pretty tough threshold to top in terms of “knowing the game”.

But why isn’t AI just inherently better? After all, I’ve seen a lot of people talk about how good the AI in Galactic Civilizations II is. And that comes down to how AI is made today versus how it used to be made.

In Civilization IV (and in Galactic Civilizations II) the game designer also wrote the AI including the worker functions. This meant that the designer knew exactly what the AI would do in various circumstances. The problem with that is that as games have gotten bigger, it has gotten a lot tougher for the designer to have time to both manage the many parts of the game’s design and find time to write the AI.

In the original Elemental: War of Magic, I wrote mainly the high level algorithms. I didn’t have time to write the AI itself. For Fallen Enchantress, I wrote the high level tactical AI and the high level strategic AI. What that means is that I have the AI call various worker functions like “AIGetAreaThreat(radius)”.

In Civilization IV, Soren Johnson, the designer, wrote the both the high level AI and all of the worker functions too. That’s very hard core and extremely rare these days. I wrote the high level AI and the worker functions in Galactic Civilizations II: Dark Avatar and that was the last game I got to do that on (in Twilight, I relied on worker functions written by the team).

The problem with AI worker functions comes down to communication between developers. It takes so little for an AI to be stupid or to seem stupid. Let me give you an example of something I worked on this evening: When the AI decides whether or not a city is defended enough, it calls “AIGetAreaThreat()”. However, after wiping out the AI one too many times because its cities were poorly defended I decided to look at this function. It turns out, it wasn’t looking inside enemy cities (i.e. if a city was stationed in a city, it didn’t “see” it).

I know when I call AIGetAreaThreat() I want to know all the units, not just the units on the map. But that’s the thing, another developer might think you only mean units on the map (such as in this case). You add enough examples like that and suddenly you end up with non-ideal AI.

The problem with writing worker AI functions is that they are really…really time consuming. And AI developers tend to be expensive. Firaxis got lucky with Soren back in Civilization IV. But that’s an atypical situation (I think Soren also wrote the AI in Civilization III as well but I could be wrong).

Now, in the case of Fallen Enchantress: Legendary Heroes, I play the game post-release so I can, in my spare time, update the AI (which is what I’m doing at 9pm on a Monday). But that’s atypical and could never be economically justified by a studio because, as-is, the AI in these games is typically more than “good enough” for most people.

“Good enough”, of course, is a symptom of a different problem and that is how disposable modern games are and the changes that attitude has had on the industry and how games are made.

so you've come to the realisation that internal communication is the source of that problem but the fix is too costly. so in order to make the AI at least challenging just hand him huge bonuses which the player doesn't have to cover the issues.

but that is just a bandaid. and a clever player will still observe weak behaviour or patterns in an AIs play and exploit that, and once you do that to a certain decree you'll never loose the game anymore.

anyway, I see no solution to this except maybe grwoing some sense of awareness to the existance of this problem and maybe this (incl. more communication) will help to avoid some problems in the first instance.

One advantage I have these days that I didn't used to have is thanks to the Internet and Skype, I can just ask Jon or Soren or Brian or some other AI coder a question and get their point of view or at the very least, cry with them.

One of the things about AI is that it tends to focus entirely on one thing, and forget everything else.

A good example: the Krynn has 200 ships. There are about 100 enemy drath ships in its territory. Instead of focusing on the enemy drath ships, more than half of the Krynn ships are heading towards a research mining base 4-5 sectors away, because the AI is programmed to kill starbases.

As a result you get silly situations where hundreds of Drath and Krynn starships are passing one another and not attacking one another.

Also, the AI builds hundreds of slow moving fighters. They would move halfway across the map to attack one enemy. But if you bribe them to attack someone else in a different area of the galaxy, the AI ships would then migrate halfway across the galaxy to attack them, ignoring the first enemy.

Through this method you can "kill" hundreds of AI fighters at no cost because they're just moving in circles around the map doing nothing.

One advantage I have these days that I didn't used to have is thanks to the Internet and Skype, I can just ask Jon or Soren or Brian or some other AI coder a question and get their point of view or at the very least, cry with them.

also thanks to the Internet, you don't have to develop and sell games like a mass market title, because maybe better AI isn't going to be appreciated by the mass market...

AI in a mass market "disposable game" may not be "economically justified" but the people who want better AI don't want a disposable game...

if fans/modders can do such a good job and if someone who is doing it part-time (eg. lead designer) can do such a "good" job then the problem is monetization strategy.

lead designers writing "good" AI as part of their job isn't exactly evidence that it's expensive.

writing AI is about solving problems (or a single problem, depending how you approach it)

designing a game gives you the upper hand because you can choose the problem

doing it post-release (like fans/modders) also gives you the upper hand because the problem isn't changing and there's a huge pool of players who have already "solved" it as humans...

I think working on the problem has diminishing returns, but I don't think we have even tried to see how far we can go before it makes no financial sense...

Digital content delivery is new. Not having to devote the entire CPU to rendering & running the simulation is new. Not having to limit yourself to writing slow-to-produce C++ code running on a single home computer is new. Not having to write static code that cannot adapt to how the world is playing is new.

All we know is that it's hard to devote resources to a niche problem when you're delivering it to people who bought the game for change on impulse (Steam, not Impulse)... but what if people spent more than $5 for a lifetime of AI? It changes the equation

I can see Frogboy's point. However the counter argument to this is that you don't see many reviews saying the AI in so-and-so strategic game is too smart (too hard maybe but that invariably means the AI is cheating up the wazoo which is usually compensation for a LACK of smarts). Whereas I frequently read reviews criticising strategy games for having poor AI. So I agree with Frogboy's final point that perhaps 'good enough' isn't really good enough.

BTW it is worth pointing out that even at Prince difficulty in Civ 5 the AI is 'cheating' a moderate amount in that it has less unhappiness, cheaper unit production, etc than the player. I don't mind that but I, like many strategy game players, would love more games to have stronger AI's so they didn't need to cheat as much.

So while I understand the economics of the problem, my vote would always be to go above and beyond on improving the AI. This is particularly true of FE/LH where I think the AI and gameplay is still hampered by quite a few bugs of the sort Frogboy has been describing in his recent posts.

As a final point I have been researching space strategy games recently (watching the Battlestar Galactica series again and have an itch for space strategy) and ran into several 2013 posts recommending people try Galactic Civilizations II because even tho it was an old game the AI was great (their words). So I think there is some pretty strong evidence that going the extra mile on strategy game AI pays off.

Why do you put in feature over feature over feature in games knowing the AI can't/only very diffilcult handle it? If you look into Dareks Civ4 Mod FFH2. It was really a nice MOD but the AI was just not able to handle it.

The "AI problem" usually is a problem for the highend of optimized-playing and very veteran players. Also, I for myself "love" to lose an game against AI just because it means the game is worth to play further and try again!

When I played Galactic Civilization II I didn't find the AI in this game good enough, especially Economy wise the AI was pretty medium. I usually try out the same games in different patch versions.

One suggestion: I think most of us more expert players stop playing when you're dominant enough and playing to the end is just boring. So why you don't add something special for the endgame which makes it worth playing further? Especially endgame in 4X is often something ending up in big managing hazzles and you usually want just end it:

- Game Rules changes over time! In bigger nations your are not able to micro cities. Everything changes from this point on

- Invasions like we had in MoO2 (even bigger once!) or the mongol attacks in total war.

- when I played total war - shogun 2 there was a nice option to handle opponents army in your "single player" game by random players from the internet. I know it far more problematic in turn based battles but still this idea could be make whole games based on it.

Stuff like that I really miss and it would so much help making endgame interesting. I think boring endgame is one of the biggest complains veteran players have. Is it so difficult to have some innovation in such games?

Well, based on when computer games started, almost Everything was innovation. As time goes on, the less and less things can be innovative. So yes, it is more difficult to make true innovation with games released at a later time.

But yes, things to make the endgame more interesting and less predictable would be very welcome IMO.

Frogboy, why don't you take charge and bring some innovation to the AI scene? I'm looking for something newer than the old heuristic-logical-semi-cheating AI. An open interface-, crowd sourced-, or neural net AI would be cool. Perhaps a combination of those. I don't care if it sucks at the beginning as long as it can improve and learn from its mistakes.

Probably just exploiting the fact that you can reload and repeat the exact same sequence and get the exact same result. Or the implementation is flawed. I find it hard to believe patterns can be found in-game that can be used to predict outcome in subsequent situations.

Personally, I don't understand what's wrong with a cheating AI in a single-player game. This requires me to be smarter than the AI, giving me a sense of accomplishment if I can still beat it. It's like climbing a mountain: the mountain and I don't have to be one and the same.

It only becomes a problem if the AI is so stupid, it consistently breaks immersion. Or if it requires so much help (cheats) it is not really playing the same game as I am.

Thanks for taking the time off-hours and digging into the AI a bit more. I can totally see your passion for this product you've put together. You've already set FE:LH as the benchmark to beat for all future 4x games, but I love that you see that benchmark being able to continue to rise.

In reading your OP, came across this: "However, after wiping out the AI one too many times because its cities were poorly defended I decided to look at this function. It turns out, it wasn’t looking inside enemy cities"

If this was something that was happening within FE:LH, is it possible that the Faction Power that is being calculated is also missing the units within cities? (just really seems like I'm on the top of the list...always....playing hard/hard settings)

Thanks for writing these up, Brad. It's always interesting to peer behind the curtain and it's one of the things I really enjoy about Stardock. Any project that requires division of labor is susceptible to the issues you're pointing out. I don't really think that using helper functions written by others is the issue (after all, the graphic designers use helper functions -- and the UI team and the audio team...).

The distinction that AI has is that debugging behavior is ridiculously hard. It's easy to bring a model up in the game and see how it's animating properly (or what have you). It's hard to bring up a game with 8 AI players and see how they're behaving. And the more complex the game, the more detailed the interactions, the more difficult it becomes to even discover bugs much less debug them.

Which is why I think one thing you left out of your analysis is AI scope and flexibility. If you're making decisions based solely on the cost-benefit slope you're going to end up with a heavily scripted AI. (And that AI is likely not going to be very good.) It's just so much easier to find bugs in scripted behavior. You go into the sandbox, set up the conditions for the script and watch.

My understanding is that with WoM/FE/LH you guys tried to build and AI engine which would theoretically be capable of playing any version of the game no matter what mods or expansions came in later. It would approach situations dynamically and independently weigh courses of actions. It would actually play the game the same way a human player did. That's was ambitious. And as you discovered, it was fraught with peril (terrible peril!).

And let's be honest, from the start you deliberately and willfully abandoned the pure economic reality in making this game. And in LH you made a great game, it took you a long time and a lot of money. But after WoM there was really no economic reason to make FE. And after FE there was no economic reason to make LH. And now, there's no economic reason to make LH better.

Except that it's a great game. And a great game deserves a great AI opponent.

Well, kinda. Its very difficult to make a random number generator that are truly random (I don't think its possible). Random number generators are really equations that produce numbers. If the numbers cannot be predicted by a person, it is considered good enough. If a person can predict or notice patterns in a random number generator, either the random number generator isn't very good, or the person has a talent.

Poor random number generators start with the same same "seed" every time, so its less a random number generator but rather a strange way to store a list of numbers. Good random number generators factor in details like date & time, buttons recently pressed, etc into the equation, so that repeating a series of steps wouldn't get the same random numbers (its unlikely to be the 23rd second at 10:31 on July 2nd, 2010 ever again).

I remember this giving me my best insight into A.I. into video games, and how a game like Starcraft and Master of Magic were different. It seems obvious, but I can see how mistakes can be made and how difficult it can be to weigh the 2 aspects (good vs fun).

I think what many of us want is more of somewhat predictable events happening throughout the game, and not a *clean up all the harass units challenge*. And that is not limited to only higher difficulties, so if some scripting is easy to add to the end game for instance, it would add a lot of depth to the game without necessarily being to difficult/time consuming (hopefully) For me the game is essentially survive the early AI onslaught early game, deal with AI harass and go on offensive mid game, and just clean up AI cities end game. The challenge disappears completely, and it seems somewhat lacking. I don't think this necessarily a difficulty thing, but a design thing, and a reasonable one to happen. (you spend most developer resources on perfecting the early game after all)

So, my suggestion is, if there was some event so to speak, where something challenging happened when you are essentially in "godmode" end game, when most of the AI challenge is over, that would spice things up significantly. You could even add that into a DLC so everyone could experience it regardless of difficulty level, and it be part of some "paid content". (add some "scaling" into the event for the AI bonuses to make it harder the higher level you play, for instance a damage modifier tied into HP etc.) Some AI "evolution into something scary" in the end game for each "classic" sovereign would be awesome for both gameplay and lore if you do it right.

I think what many of us want is more of somewhat predictable events happening throughout the game, and not a *clean up all the harass units challenge*. And that is not limited to only higher difficulties, so if some scripting is easy to add to the end game for instance, it would add a lot of depth to the game without necessarily being to difficult/time consuming (hopefully) For me the game is essentially survive the early AI onslaught early game, deal with AI harass and go on offensive mid game, and just clean up AI cities end game. The challenge disappears completely, and it seems somewhat lacking. I don't think this necessarily a difficulty thing, but a design thing, and a reasonable one to happen. (you spend most developer resources on perfecting the early game after all)

So, my suggestion is, if there was some event so to speak, where something challenging happened when you are essentially in "godmode" end game, when most of the AI challenge is over, that would spice things up significantly. You could even add that into a DLC so everyone could experience it regardless of difficulty level, and it be part of some "paid content". (add some "scaling" into the event for the AI bonuses to make it harder the higher level you play, for instance a damage modifier tied into HP etc.) Some AI "evolution into something scary" in the end game for each "classic" sovereign would be awesome for both gameplay and lore if you do it right.

Is why I was suggestion expanding wildlands in another thread. Mid to End game, the player would eventually be forced to tackle those wildlands as they expand and take over the world of Elemental (they were are recently released from the great cataclysm afterall).