The Discomfort Zone: The Hidden Potential of Valve’s AI Director

Remember Left 4 Dead? It’s highly likely you do; its 2008 release was heralded with tons of fanfare and a $10 million marketing spend that showed Valve meant business. The push paid off – as of October 2012, the game, its 2009 sequel and additional DLC have garnered the series over 12 million sales. So why, in the age of endless zombie games, did this particular zombie game do so well? And more interestingly, why has Valve not released any new installments since 2010, after rapidly developing a sequel and multiple DLC packs?

An obvious answer to the first question is the sense of authentic cooperation created by the game’s mechanics, afforded in no small part by the “Director,” the much-admired AI system that generates random enemy encounters; prompts changes in the environment; and adjusts the placement of items players find, all based on a calculated ‘stress level’ for each player. For the first time in a game, an AI entity could modify multiple elements of the level on the fly for a tailored experience every time. It’s perfect for zombie games, right?

Wrong. It’s perfect for EVERYTHING.

“We set out to create a first-person four-player game in which the action changes each time it’s played. We wanted to use procedural narrative to simulate stories because, in a multiplayer environment like this, we felt a more linear, scripted approach would not deliver in the same way.” –Gabe Newell

There’s a reason Gabe calls what the AI Director does procedural narrative, and not procedural zombie spawning. All throughout the piece he wrote for Edge as part of a PR push for L4D, he hints broadly about the implications for cooperative games of all types, as well as single-player games. The promise had already been recognized internally at Valve long before the game debuted; the rest of the industry just never caught on.

But back to L4D’s use of it in particular. One thing to note is that the AI Director approach is not dynamic difficulty. Dynamic difficulty has been around for a while, and is largely based around the idea of the system making minute adjustments to existing values in order to raise or lower the game’s difficulty. For example, In Max Payne, the system would make tweaks to enemy health and the degree of aim assistance based on how well/poorly you were dominating/getting wrecked. The end result was a tighter game that never felt too easy or too hard; but all the same, each playthrough was identical.

“This is what makes procedural narrative more of a story-telling device than, say, a simple difficulty mechanism.” -Gabe Newell

Apparently, a similar tact was used in Half-Life 2: Episode 1 and 2, where you would receive different items from crates depending on your health status at the time you smashed it. Again, more than anything else, this kind of adjustment on the fly would help level the playing field without altering the experience too much – the core experience is still the same. This is why the AI Director is different – using player activity data, it sculpts a dramatically different experience that fits perfectly with what players are capable of and that engages them on a primal level, since L4D’s focus is a survival/action/horror hybrid.

Max Payne’s dynamic difficulty system helped keep battles tense, but they were always the same.

But if you remove the zombies, health packs and clichéd apocalyptic scenarios, you’d be left with a core truth of what this system affords: instant eviction from the comfort zone. While the comfort zone is a nice place to hang out in, in life as well as in games, the returns diminish quickly since you’re essentially in a suspended state in which no learning takes place.

This is why games that, on paper, should be exciting and thrilling prove to be boring despite the number of explosions, particle effects and screaming marines that fill the screen. Disruption of the expected, when you’re pushed into a new experience and enter the discomfort zone, is the thing ultimately responsible for you learning more about the system, your recourses and your capabilities, and is what makes you better at playing the game. While it’s stressful at first, this dynamic proves to be more pleasurable in the long run.

“Going forward, we’re definitely going to use some of the things that we’ve learned – what worked and what didn’t work – with Left 4 Dead not only in multi-player but also in our single player games in the future.” –Gabe Newell

So why aren’t more games using this principle? After all, it has been almost five years since Valve debuted the Director. The only reasons I can think of are the prohibitive complexity of creating such a system, and simple laziness. Perhaps this system really is only suited for horror games, you say? How could similar systems even be implemented in other games? Let’s take a look!

Racing. Create courses with dynamic events, similar to Split/Second – though instead of having events be scripted into the tracks, enable them on the fly to correspond with the player’s performance.

Shooter. Think of a wide-open shooter with a single-player component, like Halo or Battlefield. Instead of scripted enemy waves attacking from the same direction every time, why not have them drop in via parachute or dropship, launch assaults in vehicles from over the horizon, or spawn mini-bosses where you’d normally expect a battalion?

Split/Second’s track destruction is scripted. Imagine if it was dynamic, based on the player’s stress level.

RPG. In a story-driven RPG that focuses on dialog, should you grow closer to certain NPCs, why not have them drop plot bombshells that threaten your relationship thus far – and better yet, do it in the middle of a climactic battle?

Sandbox. In a Deus Ex-style sandbox game, why not have Director-triggered EMP events temporarily cut off access to certain augmentations, forcing you to adapt to different combinations of powers you wouldn’t normally combine? Remember, the Director would know which combinations you’ve been using regularly.

Arcade. In a Super Hexagon-style reflex/puzzle game, why not sculpt purposeful obstacles for the player that build upon what they’ve just overcome, instead of simply swapping in components from a pool of prefab barriers?

Beat ‘em up. In a God of War-style melee brawler, why not have enemies switch up their attack patterns and weaknesses depending on the player’s proficiency at dispatching certain enemy types?

Of course, there’s a fine line between pushing the player into the discomfort zone and unfairly screwing with the player as a way to extend replay value. In any of the above examples, the key to creating an authentic ‘discomfort zone’ dynamic is to make sure there are plenty of options available to the player under the new circumstances.

If the goal is to encourage improvisation through immediate, unexpected constraints, this would fail if the player did not have adequate options for dealing with the new problem that’s suddenly been foisted upon them.

“One of the theories of fun we use is that the more ways in which the game is recognizing and responding to player choices, the more fun it seems.” -Gabe Newell

The secret sauce of L4D, the reason why it was (and continues to be) such a hit, lies in attaining a balance in such a system. It lies in how it makes players anxious for what’s next by making their moment-to-moment actions matter in a more truthful way than purely scripted sequences could.

For developers and designers alike, I challenge you to think of how similar Directors can keep your experiences varied, your players fully engaged and your replay value unlimited.

Agreed, AI should play a major part in the player experience, although I am not sure if the risk of variable experiences is something most developers want. ValVe is a special example in this case because they are a flat line company… which means that this type of crazy AI can get past the executives or rather lack of.

When you say that this type of concept of AI controlled experiences can be developed into other experiences you mean all genres right? The thing is… L4D did a great job of discretely using an AI director, this was possible because the game play was quite literally “shrouded in darkness”. I think there is a trap in this blog post though… If an AI is used to determine the dificulty of the game it becomes well bad. If implimented in a racing game It would hard to not notice that you accidentally made a horrible crash and then following that scene you see several other racer’s crashing… not very discrete and in my opinion makes the gameplay feel cheap. I guess it might be possible, but the problem is that many games run under the idea of suspended-disbelief where the player is just barely connected to the game universe. If AI directors were more widely implemented it would seem like there was some higher power looking down on your game “fixing” the game so that it fits your playstyle and/or skill level.

I think that AI is something to look into developing… but not for the reason of increasing the experience as far as baby-sitting the player, but rather to give diversity to the players so that their experience is special.

If I am not mistaken the L4D1/2 AI was developed to paralleled test results from beta test players with sensors. This allowed the AI to anticipate the reaction of certain events.{{ ex. Enter 1x hunter, (AI, Fear +5 to all players, +10 to the player that gets hit, -3 to any player that damages hunter before death, fear degrading funcion fear = fear current – 1 second*0.5(or 1 fear is “lost” per 2 seconds). }} I am not a ValVe employee, but based on (http://www.valvesoftware.com/company/publications.html) and some of the remarks and behaviors of their recent AIs, it seem likely that their system uses an anticipation of psychology to determine the best possible modification to the gameplay to generate the best possible experience. This doesn’t mean flat nerf/buff enemy AI like God Hand does, but rather that that event play out in a different way than if a certain event.

So, for L4D if your team is in a state of panic/separation the AI might send a smoker to encourage sticking together, and vise versa a boomer to not have the group to clumped together. If the AI detects the player is “jumpy” it might throw a charger or Hunter at it so that the team can let off a little steam, and also reward an alert players that dodge the enemy attacks which give them confidence. If a team is bullet frenzy and their accuracy is low, the AI might spawn more witches to encourage careful shooting as the player will be terrified when that dang witch freaking scares the crap out of him. 🙂

Anyways, I don’t think that AI directors should be used as a nerf/buff system that makes the gameplay streamlined, but rather to encourage a play style that is entertaining. The problem is that people very in what they find enjoyable. Some people like the Rambo Halo style gameplay, while others like Rambo Six tactical strike games with lots of coordination and strategy. One of my biggest complaints with L4D was that if you weren’t playing with friends usually I would want to go alone, but that game’s mechanics said otherwise which in the long run was probably more fun… but some might disagree.

The Best solution in my opinion would be to developed the AI to detect certain play styles and adapt to encourage the best possible experience that the player wants to play.
-Stealth, so if I went long wolf a stealth aspect should help me avoid zombies, and dependent on how well I played, the AI should give me close encounters which make me feel like a bad-ass stealth survivor. Ex. I am in a building hiding jumping from building top to bulding top, a tank walks by… He keeps on walking… Shhhhhh… 🙂
-Group, This could also be the case with grouping survivors… If your group of 4 stick together and 150+ zombies come at you and die before the might of your formation and strategy, that would be an amazing experience. Ex. You set up camp and clear half of the level by purposefully setting off the alarm to draw everyone into your death trap.
-Rambo style, AI could send zombies at you that either have dodge-able CCs or are slow enough for you to counter. It could even work if zombies were slower and this would force the player to quickly transverse the levels with only enough time to pick up the important items/objectives in the level. This styles is the hardest, but it would be quite possible, but obviously the most risky choice.
Note: L4D1/2 are great games and don’t try to accomplish this, but I think this would deliver the best experience.

“I think that AI is something to look into developing… but not for the reason of increasing the experience as far as baby-sitting the player, but rather to give diversity to the players so that their experience is special.”

I definitely agree with this. The best use of these kinds of AI may be tied to difficulty, or it may not – in the case of L4D, it’s more a function of suspense than anything else, with difficulty a natural extension of the interaction. The trick is to adapt the system in the way that makes the most sense for the genre/game in question.