Every
computer game is designed around the same central element:
the player. While the hardware and software for games may
change, the psychology underlying how players learn and react
to the game is a constant. The study of the mind has actually
come up with quite a few findings that can inform game design,
but most of these have been published in scientific journals
and other esoteric formats inaccessible to designers. Ironically,
many of these discoveries used simple computer games as tools
to explore how people learn and act under different conditions.

The
techniques that I'll discuss in this article generally fall
under the heading of behavioral psychology. Best known for
the work done on animals in the field, behavioral psychology
focuses on experiments and observable actions. One hallmark
of behavioral research is that most of the major experimental
discoveries are species-independent and can be found in anything
from birds to fish to humans. What behavioral psychologists
look for (and what will be our focus here) are general "rules"
for learning and for how minds respond to their environment.
Because of the species- and context-free nature of these rules,
they can easily be applied to novel domains such as computer
game design. Unlike game theory, which stresses how a player
should react to a situation, this article will focus on how
they really do react to certain stereotypical conditions.

What
is being offered here is not a blueprint for perfect games,
it is a primer to some of the basic ways people react to different
patterns of rewards. Every computer game is implicitly asking
its players to react in certain ways. Psychology can offer
a framework and a vocabulary for understanding what we are
already telling our players.

Contingencies
and Schedules

The
concrete translation of "What are we asking of our players?"
is "What are our contingencies?" A contingency is
a rule or set of rules governing when rewards are given out.
The anecdote about this discovery (as passed to me by one
of his students) is that one day B. F. Skinner ran low on
the small food pellets he gave the rats in his experiments.
Rather than risk running out and having to stop work for the
day, he began to provide the pellets every tenth time the
rats pressed the lever instead of every time. Experimenting
with different regimens of reward, he found that they produced
markedly different patterns of response. From this was born
a new area of psychology, and one that has some strong implications
for game design.

Some
common terms in behavioral psychology as they apply
to game design considerations:

Reinforcer:
An outcome or result, generally used to refer to a reward.
Examples: an experience point, winning a level, a bigger
gun.

Contingency:
A rule or set of rules governing when reinforcers are
given. Also referred to as a schedule of reinforcement.
Examples: a level every 1,000 experience points, a bonus
level that is only available if you kill a certain opponent.

Response:
An action on the part of the player that can fulfill
the contingency. This could be killing a monster, visiting
an area of the game board, or using a special ability.

The
contingencies in computer games are more complex, but the
analogy is clear enough. For example, players in an RPG earn
experience points to gain levels or collect bonus items to
gain extra lives. In an arcade-style game, power-ups appear
at random intervals, or only when certain conditions are met.
As in any contingency, there are actions on the part of the
participant which provide a reward under specific circumstances.
This is not to say that players are the same as rats, but
that there are general rules of learning which apply equally
to both.

Ratios
and Intervals

There
are essentially two fundamental sorts of contingencies, ratios
and intervals. Ratio schedules provide rewards after a certain
number of actions have been completed. For example, a player
might gain an extra life after killing 20 opponents. This
would be called a "fixed ratio" schedule, because
the same number of kills is required every time. Other types
of ratios will be discussed later.

One
of the most common contingencies found in games, fixed ratio
schedules typically produce a very distinct pattern in the
participant. First there is a long pause, then a steady burst
of activity as fast as possible until a reward is given. This
makes sense when one considers that the very first action
never brings a reward, so there is little incentive to make
that first kill. Once participants decide to go for the reward,
they act as fast as they can to bring the reward quickly.

The
distinct pause shown under a fixed ratio schedule can be a
real issue for game designers. Having a period of time where
there is little incentive to play the game can lead to the
player walking away. Additionally, the length of the pause
is a function of the size of the ratio (the number of actions
required), so the more actions required the longer the pause.
This means that if the ratio increases over time, such as
the increasing number of experience points required to gain
a level in Dungeons & Dragons, so does the pause. Eventually,
the pause can become infinite, and the player simply decides
it's not worth it and walks away.

As in any contingency, there are actions on the part
of the participant which provide a reward under specific
circumstances.

On
the plus side, during the pause other, less rewarding activities
often come to the fore. For example, if players know it will
take them a long time to gain their next level, they might
take the time to test a new tactic or try out different aspects
of the game.

There
are also "variable ratio" schedules, in which a
specific number of actions are required, but that number changes
every time. A player might be required to shoot down approximately
20 enemy fighters to gain an extra ship, but the precise number
is randomly generated each time. It's important to note that
the player does not know how many actions are required this
time, just the average number from previous experience.

Under
variable ratio schedules, participants typically respond with
a steady flow of activity at a reasonably high rate. While
not quite as high a rate as the burst under a fixed ratio
schedule, it is more consistent and lacks the pausing that
can cause trouble. Since it's possible (though unlikely) that
the player can gain a life for shooting down only one enemy,
there's always a reason to go hunting.

Killing opponents to gain experience points and gain
levels is one example of a ratio contingency.

In
general, variable ratio schedules produce the highest overall
rates of activity of all the schedules that I'll discuss here.
This doesn't necessarily mean they're the best, but if what
you're looking for is a high and constant rate of play, you
want a variable ratio contingency.

On the other side of the coin there are interval schedules.
Instead of providing a reward after a certain number of actions,
interval schedules provide a reward after a certain amount
of time has passed. In a "fixed interval" schedule,
the first response after a set period of time produces a reward.
For example, the game might introduce a power-up into the
playing field 30 minutes after the player collected the last
one.

Participants
usually respond to fixed interval contingencies by pausing
for a while after a reward and then gradually responding faster
and faster until another reward is given. In our power-up
example, the player would concentrate on other parts of the
game and return later to see if the new power-up had appeared.
If it hadn't, the player would wander off again. Gradually
the checks would become more frequent as the proper time approached,
until at about the right time the player is sitting there
waiting for it.

As
in the fixed ratio, there is a pause that can cause problems
for a game designer. Unlike the fixed ratio, there is no sharp
transition to a high rate of activity. Instead, there is gradual
increase as the appropriate time approaches. The pause remains,
a period where player motivation is low.

There
are also "variable interval" schedules, where the
period of time involved changes after each reward. A counterpart
to the variable ratio schedules, these also produce a steady,
continuous level of activity, although at a slower pace. As
in the variable ratio schedule, there is always a reason to
be active. The power-up mentioned in the earlier example could
reappear immediately after being collected or an hour later.
The motivation is evenly spread out over time, so there are
no low points where the players' attention might wander. The
activity is lower than in a variable ratio schedule because
the appearance is not dependent on activity. If the player
looks for the power-up 1,000 times during the interval, it
will appear no faster. Experiments have shown that we are
very good at determining which consequences are the results
of our own actions and which are not.

These
are the basic building blocks, but this is by no means an
exhaustive list. Each contingency is an arrangement of time,
activity, and reward, and there are an infinite number of
ways these elements can be combined to produce the pattern
of activity you want from your players.