Rethinking Carrots: A New Method For Measuring What Players Find Most Rewarding and Motivating About Your Game

Show Me the Money: Why the Measurement of PENS Can Surpass the Measurement of “Fun”

We’ve
outlined the core elements of the PENS approach and highlighted some of
the strong relationships with important outcomes, both with regard to
the player experience and commercial success. Next we would like to
present some more of our data in detail, taking the full PENS model
into account and looking at the power of our complete motivational
lightbox when competence, autonomy, and relatedness are working
together to create the player experience.

Remember that one of our hypotheses is that the PENS variables are
the power behind things such as enjoyment and perceived value. Hence
our data needs to show that PENS is where the deeper action is. We
accomplish this objectively through a variety of analyses, but will
here focus on correlations and regressions.

A quick note on regression analyses for those who are unfamiliar
with them. Regressions allow you to determine the relative contribution
of different factors to your outcomes. As an example, let’s say you
want to know what factors contribute to good grades in the classroom.
You find two variables predict good grades: Student motivation and
sitting towards the front of the class. To raise grades, do you focus
on student motivation or reconstructing classrooms so more students sit
near the front? Or both? It is important to know the relative
contribution of variables to outcomes, and how one (motivation) may be
behind the other (sitting at the front). Regression analysis allows us
to do just that: We can look at groups of variables and see their
relative contribution to outcomes. We can give credit where credit is
due.

So let’s now illustrate this using two factors in our data: game
enjoyment and the PENS measures. We recently finished an eight month
longitudinal study of MMO gamers in which we asked them in March about
their experience of fun/enjoyment (i.e. the usual kinds of questions
developers ask during playtesting), and also asked them about their
experience of need satisfaction in areas of autonomy, competence, and
relatedness (i.e. the PENS model). We then talked to them again eight
months later in November to see if they were still playing the game
they reported playing in March, and to get their impressions of the
game and its value.

For those still playing the game after eight months, our model
showed better predictive value than enjoyment in the correlations, even
though both related positively to enthusiasm (“This game rocks!”), and
perceived value (see Table 5)

You might think this means enjoyment measures are a
reasonable alternative to PENS and can equally predict some outcomes.
But our data shows they don’t. When we ran our regression analyses to
see who is really pulling their weight (remember the motivation v.
front of the class issue?), the PENS measure related strongly to all of
the outcomes, while enjoyment questions did not. In other words, only
our need satisfaction components predicted these important outcomes,
including sustained subscriptions, as seen in Table 6.

This directly supports our hypothesis that it is the deeper need
satisfaction that is experienced in games that leads to their success,
both in the direct emotional experience of the player (i.e. “This game
rocks!”) and in commercial outcomes (such as perceived value and future
intent to purchase). Apparently what “rocks!” about games is that they
satisfy motivational needs, supporting our contention that our
motivational lightbox lies behind the experience of enjoyment, and more
directly measures what is important to the gaming experience in order
to catalyze sustained player enthusiasm and commercial success.

How does this contribute value directly to a game’s bottom line? As
the tables above show, in this MMO study only our model was able
predict future play eight months later. Fun/enjoyment measures did not
relate to continued play. Based on the data, we estimate that an MMO
developer could increase retention (decrease churn) by 15-20% if they
tested gameplay ideas using a PENS approach, which based on today’s
subscription rates (i.e. about $15/month) roughly equates to about
$25,000 in additional revenue every month for every 10,000 players. For
a game with 100,000 subscribers, that could mean $3MM in revenue each
year (mostly dropped right to the bottom line).

Let’s look at another study that shifts the focus from MMO’s to two adventure console games…