Better Games Through Usability Evaluation and Testing

No one
wants to play games that are either frustrating or difficult for the
wrong reasons. The best way to make sure that unintended problems do
not hinder enjoying the game is to take usability into account in game
development. This article presents how this can be done and what kind
of results to expect.

Usability
is an integral part of software development and has been so for the
past 20 years. For one reason or another, usability has not gained
similar popularity in game development. This, however, is about to
change. Ease of use and optimal user experience are already important
in games and will become even more so in the future.

What is Usability?

Usability
is about maximizing effectiveness, efficiency and satisfaction. This
definition originates from the traditional software industry, but it
translates well to game development. In games, usability is about
delivering a better and deeper experience with less unnecessary
interruptions or challenges that have not been designed by the
developers.

Why is Usability Important?

There
are many reasons why usability is important in games. For one, playing
games is voluntary. If the player has to struggle with problems that
make playing less fun than doing something else, then there is nothing
to stop the player from switching off the console. This is a serious
risk as the user experience is very sensitive to usability problems.
Even the smallest glitch or hiccup in the user interface may render an
otherwise good game into a rather annoying experience. For example, if
managing the inventory in a role playing game is not fluent enough or
restarting a race in a driving game is tedious the player is not likely
to enjoy playing the game.

Another
reason why usability is important in games is the competition.
Competition in the market is fierce. The gamers can choose which game
to buy from a wide variety of titles; if the controls are not fluent in
one soccer game, there are five more titles left from which to choose.
Usability is one of the key factors that make the game stand out of the
crowd.

The
delicacy of the user experience and heavy competition actually make
usability more important in games than it is in other software. There
are not too many word processors to choose from, and having fun at work
is not usually a top priority.

There
are also other reasons why ease of use is important. One of them is
that modern games are large and complex programs. In even the most
focused games there are tons of menus and ways to interact within the
game, not to mention games like the Grand Theft Auto series.
Usability is important when making a game as easy and intuitive to play
as possible. Good examples of complex games made easy are World of Warcraft and Xbox Live games. Both have succeeded in making traditionally difficult multiplayer gaming easy.

Usability
is also important for the future of gaming. For gaming to continue to
increase its popularity, the ease of setting up games and a fluent
gaming experience are of the essence. This is because newcomers are not
familiar with the conventions and common pitfalls of gaming. For them,
learning the peculiarities that old gamers are already familiar with
can be too much.

This Article

The
focus of this article is not only to tell why usability is important in
games, but also to introduce two usability methods and the kind of
results they yield. The methods are expert evaluation and usability
testing.

It will be presented how these were applied in the development of Frozenbyte's Shadowgrounds (www.shadowgroundsgame.com) game. Adage Corporation (www.adage-usability.com)
was responsible for the usability related activities in the project.
Before moving on to the methods, the game will be presented briefly.

Frozenbyte's Shadowgrounds Game

Frozenbyte
Inc. is an independent games studio, based in Helsinki, Finland,
founded in year 2001. Frozenbyte is currently working on a PC game
called Shadowgrounds, which is to be released in the third quarter of 2005.

Shadowgrounds
is an action game viewed from the top-down/3rd person perspective (see
Figure 1). The main features of the game are adrenaline-pumping
old-school action, realistic lighting, destructible environment and
upgradeable weaponry.

In
a usability expert evaluation, usability experts review the game and
search for potential usability problems. Based on both their knowledge
and experience they review the game systematically and report the
findings.

In
a typical expert evaluation three usability experts review the game.
The experts first evaluate the game independently. After that the
experts have an evaluation meeting together. In this meeting, the
usability problems found are discussed and rated for severity. After
that a report is written where the usability problems found are listed
systematically. For each problem there is a title, severity rank, and a
detailed description of the problem. A solution to each problem is also
suggested.

This
process takes approximately a week. Preliminary results, however, can
be given in a few days. After handing in the final report, a meeting
between the usability experts and the game developers is arranged. In
this meeting, the findings are discussed in more detail, and the
usability experts can answer questions that come about from the report.

Expert evaluation is a very flexible method, and it can be done at almost any point of game development. In the case of Shadowgrounds
the expert evaluation was done approximately six months before the
planned deadline of the game. In the version evaluated there was one
playable level and the basic gameplay mechanics were implemented.
Everything was, however, not ready. For example, the destructible
environment was not fully implemented yet, voice acting was missing,
and many smaller bugs were still present.

It
is also possible to conduct an expert evaluation even in earlier stages
of game development. For example, menus and displays can already be
evaluated on the basis of paper prototypes, and potential usability
problems affecting gameplay can be spotted from the design document.
The earlier the expert evaluation is done, the easier and more cost
efficient it is to make changes.

The
depth and scope of an expert evaluation are also easy to change. For
example, if there is a desire for constant input from the usability
experts to the development process, then conducting several smaller
usability evaluations with less experts and faster reporting may be a
good idea.

Results of the Expert Evaluation

In total 135 usability problems were found in the expert evaluation of Shadowgrounds.
Out of these problems 2 were classified catastrophic, 30 severe and 60
intermediate. The remaining 43 problems were either minor or cosmetic.
The distribution of the problems is illustrated in Figure 2.

Figure 2. Classification of the usability problems found in the expert evaluation.

The
number of usability problems found may seem high, but in reality it is
not exceptional by any means. Expert evaluation gives feedback to
almost every area of game development. In a typical evaluation,
usability problems are found in menus, in-game displays, controls,
gameplay and level design. Because of this, the total number of
problems found can be quite high. Here are some examples of usability
problems along with the game developers' view on the problem.

ProblemOne color has multiple meanings in the map display.

Rating:Catastrophic

Description:The
light blue color in the map display has three different meanings. It
marks the area visited, unknown area, and areas that cannot be accessed
(e.g. mountains).

Using
the same colour to symbolize three different things makes understanding
the map difficult. Every time the user opens the map, s/he must stop
for a second and think what the colors mean in different locations.
There is also a danger that the users do not understand what the colors
mean or misinterpret the map.

SolutionUse different colors for displaying visited, unknown and inaccessible areas.

Developers' commentThis
problem could have been addressed earlier if the design document had
covered the map screen more thoroughly. In the final game, a map legend
will be displayed, and only one color (shade) will be used for one
meaning.

Problem:No feedback is given if the player cannot pick an item.

Rating:Severe

Description:Sometimes
it happens that the player cannot pick up an item because there is no
room in the inventory. If this happens, the user is not given any
feedback.

This
is problematic as the user may not know why s/he cannot pick up the
item. It is likely that the user will figure it out eventually, but the
confusion and extra effort required are likely to cause frustration.

SolutionGive
the user proper feedback in every situation where the user interacts
with the environment. If the item cannot be picked up, inform the user
about this with a sound and/or textual feedback.

Developers' commentThis
was addressed only vaguely in the design document, and was overlooked
by the programmers at the time. It is likely that this feature would
have been implemented in due time, and goes to show that sometimes
developers leave many of the smaller usability issues to the final
stretch of development.

ProblemWhen fixing an item, the feedback is displayed too far away from the location being fixed.

RatingIntermediate

DescriptionThe
player often fixes items in the game. Fixing an item is done by moving
close to the item and holding down E. A progress bar is displayed and
the item is fixed when the bar reaches the end. The progress bar is
displayed too far away from the item fixed.

Presenting
the feedback closer to the location at which the action takes place
makes it easier to understand that the feedback and action are linked
to each other. It also reduces the need for the user to move his/her
attention between the feedback and the action.

SolutionDisplay the progress bar close to the location it refers to.

Developers' commentThis
seems like a no-brainer but it went unnoticed for a long time by the
development team, because it had grown so accustomed to the game and
its features. The feature had only a vague description in the design
document (loosely translated to "a repair bar is displayed on the
screen"). Had the document been given more thought or had it been read
by usability experts before implementing the feature, the feature might
have been done correctly from the get-go.

ProblemIt is not immediately obvious whether the upgrades presented next to the weapon are installed or not.

RatingIntermediate

Description:The
upgrades available for the weapons look active even before purchasing
them. This is because they are marked with a similar color scheme as
the weapons next to them, which are active.

This
is especially problematic early in the game. Because the upgrades look
active, the user may not understand that this menu can be used to
purchase new upgrades.

SolutionMake
the upgrades that have not been purchased yet clearly inactive. This
can be done for example by using a darker shade of blue to present the
upgrades that have not been bought yet.

Developers' commentThe
weapon upgrade feature had not been fully implemented by the time of
the testing, but regardless, this particular problem had not been taken
into consideration. Later user-testing showed that this was definitely
a problem with gamers, especially in the beginning of the game.

ProblemRed lights in the computers are easily mixed with enemies displayed in the radar.

RatingMinor

DescriptionEnemies are marked with small red spots in the radar. The radar is semi-transparent: the scenery is visible through it.

In
some buildings there are computers and other equipment that have small
red lights. When it happens that such a light is visible through the
radar it is easily thought to be an enemy.

SolutionDo not use small red lights in levels, or change the method of displaying enemies in the radar.

Developers' commentSometimes
the developer can freely ignore the comments or suggestions made by the
usability report. Here, Frozenbyte decided that this problem occurs too
rarely to warrant any changes to the user interface.

After
the expert evaluation some of the most central usability problems were
fixed to prepare the game for the usability testing that was scheduled
to start two weeks later. For example, the map problem discussed
earlier and minor usability problems related to the radar were fixed.

Page 1 of 2

Related Jobs

FoxNExt Games —
San Jose,
California,
United States
[03.20.19]
Producer