Many people with a disability play games despite difficulties in relation to access or quality of experience. Better access is needed, but there has been limited industry interest. For players with motor impairments the focus has been on the controller. Numerous solutions have been developed by third parties, but all are likely unsuitable for at least some users and there remains space for radically alternative angles. Informed by my experiences as a disabled gamer, concepts of affordance and control dimensionality are used to discuss the accessibility implications of controller design from the Magnavox Odyssey to the present. Notions of incidental body-controller fit and precarious accessibility are outlined. I subsequently draw on Lévy’s theory of collective intelligence and example games Keep Talking and Nobody Explodes and Artemis Spaceship Bridge Commander to develop a model that uses asymmetrical roles and diverse input to fit individual abilities and thereby expand participation.

Many people with a disability play games despite difficulties in relation to access or quality of experience. Better access is needed, but there has been limited industry interest. For players with motor impairments the focus has been on the controller. Numerous solutions have been developed by third parties, but all are likely unsuitable for at least some users and there remains space for radically alternative angles. Informed by my experiences as a disabled gamer, concepts of affordance and control dimensionality are used to discuss the accessibility implications of controller design from the Magnavox Odyssey to the present. Notions of incidental body-controller fit and precarious accessibility are outlined. I subsequently draw on Lévy’s theory of collective intelligence and example games Keep Talking and Nobody Explodes and Artemis Spaceship Bridge Commander to develop a model that uses asymmetrical roles and diverse input to fit individual abilities and thereby expand participation.

The barriers faced by people with disabilities are often considered in terms of two theoretical models: the medical model and the social model. The medical model of disability locates disability in the mind or body of the individual “patient” and emphasises linear restoration to “normality” (Gough, 2005). This was contested (UPIAS, 1976, pp. 3-4; Locker, 1983, p. 90) and there followed a shift to a social model of disability that posited that people are disabled by the attitudes of society (Shakespeare, 2016). If arguably outdated (Owens, 2014), the social model remains widely adopted.

Under the social model, suitably designed technologies are seen to enable people with disabilities to fulfil their desires to socialise, work, learn and play (Arrigo, 2005; Cobb et al., 2002; Hasselbring & Glaser, 2000); potentially improving independence (Jewell & Atkin, 2013) and quality of life (Arrigo, 2005). As video games have increased in popularity (Statista, 2017), there has been great interest in how video games might benefit players generally (Jackson et al., 2012; Posso, 2016) and people with disabilities specifically. For example, Jiménez, Pulina and Lanfranchi (2015) review how video games can improve the cognitive abilities of players with intellectual disabilities. Elsewhere, Rowland et al. (2016) study how active video games can help improve the fitness of players with lower-limb impairments. There are numerous other examples, but enjoyment remains the main motivation for many players.

The number of players with a disability is significant: there are 215 million players in the US alone and more than 32 million state that they have a disability (Statista, 2018; The Economist, 2017). While many players with disabilities find the types of games they can play limited (Flynn & Lange, 2010; Dolinar & Fels, 2014), there are many other people with disabilities who have been unable to participate at all (Flynn & Lange, 2010). Organisations such as AbleGamers and SpecialEffect strive to improve access, but resources are finite (SpecialEffect, n.d.).

With this background, this paper will now discuss some of the ways that disability and motor impairment specifically can impact participation, including relevant personal experience. Attention will be paid to the often-precarious nature of access given the rapid evolution of video game controllers. Existing work will be critically reviewed, and a new accessibility model developed around asymmetric roles and controllers. This addresses some of the limitations of existing solutions and provides plentiful possibilities for future work.

Impairment and the interaction loop

Disabilities and experiences of disability are inherently extremely diverse, but Bierre et al. (2005) propose that four broad types of impairment commonly impact video game players:

Visual impairment

Auditory impairment

Motor impairment

Cognitive impairment

By contrast, Yuan, Folmer and Harris (2011) focus on the effects of impairment, stating that difficulties typically relate to:

ability to receive feedback

ability to determine in-game responses

ability to provide input using conventional input devices

These can be positioned within a classic human-machine interaction loop (Fig. 1). In short, the user provides control input; the system processes the input and produces an output. This output is passed back to the user at the interface. The user perceives the system output, processes it, and provides further input (Bongers, 2000).

Figure 1. The human-machine interaction loop (Bongers, 2000).

For reasons that will shortly be apparent, this article will focus on motor impairment and restricted mobility, and the ability to provide input.

While traditional video games are considered sedentary (Lu et al., 2016), they make considerable demands of players in terms of eye-hand and bimanual coordination, dexterity of response and stamina in the hands. These demands meet the abilities of the player at the interface or, more specifically, at the controller. Players with motor impairments can be limited in the type or amount of input they can provide (Yuan, Folmer & Harris, 2011) and standard controllers can therefore be a particular barrier to participation.

Affordance and control dimensionality

Two concepts can help us to consider the interaction possibilities of video game controllers and the interaction demands made of players: affordance and control dimensionality (CD). Donald Norman (1988, p. 9) used the term affordance to refer to the actions made possible by an object’s physical form and properties. However, the intangible properties of software limited its tenability and the concept was revised to emphasise a distinction between “real” and “perceived” affordances: actions that are actually possible and actions that users perceive to be possible (Norman, 1999).

A more recent concept and specific to a video game context, CD is a measure of the degree of complexity inherent in a system as a result of its interaction demands and possibilities (Mustaquim & Nyström, 2014). There are two steps (Swain, 2008, pp. 135-138), the first being to assess the primary movement scheme:

One dimension of movement (left-right) CD = 1

Two dimensions of movement (left-right, up-down) CD = 2

Three dimensions of movement (left-right, up-down, in-out) CD = 3

A second step adds additional CDs for each secondary dimension of control:

For each additional movement dimension: strafe back-forth, accelerate-brake, rewind or fast-forward time, etc. (typically use two buttons) = 1

In isolation, controllers do not have any inherent CD as this is co-dependent on the particular game design, but they do have a (maximum) potential CD.

Evolution of the controller

As the affordances (i.e. action possibilities) of controllers have generally increased, so too have their complexity and interaction demands. To this end, the table below (Table 1) shows how the potential CD of a representative selection of controllers has evolved over time.

Table 1: CD of a Representative Selection of Video Game Controllers (1972-2017)

At this point it is pertinent to note that some kinds of interface resist classification. Firstly, the potential CDs of interfaces such as the Microsoft Kinect and integrated multitouch interface/platforms such as smartphones and tablets are highly variable and primarily defined by how they are used in the context of individual games: the player can be presented with simple, one-button gameplay or more complex, multi-dimensional interaction. Second, some interfaces feature secondary sensors. For instance, the Kinect features a microphone array, while smartphones and tablets typically contain a microphone, plus a 3-D accelerometer and/or gyroscope. These can be used to replace or supplement more conventional controls but are employed in selected games only. Thirdly, there are new types of interface whose affordances are not yet well understood. For instance, brain-computer interfaces (BCIs) are only starting to edge into the mainstream (Ahn, Lee, Choi & Jun, 2014).

Nevertheless, particularly relevant to players with motor impairments is the broad correlation between increased potential CD and increased action possibilities, and heavier interaction demands. Not all games use all of the potential CD, but newfound action possibilities are typically soon exploited by game designers; at the cost of increased interaction demands and complexity.

The first controllers were very simple (low CD). The Magnavox Odyssey controller (CD = 2) had one knob for vertical movement and a second for horizontal movement. There were no buttons. The included titles Table Tennis and Ski used only the vertical knob and horizontal knob respectively. These limited affordances resulted in modest interaction demands: ideal for a population only recently introduced to video games.

The Atari Computer System joystick (CD = 2.5) allowed movement in four directions and featured one action button only, but comparatively more sophisticated affordances evolved. For instance, Asteroids enabled the player ship to independently rotate left or right, thrust forward and fire. However, these action possibilities meant that players also needed to coordinate more than one simultaneous action.

The Nintendo Entertainment System controller (CD = 4) replaced the joystick with a four-way directional pad (D-Pad). It also featured two action buttons and two secondary buttons (Diskin, 2004). The D-pad required smaller movements than the Atari joystick and so games designers could demand players provide more nimble and precise input. Additional buttons afforded more varied player actions; at the cost of increased interaction complexity. For example, Punch-Out!! made use of the A and B buttons plus D-pad to punch left and right, but also the Start button to throw an uppercut (u.a., 2017). Similarly, Double Dragon II: The Revenge used a two-button combination to deliver a jump kick (Nintendo, 1990).

A decade on, the small analogue joysticks of the Nintendo 64 (CD = 9) and Sony DualShock (CD = 12.5) controllers had relegated the D-pad to secondary functions only. The twin joysticks of the DualShock (Plunkett, 2011) fell neatly under the thumbs and enabled movement and viewpoint to be decoupled. Running in one direction while aiming in another has become a canonical feature of first-person shooter titles (McMahan, Bowman, Zielinski and Brady, 2012), but poses a significant increase in interaction complexity: the player must not only coordinate two simultaneously 2-D inputs, but also provide input that is proportional and precise, at the same time as operating multiple buttons.

Incidental fit and the precariousness of access

If increased CD can produce (or be the result of) extended affordance, an inadvertent consequence of resultantly increased interaction demands is that players with motor impairments like Microsoft employee Solomon Romney can be abruptly excluded. Born without fingers on his left hand, Romney readily adapted to the simple controls of 1980s arcade machines (Stuart, 2018) and played for over a decade. However, the expanded affordances of the PlayStation controller eventually led to unmeetable demands related to extensive use of “chaining” together sequences of button combinations (Stuart, 2018).

My own experiences are similar. Born with transverse hemimelia and bilateral fibular hemimelia, I have no left arm below the elbow except for a thumb-like protuberance on the elbow, and extensive lower limb deformity. Despite this unpromising physicality, I received a NES for my fifth birthday. With no predetermined concept of how to play, I intuitively rested the controller on the floor, then used my left foot to manipulate the D-pad and right hand to operate the buttons. I had no sense that this was unusual.

Over time I moved from the NES to the SNES, to the Sega Dreamcast and finally on to PlayStations 1 to 4. The evolution of the controller prompted my interactions to slowly develop into the comparatively more conventional style I use today (Fig. 2). The flexibility and open-endedness of these interfaces arguably facilitated this serendipitous fit: the controller did not prescribe exactly how it must be used but could instead (inadvertently) support varied bodily affordances and interaction styles.

Figure 2. The current interaction style of the author: left thumb at elbow joint operates left analogue stick and D-Pad, right hand operates right analogue stick and all buttons.

If such instances of incidental fit between unconventional bodily affordances and standard controllers are rarely mentioned in a video game context, there are antecedents elsewhere, notably in music; another domain that requires complex real-time control. Examples include pianist Paul Wittgenstein and guitarist Django Reinhardt (Dalgleish, 2014).

There is nonetheless a fundamental difference between the two domains: traditional musical instruments have evolved extremely slowly (Fletcher & Rossing, 1991: v-vi), but video games and their controllers have evolved very rapidly (Bhardwaj, 2017): relatively suddenly, players can find interaction demands unmeetable. Like Romney (Stuart, 2018), my own exclusion came unexpectedly. It related to the shift from handheld to gestural controllers. The innovative Wii remote was unproblematic in itself; the problem lay in that most games required concurrent use of a Nunchuck (second controller). While more conventional controllers can be held – if not necessarily operated – by one hand, this combination assumed that the player could both hold and operate one controller in each hand: a seemingly inconsequential difference that essentially precluded my participation.

The success of the Wii inspired related approaches. Most notably, Microsoft developed the Kinect; a camera and infrared-based motion controller for the Xbox 360 that promised whole body interaction without a physical controller (Zhang, 2012). However, it soon became apparent that the Kinect could not map its skeleton model onto my body: I had to return to using a more conventional controller only. The irony is that Nintendo and Microsoft intended these systems to broaden participation (Ulicsak, Wright & Crammer, 2009; Chen, Li, Ngo & Sun, 2011). Instead, their narrow and inflexible schemes of allowable interactions effectively disempowered one subset of possible users in order to empower another.

Related work: adapted controllers and new designs

Examples similar to the above have rarely been considered in prior literature. Instead, efforts to increase the accessibility of video games for players with motor impairments have tended to focus on three aspects:

Remapping of controls in software

Hardware modifications to standard controllers

Development of new/alternative controllers

Remapping refers to the ability of software to flexibly redistribute controls in order to suit particular player abilities and preferences. Game Accessibility Guidelines state that: “Many people […] benefit greatly from being able to move essential controls into positions that they are able to reach more easily” (Game Accessibility, u.d.). For instance, professional Street Fighter player BrolyLegs (Street Fighter, 2016) remaps controls in order to play using his face. If the ability to remap controls is implemented, a range of different mappings can be quickly tested, at little or no cost. However, remapping alone may be insufficient for some needs and is rarely implemented on consoles (Game Accessibility, u.d.).

Modifications to standard controllers have been aided by the spread of Maker culture. They vary considerably in their complexity. At one end of a continuum are modest changes to controls to make them easier to grip or press. For instance, Caleb Kraft has presented a relatively simple, modular design for 3-D printed joystick that can broadly increase controller accessibility (Kraft, 2015a), as well as modified thumbstick buttons for a player with muscular dystrophy (Kraft, 2014). Other modifications are more complex. For example, the Single Handed Gaming Controllers for Accessibility Use project by Ben Heck (u.d.) extensively modifies standard controllers to enable one-handed operation.

While modifications to standard controllers can be useful for some players they have some limitations. For instance, they can be difficult to produce in larger numbers. As Caleb Kraft (2015b) comments: “The biggest issue that I run into is time. I simply can’t keep up with the requests.” Another issue for adapted controllers is that they can be difficult to sell as they are usually informal (and relatively untested) modifications of a commercial device (Iacopetti, Fanucci, Roncella, Giusti & Scebba, 2008).

With the Nintendo Hands Free controller for the NES a rare exception (Plunkett, 2009), the main manufacturers have shown limited interest in accessible controllers. Instead, new, accessibility-focussed controller designs have typically come from third parties. Quadstick (u.d.) have produced three controllers aimed at quadriplegic players and featuring combinations of spatial, pressure and “sip/puff” sensors, and head-operated joysticks. At the other end of the body, Gyorgy Levay and team developed the Game Enhancing Augmented Reality controller; a padded device operated by the feet (Cragg, 2016).

More flexible, modular approaches have been proposed by Iacopetti et al. (2008) and, most recently, Microsoft. The Microsoft Adaptive Controller is informed by its earlier Xbox Elite controller: a professional esports controller adopted by some players with limited mobility (Stark & Sarkar, 2018). The Adaptive Controller is not aimed at a specific disability but provides a flexible hub for additional input devices (Microsoft, 2018; Englard, 2018). Released in late 2018, its customisability has the potential to meet the needs of disparate players, without the sometimes-prohibitive costs of bespoke production.

Entirely new classes of controls continue to emerge, and some have been applied to a video game accessibility context. For instance, Brain-Computer Interfaces (BCI) emerged in the early 1970s (Vidal, 1973), but more refined, widely available and lower cost designs have appeared only in the last decade. As BCIs – primarily using electroencephalographic signals – have become more readily available, they have been used as video game controllers (Ahn, Lee, Choi & Jun, 2014). Researchers have also started to explore the potential of BCIs for disabled gamers (Maby et al., 2012), but they remain nascent and beyond further coverage in this paper.

Asymmetrical roles and asymmetrical controllers

This article has identified numerous ideas and developments around controllers and accessibility. All are likely imperfect to at least some people and universal accessibility is probably unachievable (Barlet & Spohn, 2012; Vanderheiden & Henry, 2003). As such, alongside additional considerations such as cost, the need for setup help and hard-to-predict individual preferences, it is surely fruitful to continue to explore multiple directions.

While attempts to make games more accessible to players with motor impairments have focussed on controllers and mappings, game mechanics have also been considered. For example, Barlet and Spohn (2012) discuss the power of adaptive difficulty. More generally, the social mechanics of multiplayer games have been deliberated (Quandt & Kröger, 2013; Siitonen, 2007), but standard controllers are usually assumed. There also remains scant consideration of how multiplayer experiences can inform controller design; or how they might inform the interface for motor impaired players.

A rare exception is a user-selectable Xbox One feature called Copilot. This links two controllers so that they can be used as one, in order to provide assistance as needed (Englard, 2018). The Copilot model can be criticised on at least three grounds. First, the pilot-co-pilot relationship has an inbuilt power dynamic; the pilot player is beholden to a “better” player for help. Second, all other aspects of the game remain identical to the single player version: the potentials of multiple players are not explored. Third, social isolation is an issue for many people with disabilities (Scope, 2017), but Copilot allows for cooperation in-person only; it does not exploit the social potentials of the internet.

A less restrictive basis is provided by Pierre Lévy (1997, p.13) and the notion of collective intelligence: “a form of universally distributed intelligence, constantly enhanced, coordinated in real time, and resulting in the effective mobilization of skills.” Proposed at a more utopian time for the internet, collective intelligence posits a shift from human-machine interaction to collaborative human-human interactions. Particularly relevant for its implicit embrace of diversity is the assertion that: “No one knows everything, everyone knows something, all knowledge resides in humanity” (Lévy, 1997, p.13); a notion reinforced by the statement: “Before we can mobilize skills, we have to identify them. And to do so, we have to recognize them in all their diversity” (Lévy, 1997, p.13).

Lévy (1997) already intends “intelligence” to refer to a duality; not only the construction of ideas but also the construction of people (i.e. society) (p. 10). Perhaps it can be extended further still, to account for how brain-centric views of cognition have been increasingly challenged by embodiment: the notion that the body is needed for intelligence (Pfeifer & Bongard, 2007, xvii). Related, Paul Dourish (2001) has proposed embodiment as a foundation for human-machine interaction. If this shift can be accepted, it is possible to reframe Lévy’s statement to conceive a model of interaction whereby no-one can do everything, but everyone can do something.

An example of this kind of asymmetric collaborative play in a musical context is Harmony Space: a multi-user system created by Simon Holland (1993) that enables collaborative roles to be split in numerous ways. For instance, non-traditional roles such as steering the root note, changes to key, chord size and inversions, can be distributed across one or more players. Many pieces of music require only two or three roles, but interplay can still deliver rich sequences. A more recent version of Harmony Space (Bouwer, Holland & Dalgleish, 2013) also expands the variety of controller types that can be used, from dance mats to MIDI foot pedals. This added flexibility enables players to choose the most appropriate controller for their selected role and abilities.

There are also examples of video games that make use of asymmetrical roles. For instance, Gandolfi (2018) explores how players use asymmetrical roles to develop cooperative and computational thinking in the online multiplayer games Overwatch, For Honor and Tom Clancy’s Rainbow Six: Siege. However, these are not aimed at players with a disability and the expectation is for all players to use a similar controller.

More useful in an accessibility context are Keep Talking and Nobody Explodes (KTNE) and Artemis Spaceship Bridge Commander (ASBC). KTNE features one player as the Defuser and one or more other players as Experts. The Defuser must diffuse a bomb before time runs out but must be guided by the Experts. The Experts have access to the instructions but cannot see or interact with the bomb directly. Controller options for the Defuser include keyboard and mouse, touchscreen and gamepads, and the Experts and Defuser communicate verbally, in person or online. A number of design decisions help to increase the game’s accessibility. First, players are able to select from two roles that are equally vital but highly asymmetric in their interaction demands. Second, the Defuser role supports several different input devices, and these may further increase the diversity of compatible bodily affordances. Third, a “free play” mode enables the countdown timer to be adjusted; useful if a player requires extended time to provide input.

ASBC requires multiple (ideally six) players, and at least one Microsoft Windows computer. The six available roles are highly asymmetric in the skills and abilities they require for players to succeed (Justin, 2013). While the Captain must use a Windows machine, mobile devices can be used for the other roles if desired. Regardless of the devices used, inter-player voice communication is the primary way to coordinate complex ship operations.

A strength of both titles is that while roles are highly asymmetrical, fundamental principles such as reward, goals, challenge and meaningful play (Barrett, Swain, Gatzidis & Mecheraoui, 2016) are maintained. While ASBC is more restrictive in some respects than KTNE, its more extensive roles also allow for more extended differentiation in role requirements and interaction demands. That all but the player in the Captain role can choose from mobile, laptop or desktop platforms enables players to use a wide variety of devices and to customise interaction to suit individual requirements.

The “asymmetrical roles and asymmetrical controllers” (ARAC) model exhibited by KTNE and ASBC is still to be formally tested in the context of games and disability. Indeed, there are few if any documented cases of informal use. Nevertheless, the asymmetrical roles and asymmetrical controllers can be seen to oppose the notion of “parallel game universes” developed by Grammenos, Savidis and Stephanidis (2009). This aims to create subspaces of differential difficulty in order for both players with and without impairments to play cooperatively or against each other in a way that provides equivalency of difficulty (Grammenos et al., 2009). However, in order to achieve a balance, the player with impairments is provided with a simplified and arguably lesser version of the “full” experience.

Rather than segregate players with disabilities by placing them in exclusive sub-sections that provide “cut-down” versions of the canonical experience in an attempt to manage challenge and difficulty, the ARAC model has all players – impaired or otherwise – play in the same space. Players also all engage in the same tasks (there are no simplified iterations) but do so from different perspectives. More specifically, players can adopt different roles and input modalities that best suit their individual abilities. For instance, players with motor impairments might use voice input to direct the actions of other players or use physical input devices in non-real-time (i.e. less temporally-critical) roles.

Conclusions and future work

This paper has outlined the evolving nature of developments around video games and accessibility, highlighted some current limitations, and outlined the ARAC model as a potential solution to some of these issues. A particular problem for any accessibility model is a persistent lack of adoption, seemingly as a result of apathy on the part of industry. This apathy is likely at least partly financially motivated in that, despite the apparent size of the market, prior developments have tended to be one-offs: bespoke adaptations or entirely bespoke designs for individual players. These are usually relatively costly and require specialised expertise to develop but have few of the economies of large-scale production and limited mainstream potential. The ARAC model has promise in this respect: rather than craft one-off controllers to adapt titles that would ordinarily be inaccessible, ARAC improves accessibility by making diversity of roles and input a fundamental tenet of game design.

From an accessibility perspective, it is important that a wide variety of input devices are supported so that players can try out and mix-and-match different combinations until individually optimised solutions are found. Moreover, asymmetrical roles inherently imply the use of a range of input devices: different roles have different affordances and interaction demands, that in turn imply particular types of input. Although controllers are often treated as generically interchangeable, KTNE already demonstrates how matching of game mechanics and input modalities can produce gameplay that is original and accessible.

It is important to note that this does not necessarily mean simple or simplistic gameplay. Although, as in Harmony Space (Holland, 1993), individual roles may have modest interaction demands (i.e. low CD), heterogeneous but well-coordinated combinations of players can adeptly complete very complex tasks: as is also the case with complex problems in the real-world (Kirschner, Paas, Kirschner and Janssen (2011)).

If roles can already be seen to suggest some types of controller over others, matches between controllers and unconventional bodily affordances remain poorly understood. For instance, if simpler (low CD) controllers can allow for more flexibility of use and may therefore accommodate diverse users, it is not yet clear how to predict matches between bodies and controllers new or old, bespoke or mass-produced. Thus, trial and error remain a necessity for most motor impaired players and disabled players more broadly. As such, there is still much to be done. Pertinent future directions include:

Identification of additional games that have the potential to be case studies, and to study their player-game relationships longitudinally and preferably in-the-wild so that issues can be understood in situ.

Exploration of particular aspects within the broader ARAC model to enable more certainty in implementation and leading to guidelines for game designers and the design of new games. For instance:

development of a framework for assessing the likely fit between (old and new) controllers and diverse user needs;

examination of how to best match player roles to a variety of interfaces.

Development of interfaces that reflect how the effects of disability can change over time (even day to day) and adjust their response accordingly (i.e. adaptive interfaces).

Education of games industry staff, particularly those with strategic responsibilities, in order to increase awareness of accessibility issues. This may be crucial if accessibility is to be “designed in” to games from the outset of their development.

Cragg, O. (2016). ‘Handi-capable’ video game controller lets disabled gamers play without hands. International Business Times, 24th June 2016. Retrieved from https://www.ibtimes.co.uk/handi-capable-video-game-controller-lets-disabled-gamers-play-without-hands-1567256

Dalgleish, M. (2014). Reconsidering process: bringing thoughtfulness to the design of digital musical instruments for disabled users. In Proceedings of INTER-FACE: International Conference on Live Interfaces.

The Economist (2017). Games for people with disabilities. The Economist, 30 September 2017. Retrieved from https://www.economist.com/science-and-technology/2017/09/30/games-for-people-with-disabilities

Hue is a successful digital game from the independent game company, Fiddlesticks, that is focused around color theory. The game’s core concept, however, made digital game play impossible for players who are colorblind. Hue’s built-in and customizable game modes allow colorblind players to use patterns instead of colors to play the game. Through an interview with Hue’s creator Henry Hoffman, this article details Hoffman’s design process and proposes a framework for accessible game design based in universal design principles.

Hue is a successful digital game from the independent game company, Fiddlesticks, that is focused around color theory. The game’s core concept, however, made digital game play impossible for players who are colorblind. Hue’s built-in and customizable game modes allow colorblind players to use patterns instead of colors to play the game. Through an interview with Hue’s creator Henry Hoffman, this article details Hoffman’s design process and proposes a framework for accessible game design based in universal design principles.

Digital games are currently one of the largest sectors of the entertainment industry, with a U.S. market value of $18.4 billion in 2017 (Statista, 2018). Bierre et al. (2014) note, however, that the digital game industry excludes a significant portion of consumers because of the difficulty of making many games accessible to players with disabilities. Yuan et al. (2011, p. 86) estimate the number of people whose ability to play digital games is impaired by disabilities to be 32.2 million, or 1% of the total U.S. population. While assistive technology solutions, such as screen readers, magnifiers, voice recognition systems, and adaptive controllers and keyboards exist, many of these technologies are not compatible with widely used game consoles and popular games themselves. There is a need, then, to consider not only the ways to adapt a game for players with disabilities after it is produced, but to integrate considerations of disability within the design process itself.

This paper considers the process through which one digital game designer approached adaptations of the popular digital game Hue (2016) for players with one particular disability: color blindness. Through an analysis of an interview with the game designer Henry Hoffman, this article considers the ways that Hoffman’s design process follows principles of Universal Design (UD), an approach to disability adaptation that considers accessibility at the beginning of the design process. Hoffman’s example provides a model for other game designers to follow in developing successful games that are accessible for players with specific disabilities.

Accessibility in Gaming Design

While accessibility and accessible design are growing areas of concern in education, education technology, and other areas of digital technology, digital games are a bit behind in addressing the needs of individuals with disabilities. The W3C set accessibility standards for websites worldwide, but digital games have more specific and often idiosyncratic requirements. Yuan et al. (2011) note the difficulty in improving accessibility in digital games; technologies and consoles change frequently and use different configurations and hardware. Digital games also are doing more than simply presenting information to the user, and they also require interaction through user feedback (Yuan, 2011, p. 82). Some players can use alternate input devices, including eye controllers and vocal joysticks, but these devices usually have limitations on the amount and type of input that the device can record.

Yuan et al. (2011) propose that games that need just one form of input, called one-switch games, can be easiest to make adaptable, especially for players with visual or motor impairments who use alternative input devices; a few of these popular games include remakes of earlier classics, including Frogger, Sudoku Access, and Gordon’s Trigger Finger, a modification of the FPS game Half-Life 2. Bierre et al. (2014) also surveyed game developers on specific games that were designed with accessibility in mind. Like Yuan et al., these scholars mentioned a number of games that have been adapted from commercial titles in order to shape gaming experiences for individuals with specific types of disabilities. They also described popular commercial titles with adaptations for players with disabilities, including Half Life 2, with closed captioning options tested in collaboration with deaf players; Doom 3, with a closed caption modification developed in collaboration with players after the game’s release; Terraformers, created with a sonar system for vision-impaired players; and F355 Ferrari Challenge, with an intelligent braking system for players with slower reaction times (p. 8-9).

Color Blindness in Digital Games

Color blindness, which affects nearly 15% of Caucasian males (Zammitto, 2008, p. 272), has not only proven problematic for digital game creators, but is often ignored for commercial or production reasons, or much less even considered to be an issue. When digital games do adapt to players’ needs, rarely do they make any news about it, nor is it much beyond a cursory notification in the game’s instruction manual. Scholars such as Tanuwidjaja et al. (2014) have investigated the issue at length, designing Chroma, a Google Glass inspired technology to provide “real-time textual feedback about the color at the center point of the scene” (Tanuwidjaja et al., 2014, p. 801). Another app on iTunes Tanuwidjaja et al. (2014) mention is HueVue, which helped the colorblind “to identify, match and coordinate colors” in addition to providing detailed color information just by touching any image. (Cnet, 2010).

Several other studies have examined these issues within game design (Duvall, 2001), players with disabilities (Lim and Nardi, 2011), and even eye testing in children (Gaggi and Ciman, 2016). While these technologies do make exceptions for individuals who are colorblind and make various media more palatable and usable by those with this particular disability, they invariably work as add-ons, as secondary technologies. Disability studies scholar Jay Dolmage (2008) calls such approaches “retrofitting”:

To retrofit is to add a component or accessory to something that has already been manufactured or built. This retrofit does not necessarily make the product function, does not necessarily fix a faulty product, but it acts as a sort of correction. (p. 20)

While these technologies can assist players who are colorblind in playing the digital game more effectively, they do not solve the underlying problem that makes the digital game inaccessible to a portion of the gaming market.

Universal Design

Universal design (UD) is a broad set of design principles used in architecture, technology, and education to create spaces and objects that take into consideration the needs of differently-abled and older adults. The core tenant of universal design is to start from the earliest stages of a product’s design to take into consideration how different sorts of users might use the space or object. By foregrounding concerns of accessibility, designers can create products that work for the widest range of people possible. The most recognizable and perhaps ubiquitous of these adaptations are curb cutouts. The Center for Universal Design at North Carolina State University (1997) has developed the following best principles for universal design:

Equitable use

Flexibility in use

Simple and intuitive use

Perceptible information

Tolerance for Error

Low physical effort

Size and space for approach and use

These principles, as a whole, allow for multiple approaches and means for reaching an objective. Whether it is creating a space with wide hallways and movable furniture to accommodate wheelchairs and other assistive devices, or allowing for user error and multiple approaches, the overall objective is to provide flexible, user-friendly objects for all individuals, not just those with disabilities.

While universal design principles were first developed for architecture, they have been applied to other fields, including education and educational technology. UD approaches in these areas may be the most productive to consider for digital game design. Dolmage (2015) argues that anyone utilizing universal design for education needs to consider three broad principles in its creation that will avoid problems of retrofitting and ensure the most accessible approaches:

Multiple means of representation, to give learners various ways of acquiring information and knowledge,

Multiple means of expression, to provide learners alternatives for demonstrating what they know,

While Dolmage’s framework is specific to instructional design and delivery, the principles are broad enough to provide a flexible framework for game designers. Considering multiple means of the representation of information as well as multiple means of engagement will allow digital game players to customize digital games to fit their needs, whether it is closed captioning on a digital game (as a differentiated means of representation) or through multiple possibilities for player controls or inputs (as multiple means of interaction). Taking these issues into account early in the design process provides more flexibility and possibilities for meeting accessibility needs than those available later in the process.

Hue

Indie developer Fiddlesticks and publisher Curve Digital made their commitment to an adaptive technology from the start with their digital game Hue, a runaway hit, both commercially and critically. As the protagonist Hue, the player explores a gothic story of a boy searching for his missing mother through world of black and white environments that can be manipulated with a color wheel that is slowly filled in with various colors as the game progresses. Obstacles can disappear and reappear with a turn of the wheel, or reveal new puzzles to solve, as the game increases in difficulty all while adding eight colors: Blue, Dark Blue, Purple, Pink, Orange, Red, Yellow, and Lime. The very first press release describes the game as, “A heartfelt story that touches on themes of love, loss, existence and remorse” with “Over 30 original music tracks, composed exclusively for Hue.” The designers explicitly state that Hue features “full colorblind support, using symbols as a color aid” (Hue, 2016).

Creator Henry Hoffman has noted this was a key consideration in making the game. “After much research, testing, and seeking advice from color-blind communities, we devised a simple symbol system which we hope will make the game accessible to all. We really want as many people as possible to enjoy Hue” (Musa, 2016). The rest of this article details the design process for this specific digital game specifically.

Methodology

In order to more fully consider the ways that accessibility for players who are colorblind colorblind players was incorporated into the design process for Hue, I conducted an interview with Hue creator Henry Hoffman, the Creative Director and Co-Founder of indie game developer, Fiddlesticks. I spoke with Hoffman at length over Skype and asked questions regarding the development of Hue and the design process. The interview was transcribed, and I analyzed the transcript for common themes. While the experience of one designer cannot be extrapolated to the experiences of other game designers, as a case study Hoffman’s experience can identify trends and questions for further inquiry.

Interview Details

Hoffman explained that the central concept of the digital game was color theory. He noted that he begins a game around a central concept or mechanic, and then builds the game from there:

I’ve got a very consistent approach to game design in that what I find most interesting about game design is coming up with new ways to interact with things that haven’t been interacted with before. So, for me, it’s all about devising interesting game mechanics that haven’t been explored. [. . .] In Hue’s case, what I was doing is I had PhotoShop open I think, and I had sort of a color fill background, and I had a layer with something on, and I was just sliding the hue slider around on one of the layers, and I noticed that as the hue of the foreground layer matched the hue of the background layer, it kind of visibly melted and disappeared into the background. And I thought it would be interesting, as a game mechanic, if it also physically sort of disappeared.

Hue as a game, then, was based around color as a central concept. Zammitto (2008) has noted the ways that digital games traditionally have relied on color to communicate meaning, for example, using the color red for danger when the player’s health bar gets low. In Hue, however, the central game mechanic requires the player to switch between color backgrounds. In the game, as noted above, players unlock an array of colors and switch between them as they play, making the objects appear and disappear based on their own colors. The game’s central concept is based on color, so the game has an immediate accessibility problem for those players with colorblindness.

Hoffman reported that he discovered this issue not in the initial design process, but during initial testing:

So, I don’t have a background in game accessibility at all, so I was just kind of thrust into this without any real notion that this was going to happen. […] When I started taking it to game conventions very early on, I think we had a five-minute demo that I put together in two weeks, and I took it to game convention just to get some player feedback, and it was immediately apparent that a large amount, a substantial amount or a percentage of people were struggling to play the game, and that was due to their inability to differentiate between different colors due to colorblindness–various forms of colorblindness.

Hoffman noted that he doesn’t necessarily know anyone with colorblindness, and he had not initially anticipated that it would be an issue with this particular game. As a small developer, Fiddlesticks does not have a quality assurance (QA) department, but the company instead tests their games at various conventions and events with real players, which is where he first found colorblindness to be a problem with Hue.

After this discovery, Hoffman sought out players who are colorblind for advice on how to address issues of accessibility with this group, as well as game testing:

On Reddit, there’s a colorblind subreddit, and I posted a video, a really early video of the gameplay, and I was like, “Guys, I think I created your worst nightmare. Can you please help me?” And they provided loads of feedback, lots of people offering to do game testing. People came up with the idea of using symbols, using patterns. I actually experimented very early on with using patterns, which are a much simpler aesthetic, having sort of a pattern overlay over each different color and being able to match those patterns. But then as the aesthetic developed and we got these intricate details, overlaying patterns became too much visual noise, and even people who weren’t colorblind struggled to play the game. So, I was like, okay, that’s not going to work. So I ended up devising sort of this symbol-based system, my own symbol-based system, which ended up getting a substantial amount of press, which I’m very happy about.

Universal design principles advocate for an inclusion of users in the design process, as Dolmage (2015) has noted. While Hoffman did not anticipate accessibility issues when designing Hue, he was quick to reach out to individuals with colorblindness to come up with a solution that worked for them. He included these users in the design and troubleshooting process in order to make sure that his solution was 1) flexible; 2) intuitive; and 3) easy to use, all key principles of universal design. What is especially notable about this example is that the solution came not from Hoffman or the other game designers, but instead was a suggestion from the community itself.

Hoffman reported that he found these players, particularly players who are colorblind, essential to the design and development process:

I think we wouldn’t have even recognized that there was an accessibility issue unless we had that dialogue in the very beginning. So, it was super important to get people in front of it. And I think the aesthetic – and the world itself, the world building itself – constantly developed when we were showing it to people. We showed it at 40 different events throughout the development of Hue, and in that process, we just got such huge amounts of feedback.

Universal design principles, as described by the Center for Universal Design (Connell, 1997), at their core are simply best practices for design. Applying these principles to digital game design, involving users at all stages of the design process, and incorporating their experiences, can create better gaming products and experiences. In this example, Hoffman found the testing phases with players to be important not only for issues of accessibility, but also for refining other elements of the digital game as well.

He also found the accessible features to be incredibly successful; players rated the adaptations highly, and they were also utilized by other players, not just those with colorblindness:

There’s been a huge response. We’ve had colorblind streamers who have streamed the game and sort of expressed support. There’s been players that have got no vision impairment whatsoever who turn on colorblind mode just because they prefer it, because it’s another form of visual reinforcement.

While many features of accessible design, including the curb cutouts mentioned above, are created for those users who need accommodations, they assist all users, and many individuals who do not have specific disabilities also use and appreciate the adaptive features.

These design experiences have given Hoffman a sense of the importance of building accessible games, and not just for players who are colorblind He also described another situation that caused him to understand the need for flexible, accessible design:

When I came into this, I had a vague awareness of accessibility but I didn’t realize quite how important it was and how pervasive some of these things that people can suffer from are. And I think one of the most notable things I remember is our showcasing … in London, and there was a player with a physical disability who wasn’t able to use a keyboard and mouse, and I don’t think they were able to use an analog stick on a controller. So, at that point we didn’t have accessible controls, so he wasn’t able to play the game at all. He was really disappointed, and you could see that he was really excited to play the game, and then the disappointment when he physically wasn’t able to. That, for me, was a real eye-opener, and because of that we added accessible controls, which allows players to sort of personalize the controls so that they can play how they like.

This comment demonstrates the importance of building flexibility into the game design process wherever possible. While many players may not need or use these controls, they can be crucial for those who do need them. Hoffman’s experience here demonstrates how he learned firsthand the importance of accessible game design.

While the importance of accessible design is clear, these adaptations can certainly cost money. For small developers reliant on funders for project development, decisions about accessibility ultimately come down to cost. Hoffman noted that the positive attention Hue received because of its accessibility features have made the game more commercially successful:

We even released promotional videos sort of speaking game accessibility, and if you look at MetaCritic, what’s really interesting, and this is something that I say often when giving talks about accessibility features is that, as a small studio and developer, when you’ve got other people’s money on the line, you need to make a business case for accessibility features. And if you’re willing to accept that your MetaCritic rating directly influences your sales, if you go through our MetaCritic reviews, every single positive review praises the accessibility features. So, I think there’s a huge business case.

While building accessibility accommodations into digital games can open them to a different audience, they can also make the games more marketable. That feature, more than ethics of accessibility, can be a driving force in increasing accessibility in digital games.

Hoffman described accessible game design as a growing area, and one driven by this commercial viability:

I think a lot more people are starting to realize that if you make games as well, because you’re making games and small elements of these games aren’t sort of accessible, and they’re realizing that players aren’t able to play the games and those players are disappointed, so I think there’s a kind of a collective awakening. […] I think the bigger studios are going to be looking at the bottom line, and I think the game designers are probably going to be pushing for making the game accessible so more players can enjoy the game. I think there’s going to be arguments coming from all sides because it just makes sense. Like there’s no argument against it at this point.

As Neely (2017) argues, digital game designers can no longer afford to ignore concerns of disability and accessibility when it comes to design. The broader appeal of digital games, and the need to make them adaptable to players’ abilities, make this a required issue in the marketplace.

While making accessible games are important to Hoffman, accessible design is a secondary concern to the overall game design itself:

But I think, for me, what’s important about accessibility is that you’re not compromising the game design vision in order to make a game more accessible. What you’re doing is just allowing as many people to play your vision as possible without compromising the game. For me, game design comes first, and making it accessible is sort of a really high priority after that.

Hoffman’s primary concern is to build a digital game that fulfills his vision, and then he works to build accessibility accommodations. While they do not come first in his own game design, when integrated early enough in the process they can enhance and even improve the game overall. In the case of Hue, building a pattern-based alternative to the color backgrounds and the subsequent testing with players with different abilities improved the game as a whole. It also created an important market and marketing approach for the game after its release. Working to make Hue more accessible built a better game.

Henry Hoffman provides just one example of a digital game designer who was confronted with issues of accessibility and worked to adapt. We can learn a good amount about accessibility issues in design, however, by considering his experience. The main findings of this interview are as follows:

Hoffman described the central concept of Hue to be color theory.

Hoffman first discovered accessibility issues with color during initial game testing with an early version of Hue.

Hoffman approached players with colorblindness to develop a solution for his game design and involved these players in the design process.

Hoffman solved these accessibility issues by offering a pattern option rather than a color option.

Hoffman saw accessibility as a central concern for all digital game designers, one that will help a game to reach more audiences and also to perform more successfully in the competitive digital game market.

Hoffman noted that he came to these issues in game design organically. Because he emphasized color as his central game mechanic, he soon learned that this design feature was a limitation for many players. His process in solving this problem by involving players with colorblindness themselves is a strategy recommended by many experts in universal design and can serve as an example for other designers.

Conclusions

Henry Hoffman’s introduction to issues of accessibility in digital game design was based on his first-hand experience with different players. While designing games for individuals with disabilities was not a primary goal of his, he turned to accessible design as a means to solve a problem. While Hoffman’s experience cannot stand as representative for all designers, and we cannot extrapolate beyond his experience, his emphasis on both the positive press he received from Hue’s accessibility features, as well as the financial argument for accessible design brings up issues of concern for many in the digital game design industry. Pitaru (2008) notes the importance of accessible game design, which is still a neglected issue in industry. As Pitaru argues, “even the slightest impairments can severely compromise [players] ability to play mainstream games” (p. 75). As accessible design is becoming a larger concern, and even a market for digital games, it’s worth considering not only the process of game designers in approaching accessibility accommodations, but also ways to incorporate accessible, and in fact universal, design throughout the creation process.

While these concerns were not in the forefront of his mind or his experience, Hoffman’s approach to Hue demonstrates the flexibility and usefulness of universal design principles for game design. As Hoffman described, he had a short mockup of the game when he began testing it with different groups of players and discovered the color issue for players who are colorblind. This realization early enough in the design process allowed Hoffman to incorporate the needs, concerns, and even ideas of players who are colorblind in order to create a more accessible gaming experience. As advocated for by many universal design proponents (Connell, 1997; Dolmage, 2008, 2015), Hoffman worked with the community of individuals who are colorblind, and the solution to his accessibility problem came directly from this community itself. While the pattern innovation helped players who are colorblind, it also helped other players as well in providing customization and allowing for ease of use. Hue has won several awards: the Develop Indie Showcase 2015 as well as the Develop Indie Showcase, Game of Show and Best Art at Casual Connect USA, and Best Design at Casual Connect Europe. Hue serves as a model for the ways that digital games, even those from small and indie developers, can excel both creatively and commercially by embracing accessible design.

Universal design principles and approaches present what I argue is a productive approach for digital game design. Jay Dolmage (2008, 2015) advocates for design approaches that allow for multiple means of input, multiple means of output, and multiple means of interaction. Considering these approaches at the beginning of the design process can not only avoid the issue of retrofitting, but it can also ensure ease of use for players with a wide range of disabilities. Testing the game throughout the process with a wide range of players will allow designers to see these approaches in action and allow for troubleshooting as well.

Hue and Hoffman’s experience also emphasizes the importance of leaning on the community of individuals with disabilities for support, not only for testing the game, but also in solving accessibility challenges. It is notable that in this case, as described by Hoffman, the ultimate solution for issues of colorblindness came from players who are colorblind themselves. This experience demonstrates how crucial it is to include the community of individuals with disabilities into the design process, in order to create flexible games with multiple means of both representation and interaction. As accessibility becomes a larger and larger issue within digital gaming, as Pitaru (2008) and Neely (2017) argue, building a comprehensive approach to accessible game design is becoming more crucial. In these ways, Hue can serve as a model for successful possibilities for other game designers.

Pitaru, A. (2008). E is for everyone: The case for inclusive game design.. In K Salen, (Ed.) The Ecology of Games: Connecting Youth, Games, and Learning. The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Press, 2008. 67–88. doi: 10.1162/dmal.9780262693646.067

Statista (2018). Value of the video game market in the United States from 2011 to 2020. Retrieved from: https://www.statista.com/statistics/246892/value-of-the-video-game-market-in-the-us/

]]>Benjamin Vercellone (Missouri Department of Social Services), John Shelestak (Department of Biological Sciences, Kent State University), Yaser Dhaher (Department of Mathematics, Kent State University) and Robert Clements (Department of Biological Sciences, Kent State University)

Abstract

Vision enables a person to simultaneously perceive all parts of an object in its totality, relationships to other objects for scene navigation, identification and planning. These types of perceptual processes are especially important for comprehending scenes in gaming environments and virtual representations. While individuals with visual impairments may have reduced abilities to visualize objects using conventional sight, an increasing body of evidence indicates that compensatory neural mechanisms exist whereby tactile stimulation and additional senses can activate visual pathways to generate accurate representations. The current thrust to develop and deploy virtual and augmented reality systems capable of transporting users to different worlds has been associated with a push to include additional senses for even greater immersion. Clearly, the inclusion of these additional modalities will provide a method for individuals with visual impairments to enjoy and interact with the virtual worlds and content in general. Importantly, adoption and development of these methods will have a dramatic impact on inclusion in gaming and learning environments, and especially in relation to science, technology, engineering, and mathematical (STEM) education. While the development, standardization and implementation of these systems is still in the very early stages, gaming devices and open source tools are continuing to emerge that will accelerate the adoption and integration of the novel modes of computer interaction and provide new ways for individuals (with visual impairments) to experience digital content. Here we present the rationale and benefits for using this type of multimodal interaction for individuals with visual impairments as well as the current state of the art in haptic interfaces for gaming, education and the relationships to enhanced end user immersion and learning outcomes. As these technologies and devices are adopted and evolve, they are poised to have a dramatic impact on entertainment, education and quality of life of individuals with visual impairments.

Successfully navigating and interacting with video game environments is naturally tied to the ability of the player to visually discern objects on the display and understand spatial relationships between these objects. As such, the realm of gaming has been heavily focused on increasing rendering quality to provide greater acuity and immersion for the user. This has undoubtedly pushed both the hardware and software forward, however, little of this development effort has been focused on how to convey in game elements without the use of vision, specifically for gamers who are visually impaired. Touch based methods have been classically used by the blind as a substitution for vision, with the most obvious example being the braille method of tactile writing whereby a system of raised bumps organized into six block cells, typically on embossed paper, are used to represent characters and punctuation. Invented by Louis Braille (Duxbury Systems, 2017) in the early 19th century this method has stood the test of time affirming the utility of touch, or haptic, sensations as an alternate method to convey visual concepts to the blind. In the realm of computer interaction for gaming and visualization of complex imagery, haptics have been integrated for many years via “rumble pads” and force-feedback devices providing users the ability to crudely physically sense digital events (Orozco, 2012). While these systems offer a level of multi-modal interaction for the sighted user, they have had limited use for seeing impaired individuals with most games for the visually impaired based around the use of sound for cue identification. However, tactile games for individuals with visual impairments based off popular WII console games (VI-bowling and VI-tennis) have been successfully developed using vibration to indicate ball timing, controller position and object placement (Tony, 2010). The crudeness of these cues and the difficulty in isolating them spatially limits the vibration only approach. It does affirm the principal and ability of using tactile feedback to accurately represent spatial concepts within the gaming realm. This would be especially true for haptic interfaces referenced to real world space with rapid refresh-rates that could provide much of the information that a sighted gamer would get visually.

While individuals with visual impairments may have reduced abilities to visualize objects using conventional sight, an increasing body of evidence indicates that compensatory mechanisms exist whereby tactile stimulation and additional senses can activate the visual system to generate accurate spatial representations. Indeed, reading braille is known to activate the visual cortex (Burton, 2006) of the brain in the blind (a region that typically codes visual information) and reflects the recoding of tactile sensations into “visual” signals. Studies also indicate that blind participants are typically able to better discriminate 3D shapes via tactile and haptic interaction (Norman, 2011) than sighted persons. Research evidence indicates the blind have different patterns of activation in the brain during haptic tasks (Roder, 2007) as well as during vibrotactile stimulation (Burton, 2004). Moreover, large-scale changes manifested in both structural and functional differences in brain connectivity indicate that the changes are not limited to the visual areas of the brain but reflect large changes in associated areas and “visual” perception as a whole. The idea of perceptual learning and substitution of sensory signals (where visual input is substituted for touch) can afford the ability of “sight” to subjects who are physically incapable of doing so. The remarkable capacity of the human brain to reorganize itself results in long lasting changes. This type of reorganization of visual pathways affirms the utility of a haptic approach to represent virtual environments to individuals with visual impairments. This will not only provide a means for additional brain reorganization but also that inherent differences in their brain structure suggests that perceptual learning will be more beneficial in this population with a dramatic impact on in game orientation and perception.

With the advent of new devices facilitating the generation of complex spatial and temporal representations of real world and virtual scenes, as well as the incorporation of tactile devices that allow physical interaction with computer generated imagery has emerged a novel platform for augmenting seeing impaired interfaces. Coupling this with evidence that individuals with visual impairments are able to acquire spatial knowledge using ancillary neural pathways via harnessing additional senses indicates that this is a promising and potentially game changing avenue to enhance user engagement and immersion. Here we detail the current state of haptic technologies as it relates to using these methods for encoding spatial information to individuals with visual impairments for both gaming and education. We begin with a discussion of the available hardware, and then we propose how these technologies could be used to convey in game information and learning as well as identify barriers and potential solutions. Finally, we address future developments and next steps for dealing with these innovations in the long term.

New Technologies at Stake

Haptic Media

Simple haptic feedback has been a part of gaming now for many years, but newer technologies are allowing for a larger range of haptic sensations across different digital experiences. Newer controllers, such as the Steam controller have improved on this feedback, allowing for a greater range of in-game information through finer control of the haptic output including haptic information on speed, boundaries, textures, and in-game actions (Steam Store, 2015). Microsoft is developing various haptic controllers to achieve various sensations such as texture, shear, variable stiffness, as well as touch and grasp, to enhance the level and variety of stimulation provided by a VR experience (Strasnick, 2018; Whitmire, 2018). Haptic styluses offer a different handheld experience via force feedback instead of vibrotactile feedback and can be used to interact with objects, or as a physical interface for more advanced simulations (Steinberg, 2007). Further, advances in wearable haptics; gloves, vests and full body suits have also expanded the potential experience of video games. Gloves allow for different actuators to be woven into the fabric or attached as an exoskeleton to achieve both vibro-tactile and force feedback (Virtual Motion Labs, 2018; VRGluv, 2017). The force feedback from the exoskeleton can define the edges of hard surfaces while vibrotactile actuators give information surface texture. In addition to exoskeleton-based force feedback, some gloves feature pneumatic actuators that can provide pressure directly to the skin, creating a realistic feeling of skin displacement.

Beyond gloves, the same technology is being applied to wearable vests and full body suits to create the most complete and immersive feel possible. Vests allow for targeted sensation to the body depending on where the stimulation is received in a game or VR environment (Kor-Fx, 2014; Hardlight VR, 2017; Woojer, 2018). Electrical nerve and muscle stimulation is also used for haptic feedback and also provides temperature control for changing virtual environments (Teslasuit, 2018). One practical problem with VR gaming is that moving around a space while wearing a headset can be difficult and dangerous. Haptic or omni-directional treadmills help fix this problem by anchoring the user to an area while still allowing for unfettered movement in any direction. More commonplace treadmills are also integrating haptic force feedback to give the sensation of moving through a real place while running on a treadmill (Nordicktrack, 2018). Indeed, a great deal of focus has also been spent on developing haptic touchscreens and incorporating this hardware into experiences for blind users. Newer haptic feedback seeks to alter the interaction between screens and fingers to create a more diverse sense of touch such as friction or texture simulation (Tanvas, 2018). Other developments include the manipulation of ultrasound to achieve tactile stimulation (Hap2U, 2018) with potential to create 3D surfaces in midair. Clearly as these new haptic technologies evolve and standardize the impact on immersive gaming and blind user interaction will be broad.

Visually-Impaired Specific Devices

A number of tactile hardware implementations specifically designed for individuals with visual impairments also exist. The Graphiti is tactile device consisting of an array equidistant pins with variable height and can be connected to a smartphone to control the height of the pins to represent text or camera images (Graphiti, 2018). The variable pin height could conceivably be used to represent limited information regarding the Z-axis, color, or whatever else someone may reasonably desire, such as an in game map. The system also offers a way to zoom in and out which is important from a tactile perspective, since human fingers cannot perceive and interpret information as minute and dense as the human eyes. The BrainPort V100 (Wicab, 2018) provides real world raw geometric information based on wearable camera input via a dongle inserted in the mouth that provides electro-tactile stimulation to the tongue. Bubble like patterns convey spatial information on the tongue and the user learns about shape, direction, relative distance, and size. BLITAB have successfully created a tactile braille tablet that utilizes “tixels” that dynamically rise above the surface of the display and can convert text to braille (Blitlab, 2018). The hardware also provides audio cues for the user but is limited in its ability to convey graphics, and objects in 3D space. BlindPAD offers a similar technology consisting of an array of electromagnetic “taxels” with 8mM pitch and a rapid refresh rate (Blindpad, 2018). The interactive technology is able to display graphics, maps and symbols as a tactile representation. This system has been successfully used in the education realm as well as can enhance users with visual impairments sense of space and their knowledge of unknown places. These latter points would be especially helpful if used for gaming to enhance blind users’ ability to navigate and investigate virtual environments and worlds.

While these systems would clearly be beneficial for reading braille and other 2D applications, it would appear spatial haptics currently in development for VR applications would be capable of creating 3D touch sensations closer to real world experiences. The enhancement in the variety of tactile stimulation available only deepens the experiences and the value these technologies can provide. It is clear that the development of the specific devices for individuals who are blind have been based around familiar technologies typically used to convey text; raised bumps on a flat surface. Nevertheless, creation of a common interface for these devices to video game environments or primitives (potentially by direct access to the graphics depth buffer) would create a new platform for both education and entertainment that would definitely empower its users.

Augmenting Visually-Impaired Gaming

This explosive development in hardware opens new avenues to convey spatial and game specific information in real time to gamers who have visual impairments. We have identified areas to specifically exploit these technologies to support in-game navigation, information extraction and user interaction. While some methods are specific to a certain device, the same principles would work at some scale using any haptic feedback from rumble pack in a controller to a fully-fledged VR vest.

Navigation: Fundamental to almost all gaming experiences is the requirement to correctly navigate and understand virtual environments. Tactile maps, proximity cues and new in game tools could enhance blind gameplay.

Tactile maps used by the blind have been shown to augment navigation when used during and prior to route traversal. However, the specific touch cues must be carefully considered when representing different types of information (roads, water, grass). Undoubtedly, a tactile approach to facilitate navigation of gaming environments would also offer similar benefits to the real-world case. It has been shown that the haptic approach when coupled with additional modalities such as audio can provide a method for individuals/gamers with visual impairments to generate cognitive maps of virtual environments using multimodal cues (Lahav, 2012), this is one of the major challenges for navigating gaming environments. Gaming environments (top down map) represented on a connected cell phone or other tactile tablet device (listed above) and interacted with via touch (vibration, tactile pixels) would provide an inexpensive solution. Here, haptic cues would orient the user using published frameworks for providing haptic access to 2D maps (Kostopoulos, 2007) to enhance knowledge and traversal of gaming environments.

Proximity cues. As well as knowledge of entire gaming maps, information about discrete and proximal in game cues (walls, surfaces, barriers) is essential to spatial awareness, navigation and gameplay. Haptic outputs (within a vest for example)) at discrete regions over the body activated by cues extracted from the in-game display would rapidly provide feedback about in game surfaces. Using this strategy, a wall detected on the left side of the in-game player would activate a haptic sensor on the left side of the player. Haptic output modulation (such as vibration frequency and amplitude) would code for object distance and textures using proximity sensors as has been shown successful in physical applications (Keys, 2015).

New tools. Here we propose the development of new methods for users to probe spatial environments. Specifically creating methods for users to digitally explore regions of the virtual environment and providing targeted haptic feedback. Specific tools such as a haptic sonar, floor probe and virtual compass whereby the users can target specific areas of the screen and be provided feedback regarding structural elements and orientation. The proposed haptic sonar would probe a customizable portion of the environment and activate an output (a rumble pack for example) if an object is within the frustum of the probe. Floor probes would more specifically sample areas directly in front of the user (floor) within the game and indicate hazards (water, textures, holes) by varying haptic outputs to facilitate locomotion. Orientation is key to navigating any environment, and the creation of a haptic compass to identify gaze direction in the form of a necklace (or array of actuators around the body) where feedback rotates around the ring and indicates north. These three strategies have been shown to help blind navigation in real-life applications (Quest, 2018; Choiniere, 2017; Visell, 2009) suggesting they may beneficial for navigating virtual environments.

In-Game Information: Layered upon game scene spatial information are characters, entities, game packs, enemies and other relevant dynamic game information. In order to provide a separate stream of information a distinct and generalized in game warning and information system is proposed. The information system should convey location, type of entity, and utility on a scale (good, indifferent, bad) to the user. Notifications would serve to provide spatial output to the user in the form of actuators at geographically separate points (around the body to indicate position for a vest or vibrator array for a glove). The nature of actuator output would convey the type of in-game entity at the specific location. For example, a health pack would initiate a series of short low intermittent vibration pulses but an enemy would cause fully maximal and continuous vibration at the actuator. Utilizing strength of haptic output to represent danger (good, bad, indifferent) and frequency to represent type of entity is a potentially viable solution.

User Interaction: A major factor contributing to immersive gaming is the ability to interact with digital scenes and in-game characters. Different to navigation and in game cue identification, interaction typically includes feedback from the entity, such as dialog, a response (enemy fires) or change in the entity (health pack disappears). At least two types of interaction are possible, the first is scene object interaction (e.g. pick up a key or a health pack) and the second is dynamic entity interaction (engage an enemy, interface with a non-playing, or friendly, character). With the former, interaction is relatively passive and layering this information on top of the navigational cues above makes sense whereby a separate tone (or haptic output) indicates the passive entities response. However, when engaging enemies, or other characters, active interaction requires additional feedback and dynamic responses. Here, a “lock in” mode, whereby all haptic resources are dedicated to interacting with the dynamic entity would provide directed attention to the entity. Under this system, the in-game notification system would identify an entities utility to the user via haptic cues (very good or very bad) and the user can initiate interaction, or “lock in” all haptic resources, to track the object and map it to spatial actuators and convey entity specific information to the user (hostility, temporal changes, weapon activity). Once the user “un-couples” attention (or the character disappears from the area), the full array of scene probing once again is reestablished. This method is designed to “silence” inconsequential haptic information during times of critical engagement to increase clarity and resolution of the engagement.

Possible limitations

Many of the approaches above have been shown to benefit individuals with visual impairments interactions with real-world scenes and as such provide a potentially beneficial framework for virtual scenarios. However, caution must be taken when trying to implement such a layered and complex haptic system. The ability to multiplex different signals (using amplitude/frequency modulation as well as additional modalities) and silence less relevant information (via “locking in” attention and feedback) is key to the successful application of a system with a relatively limited set of physical outputs compared to inputs. Care should be taken to reduce overload to the user, potentially accomplished by using additional modalities to convey specific notifications or events (a bell to indicate a health pack). Haptic resolution (dynamic response and spatial constraints) is also a major factor that should be considered when localizing feedback systems on the body. It is known that our ability to discriminate between two points varies over position on the skin (Purves, 2001) as such placement of haptic actuators is of great importance. These numbers vary from a 5mm threshold on the thumb to 45 mm two-point discrimination threshold along the shoulder, suggesting that haptic placement as well as device capabilities should be considered when localizing haptic sources on the body.

Potential Learning Applications

Importantly, adoption and development of these methods will also have a dramatic impact on inclusion in gaming and learning environments, especially in relation to STEM education. Understanding spatial properties including shape, size, distance and orientation is foundational and essential to developing spatial thinking skills and understanding many topics including algebra, trigonometry, calculus, chemistry, physics, biology and higher mathematics. Typically, students use sight to internalize geometric and spatial properties, however, the students with visually impairments need to have a different mode of experiencing this. While the development, standardization and implementation of these systems is still in the very early stages, gaming devices and open source tools are continuing to emerge that will accelerate the adoption and integration of the novel modes of computer interaction and provide new ways for students to experience digital content. The use of tactile graphics which employ the sense of touch rather than vision to convey spatial properties have been used to deliver mathematics instruction (Brawand, 2016), biology (Reynaga-Pena, 2015), chemistry (Copolo, 1995) and physics (Holt, 2018). 3D printing technology is also a valid method for delivering science content (Grice, 2015; Kolitsky, 2014) to blind students but clearly suffers from an inability to modify representations that are typically static, with limited use in the gaming sphere. Tools are becoming to emerge that standardize the procedure of generating tactile graphical representations (Pather, 2014); however, more are clearly required to broaden adoption of these practices. While many of these early studies incorporated printed static models or embossed paper, with the explosion of hardware and software computer haptics are increasingly being used to convey spatial information using force feedback in real world space but further development of both platforms and content are integral to expanding. With support for vibration output on many tablet and cellphone devices it is one of the simplest ways to incorporate tactile response is via activating this output (Awada, 2013; Diagram Center, 2017) and successfully conveys mathematics concepts (Cayton-Hodges, 2012), graphics and additional STEM content (Hakkinen, 2013) but not without limitations (Klatzky, 2014). Most STEM based studies have relied on surface haptic approaches while less have incorporated spatial haptic feedback systems (Nikolakis, 2004; Lahav, 2012; Evett, 2009) but it is known to facilitate knowledge acquisition of 3-dimensional objects (Jones, 2005). By incorporating 3-dimensional space co-registered to the real world into the haptic workflow clearly provides a more fundamentally “real” experience. Further, it results in an explosive increase in the potential amount of information that can be conveyed to the user or gamer. It is clear that the use of additional senses can help provide cognitive representations of environments and spatial structures and as VR gaming technologies evolve it is clear they need to be incorporated into applications to augment seeing impaired learning in STEM.

Future Directions

Our lab and others are currently investigating new directions in haptic interaction for navigating physical and virtual environments as well as identifying geometric primitives using multimodal cues and triggers. Recent studies in our lab have been focused on evaluating tactile (touch) feedback for the instruction of students who are blind or visually impaired. Pilot studies using the GeoMagic device have affirmed the utility of the method where blind users are able to autonomously recognize designed representations and specific shapes in only a few seconds. This suggests a clear benefit for teaching geometry and basic mathematical principles, something we are currently exploring. Coupling this with the fact that our infrastructure can both represent dynamic objects that change over time and new models can be simply built or downloaded. Our excitement for the method is hampered by the fact that the technology does not simulate the physical sense of touch in a natural or indeed accessible way. We acknowledge that devices do exist that can simulate touch via spatial force feedback but cost and clearly prohibits the use and mass rollout of such devices. This is coupled with the fact that software is extremely limited and requires extensive programming knowledge to be extended. As such, using off the shelf technology we have also developed a hand tracking system with vibrational feedback for touching virtual objects. Using vibration motors attached to each fingertip the user is able to probe a virtual object in space and sense 3D objects (Dhaher, 2017). We intend to evaluate the technology and develop lessons for the students with visual impairments by deploying these instruments to learn foundational spatial concepts. In addition, we are actively developing interfaces for probing external and virtual environments with tactile feedback for navigation and virtual tele-presence applications. Using scene depth imaging and physical triggers located at strategic points on the body, users can continuously physically sense details regarding proximal objects and walls in the (virtual) environment. For example, vibration on the right side of the body indicates presence of an object in the right part of the environment. The use of more specialized “virtual” probes in development for scanning depth buffers and providing physical and multimodal feedback. Importantly, this thrust is toward the development of a common interface for navigating (gaming) environments using (virtual) environment probes with accompanied audio and touch based feedback. The end of goal of which is to create a method for probing running applications on the graphics processing unit (GPU) to map inputs and outputs to multi-modal cues (sound, vibration).

As is true for the integration of any technology portability, compatibility and easy end user access to functionality is critical to adoption. The current state of haptic development is in its early stages and as such relatively disparate with most projects based on creating individual applications rather than platforms for haptic integration into existing software. The development of end-user friendly middleware, or other programming interfaces, to provide a means to generalize the access of both non-specific software and output signals to any type of (haptic) device is critical. A number of platforms for haptic interaction and programming have emerged relatively recently but are still beyond the reach of most end users since they require programming software and detailed knowledge. Openhaptics Professional (3DSystems, 2018) is a commercially available programming interface that is designed to facilitate creation of a wide range of software incorporating haptic feedback and interaction. CHAI3D (Chai3D, 2018) is a similar open source project that is designed to be compatible with a large amount of existing hardware devices and incorporates the ability to extend support for custom or new hardware. The interface exists as a C++ simulation environment that supports a number of core functions for volume rendering, visualizing CAD files and beyond. Both interfaces suffer from the significant requirement of programming knowledge as well as each project is essentially its own specific application. A relatively mature open source library, the Virtual Reality Peripheral Network (Taylor, 2001), supported by the National Institutes of Health allows access to VR tracking devices as well as a limited number of hardware haptic implementations. VRPN aims to be a device-independent interface to virtual reality peripherals and provides a way for applications to communicate via the VRPN’s client-server architecture. The system provides an excellent way to interface hardware tracking, controllers and human interface devices. However, VRPN still requires specific modules for each hardware device as well as client module for the running software application. The Open Source Virtual Reality (OSVR, 2018) software development kit (SDK) allows developers to create applications with access to all supported VR headsets and controllers by including libraries during the creation of applications. This limits the scope of OSVR as well as denying end users with no programming knowledge the ability to effectively use the software.

A number of solutions have emerged that have solved some of these issues and allow running software applications to have no knowledge of the specific targeted hardware while still providing interaction. The open source project OpenVR (OpenVR, 2018) is a programming interface and runtime binary created by Valve (Valve, 2018) that permits many types of VR hardware (including haptic devices) to communicate with running applications. Importantly, the system is compatible with SteamVR (SteamVR, 2018) and can be installed from their portal but is still limited to compatible devices and games/applications available within the SteamVR portal. An older and apparently unmaintained interface does allow software agnostic access to arbitrary hardware. The Glovepie system uses an input/output mapping system to allow end users the ability to map arbitrary controller inputs to control running applications (GlovePie, 2010). It should be noted that none of the above libraries explicitly support devices created for the blind, an issue that clearly needs to be remedied to move seeing impaired gaming forward. Some progress has been made towards allowing any running game or application to have access to any piece of haptic hardware, but these implementations are not broad enough in scope (minimal application/hardware support) and potentially difficult to implement. In addition, the abstraction layers are designed for interaction using the inherent interface and methods built into the controlling software (for example, a device can only be used to control typical in game activities such running or jumping) rather than the ability to generally probe the games 3D environment and structure. To accomplish this it will likely be required to extract information from running applications at a lower level, for example via the graphic processing units’ (GPU) depth buffer. This could provide low level access for developed environmental “probes” or tools that are implemented as a simple installable binary providing (haptic) devices access to the display lists of the running application. These digital probes could then be simply translated into pertinent outputs for (blind) users that activates and supports both the hardware specifically designed for the seeing impaired as well as the next generation of VR haptic devices.

Conclusions

Here we present the rationale and benefits for using multimodal interaction for gamers with visual impairments as well as the current state of the art in haptic interfaces for gaming, education and the relationships to enhanced end user immersion and learning outcomes. As these technologies and devices are adopted and evolve they are poised to have a dramatic impact on the entertainment, education and quality of life of individuals with visual impairments. However, accessibility, software implementations and a common interface point would significantly help broaden the impact. As far as the latter, an underlying interface, or middleware library, built upon a standardized graphic libraries (such as OpenGL) that provides simple cross platform haptic access to depth buffers, geometric primitives and the virtual environment would simplify, enhance and expedite the incorporation of new methods of tactile computer interaction. More importantly, it would function with any running game without modification. It is clear that the human eye is able to acquire much more information temporally than the somatosensory system and touch based representations of color may be arbitrary and difficult to convey but haptic representations will clearly offer a much more natural interface for video games for individuals with visual impairments. The key will be harnessing next generation devices via a simple extraction layer to provide meaningful and compatible haptic outputs. The 2016 Disability Status report (Erickson, 2016) indicates that 2.4% (7,675,600 people) of non-institutionalized people in the US population reported a visual disability. In addition, it is estimated that there are 36 million blind persons worldwide with a further 217 million with moderate/severe impairment (Bourne, 2017) indicating that the development and integration of new haptic interfaces (both hardware and software) is warranted to support the entertainment and education of this population of potential gamers.

OpenVR (2018). OpenVR is an API and runtime that allows access to VR hardware from multiple vendors without requiring that applications have specific knowledge of the hardware they are targeting. Retrieved from https://github.com/ValveSoftware/openvr

This paper explores the need for a competitive game for sighted and blind players alike, its effectiveness in fighting prejudices and how it can contribute and facilitate the inclusion of people with visual disability, when played by heterogeneous groups of players. The first part presents the state of the art about competitive games for blind and low vision users. Focusing on the typologies of games already on the market, and on the apparent lack of fast-paced multiplayer games, it shows a current space for research and development. Then, we present WaTa Fight!, a competitive casual party game that can be used by sighted and blind players alike, outlining its development, from its concept to its design and testing – conducted via focus groups and survey. Finally, we discuss the results obtained in terms of the game’s effectiveness in addressing the visual impairment topic and in fostering integration.

This paper explores the need for a competitive game for sighted and blind players alike, its effectiveness in fighting prejudices and how it can contribute and facilitate the inclusion of people with visual disability, when played by heterogeneous groups of players. The first part presents the state of the art about competitive games for blind and low vision users. Focusing on the typologies of games already on the market, and on the apparent lack of fast-paced multiplayer games, it shows a current space for research and development. Then, we present WaTa Fight!, a competitive casual party game that can be used by sighted and blind players alike, outlining its development, from its concept to its design and testing – conducted via focus groups and survey. Finally, we discuss the results obtained in terms of the game’s effectiveness in addressing the visual impairment topic and in fostering integration.

In between the 19th and 20th century, the queen Carmen-Sylva of Romania aimed to create a town for blind people only. The queen thought this proposal would improve the life of those who are blind, as she identified this condition with the inability of living a complete life in the society (Alliegro, 1991, p. 13).

Such a line of thought was by no means an exception. For example, in 1865 the newly born Kingdom of Italy, with the article 340 of the Civil Code, placed blind and deaf citizens under the legal guardianship of tutors, aggravating their condition of deprivation. Today, most of the States’ legislation in the matter of disability recognizes the importance of the integration of individuals with special needs in the society (Mangiatordi, 2017, pp. 19-20; p. 45; Istat, 2013), but a significant part of the media still uses old tropes to depict disability, often including damaging stereotypes as many pop media portray blind and low vision people as incapable of behaving normally (Barnes, 1992). The cartoon series Mr. Magoo is a well-known, evident example of such tendency: in each episode the old man reveals himself incapable of understanding any dangerous situation he faces, prevailing out of pure luck. This stereotype, done for comedic effects, is so damaging for the blind and low vision communities that they protested upon the theatrical release of the live action Mr. Magoo movie in 1997 (Maurer, 1997).

The problem with the media depiction of disability is often related to its exclusiveness: the person with disability is represented as different from the supposed normality and hence detached from the society (Barnes, 1992, p.19). In order to expose the socio-cultural bias and truly integrate individuals with disability into our society, it is necessary to fight prejudices by creating heterogeneous communities and reducing those social barriers to their everyday activities (d’Allonzo, 2008, pp. 52-61).

Among the various media, games also dealt with individuals with special needs and their issues in such areas as single- to multi-player, from audio games to board games and video games. The interest of the community in this subject has expanded over the years, as more researchers investigate different forms of gaming and playing. On the one side the academy shows specific interest towards education and learning (Gandolfi et al., 2016); on the other, the community of practice (especially indie companies) designs games that place non-impaired people into someone else’ shoes (Blind Arena Tournament, 2017; Perception, 2017; Footsteps, 2016; Blindfield, 2016).

As the games for the social change field of practice and literature demonstrate, games can be a way to inform and persuade the general public in a cause (Bogost, 2007; Kaufman & Flanagan, 2015; Bertolo & Mariani, 2014; Mariani, 2016). So far, except for a few examples, games for social change tend to rarely be competitive, but rather individual. This limits interaction and exchange with other players. Challenging such trend, the attention has been drawn on competitive online games as sources of thrilling interaction and deep engagement.

Indeed, community creation and bonding are key components of successful competitive online games as in those actions are often sensationalized to widen the community with an audience of spectators (Hamari & Sjöblom, 2017; Gandolfi, 2016; Cheung & Huang, 2011). Therefore, competitive gaming may be a way to fight the prejudices about the inability of blind and low vision people, helping the process of integration.

In this scenario, mobile gaming accounts as one of the most promising frontiers and flourishing branches of the game market (of which it represents about 50%), confirming a double-digit growth since the launch of the first smartphone in 2007 (Newzoo, 2018). The worldwide distribution of mobile devices, their extensive and growing accessibility (affordable prices, user-friendly interface, and a good usability in general), alongside with the improvement of the communication infrastructures, and the abundance of titles in the app stores platforms nurtured a healthy and promising environment.

Surfing between game studies and design research, this article investigates how a game can address the issue of blindness or impaired vision through a game aimed at fostering social inclusion and comprehension. From the analysis of games for blind and low-vision players on digital devices, the article describes the design process of a competitive audio game prototype, including the tests for evaluating its usability, playability, and effectiveness in helping the integration of people who are blind or have low-vision. WaTa Fight! is the outcome of a MSc thesis conducted at Politecnico di Milano, School and Department of Design, and it belongs to a broader research addressing the topic of social change in general, and social inclusion and communitarian comprehension more specifically, held in ImagisLab research group by Maresa Bertolo, Ilaria Mariani, and students. As other thesis researches developed in ImagisLab, the work presented here has been structured as (1) a phase of literature review and case studies analysis; (2) a design phase of the prototype conducted as an iterative process; and (3) testing with end-users, enquiring the game both as an interactive product and a communicative system able to deploy meaning through its gameplay.

Competitive games for blind and low vision players

The decision to develop a competitive game depends on the market analysis. As related pieces of information could not be found in literature, the research methodically examines the titles collected in the Audiogames.net database aiming at a better understanding of the genres of audio games in use. Among the several online databases and repositories of games suitable for blind players, Audiogames.net stands for being cross-platform, presenting titles for PC and mobile devices alike, and unlike most of the other similar services, for being independent of any development company. Moreover, Audiogames.net offers an indexation of the titles that facilitates the market analysis process. In the initial research phase, each game category in the archive has been listed, ruminating on the size of the group and the attributes of the content. To further represent the site content and nurture the analysis process, the games have been grouped into specific clusters that classify the experience offered by the audio games listed (fig. 1).

A line of reasoning that is transversal to the clusters identified, the games on Audiogames.net resulted to be mostly narrative and slow paced, often shaped as online multiplayer text-based games, in the fashion of Multi-User Dungeons. Further diving into the analysis, it emerged that the repository does not include any case of competitive, multiplayer, fast-paced, audio-based game; such vacuum emerged as a potential space for design. Therefore, it has been conceived as a game concept that adheres to the identified needs and fills a design space with interesting untapped possibilities. To maximize mutual exchange and encourage socialization, it has been designed as a party game: quick to play, with easy-to-grasp rules, based on soft skills such as agility and concentration, and with a special focus on the interaction between players. In addition, it has been conceived “for players”, regardless of their ability to see.

Figure 1. The 648 games on audiogames.net categorized into clusters.

Wata Fight! An inclusive competitive party game

Targeting individuals who are both sighted and visually impaired dictates special requirements. On the ground of the competitive audio-based game WaTa Fight! there is the study of video games and games directed to the general public as well as of interfaces for users who are visually impaired. Considering the aforementioned reasons for conceiving a casual party gaming with competitive elements, the main sources of inspirations were the series of Nintendo party games 123 Switch (2017). These mini-games characterized by short matches focus on the speed and precision of the participant’s movements, with almost no use of graphics. Considering the market, the target(s) and the intent of soliciting integration and dialogue, the game and its gameplay do not pivot around visual elements. In the game, players must perform actions faster or more precisely than their opponents to gain points. Moreover, the narrative itself relies on a strong metaphor that subtly enhances the game meaning, without stressing its topic (Kaufman & Flanagan, 2015; Mariani, 2016).

Figure 2. WaTa Fight! in action

Narrative and gameplay

WaTa Fight! is a competitive casual game where players are Ninjas that battle for glory in a Great Martial Art Tournament. As Ninjas hiding ability is legendary, no one can see them. In a match, two players face each other by tapping on a smartphone screen faster than the opponent in order to perform an attack. To fight, each player uses a smartphone in landscape position, held between the hands, with the screen facing the opponent, equalizing a player with and without visual disability.

On the smartphone screen, there are only two large buttons, named Wa and Ta respectively (fig. 2). The two Ninjas fight by pressing them to attack the opponent with a Wa or a Ta move, or even both to perform a WaTa attack: a powerful move combining the two base ones. According to the command pressed, the smartphone shouts the ongoing attack, giving the opponent the possibility to parry by quickly tapping the same attack on her/his smartphone.

Figure 3. Wa and Ta buttons on the screen.

According to the fictional world of reference, Ninjas are epic fighters, and as such, they can be wounded only in the Honor. Players start with 3 Honor Points. Each successful attack performed steals 2 Honor Points to the opponent; however, the opponent may parry, blocking the attack effect and stealing back 1 Honor Point. The match ends after a set time, and the winner is the contestant with the highest Honor score.

As only two players participate in each match, other people are encouraged to spectate the action, even disturbing the players’ actions with their voices. To this end, it is also possible, but optional, to connect the game to a computer screen or a smart TV to visualize a leaderboard displaying the current game statuses.

Towards a “comfortable” interface

Focusing on the visual and interactive aspects, the game touch-based interface required several sessions of reflections and field experimentation. It was loosely inspired by BrailleTouch 2 (Frey, Southern & Romero, 2011), a mobile texting application for blind and low-vision users. Developed by a team from Georgia Tech, this Braille-like texting app uses a six-button wireframe on a landscape positioned smartphone to simulate the structure of the Braille writing system and support visually impaired people to easily type on a screen, without even looking at it.

The game prototype was originally conceived with an eight-button interface, with 4 buttons on each side of the screen. However, still in the early concept phase, the studies in literature (Frey, Southern & Romero, 2011) led to the reduction of the number of buttons – initially from 8 to 6, as in the case study mentioned above. However, timely early tests with end-users called for a further simplification of the structure. Blind and low-vision persons proficient in Braille, as well as sighted persons participated to a set of very early tests consisting of tapping on the screen to complete some series, interacting with a different amount of buttons. Testers were asked to interact with two interfaces: the first had 4 buttons, the second had 6. The results were key in terms of design, defining how many buttons players were able to “comfortably manage” in the meanwhile, as well as their degree of feeling at their ease in using the button-based interface. All the players tested the four-button version without experiencing awkwardness or discomfort. On the other hand, all players but one stated that they felt “uncomfortable” in managing six buttons, being reluctant even to try such wireframe version of the interface, envisioning the interaction as distressful and unpleasant. The six-button interface was perceived as complex and also “scary” by some of the playtesters that asserted that the mere request made them feel distressed. Based on this feedback and subsequent analysis, the first and current playable prototype of the game uses only two buttons, one for each side of the device (fig. 2). The decision was due to the typology of game (party game) and to the time necessary to get adjusted to four potential combinations and their feedback. The analysis took into consideration that individuals with low-vision or who are blind cannot take a look at an explanation screen, but rather have to hear the voice tutorial. In essence, explaining how to use 4 buttons plus their combinations requires a certain amount of time, and significant attention for memorizing the sequences and their meaning. Because WaTa Fight! is a party game directed toward a casual audience, a fast learning curve is optimal and an audio-based learning process is more methodical and slow (Rashtian in Nasar & Evans-Cowley, 2007, p. 45).

Testing with end users. Research methodology

Since its early prototype phase, the game was tested in its effectiveness as a communication system with social aims as well as in its playability both in terms of user experience (UX) and user interface (UI), considering its peculiar, mixed audience.

Structure of the enquiry

To understand and assess the effectiveness of the directions and the conceptualization of the game, independently of their ability or disability to see, players have been qualitatively investigated employing a set of specifically designed tools (Mariani, 2016):

observed through a real-time rapid ethnography (Millen, 2000);

recorded for an in-depth follow-up analysis of UX and UI;

asked to fill a visual analog scale (VAS) questionnaires3. Because of the players, it has been shaped as regular questions with the scale printed on paper for sighted playtesters, or as speak-aloud questions with tactile scales for the sight-impaired ones;

and later involved in focus groups with semi-structured questions to further unpack specific topics of relevance.

The playtests, held in three sessions, involved a total amount of 14 players (m: 6; f: 8), of which 9 individuals were sighted, and 6 players were legally blind. In this phase the involvement of blind and low-vision end-users was pivotal to verify the usability and playability of the game in primis (as explained above), but also its ability to trigger social integration among the players, independently of their visual ability. To ensure an optimal experience and the inclusion of the final audience, the focus groups were organized with and supported by the Unione Italiana Ciechi e Ipovedenti (the Italian Blind and Low Vision Association) of Novara. The field tests occurred from May to June 2018, as shown in fig. 3. Each playtest and the focus groups have been recorded and shot to capture the full event, aiming at a follow-up in-depth analysis; in parallel, the researcher in charge of the observation conducted rapid ethnographies.

Figure 4. A visualization of the three playtests made stating the number of participants, their gender, sight condition and the tools used at the encounter.

The first version of the game has been playtested on May 2018 and involved 5 attendants, all males and legally blind, from 20 to 45 years old. Because of the nature and timing of this investigation that occurred in the very early stage of the design phase, this playtest mainly consisted into a field experimentation about players’ ability to press different number of buttons and interact with them through time. As a result, this first playtest, as described above, resulted to be more a usability study than a real analysis on the game effectiveness in prompting social integration. As suggested in the literature (Mariani, 2016, p. 79) each playtest was followed by a questionnaire and then a focus group (with semi-structured questions), whose tone was kept conversational and relaxed in order to stimulate the participants to express themselves freely, encouraging the discussion potentially beneficial for the research.

Sections of the questionnaire

The questionnaire consists of a series of VAS evaluating the experience on specific topics. The questions are arranged in the following sections:

user profiling;

analysis of the perceived feelings;

significant traits of the game;

particular aspects of the experience.

Apart from the profiling set, each question is meant to assess the effectiveness of the game in pursuing design goals, both in terms of UX and UI.

Topics of the focus group

A set of questions of the focus group is tailored to investigate the interaction with the interface, the feelings aroused under stressful conditions (fast-pace matches), and the resulting state of mind, considering the presence of several players on site with different visual ability. To sum up, the questions cover:

competition and the experience of general pleasurability;

social interactions (with players as well as with non players observing the game and interacting with those who play);

possible emergent experiences.

Then, further questions regard and expand the following topics, which provide detailed knowledge about specific aspects of the game:

the game duration;

the possibility of playing again with friends;

the experience as a spectator during other players’ matches.

The game duration is linked to the functioning of the game flow and the difficulty curve, or the experience fun factor, as the time perception of a player changes in accordance to its immersion in the game activities (Rockholz, 2014). The possibility to play the game with friends allows to better comprehend the quality of the experience and possible feelings of empowerment. Finally, the last point analyzes the experience from the perspective of the game spectator, namely of the one who observes the player, rather than the player itself, and the game communication possibilities of such users.

Results

This paragraph outlines the results coming from the questionnaires and the focus groups conducted during three sessions of playtesting 4. The first playtest was made with six legally blind participants. Among them, only one had some video-game experience, while most of the participants had at least occasionally played board games with the help of sighted friends – most notably Taboo, Clue, and Monopoly. Significantly, none of the participants declared any interest in playing with the smartphone in their everyday life, but they all stated to have good computer skills and to be interested in technology. That said, the first field experimentation was conducted with a wireframe interface, mainly serving as a usability test, aiming at understanding the players’ attitude towards the game, as well as their mood and feelings. Despite the still prototypal phase, the data defined the number of buttons players felt comfortable handling at the same time, and suggested a good level of emotive participation. The game length was perceived differently by those who won and those who lost; appropriate amount of time was perceived by winning players, and too short was perceived by the losing ones. However, speaking of pleasurability, the game was unanimously perceived as enjoyable and pleasant: rules were considered as easy to understand and remember, and the gameplay provided entertainment and recreation. These results confirmed what observed via ethnography, regarding the general mood and feeling.

Game Master: Would you like someone else to try the game?

P: YES [in unison]

F: YES [in unison]

G: YES [in unison]

F: [continuing] Yes, … honestly, I would make some of my sighted friends try it…[thinking aloud]

G: …To beat them up [completing the other one sentence]

The answer provided by F was a sort of thinking aloud process that underlined the desire to share the experience with someone else. However, what seems key is the fact of being allowed to play with a sighted person, without disadvantages of any sort. On the opposite, as stressed by G, who kind of completed F’s thought, this game almost becomes a way to “beat them up”. As if the fact of being visually impaired, for once, is giving an advantage, rather than being a handicap or a limitation. In these terms, WaTa Fight! turned out to facilitate interaction and social gatherings, overcoming the frequent issue due to visual disability. That is because to be good at this game is not important to be able to see, but to have reflexes, memory, and readiness.

Then, the participants of the focus groups claimed they also enjoyed the experience as game spectators. a data that seems in line with what was observed. However, spectators’ attitude and behaviour varied depending of the composition of the group. The spectatorship of a match with a mixed audience turned out more aggressive and “present” than that involving only non sight-impaired players. The playtest with just legally blind players saw the participants stand in silence, listening to the game audio in order to better understand the match status, whispering each other the score after each move: blind spectators cannot see the score, and therefore their observation is more centered around memory efforts. Playtests with mixed audiences were more effective in facilitating interaction and, above all, integration – that is further nurtured by the fact of going through a shared, meaningful, hands-on experience. It is also worth adding that the level of participation and interaction has varied according to the character, disposition and charisma of the players involved, and in consequence to the atmosphere that has been created. This serves to underline how the nature of the personalities involved has strongly influenced the perception of the game, in terms of experience.

The second and third playtest encounters were held in June and July 2018. One involved sighted participants only (n: 3 m and 4 f; age range: 22-32), the other mixed legally blind (n: 3 m; age range: 25-32) and sighted ones (n: 2 f; age range: 26-28). All but one considered themselves expert gamers, and 3 out of 5 affirmed to have at least some information about visual impairment. The two sighted playtesters evaluated the experience as positive and pleasurable. The only exception was a participant who won all the matches, and defined the game “too easy” – it has to be said that the player was highly coordinated and fast.

The game narrative and the tone of voice, in general, were unanimously considered fitting to the project goals. Choosing the fictional world of Ninjas contributed in terms of meaning making: the game activities gained indeed meaning because of the narrative context within they were set. As a matter of fact, screaming attacks as Wa and Ta and WaTa was not perceived as awkward or embarrassing. In parallel, also the game aesthetic has been appreciated and commented as a further trigger for enjoying the game, stimulating more playful interactions.

In parallel, remarkable information emerged as regarding the topic of sociability, showing encouraging results from both sighted and blind playtesters: all the participants stated that the game nurtured sociality, helping them to release some inhibitions in bonding with each other. However, the direct observation of the matches of the last playtest with 7, mixed participants (n: 3 m; 4 f) revealed that the sighted attendants were more active than the visually impaired counterpart, and prone to actively distract the opponents. Most of the participants attempted to disturb the game tickling and poking the other players to make them fail. When asked, the playtesters explained such behavior as resulting from a general discomfort using sound-based disturb attempt: the touch sense was therefore considered the more effective in the task.

Covering the game ability to convey meanings and nurture mutual comprehension, and hence its effectiveness as a communication system able to facilitate the integration of visually impaired people, WaTa Fight! seems promising. 16 playtesters on 18 stated they felt a general sense of openness towards the topic of integration after the game experience.

Conclusions and future developments

Nonetheless, the game is still in a prototype stage, with only the core gameplay implemented. A more complete product is needed in order to to proceed with the experimentation and in order to determine a complete idea of the full possibilities offered by the game. However, it is still possible to take stock from what we observed up to date. Even in a prototypal state and with a small test sample, the results are encouraging.

First of all, the information collected during the observation, as the general understanding of technology, the dynamics generated, and the basic gaming experience of the participants reinforced the initial idea of choosing a situated party game for mobile devices.

Data from the focus groups and the survey indicates that the participants have shown signs of openness and sympathy toward the theme and the other group members. The most effective and satisfying results were collected when the game was played by both blind and sighted players. In particular, the reaction of blind players is meaningful. They showed a clear interest in playing the game again, especially with sighted opponents, underlining the game high potentialities for facilitating inclusion and integration of people with vision impairment in groups of sighted individuals. Matter of factly, the game resulted an effective means for stimulating experiences that turned into constructive discussion and comprehension. It prompted players and spectators to talk about disabilities because of the constraints it imposed on the in-game actions through their mechanics and aesthetics. Games can use procedurality (Bogost, 2007; Sicart, 2011; Mariani, 2016), mechanics and also a fictional world to simulate disabilities, giving players an understanding of the difficulties encountered by a segment of the population.

The results are consistent with what corroborated by both literature (Gandolfi et al., 2016) and field research that suggest that games can be of help explaining disability. They are especially aligned with the social approach promoted by the WHO that sees the disability as a quality of the interaction between people with special needs and an environment that restrains them in pursuing personal goals (WHO, 2011). This game shifts attention from the commonly primary sense, sight, becoming easy or difficult independently of the ability to see, but relying only on coordination. Consequently, and for obvious reasons, it is not suitable for certain types of motor disabilities, on which however we have not carried out any test.

Then, because of the target, a key element is that the game should be available on an easy-to-use platform, using a user-friendly interface able to smooth as much as possible the interactions and lessen the visual imbalances among players.

Acknowledgements

We would like to express deep gratitude to Giuseppe Laganà and the Unione Ciechi e Ipovedenti of Novara, for their support and patience in helping with the game testing. Our grateful thanks are also extended to Francesco Venco for his guidance in making the game infrastructure and for his useful critiques during research work. We are very grateful to Gianluca Poma for the support. We thank Vittorio Silini for giving his voice to the game prototype, the sound designer Marius Arcioni for the music of the project, and the video maker Giulia Emma Nocerino for her help in documenting the playtests.

There are two disparate ways to describe the relationship between digital games and special needs (i.e., physical, cognitive and even socio-cultural conditions than require specific interventions in everyday life routines, learning activities, and general accessibility). On one hand, it can be argued that the sector is becoming more inclusive. For instance, assistive technologies are gaining a foothold in the game industry with innovative hardware (e.g., the Microsoft Adaptive Controller), focused efforts of researchers and practitioners (e.g., the IGDA game accessibility interest group or the Games For Health conferences), increased customization interfaces and input systems (e.g., those offered in the games Overwatch or Uncharted 4), and focused funding initiatives (e.g., AbleGamers Charity and Special Effect).Conversely, one could also argue that the concerns of individuals with special needs represent an overlooked area. For example, toxicity and disruptive behaviors across game audiences (e.g., “Gamergate” see Mortesen, 2016) represent additional sources of biases, games are not accessible to all players, and the literature about special needs and gaming is scarce (with the notable exceptions of Carr, 2014; Champlin, 2014; Ledder, 2015). Additional research is required to respond to these opposing perspective as well as to further impact policy and practice. There are least four reasons to justify such a claim.

There are two disparate ways to describe the relationship between digital games and special needs (i.e., physical, cognitive and even socio-cultural conditions than require specific interventions in everyday life routines, learning activities, and general accessibility). On one hand, it can be argued that the sector is becoming more inclusive. For instance, assistive technologies are gaining a foothold in the game industry with innovative hardware (e.g., the Microsoft Adaptive Controller), focused efforts of researchers and practitioners (e.g., the IGDA game accessibility interest group or the Games For Health conferences), increased customization interfaces and input systems (e.g., those offered in the games Overwatch or Uncharted 4), and focused funding initiatives (e.g., AbleGamers Charity and Special Effect).Conversely, one could also argue that the concerns of individuals with special needs represent an overlooked area. For example, toxicity and disruptive behaviors across game audiences (e.g., “Gamergate” see Mortesen, 2016) represent additional sources of biases, games are not accessible to all players, and the literature about special needs and gaming is scarce (with the notable exceptions of Carr, 2014; Champlin, 2014; Ledder, 2015). Additional research is required to respond to these opposing perspective as well as to further impact policy and practice. There are least four reasons to justify such a claim.

Video games are at the forefront of technological adoption (Duggan, 2015). Given their ubiquity, they are ideal testing grounds for problematizing current interactive affordances and patterns and developing new and more inclusive solutions.

Video games and interactive media shape society and culture (Ferdig, 2018). They convey representations, ideologies, biases, and viewpoints. Gamers and game developers take a stand that is not as neutral as it may appear at first glance (Gandolfi & Ferdig, 2018). Shedding light on how the medium deals with the issues faced by individuals with special needs becomes crucial for understanding and perhaps changing social perspectives (from reiterating stereotypes to suggesting alternative perspectives).

The combination of technical and cultural perspectives can effectively support two leading approaches to individuals with special needs – i.e., the social model (Bickenbacha, Chatterji, Badley, & Üstünet al., 1999) and the cultural lens (Wolbring, 2008; Campbell, 2009). The former refers to efforts aimed at making society more inclusive (equal possibilities, no barriers), while the latter addresses prejudices and constructed ideas of normality and abnormality. By combining these two foci, analyzing digital entertainment may become an ideal battleground for reflecting on disability and difference while promoting the development of proactive initiatives.

Videogames can potentially support special education and learners with disabilities, from improving physical and social skills to facilitating communication and self-organization (e.g., Saridaki, Gouscos, & Meimaris, 2016).

The goal of this special issue is to provide insights and guidelines for realizing and responding to this potential. The five articles collected address several aspects of the interplay between digital games and individuals with special needs. Aside from their topical differences, these contributions seem to share an underlying value given to the inclusion of individuals with disabilities in the world of gamers. The authors also collectively recognize the fact that games should be created with affordances that allow for universal access.
In his article on inclusive interfaces, Dalgliesh effectively expresses this viewpoint: “notions of incidental body-controller fit and precarious accessibility are outlined to develop a model that uses asymmetrical roles and diverse input to fit individual abilities and thereby expand participation”. Dalgliesh also recognizes the dignity of the human person in creating games where individuals can participate as equals; no one wants to be the unequal partner who is helped along in a childish manner. Dalgleish argues:

While roles are highly asymmetrical, fundamental principles such as reward, goals, challenge and meaningful play … are maintained… Rather than segregate impaired players by placing them in exclusive sub-sections that provide “cut-down” versions of the canonical experience in an attempt to manage challenge and difficulty, the ARAC model has all players – impaired or otherwise – play in the same space.

This is especially relevant in online and shared play, where social interactions and exchanges are relevant and support the whole gaming experience. Imbriani and colleagues claim that “community creation and bonding are key components of successful competitive online games as in those actions are often sensationalized to widen the community with an audience of spectators”. According to Schrier, a “learning community can help to encourage connections among disparate groups, as well as encourage a sense of belonging and inclusion in a game community, which may contribute to empathy, perspective-taking, learning, and positive exposure to others’ backgrounds and cultures, and greater self-efficacy and social support….games that support intergroup cooperation may reduce bias, particularly in multiplayer games online”.
Accessibility issues remain a priority to address, and four articles directly deal with this topic from different angles and perspectives. Plothe advocates that the construction of games should begin with the idea of universal design, thus limiting the retrofitting of games. Dalgleish’s article is characterized by an emphasis on game controllers as a bearer of inclusions/exclusion for players with disabilities. Vercellone, Shelestak, Dhaher and Clements uncover how haptic technology may make a difference in providing more inclusive interactive experiences. Imbriani, Mariani, and Bertolo focus on how inclusive game mechanics can entail a shared ground between sighted and players who are visually impaired. The authors seem to share the belief that accessibility-related developments can positively impact cultural, empathic, and learning outcomes.
Indeed, the intrinsic work of creating virtual realities in game-like environments could have the potential for increasing awareness of negative bias and improving social interactions and mutual understanding. This opportunity emerges in all the five articles, most notably in Schrier’s article:

Some games may help to immerse people into virtual worlds and new roles and identities […], which may encourage consideration of others’ experiences, feelings, and perspectives […] . Games may help people express and experiment with their own identities and others’ identities […], and may enable people to communicate and interact with people from other cultures, with other types of needs, and with different types of experiences.”

This collection draws from different disciplines (e.g., educational technology, computer science, games studies, design, and biology). As such, the articles provide a variegated array of implications, spanning pedagogical strategies, game design suggestions, and technical insights. Such a wide scope is fundamental for addressing the interplay between video games and special needs in its entire complexity and richness.
In addition to the research presented in this special issue, there are four next steps to continue to support work in this area.

Game analyses often emphasize representation and aesthetics (e.g., Carr, 2014; Lynch et al., 2016). This refers to the interactive component of the medium, from rules to heuristics; these features are not neutral but rather they can communicate specific biases and schemes (Gandolfi & Sciannamblo, 2018). There is also value in exploring ludic mechanics and media environments. This would include studies of online platforms like Twitch.tv, Steam, and Reddit, where gaming communities gather and debate. For instance, Twitch.tv is strongly supporting streamers with special needs in collaboration with the association AbleGamers, but it has been the stage of toxic behaviours against players who are disabled (see the case of Adam “Lo0p” Bahriz, a legally blind and deaf streamer who was bullied by his own game mates during a live match of Counter Strike: GO) (Jackson, 2017).

Aside from some exceptions (e.g., see https://spedapps.kent.edu/ for mobile games and the article by Vercellone et al. in this special issue), the study of video games for special education is nascent. More research is needed that specifically spans different disabilities, genres, and pedagogies. Mainstream games should be investigated targeting their instructional potential and affordances, which can be relevant due to their popularity.

Researchers in this special issue studied digital gaming and its ability to foster empathy and perspective-taking (see Imbriani et al.; Schrier). Research should capitalize on this work to further explore affective, emotional, and cultural outcomes related to special needs.

Technology companies and scholars should partner in research and development efforts to further explore assistive technologies for gaming. Despite some early efforts and contributions by authors in this special issue, the field is lacking accessible software and hardware.

We conclude this special issue by thanking the contributing authors as well as the reviewers who spent significant time during the review process providing suggestions and insights. Finally, we are very grateful to the associate editors of GAME, who have supported this special issue and its cause since the beginning.

Richard E. Ferdig
rferdig@kent.edu
Research Center for Educational Technology, Kent State University

]]>G|A|M|E Issue 7, 2018 – Digital Games for Special Needs; Special Needs for Digital Gameshttps://www.gamejournal.it/game-issue-7-2018-digital-games-for-special-needs-special-needs-for-digital-games/
Thu, 21 Mar 2019 16:11:43 +0000https://www.gamejournal.it/?p=3522Issue 7, 2018 – Digital Games for Special Needs; Special Needs for Digital Games

How can games be used to reduce biases and biased behavior toward those people with special needs—if at all? In this paper I contextualize biases toward people with special needs, and also investigate possible intervention to encourage bias reduction. I also provide particular attention to educator biases around students with special needs, and how games may support interventions and professional development to reduce biases and biased behavior. Based on this, and the limited research on games, empathy, perspective-taking and bias reduction, I describe five possible strengths and four possible limitations of using games for these purposes.

How can games be used to reduce biases and biased behavior toward those people with special needs—if at all? In this paper I contextualize biases toward people with special needs, and also investigate possible intervention to encourage bias reduction. I also provide particular attention to educator biases around students with special needs, and how games may support interventions and professional development to reduce biases and biased behavior. Based on this, and the limited research on games, empathy, perspective-taking and bias reduction, I describe five possible strengths and four possible limitations of using games for these purposes.

Keywords: Games, Design, Bias, Special Needs, Empathy, Video Games.

Introduction

Can games help to reduce biases, or the practices and behaviors related to these biases? If so, which characteristics of games may encourage or limit the reduction of biases, such as biases toward people with special needs? Games are starting to be used for bias reduction and the lessening of stigma, including non-digital games (Wong & Li, 2015) and digital games (Roussos & Dovidio, 2016; Simonovits, Kézdi & Karos, 2018). Finding interventions and experiences that support bias reduction and change behavior is becoming even more pertinent as schools become more diverse in student population (Lee, 2010) and as the number of children characterized as having special needs continues to increase (according to the IDEA act statistics, the number of children ages 3 to 21 that are served doubled from 1976 to 2015 (US DOE, 2018)). Moreover, between 2016 and 2018, there has been an increase in bias-related incidents, including hate crimes, cyber harassment, and a lack of civil discourse online (ADL, 2018).

In this article, I will review research on bias, anti-bias education, and bias intervention techniques, as well as research on games that aim to reduce bias, as well as games more generally. Games and gaming culture have even been seen as driving an increase in bias and expressions of bias, and has been suggested as causing antisocial behavior, such as aggression, violence, and addiction (WHO, 2017). Rather, in this paper I seek to understand how we may be able to reduce bias, unravel any strengths and limitations, and make recommendations for going forward. Finally, I will apply particular attention to research on educators and biases, and investigate research on biased behavior and attitudes toward students with special needs.

Biases and Special Needs

What are special needs? In this paper I define special needs as specific needs outside of what is typically designed for, whether a game, intervention, learning experience, or classroom environment. Regardless of whether these needs should be designed for, they may not be considered in the design process, as designers may seek to create a “one size fits all” environment. For instance, a person with special needs could refer to a person with autism spectrum disorder, or someone, like my son, who is developmentally delayed and needs to wear prosthetics. It could also mean different types of learning disabilities, hearing or vision issues, a sensory disorder or ADHD.

Further, what are biases? A bias is an, “inclination or preference either for or against an individual or group that interferes with impartial judgment” (ADL, 2018). Researchers distinguish between implicit biases and explicit biases, with implicit being less consciously applied and explicit more deliberate. Both types of biases can affect behavior (Bai & Ertmer, 2008), judgments (Campbell, 2015), and sense of self or self-efficacy (Burgess & Greaves, 2009), although it is still unclear whether activation of stereotypes is causing the application and judgment of a group (Glock & Krolak-Schwerdt, 2013).

Besides the term bias, there are other related terms: stereotype, prejudice, stigma, and discrimination. Stereotypes are automatic and (often) evaluative judgments of a specific group (e.g., a gender or racial group as being associated with particular characteristics) (Campbell, 2015). Prejudice is defined as a systematically unfavorable attitude or belief toward a specific group of people, whereas discrimination is the “operationalization” of these prejudices in the form of negative actions and behavior toward that group (Adachi, et al, 2015). As Powell (2014) explains, stigma is a mixture of “stereotypic beliefs, prejudicial attitudes, and discriminatory actions” directed toward any specific group of people, which also stems from and varies by its social context (Lockenvitz, 2016).

All people, including teachers, parents, peers, and even young children, may have different types of biases. Biases may be based on gender, race, ethnicity, perceived social class, nationality, and special education needs (Campbell, 2015). These biases may influence a person’s behavior (Bai & Ertmer, 2008) and may also have an affect how that person interprets, assesses, and interacts with others (Campbell, 2015; Overby; Carrell & Bernthal, 2007).

Teaching and Biases

Do teachers and other educators have biases about and biased behavior toward students with different backgrounds and needs? Teachers may assess their students differently based on their biases (Campbell, 2015). For instance, Campbell (2015) looked at teacher judgments of their students and found that students with any special needs diagnoses were less likely to be found to be rated “above average” for reading and math ability than those without any special needs. The researcher also found the same pattern for lower-income students, boys (reading), girls (math), and all ethnicities except white and Indian (Campbell, 2015). Likewise, research by Lavy & Sand (2016) suggested that gender biases were implicated in long-term consequences for domains such as STEM and boys’ achievement (positive effect) and girls’ achievement (negative) (Lavy & Sand, 2016).

Teachers also specifically have biases about people with different types of special needs. For instance, researchers looked at biases about students with speech sound disorders; and their results suggested that teachers may judge students’ school performance based on intelligibility and speaker pitch (Overby, Carrell, & Bernthal, 2007), such as judging those students with moderately intelligible low-pitched speech as having more behavior problems (Overby, Carrell, & Bernthal, 2007). People may have biases about adults and children with speech and communication disorders, such as a lisp—and judge them as not being as intelligent, even if the disorder has no relationship with intelligence (Lockenvitz, 2016). Teachers may also have biases about the competence of students with autism spectrum disorder and developmental delays (Travers & Ayres, 2015). An older model of special needs was based on an idea that there was a “deficit” or gap in ability in children; whereas a newer model is one of presuming competence and seeing those people with autism spectrum disorder as being capable of feeling and thinking (Travers & Ayres, 2015).

Researchers also uncovered biases related to who gets referred for treatment and/or further support for special education and special needs. One’s professional judgment of behavioral issues or academic competence is affected by their biases (Travers & Ayres, 2015). For instance, males are more often referred for learning disabilities services and support than females by teachers (Flynn and Rahbar, 1994) suggesting gender bias in beliefs around learning disabilities and in interpreting behaviors associated with this (Anderson, 1997). On the other hand, some studies suggested that teachers’ judgments and behavior were not driven by racial and socioeconomic bias (Abidin & Robinson, 2002).

Teachers are not the only ones with biases and are no more or less biased than other populations (Campbell, 2015). Biases also start young. Research by Quian, et al., (2015) suggested that implicit racial biases were present in three to five-year-old children in China and Cameroon. Parents have implicit and explicit biases against children with obesity (Lydecker, O’Brien, & Grilo, 2018). Biases do not always lead to negative or problematic outcomes. Researchers found a leniency bias that favored applicants to jobs who had a physical disability or handicap (Brechner, Bragger & Kutcher 2006).

The relationship among bias, attitudes, behavior, and social context, however, is complicated. Biases that educators may have also influence, and are influenced by, biases of students. People may be both objects of stereotypes and biases, as well as perpetrators or agents of biases and stereotypes, even at the same time (Castillo, Cámara & Eguizábal, 2011). What may be problematic or stigmatized behavior or characteristics in one social context may not be stigmatized in another time, place, or social situation (Lockenvitz, 2016).

Bias and Identity

The relationship becomes even more complicated as one’s own identity, other’s biases and one’s internalized biases, and structural and systematic inequality, can even one’s affect performance and overall educational attainment (Burgess & Greaves, 2009). Internalized biases and stereotypes about one’s identity or group identity, may also affect one’s performance. Stereotype threat, the risk of living up to the negative stereotypes that an individual hears about one’s group or one’s identity, affects performance and achievement, and affects marginalized minority groups (such as women and people of color) and may lead to disengagement in STEM and other domains (Woodcock, et al., 2016; Woodcock, et al, 2012; Steele & Aronson, 1995).

Biases are also embedded in design itself—whether of the classroom, the learning experience, or a game or interactive application. For instance, biases have been found in artificially intelligent algorithms, which then become embedded in platforms that rely on them, such as search engines, face recognition, or social media platforms (Howard & Borenstein, 2017). Models and systematic educational approaches may also be embedded with the biases of the time and the social context of its creators. As mentioned earlier, more recent models of presumed competence replaced previous approaches toward those with special needs as if they have a deficit (Travers & Ayres, 2015); and each model or theoretical approach comes with its own set of biases and ways of seeing the world.

How do we reduce biases? What can design do to reduce biases and support greater equity, accessibility, and support for all human beings? But how do we mitigate disrespect for others and reduce negative social behaviors, such as bias? In the section, I will discuss findings and studies that investigate bias reduction, much of which is contradictory, and apply it to games.

Games and Bias Reduction

There is limited research on how to successfully reduce biases and biased behavior, and which strategies are most effective. Likewise, there is even less research on how to reduce biases using games and/or other designed playful experiences. In this section, I will share and apply the existing research on bias reduction and games, and identify the possible strengths and limitations of games, particularly in relation to helping teachers and other education professionals reduce biases in relation to students and colleagues with special needs. I will also explore open questions and gaps, and recommended areas for further research.

Possible Strengths

Games can help to support a professional development experience for practitioners, or can function as part of the intervention

A game can be part of a professional development experience that is virtual, in-person, or a hybrid of the two. This experience, depending on how it is designed, can be the entirety of the intervention, or can be part of it, and support other aspects of an intervention. How the entire intervention is designed matters, just as how a game is designed also matters. While some interventions, workshops, and even simulations have worked to reduce biases (Simonovits, Kézdi & Karos, 2018), many do not (Nario-Redmond, Gospodinov, & Cobb, 2017). It often depends on how the intervention is designed, rather than the fact that there is an intervention. For instance, there are also many “trendy” interventions that do not work, such as for those with autism spectrum disorder (Travers & Ayers, 2015).

Researchers have begun to drill down into what types of interventions work for which types of biases, populations, and prior experiences (Castillo, Cámara & Eguizábal, 2011). Brecher, Bragger & Kutcher (2006) looked at job interviews, and their findings suggested that structuring an interview (standardizing the interview in some way to decide on questions and evaluation rubrics and methodologies (Campion, et al., 1997) helped to reduce biases in favor of applicants with physical disabilities. Williams, et al. (2018) looked at medical schools and found that those programs with a lower incidence of bias were found to have “longitudinal reflective small group sessions; non-accusatory approach to training in diversity; longitudinal, integrated diversity curriculum; admissions priorities and action steps toward a diverse student body; and school service orientation to the community” (p. 1). Though this does not suggest causation, it does suggest possible features to address in future training environments. They recommend, for one, the creation of learning communities as this would support reflective small groups over time (Williams, et al., 2018).

How can games, specifically, be incorporated into interventions? First, either analog or digital games can be used as part of in-person in workshops or other interventions to support learning and interaction among participants. The ADL’s anti-bias teacher training, for instance, includes non-digital games and playful interactions, such as “Things in Common,” a game where players need to figure out aspects of their identity that they have in common before the time runs out (ADL, 2018). Adathi, et al. (2015) explains that games can serve as intervention tools to reduce bias, particularly when played cooperatively, even if the participants are in different locations. This may be particularly effective when people on the same team are aware that they are working with people from “out-groups” (Adathi, et al., 2015). While a completely virtual game that functions as an intervention is not an exact replication of an in-person, interactive professional development experience, it may have instructional value, and its own strengths and weaknesses. Games, for instance, have been used effectively for instructional purposes to teach both skills and content knowledge in a variety of areas, from science, technology, engineering and mathematics (STEM) to English language arts (ELA) to music (Schrier, 2018). Quandary, created by Learning Game Networks, helps teach middle and high school students how to build arguments and interpret evidence, by placing the player in a new society, where they have to help decide how to solve problems in the new society.

Games can possibly cultivate the practice of empathy, perspective taking, and compassion

Research on bias reduction, particularly racial and ethnic biases, often points to encouraging empathy and perspective-taking (Batson, 1987), which has helped improve attitudes toward marginalized groups (Dovidio, Pagotto, & Hebl, 2011). Though there are many different definitions of empathy, in general, empathy has affective and cognitive components and is defined as caring about what someone else is going through, thinking, or feeling (Schrier & Farber, in progress). Recent research on bias suggests that people tend to treat others with more empathy if they feel like they are more “like them” (in one’s in-group) as opposed to those who they think are different (an “out-group”) (Darvasi, 2016). Seeing someone as being in one’s “in-group” or in an “out-group” can be based on one’s abilities, race, gender identity, religious affiliation, or even sports or fan interests. Intergroup contact has been a common tool for helping people reduce their biases about other people (Castillo, Cámara, Eguizábal (2011 Allport, 1954; Pettigrew & Tropp, 2005, 2006), in that people become more accustomed to others by spending more time with them, and begin to see someone as more of a member of an “in-group.” For instance, Powell (2014) looked at stigma around mental illness, and found that those who have exposure to mental illness are typically those who have less anxiety around it, and more positive beliefs and fewer negative beliefs about mental illness and its prognosis.

Perspective-taking activities may also increase connections among disparate groups. Castillo, Cámara, Eguizábal (2011) describes perspective-taking as the process of cognitively thinking through what one “cognitive approximation between the self and members of the stereotyped group and between the ingroup and the outgroup” (p. 168). Such perspective-taking could lead to more positive views of “out-groups,” reduced communication of stereotypes and stereotypical representations (Galinsky & Moskowitz, 2000; Galinsky et al., 2005, 2008; Castillo, Cámara, Eguizábal, 2011). Perspective-taking has also been associated with a greater willingness to engage with others that are from another group (Wang, et al, 2014). When engaged in perspective-taking, people will start to ascribe the perspectives of the other as their own, and vice versa, which in turns strengthens connectedness between the two groups (Galinsky, et al, 2008). Even just feeling that someone else is taking our perspective may be beneficial as taking another’s perspective, as it could lead to more connection and feeling of overlap with the other person and greater empathy for them (Goldstein, et al., 2014). Self-esteem and empathic feelings may also interrelate to the effectiveness of perspective-taking (Vescio, et al., 2003; Galinsky & Ku, 2004). Vescio, et al (2003) had participants take on the perspective of an African American student, and the participants self-reported more empathy and more positive attitudes toward African Americans in general following this exercise (Vescio, et al, 2003; Castillo, Cámara & Eguizábal, 2011). Likewise, Castillo, Cámara, Eguizábal (2011) studied older adults and provided a story about a Moroccan immigrant and led through perspective-taking exercises. Their research suggested that perspective taking was moderately effective for reducing stereotyping and that this research has implications on training programs to support the reduction of intergroup biases among older people, for example.

Familiarity, exposure, and contact with others may also help (Steele, Maruyama, & Galynker, 2010). For instance, teacher training, workshops, and professional development opportunities that use these types of methodologies have been shown to enhance more positive attitudes in people, such as preservice teachers toward linguistically diverse populations (Cho & DeCastro- Ambrosetti, 2005).

How can games further encourage empathy, perspective-taking, exposure, and other related practices, such as people with special needs, or people with different backgrounds, perspectives or experiences? Many researchers have pointed out the potential of virtual spaces to support the practice of empathy, bias reduction and compassion (Faber & Schrier, 2017; Aviles, 2017; Schrier & Shaenfield, 2015; Schrier, in progress). Most research has considered how to apply the aforementioned theories and practices around reducing intergroup conflict and biases, and applying them to virtual worlds and games (Yee & Bailenson, 2007; Faber & Schrier, 2017; Aviles, 2017).

For instance, some games may help to immerse people into virtual worlds and new roles and identities (Schrier, 2014), which may encourage consideration of others’ experiences, feelings, and perspectives (Faber & Schrier, 2017). Games may help people express and experiment with their own identities and others’ identities (Schrier & Faber, 2018), and may enable people to communicate and interact with people from other cultures, with other types of needs, and with different types of experiences (Schrier & Shaenfield, 2015). As Aviles (2017) writes, “If one reason individuals self-segregate in the physical world is because they have no prior experience interacting with diverse individuals, the virtual world can be a safe place to gain this experience and lead to further contact in the real world. In addition, virtual contact with diverse representations of users may be enough to enact the positive qualities associated with contact in the physical world.” (Aviles, 2017, p. 3).

Games may help motivate players to consider others’ perspectives and to share their own views (Schrier, et al., 2014), and to work with others to solve real-world problems (Schrier, 2016), which may help to build empathy for those in an “out-group” by making them more of an “in-group.” For instance, in games, players can learn about other’s perspectives and connect with (virtual) people and stories in which they may not typically interact (Farber & Schrier, 2017). Research has suggested that even virtual perspective-taking can decrease bias and negative attitudes toward “out-groups,” and also enhance compassion and helpfulness toward others (Hasler, et al., 2014). Though this is complex (Nario-Redmond, et al., 2017). Simonovits, et al. (2018) created a perspective-taking game that significantly reduced prejudice toward stigmatized groups in Hungary (Roma and refugees); this reduction lasted at least a month, and also resulted in changes in voting behavior. In Revolution: 1979, players are able to take on the role of a person from history during the events leading up to and during the Iranian Revolution of 1979. The game incorporates authentic perspectives from this time period and moment, and helps players to better understand the types of issues that led to the uprising, as well as how that may affect culture and politics today. Another example, The Migrant Trail, is a game that allows players to explore two different perspectives on migration from Mexico to Texas, and shows both the dangers of being a migrant, as well as the difficult decisions in being a border patrol agent. Games can not only share perspectives, but also help us express and appreciate emotions. In That Dragon, Cancer, a game about a (real-life) family grappling with their young son’s cancer, players can connect through themes of loss, family, and love (Farber & Schrier, 2017).

Moreover, researchers also found that they could redesign the features of a platform to counteract the tendency of people to make judgments based on social biases and trust people more if they are more similar to them (Abrahao, et al., 2017). Abrahao, et al. (2017) found that they were able to increase trustworthiness of dissimilar people from the participants on the Airbnb platform by including a reputation system.

Games can foster a learning community and enhance exposure to others through social interaction and cooperatiive problem solving and working on tasks

Games can support the creation and interactions of a learning community, or a community of practice (Wenger, 1998). For instance, games can help participants interact with each other around topics of interest (Crowley & Jacobs, 2002), can help participants collaborate on solving problems (Schrier, 2016), and can encourage more experienced players and practitioners to train novices, or to trade knowledge around different types of skills (Steinkuehler & Oh, 2012). Players (such as those of World of Warcraft) may develop shared norms by participating in these communities (Steinkuehler, 2007) and can help players acquire the vocabulary, tools, and epistemic understandings of those communities (Shaffer, 2006; Squire, 2011). As mentioned earlier, a learning community can help to encourage connections among disparate groups, as well as encourage a sense of belonging and inclusion in a game community, which may contribute to empathy, perspective-taking, learning, and positive exposure to others’ backgrounds and cultures, and greater self-efficacy and social support (Schrier 2016; McGonigal, 2011).

Games may also allow players to work with and partner with others across distance and time, who are not only not part of their everyday peer group, but may even be from the outgroup. If players are depending on each other, and need to communicate with each other on a shared task, they are able to build a more collaborative, altruistic relationship with each other (Schrier & Shaenfield, 2015; Adachi, et al., 2016). For example, in Eyewire and Foldit, players work on real-world problems (such as understanding brain cell and protein structures, respectively) and communicate with each other through the game’s platforms, and often form learning and problem-solving communities, which have helped to contribute to significant scientific discoveries (Schrier, 2016). Intergroup contact enhances empathy for the out-group and intergroup empathy decreases prejudice for the out-group (Adachi et al., 2015).

However, competitive and/or violent games may not have the same effect as collaborative and/or cooperative ones (Adathi et al., 2015). Adathi et al (2016) looked at violent video games and those who played it cooperatively were able to have more improved out-group attitudes than those who played them solo. Those playing a violent or nonviolent game, cooperatively, were more likely to increase attitudes toward the outgroup versus those playing either game by themselves (Adathi et al., 2016; Greitemeyer, 2013). It is unclear as to the mechanism for why cooperative play enhances these attitudes, and often these types of cooperative interactions among disparate groups are avoided (Adathi et al., 2015). Thus, while research by Adachi, Hodson & Hoffarth (2015) suggested that competitive games are associated with enhanced intergroup bias, games that support intergroup cooperation may reduce bias, particularly in multiplayer games online.

Players in a game are solving problems simply by the very act of playing a game. Players need to figure out how to overcome obstacles, avoid enemies, or reach specific goals. Players in a game could solve these types of problems on their own, or as mentioned in the previous subsection, they could solve those problems with others. Problems could be solved using a “jigsaw” method or having different players have different roles, each of which helps to solve part of the problem. However, the entire team needs to work together to solve the problem. By having players solve problems with others, players start to see that people have different abilities, and different ways of contributing. This may help players to be more open to different types of skillsets and experiences and help to make people feel more “in group” with those who are in an outgroup.

In research on role-playing games, such as Fable III, Schrier (2016) found that players used more empathy-related skills, including perspective-taking, after spending time playing and overcoming obstacles with others, even virtual characters (Schrier, 2016). Schrier & Shaenfield (2015) also found that playing collaboratively with another (real) person in a game increased one’s willingness to engage with the collaborator, befriend them, and enhanced more compassion and empathy toward them (Schrier & Shaenfield, 2015). In the multiplayer online game and cooperative game Way, players tended to connect, forge friendships, and identify each other’s emotions to support collaborative problem solving (Schrier & Shaenfield, 2015).

Games may allow expression and experimentation with identity, and help to increase identity self-efficacy

Games can enable people to play as a role and shape their role’s identity, as well as gain confidence and self-efficacy in their identity. Games may also enable players to interact with other types of roles and identities as well. For instance, in some games players can shape their avatar (or their representation in the game), they can role-play as other identities, they can “level up” by practicing certain skills (e.g., magic, weaponry), and/or can make choices or interact with other characters or players to shape their own role in the game. In Mass Effect, you can make choices that lead more to a “renegade” character, or one who tends to disobey rules and social norms to get things done, or you can make choices that lead more to a “paragon” character, or one who follows the rules and social norms on their path. In the Walking Dead (Telltale) game, players make choices on how to support other characters during zombie attacks, which affect how other characters treat the player’s avatar and also shapes how events unfold in the game. In the Fable series, players can make ethical choices, as well as choices on how to rule their land (as in Fable III) and their decisions directly impact how other characters (NPCs or non-player characters) treat them, or how they look (the avatar or character representing the player starts to look more devilish or more angelic, or more scarred or more clear-skinned depending on their choices. They also might get brawnier or more tattooed depending on weapons used or abilities earned. Expressing one’s own identity, experimenting with different identities, and learning about other’s identities has been implicated in supporting empathy (Banakou et al., 2016) as well as in reducing bias (ADL, 2018) in that it helps to make any differences seem less “othered” and more understood and appreciated (Darvasi, 2017).

Who someone plays as (their avatar) may affect their biases as well. Behm-Morawitz, Pennell, & Speno (2016) investigated avatar creation and a game to understand whether participants took on the perspectives of others. They found that creating and being a Black avatar in the game helped to support more favorable attitudes about African American men (though not women) and more support for policies that support minorities, than playing as a White avatar (Behm-Morawitz et al., 2016). Banakou et al., (2016) also found that White people’s implicit racial bias against Blacks decreased after virtually embodying a Black avatar. Likewise, in another study, Peck et al., (2013) found that light-skinned players inhabiting a darker skinned avatar reduce their biased associations with darker-skinned people. However, the game context may matter. Yang et al., (2014) found that in a violent video game, playing as a Black avatar (instead of a White one) enhanced negative beliefs toward Blacks, and the players behaved more aggressively in the game. On the other hand, Aviles tried two different theoretical approaches to reducing prejudice using avatars in a virtual world (the Proteus effect (or the idea that how an avatar looks affects one’s own behavior), as well as a theory of intergroup contact) and found that neither were effective (2017).

Increasing self-efficacy around one’s identity can also help people to persist in systematically biased environments. Enrichment programs can excite interest in STEM by enhancing scientific curiosity (Ogle et al., 2017), which in turn leads to high scores of self-efficacy around science and also higher STEM knowledge (Ogle et al., 2017). Leonard et al., (2016) used robotics and game design to enhance self-efficacy around STEM for middle school girls and indigenous populations. They found that students who participated in blended robotics/gaming clubs had higher self-efficacy scores related to the construct of videogaming (Leonard et al., 2016) and created effective game prototypes. Siritunga et al., (2011) taught biology using a culturally and personally relevant model and found that half or more were confident or very confident in their results, and an increase in content knowledge occurred after the module.

Well-designed games scaffold (or support) content over time, and level it appropriately, so players are only getting what they need at each moment to do what they need to do to reach a goal (Fullerton, 2016). This type of design avoids overwhelming cognitive load, where a player has too much information to attend to and does not know what to attend to at each moment. It also helps to balance the players’ mastery of material with their ability to practice it without being too bored, such that they end up in a “flow state” (Csikszentmihalyi, 1990) or centered state where they feel both a sense of mastery without a sense of tedium and boredom. This state also relates to a feeling of higher self-efficacy or the feeling that one is able to do what they need to do and reach the goals they need to reach. Encouraging a higher self-efficacy may also support people’s ability to help others and engage in prosocial behavior through the game or perhaps even beyond the game.

Games can encourage moments of reflection and give clear and systemic feedback

Part of the learning process is not only acquiring and using information and skills, but reflecting on the learning process itself, and how it might be changing one’s views, behaviors, and perspectives (Mezirow, 1996). It is important for professionals to not only take actions, but to reflect on the consequences and outcomes of their choices, and how they may act differently in the future. The reflective process also helps to crystallize new knowledge and allows us to think about how we think, or engage in metacognitive processes, which further critical awareness, flexibility, and possibilities for future adaptation (Bransford, Brown, & Cocking, 2000).

How can games support reflection and reflective moments, especially when games often require the player to keep moving forward toward a goal. While not all games provide points where a player needs to reflect on their choices, games can provide outcomes, consequences and feedback on choices, which can be used to help players reflect on those choices and consider how they would make choices differently. For instance, in Life is Strange, the game takes the player back to moments where they made a decision and has them reflect on the decision and its consequences, and then invites them to make a different decision to see how the consequences play out (Schrier, 2018). Outcomes could be large (a town disappears, a person commits suicide, a person gets murdered) or smaller (a woman gets bullied). Life is Strange also includes gameplay where a player can decide to sit on a bench and just look around for a while, suggesting to the player that they can take breaks from the storyline and gameplay, the reconsider their actions, and engage in the present moment.

Clear and comprehensible feedback is important in any learning situation. People need to know how their actions and choices are evaluated, and in what ways their behaviors affect the world. As they receive more feedback, and more specific feedback, this helps them adapt and reorient themselves to better meet the goals and needs of a learning environment. Well-designed games are typically adept at giving this type of feedback, and enabling players to know how their choices are valued in the game, what the effects are, and whether and how well they are doing at reaching their in-game goals. The more precise, personalized, relevant, and “just-in-time” this feedback is, the more useful it is to the player. For instance, in Lim, players play as a colored square who is moving through a board, which is filled with rooms that have groups of squares that are different than the players square. As they move, the other squares have different reactions to the player, such as bumping into the square, pushing it away, or attacking it, and the player gets immediate feedback on how their square will be treated. The player can choose to “pass” as different colored square, or adjust their square’s color temporarily, and this has an immediate result of the other squares being less combative and more tolerant. Using these simple interactions and feedback, the game is able to express an abstraction of the effect of individual biases on a person.

Likewise, in a complex system like a game, players can experience change dynamically rather than just linearly. The feedback that players receive does not have to be linear (1:1), in that players only get to try something or make a choice, and then get a clear piece of feedback returned. Feedback in games can also be provided in a such as way that players can understand consequences and effects in a more dynamic way. For one, games can help to simulate and express complex systems, and enable players to interact within those systems, helping them to visualize and experience shorter- and longer-term effects. But players can push on and play with the boundaries of these systems, try out “what ifs,” enact behaviors, and perform within the system, such that they begin to interact with more holistic feedback provided on a more system-wide level (Schrier, 2016). Through a game, players can also experience and comprehend more complex, messy consequences—even ones that happen over time or have multiple levels of interactions. As a result, players can build systems thinking skills and the ability to see how different behaviors, biases, and attitudes may affect others, not only on a one-on-one basis, but at a system-wide level. For instance, though it’s not a game, per se, Parable of the Polygons is an interactive simulation that shows how biases affect interactions on a more systemic level. Participants are asked to move objects around to show how individual biases may result in more system-wide and institutional biases.

Thus, these types of simulations can help establish contexts to support intergroup relations and a reduction in bias (Adathi et al., 2015). Adathi et al. (2015) explain how enabling students who identify as heterosexual to imagine, discuss, and consider life on a different planet for those from a marginalized minority group helps to reduce prejudice against LGBTQ+ populations in real life. The context of imagining the short-term and longer-term effects for a group help to reduce out-group prejudices and enhance trust among groups (Hodson, Choma, & Costello, 2009; Hodson, Dube, & Choma, 2015; Adathi, et al., 2015).

Possible limitations

While there are many possible strengths of games in supporting bias reduction and changes in empathetic behaviors, there are also many limitations. For one, many games are not designed specifically to reduce biases or support perspective-taking. The design of the game, rather than the fact that it is a game itself, matters more in how well it is effective. The following are general limitation of games, however, the biggest limitation is simply that they are very dependent on how they are designed, which is complex in understanding.

Like any designed intervention, a game may not meet its goals

Biases and biased behavior are highly complex and dynamic, and require similarly complex and dynamic solutions. Can any intervention, whether a game or not, even reduce biases and change behavior? Researchers have debated whether biases and stereotypes are always automatic, and whether we will be able to actually reduce implicit biases (Devine, 1989), stereotypes, or prejudices, as they are too ingrained and may also affect social behavior (Quillian, 2006, 2008; Castillo, Cámara, Eguizábal, 2011). Other research suggests that the activation and application of the stereotype can be adjusted, and even automatic biases can be changed, with trainings and interventions (Castillo, Cámara, Eguizábal, 2011). As such, research on interventions is often contradictory, as it depends on the design of the intervention, the interaction with a specific audience, and the context of its use, among many other factors. Any type of intervention has limitations, and may be ineffective or unsuccessful in reaching its goals. While some games may take place over many months or even years, depending on the level of engagement of the player, it is difficult to design a full game experience that is so versatile and engaging for all players that it meets the goals of the intervention, fits appropriately into the context, and meets the needs of the audience. Design and context matter (Gentile, 2011) and may interact with how people play the game, treat each other, and their attitudes and behaviors toward each other (Adachi, et al 2015). The same game played in a different context may not be as successful. How a game is played (with others, alone, in a classroom) may also influence outcomes as well as what type of game is played (such as a cooperative or prosocial game) (Adachi et al., 2015). Games, for instance, need the right types of support, and are not “one-size-fits all” “out-of-the-box” solutions. They often require an appropriate teacher or mentor to help shape the game and curriculum to each other.

Overall, the simulations used to reduce biases have had mixed and limited results. Nario-Redmond et al. (2017) explain that a review of empirical evidence for disability awareness simulations resulted in only four prior studies and only one suggested that upper elementary school students had enhanced knowledge and understanding for people with a disability (2017). Nario-Redmond et al. (2017) used a virtual reality game to show the perspective of someone with a wheelchair and found that the game was not effective for enhancing constructive behavior toward those with disabilities.

Games may even further activate and spur biases, particularly competitive ones further expressing how the design of the game matters. For instance, research by Greitemeyer (2013) suggests that violent game play may enhance intergroup biases, however, the underlying mechanism for this is unclear as there have been mixed results as to whether violent video games cause real-world violence or aggression. Greitemeyer & Mügge (2014) also explore in another meta-analytic study how violent games are associated with antisocial behavior, whereas prosocial games are associated with prosocial behavior. Likewise, research by Adachi, Hodson & Hoffarth (2015) suggested that competitive games are associated with enhanced intergroup bias, whereas games that support intergroup cooperation may reduce bias.

Games, players, and the cultural context around them are embedded with biases

Games are built with and embedded in societal and individual biases, even unwittingly. All games, just like any designed experiences, embed the biases of how they were designed, who designed them, and the sociocultural context in which they were designed (Deng, Joshi, & Galliers, 2016; Flanagan, 2009; Flanagan & Nissenbaum, 2014). Just the fact that we are using a game to try to reduce biases is a type of bias in that it suggests that games may have value in this way. Moreover, the communities around and within games have their own norms, values, and biases that are embedded in their design, and how they emerge over time. Games have also been cited as places where biases and intergroup conflict persist and are propagated (Adachi et al., 2015; Greitemeyer & Mügge, 2014). While most game players and communities are respectful, mature and engage in appropriate language, some game players and communities participate in toxic behavior, or purposefully volatile or problematic communications, such as bad language, mocking, negative tone, taunting, or even biased remarks or stereotyping, the amount of which may even vary by different game communities and fanbases. While many games, particularly more collaborative games, may not be as supportive of antisocial behavior, and some game companies have been trying to reduce this using various strategies, there remains a stigma around games themselves as being propagators of bias and antipathy—the very behaviors and attitudes that we hope to address and even reduce. This stigma, a bias in itself against games, affects how games are interpreted and the types of ways they are used (Schrier, 2016).

A game may be well-intentioned, have appropriate goals and operationalize change in an appropriate way, but still fail to meet its goals. This is because players come into game contexts with their own myths, misconceptions, and biases around people, relationships, and social structures, and it is hard to predict how people will complexly affect the game system, and be affected by the game system, dynamically. In fact, games have been used as environments in which to study and understand intergroup conflict and biases more generally (Greitemeyer & Mügge, 2014; Adachi et al., 2015; Ferguson, 2015; Pryzbylski et al., 2014). Most of the research on games and prejudice has been around using it as a site for understanding implicit bias, rather than on how to constructively support less bias for another group. A number of studies looked at shooter games, where players need to quickly decide who to shoot – they need to shoot at armed targets and avoid shooting at unarmed ones as they appear on the screen (Adachi, et al 2015). For instance, researchers have found that there is a bias toward shooting unarmed Black rather than White targets (Adachi et al., 2015; Correll, Hudson, Guillermo & Ma, 2014) and have also been able to decrease this bias in the short-term by presenting counter-stereotypical scenarios to the police officers (Adachi et al., 2015; Sim, Correll & Sadler, 2014).

Players may enact and perform their own individual biases in games, or even express biases that emerge from the particular design of a game experience. Yang et al. (2014) found that those playing a violent game as a black avatar versus a white avatar had more negative and stereotyped attitudes toward Blacks if they had a black avatar in the violent game, but in the nonviolent game, neither condition had more stereotyped attitudes toward African Americans after playing the game. The design of the game interacts with already present societal stereotypes and could activate them.

Players may bring their preconceived notions, myths and biases to the game. Roussos & Dovidio (2016) used the game, SPENT, to understand whether players would reduce biases against those with financial insecurity by playing a game about the poor. They found that those players who came in with the preconceived notion of a “meritocracy” (in that systems reward those who work the hardest and are the most deserving based on personal characteristics), ended up having less empathetic attitudes toward those in poverty than those who did not have the meritocracy preconceived notion (Roussos & Dovidio, 2016; Farber & Schrier, 2017).

Players and designers need to consider ethical ramifications

Those who are using or playing games to support bias reduction also need to understand and reflect on the ethical ramifications of their designs or gameplay. For instance, players may need to be mindful of how they experiment with and/or try on others’ identities. Players also need to be respectful of how they take on or engage in dialogue with the perspectives of others. One major critique of games and virtual reality when players take on roles of marginalized individuals and engage in perspective-taking is that the player is really a “tourist” who briefly takes on a fictionalized version of an individual or group, and then mistakenly think they perfectly understand the perspectives of others, while also being able to slip off that identity without feeling real consequences. This may then reinforce misconceptions and make it even more likely to dismiss real individual’s perspectives or more systemic inequities.

Another myth might be that games are even able to change biases or behaviors, and that games, like other technologies, are all-knowing, all-powerful and magical. This has ethical implications because it obfuscates its flaws, and also how important the context, player, and other factors are in making a game successful. Tailoring and communicating these expectations properly will help to ensure that the game is appropriately contextualized and that ethically transparent. It will also help ensure that the game is effective as it focuses on the humanness of games, rather than its cold, technological prowess. Games may not always effectively teach or change behavior per se, but they can possibly give us experiences with people we may not come in contact with often and give glimpses into different aspects of humanity, even if they cannot perfectly simulate a system or how someone feels, thinks, dreams or acts.

Designers need to address how to best cultivate the practice of empathy

There are possible general limitations to using a game for the cultivation of empathy, compassion, which should be considered (Farber & Schrier, 2017). For one, putting someone in “another’s shoes” or simulating their life may increase empathy, but also distress and overwhelming negative emotion. In Against Empathy, Bloom (2016) argues that empathy can backfire (in part) if people get too immersed in their own response to someone else’s pain or suffering, that they cannot properly and appropriately help them. For instance, Nario-Redmond, Gospodinov, & Cobb (2017) found that after showing two simulations about a person with a disability such an individual with dyslexia or mobility issues, the participants felt more upset, anxious, ashamed and helpless than before. Although their empathy toward people who are disabled increased, their openness to interacting with people with disabilities did not get better, and they felt that they would be less effective in interacting with someone with a disability (Nario-Redmond et al., 2017).

Galinsky et al. (2008) argue that perspective-taking helps participants take on the so-called stereotypical behavior of the other (“out-group”) (both the positive and negative ones). Perspective-taking may even “backfire” and lead to negative outcomes when a person feels threatened (Sassenrather, Hodges & Pfattheicher, 2016) or even where there are good intentions (Holoien, 2014). Perspective-taking may also work well in Western cultures, but may not be as effective in East Asian cultures (Wang et al., 2018). Perspective-taking may help enhance understanding, but the process may not predict the other person’s actual views, mental state, emotions, and attitudes; whereas communicating with the other person and learning about them, and gaining their perspective (exchanging perspectives), did help to increase accuracy of understanding the other person (Eyal, Steffel, & Epley, 2018). For instance, Gloor & Puhl (2016) looks at strategies for reducing weight bias and found that empathy-induction and perspective-taking conditions both enhanced more empathy for people with obesity than the other conditions, but may not reduce overall stigma about weight (Gloor & Puhl, 2016). Moreover, “imagining oneself in the place of others—rather than taking the other’s perspective—is less effective at inducing empathy and help” (Nario-Redmond et al., 2017). In sum, perspective-taking is complex in that it can enhance connection among groups and between those of in-groups and out-groups, but it can also contribute to increased flawed understandings of the other, and enhanced use of stereotypical evaluations and behaviors associated with the out-group (Nario-Redmond et al., 2017).

Moreover, research has suggested that players need to build relationships and interpersonal connections over time to support empathy and perspective-taking (Schrier, 2014), such as in the case of Fable III, or need to interact with them in a way that they mutually benefit and play together (Schrier & Shaenfield, 2015), such as in the case of Way. Players need to have the time to authentically build intimacy with other players or even non-player characters (NPCs), or virtual characters that are not controlled by a player.

Finally, a game may be well-designed, but it if no one plays it, it will not be able to support any types of experiences for its players. Players need to be motivated to play a game that encourages empathy, and be open to the types of interactions, choices, relationships, and outcomes in the game. Moreover, these choices need to be meaningful to the player—not just in terms of how the game is played (in that the choices affect something in the game itself), but that it is also meaningful to the player and what is personally relevant to them (Schrier, 2018). This may be many different things to many different players, so it is important to allow for a variety of play styles and experiences such that players can form their own personal connections to the experiences.

Conclusions and next steps

This paper seeks to analyze the recent research on bias education and couple it with research on games and bias reduction (with particular attention to special needs), to help make recommendations of how to better design game and related environments to support bias reduction and less biased behavior toward others. There are many complex and dynamic interactions that both spur and reduce bias, prejudice and discrimination. This complexity is also seen in interventions such as games, virtual worlds, and simulations, in that it’s never as simple as “games reduce or increase bias,” but it’s under what conditions, by whom, how, and when. I described five possible strengths and four possible limitations of using games for bias reduction, particularly in relation to special needs. However, almost any aspect of a game could possibly be a strength or a weakness, as games and their players form a dynamic system that depend on many factors, including social context and cultural biases of the time, place, and people.

As next steps, each of the strengths and limitations should be empirically studied and further investigated. For instance, we may want to take each of our strengths and weaknesses and ask additional questions. In terms of strengths, the following are questions we may want to pursue further, as these will have implications for designers of games to reduce biases, players of these games, and people who are creating interventions for teacher educators to support their professional development:

Games can help to support a professional development experience for practitioners, or can function as part of the intervention. Which types of games work best within different types of interventions for bias reduction and special needs educators? How can games be best integrated into professional development for teachers and educators; what are the factors and configurations that are most effective?

Games can possibly cultivate the practice of empathy, perspective taking, and compassion. What are the specific design elements and gameplay that best support the cultivation of these skills, and particularly, in how they may help to reduce biases? How does storytelling, emotional expression, character development, relationships, reflection, transportation and other factors specifically affect this practice?

Games can foster a learning community and enhance exposure to others through social interaction and cooperative problem solving and working on tasks. How can we best support, manage and cultivate useful learning communities using games? How do we provide the appropriate scaffolds for problem solving, and encourage constructive social interactions among the players?

Games may allow expression and experimentation with identity, and help to increase identity self-efficacy. How do we support players to safely experiment with their own identities and other identities, as well as other perspectives? How do we best enhance player’s self-efficacy and confidence in their own roles to better support bias reduction through games?

Games can encourage moments of reflection and give clear and systemic feedback. What are the most useful ways to provide feedback and how can we use messaging, communication and interactions with the game’s system to further help players grapple with bias? How can we build in reflective moments that will continue to crystallize knowledge and enhance critical thinking of one’s own practices and biases.

Freedman, G. & Flanagan, M. (2017). From dictators to avatars: Furthering social and personality psychology through game methods. Social and Personality Psychology Compass, 11(12): https://doi.org/10.1111/spc3.12368

Galinsky, A.D. & Ku, G. (2004). The effects of perspective-taking on prejudice: the moderating role of self-evaluation. Personality & Social Psychology Bulletin, 30(5):594-604.

Greitemeyer, T., & Mügge, D. O. (2014). Video games do affect social outcomes: A metaanalytic review of the effects of violent and prosocial video game play. Personality and Social Psychology Bulletin, 40, 578–589. http://dx.doi.org/10.1177/0146167213520459.

Ogle, J.P., Hyllegard, K.H., Rambo-Hernandez, K. & Park, J. (2017). Building middle school girls’ self-efficacy, knowledge, and interest in math and science through the integration of fashion and STEM. Journal of Family and Consumer Sciences, 109(4): 33-40.

Roussos, G., & Dovidio, J. F. (2016). Playing below the poverty line: Investigating an online game as a way to reduce prejudice toward the poor. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 10(2), 1-24

Cover Art:Extase – a music game designed by R. Herbulot; programmed by P. Dublanchet; graphics by M. Rho; head design by P. Ulrich; music by P. Eidel & A. Devos and the Bulgarian Voices, courtesy Virgin Musique; interactive music by S. Picq; Virgin Loisirs, France 1991.

]]>Issue 6, 2017 – Hear the Music, Play the GameMusic and Game Design: Interplays and Perspectives

Edited by H. C. Rietveld & M. B. Carbone

Cover Art:Extase – a music game designed by R. Herbulot; programmed by P. Dublanchet; graphics by M. Rho; head design by P. Ulrich; music by P. Eidel & A. Devos and the Bulgarian Voices, courtesy Virgin Musique; interactive music by S. Picq; Virgin Loisirs, France 1991.

There is a growing recognition of the role of music in games by the gaming industry, game fans, and journalists. Several conferences have been established on the roles of music and sound in video games, such as the industry-focused GameSoundCon, first initiated in Los Angeles in 2009, and Game Music Connect, that has taken place annually between 2013-15 in London. Simultaneously, the study of music and audio in games is gaining interest in game studies. For example, Rob Hubbard, most famous for his work on the Commodore 64 system, has been recognized with an honorary degree by Abertay University in Dundee, Scotland (Wawro, 2016). The tendency, however, is not only in response to the industry. It is also in line with an “Auditory Turn” in the humanities and social sciences, providing an alternative sensory approach to a notable visual dominance in the humanities and in media and cultural studies (Herzogenrath, 2017).

There is a growing recognition of the role of music in games by the gaming industry, game fans, and journalists. Several conferences have been established on the roles of music and sound in video games, such as the industry-focused GameSoundCon, first initiated in Los Angeles in 2009, and Game Music Connect, that has taken place annually between 2013-15 in London. Simultaneously, the study of music and audio in games is gaining interest in game studies. For example, Rob Hubbard, most famous for his work on the Commodore 64 system, has been recognized with an honorary degree by Abertay University in Dundee, Scotland (Wawro, 2016). The tendency, however, is not only in response to the industry. It is also in line with an “Auditory Turn” in the humanities and social sciences, providing an alternative sensory approach to a notable visual dominance in the humanities and in media and cultural studies (Herzogenrath, 2017).

Sound has, of course, always been a crucial aspect of gaming audio-visuals. Far from merely accompanying a game, the auditory elements bring life into the game interface.

In line with the auditory turn, the past few years have seen an explosion of studies of sound and music in games. Karen Collins’ work has set the wheels in motion in 2008 with an edited collection. In the same year as her the landmark publication, Game Sound, offered a systematic understanding of game music. Organizations and study groups also emerged as part of an interest in game sound that covers a broad, multidisciplinary field. The annual North American Conference on Video Game Music also held its first event in 2017. Another specific focus on game music has been offered by the annual Ludomusicology conference, which began its work as an RMA (Royal Musical Association) study day in the UK. A Ludomusicology special journal issue for The Soundtrack 8/1-2 was published in 2015, and in September 2017 the Journal of Sound & Music in Games (JSMG) was announced with Mark Sweeney (interviewed in this issue of GAME) as its Director and Michael Austin as Secretary. The study area of Ludomusicology addresses the video game music from the perspective of musicology, as can be found further in a recent collection edited by Kamp, Summers, and Sweeney (2016). In addition to musicology, media and cultural studies provided a wider social perspective on games and music. These include works by Austin (2016) and by Donnelly, Gibbons, and Lerner (2014), both reviewed in this issue of GAME. Further publications are now appearing in the field, including a comprehensive textbook by Tim Summers (2016), also reviewed in this issue.

The above overview of recent approaches to game sound is far from exhaustive, but it provides an insight into the importance of reappraising the sonic element in games. Sound has, of course, always been a crucial aspect of gaming audio-visuals. Far from merely accompanying a game, the auditory elements bring life into the game interface. Sound is a sonic vibration that produces embodied affect. It also elicits interpretations, and provides the player an immersed sonic sense of architectural space. Sound effectively build the game space. The sonic dimension has always taken part in orientating game play perspectives, positions, and rhythms of interaction, from the ominous march of Space Invaders (Taito, 1978) to the more recent experiences of games based on virtual and augmented reality technologies. Music, moreover, provides sound with the potential for temporal and harmonic forms. Game music is a necessary element of the immersive dramatic pace and rhythm of many games. It would be very hard to think of an experimental music shooter like Rez (UGA 2001) without its central sonic element, and it would also be hard to remember a game like Streets of Rage 2 (Sega 1992) without its Yuko Koshiro soundtrack, or Super Mario Bros without its Koji Kondo score (Nintendo 1983). Game studies has only scratched the surface of the importance of music. Interestingly, elements of change come from scholars from the borders of the gaming field. Schartmann, in his study on Super Mario Bros’ soundtrack (2015) provides a holistic, contextual analysis of the success of the game that does justice to its audio-visual-interactive complexity. More simplistic analyses from game studies, on the contrary, seem to forget the sonic dimension of the game, describing it often from the mythical perspective of the genius game designer/solo artist, overlooking some of the manifold, eminently choral elements, agents, and contextual elements that made it possible (deWinter, 2015).

A reappraisal of the importance of the auditory elements in games could has two important consequences: first, it could challenge dominant definitions of video games, suggesting a more nuanced view of the medium characterized by a recognition of its hybrid and polymedia forms; second, it paves the way for alternate histories of games, in which music and sound would regain their apparent, but often overlooked, centrality in players’ experience.

Forthcoming studies on games and music are likely to highlight and look at games from an auditory positionality, as Schartmann (2015) did with Super Mario Bros. The directions are manifold. Sound and music are important elements of narration. They can contribute to the story diegetically, with music created in the dramatic space–think of the use of radio in the Grand Theft Auto series. They can also play their role non-diegetically as an accompaniment that is not always hidden as underscore, but that can become distinctive and memorable–think of the gloomy synths of Shadow of the Beast (Reflection/Psygnosis 1989). Even more so, the interactive soundscape of the new Doom (id Software, 2016) effectively co-constructs the gaming experience: far from sitting behind the gameplay, it is a dynamic and integral part of the pace of the shooter that works alongside the environment and interaction, and an essential element for its brutal elements of pathos. Doom’s music effectively shapes the game with non-linear solutions, taking into full effect previous experimentation with dynamic soundtracks in games. Experimental games like Extase (1991), featured on the cover of our issue, are among the earliest and more radical examples of how music can be the game experience. Designed by R. Herbulot with P. Dublanchet, M. Rho, and P. Ulrich, Extase features interactive music by Stephane Picq as a key to its success as a music puzzle that works as an interactive soundscape.

Yet, game music also lives on outside the game itself, in various guises. Broadly speaking, the sonic elements add to the emotional and cultural dimensions of the game through a wide range of paramusical fields of connotation (Tagg, 2015): elements that take part in defining any gaming text. In effect,As Kamp (2016) shows, music is in menus, start screens and other circumstantial components outside the diegesis of the game. Moreover, orchestral and pop performances are popular within specific game subcultures (Carbone & Ruffino, 2013), which throws up debates regarding which version may be more authentic, the original game version or the performed full vision of the composer (Gibbons, 2015). As Mike Gordon, the composer and producer of Doom’s 2016 OST (original soundtrack) puts it: “I think video game music should always be able to find some sort of place outside the game; […] that should always be the ultimate goal” (O’Dwyer, 2016, 20:30). In effect, Doom’s music is so intertwined with the gameplay as to raise the question of whether or not it can be fully appreciated outside of the game (and vice versa)––in addition to reminding of the deep connections between early first-person shooters like the original Doom (id Software 1991) and metal and industrial music from the 80s and 90s.

Game music not only functions as a reminder of games played but is also used to promote the games. By becoming a defining part of gaming franchises, game music can become a successful product in itself. In this sense one literally hears the music, and next plays the game. Original soundtracks can be found in digital formats and, perhaps unexpectedly, on vinyl aimed at distinct collectors’ markets (see, for example, Napolitano, 2012), as well as on dedicated online sites that offer game music soundtracks (for example, Spotify’s dedicated VGM channel––Vincent, 2016). The pleasure of game engagement is further extended through OST remixes by game fans, subcultural activities that may well become independent of gameplay as techno, grime, dub step, trap, and hip-hop remixes of games such as the Zelda techno and the Super Mario trap; such experimentations abound on social media sites.

Game music inspires a particular sonic aesthetic in electronic music production by a generation that has grown up within games’ cultures. Computer game music is now embedded into the very fabric of electronic music genres and concomitant music cultures.

There is a continuous dialogue, moreover, between games and a wide range of musical styles, from classical to popular, and from fan-based to avant-garde experimentation. As video game music lives on in the sound of contemporary popular music, the chiptune scene particularly celebrates the early low-res game sound, applying this to new contexts. Similarly, perhaps, a reordering of cultural memory takes place in the reuse of game technologies. For example, the obsolete Gameboy handheld game was hacked in the late 90s by Oliver Wittchow as performative musical instrument (Wittchow, 2014), emphasizing the ambiguity between gameplay and obscured music performance (McAlpine, 2016), which is further worked out as a training ground for digital music performance (BeatLab Academy, 2016).

Not surprisingly, game music inspires a particular sonic aesthetic in electronic music production by a generation that has grown up within games’ cultures. Computer game music is now embedded into the very fabric of electronic music genres and concomitant music cultures. With reference to the grime music scene in the UK (a music style based on a genealogical mix of electronic dance music and hip-hop), Rob Gallagher demonstrates in this issue, how a generation of music makers that have grown up with game culture and digital music software and now weave this experience into their music. Other examples of game music inspired genres include hip-hop (Dirs. Dwyer & Neill, 2014; Vice Staff, 2014), gabba/gabber house (Schouwenburg, 2013), and a range of other electronica (Hinton, 2017). Making use of MIDI (Music Instrument Digital Interface) that became available during the mid-80s on sound cards, music composition software was developed for the same computers as video games (Manning, 1994). For example, C-Lab’s relatively short-lived 1986 Supertrack for the Commodore 64 micro (Jenkins, 1986), followed by C-Lab’s Creator (Trask, 1987), pre-runner of Notator Logic and Logic Pro, and Steinberg’s Cubase (Lord, 1989) for the Atari ST home computer, which attractively included a MIDI-to-PC port. Such music production software treats musemes (distinct musical components––see Tagg, 2013) as building blocks that are sequenced and triggered. Also, the sequential visual display of music software offers a graphic interface that reminds of music and dance games in terms of moving along musemes on a linear timeline. Mobile gameplay and music apps add a different dimension to this, as musical elements merge with finger movements.

The specific characteristics of interactive and immersive player engagement with non-linear music composition and adaptive audio set it apart from linear music composition, however. This is illustrated in detail in the BEEP research project, in which Karen Collins and her team video-document interviews with game composers around the world. Engagement with games, game music and game culture is also addressed from players’ perspectives in her work on player interaction (Collins, 2013). In this issue, interview clips from the BEEP project are linked to a playlist of her favourite game music, showing a dynamic connection between the personal experience of game music and research in the area. In this issue, the topic of interactive game sound is further addressed by Tom Langhorst, with a focus on sound effects that provide believable action cues, and by Zander Hulme’s investigation of the issue of crossfading between audio components during gameplay. In his recently published monograph, Rob Gallagher (2017) suggests that a socially produced embodied alignment occurs between gamer and the rhythm of the game, a type of “entrainment”. Musical pulse and rhythm can significantly enhance the experience of entrainment produced within the rhythms of gameplay interaction through both seductive flow and the challenge of rupture. A type of interactive dialogue is set up in this way between the game environment and player, similarly to the way in which between DJs and dancers become part of a responsive network (Ferreira, 2008; Rietveld, 2016).

Providing a varied series of perspectives on the many directions in which the study of the auditory dimension could bring game studies, as well as games, our edited collection offers a glimpse into its “polyphonic” and still vastly underexplored fields, identifying some of them, and suggesting a long-term cooperation and interplay between music and game studies.

Another critical aspect of game music is its enmeshment in narratives of gaming history that focus on technology and sound to celebrate innovation and appease a nostalgic sense of affection for video games. In this special issue of GAME, technostalgia is present in the discussions by James Newman, Kenny McAlpine and Tom Langhorst, each of whom addresses, in various ways, issues that relate to the aesthetics of low-resolution digital sound, which hail back to the early days of gaming. James Newman has worked extensively on the ephemeral ontology of video games as hardware becomes obsolete and software is superseded, and the challenges as well as questions this brings to game preservation (Newman 2012; see also Newman 2004 and 2008). In his study for this issue, Newman focused on the relations between technology and the musical, and on how composers like Rob Hubbard and Martin Galway went on to shape the sound of video game music for generations of players. From the perspective of sound design, Langhorst contributes to the related issue of the relationship between visual and audio realism as a designer, discussing game experience through sound in early games such as Pong (Atari 1972). Like Newman, Kenneth McAlpine also explores constraints faced by early programmers. McAlpine focuses on designers working with 48k Sinclair ZX Spectrum and argues that their ingenuity turned limitations into creativity an innovation, effectively shaping an early sound of video games that would go down in history as well as influence modern developments like chiptune music.

Further contributions in our issue show examples of some of the many directions of research that the auditory element of design brings to the attention of game studies. Rob Gallagher’s paper, as already discussed, shows how a generation of music makers grew up with game culture weaved this experience into their music. Frederico Peñate Domínguez addresses what he calls “Nazi rock ‘n’ roll”, an imaginary American popular music used to promote Wolfenstein: The New Order, which simultaneously mythologizes Nazi culture through false musical memory. Peñate Domínguez discusses how music worked as an essential aspect through which the programmers were able to create an alternate, immersive, heterotopic post-WW2 history and to promote the game through it. In his article for this issue, Zander Hulme focuses on more technical aspects related to this issue. He discusses how the implementation of adaptive musical through dynamic, imbricate audio could further increase the ability of composers to immerse players in gameplay. Other contributions in this issue include reviews of recent books on the subject of games and music, an interview with the Ludomusicology research group, and an original playlist on memorable moments in game sound history by Karen Collins. Providing a varied series of perspectives on the many directions in which the study of the auditory dimension could bring game studies, as well as games, our edited collection does not aim to provide an exhaustive or linear history of game music. Rather, it offers a glimpse into its “polyphonic” and still vastly underexplored fields, identifying some of them, and suggesting a long-term cooperation and interplay between music and game studies.

References

Austin, M. (Ed) (2016). Music Video Games: Performance, Politics and Play. London and New York: Bloomsbury.

Authors’ Info
Hillegonda C Rietveld is Professor of Sonic Culture at London South Bank University and, between 2011-17, Chief Editor of IASPM Journal, the journal of the International Association for the Study of Popular Music.

Marco Benoît Carbone (PhD, UCL) is an Associate Lecturer in Media Communications at the University of the Arts, London College of Communication. He is a member of the UK and Italian branches of DiGRA – Digital Games Research Association.

The sounds of video games have changed tremendously over their history, from about 1971 until now. Having been born approximately when commercial video games were first created and then growing up alongside the games as they evolved, the history of video games is very much a history of my own soundscape as it changed over the years. I had a Pong clone (a Telstar Alpha, made by Coleco) in the late 1970s, an IBM PC (the 5150, released in 1983), an Atari 2600, and then later Nintendo NES, Sega Genesis, PlayStation, and so on. What consoles I didn’t own, my neighbors usually had (the Apple II, Colecovision, Intellivision, and so on), or my school (the Commodore 64), and I spent many hours of my childhood playing these games.

The sounds of video games have changed tremendously over their history, from about 1971 until now. Having been born approximately when commercial video games were first created and then growing up alongside the games as they evolved, the history of video games is very much a history of my own soundscape as it changed over the years. I had a Pong clone (a Telstar Alpha, made by Coleco) in the late 1970s, an IBM PC (the 5150, released in 1983), an Atari 2600, and then later Nintendo NES, Sega Genesis, PlayStation, and so on. What consoles I didn’t own, my neighbors usually had (the Apple II, Colecovision, Intellivision, and so on), or my school (the Commodore 64), and I spent many hours of my childhood playing these games.

the games listed below are the games that stuck in my head over the decades of playing games myself, and became important to me personally through the hours I spent with them. Some have admittedly “terrible” sound by today’s standards, but to me, that’s part of their charm, and growing up with these games means they hold a special nostalgia that I believe is best captured in their sound, even more than their graphics.

Being asked to make a playlist for game sound has always been a challenge for me. I have stayed away from lists as much as possible in the past, in the interests of not developing a canon for video game music. To me, there are far too many games deserving of our scholarly attention, and yet we in the field already seem to be focusing on just a few games, and a few genres of games, at the expense of others. Despite my long-time interest and expertise in the field, I also don’t feel that I am qualified to decide which games are particularly important, or which are especially worth listening to or highlighting. My list below, then, isn’t a “most important” or even “particularly notable” games when it comes to their sound and music. The list doesn’t cover all of the consoles by any means, and anyone looking for a “best of” is sure to be disappointed by some of my inclusions and omissions. I take the title of my article from “Desert Island Discs”, a radio show created in 1942 by Roy Plomley for BBC Radio 4, which would bring a celebrity guest onto the program and ask them to choose eight recordings to take with them if they were cast away on a desert island. They were not meant to be bests of anything, only personal picks that held some meaning, which the guest would explain.

Therefore, the games listed below are the games that stuck in my head over the decades of playing games myself, and became important to me personally through the hours I spent with them. Some have admittedly “terrible” sound by today’s standards, but to me, that’s part of their charm, and growing up with these games means they hold a special nostalgia that I believe is best captured in their sound, even more than their graphics. Some were games whose composers I have since met, befriended and/or interviewed and are now of interest to me for that reason. Some indeed are usually in top ten lists or were the “first to do something” games, but are chosen here simply for their place in my heart, not for their place in history.

The list is supplemented by interview material I gathered during a year of filming interviews for the Beep documentary film (Ehtonal 2016). This documentary is a history of game sound from the penny arcades to today, and involved over 80 interviews with composers, sound designers, voice actors, and others involved in game sound from around the world (shot in UK, Canada, US and Japan, but also including interviews with composers from Germany, Spain, France and the Netherlands). Each full interview is released as an independent webisode, as well as in my two-volume book series Beep (Ehtonal 2016). The clips described here have also been assembled into one video that can be streamed or downloaded from our Vimeo website.

Pong was originally developed in 1972, and was the first commercially successful video game. The home version was sold through Sears starting in mid-1975, but it was one of Coleco’s Telstar models that my Uncle purchased one Christmas in the late 1970s that was my first foray into video gaming. The game had no music, and only made simple analogue beeping sounds of different frequencies for the four games (all simple variations on Pong) built into the system, but to us children at the time, being able to control something on the television set cannot be overestimated in importance. The game was captivating, and we spent many hours listening to the beeps. Compared to the Magnavox Odyssey, which was also available as a home console at the time, the Pong clones were a huge leap forward for the simple fact that they had sound at all. There isn’t a whole lot of interest here sonically, but I can still remember hearing those sounds in my head a long time after the game was set aside.

Floppy Frenzy was a little-known game that must have been one of the very first educational video games. My father brought the game home with the computer, to teach us children not to put the 5¼-inch floppy disks near magnets or dust. Movement triggers the soundtrack: stop the diskette from moving, and the sound stops. Move, and the sound continues its monotonous, never-ending beeping—a trait also found in DigDug (Namco) that came out the same year. It’s unlikely that the games were created in the knowledge of what each other was doing, since Windmill was Canadian and Namco based in Japan, so the idea of tying game sound to the action of the main character (very much Mickey-Mousing the sound) may have been independently created, simultaneously. The game also had little bonus, high score and death jingles. It’s the death song that sticks with me more than anything (heard at approximately 9:45 into this video): not only did players use to spend a lot of time “dying” in early video games but to me, today, the song epitomizes my experience of audio on the IBM PC: a single monophonic channel playing simple on/off square wave sounds in very short melodies. This was PC sound before soundcards: monophonic, without sound envelope generators, and dreadfully annoying, which is probably where the original option to turn the sound off in games (still present in most current games) came from.

Tapeworm still fascinates me. Its atonal melody that introduced the characters of the game at the start was a wonderful representation of the limitations of the Atari 2600 TIA soundchip. Sounds on the Atari chip were created by dividing the system clock into 32 notes. These notes didn’t align with just tuning, and the selection of each original note and waveform determined the division of frequencies of the other 32. This meant that each tuning was completely different depending on the first note selection, and for the most part completely atonal, resulting in some bizarre musical creations. I believe that the many hours of my childhood spent in front of the machine were a strong influence on my later musical tastes and even on the development of electronic music in the 1980s general (see Collins 2006).

Beep video clip: The first clip in the video features several of our interviewees talking about the infamous TIA soundchip and its difficulties. First is Brendan Becker, a chiptune composer known as “Inverse Phase”, famous for his Nine Inch Nails 8-bit cover album, Pretty Eight Machine. This interview was recorded at the annual MAGFest (Music and Games Festival) in January 2015 in Maryland. The second clip is pinball/game composer David Thiel, recorded at his home studio in Seattle in May 2015. The third is founder of music software company Plogue Art et Technologie, David Viens. Plogue makes the popular soundchip emulator software called chipsounds, and the clip was recorded in Montreal in June 2015. It’s easy to see that all of these folks have a love-hate relationship to the TIA chip: part of its appeal was its mix of very awkward tuning and rich waveforms.

The Commodore 64 had a much more advanced sound chip than its competitors. I didn’t have a C64 at my house, but we had one at school and were allowed to play games at lunch time and during breaks. I remember playing Where in the World is Carmen Sandiego? (Brøderbund 1985) a lot, but not Monty in those days. However, when I began research for my book Game Sound (2008), I played just about every game ever released for the C64, and Monty’s theme was one of the songs that stood out for me amongst the lot. Created by Rob Hubbard, the theme uses Hubbard’s module format, a way of maximizing the amount of music that could fit into a game by using loops, instrument changes (selections of sounds by picking waveforms and using envelope generators) and transpositions. Because the size of media and RAM was limited, there was a limited amount of space allocated to music. Through simple transposition statements, the same blocks of code could be re-used without the music sounding so repetitive.

Beep video clip: In this clip, German game composer Chris Huelsbeck talks about the Commodore 64 soundchip, followed by Charles Deenen, who explains some of the RAM issues. Both interviews were recorded in San Francisco in March 2015.

Metroid, along with Super Metroid on the SNES was, and still is one of my favourite games. The Metroid soundtrack was so different to nearly everything else that came out on the Nintendo. It used the NES soundchip to full advantage, but more than that, it didn’t have the poppy “chippy” sound of so much of the music. The bass channel creates pedal tones that the melody rests on for the Brinstar level, but it’s the Ridley and Kraid stages, with its sparse ambience, that captivated me as a young gamer. Not only did the music inspire in me a love of game audio, it’s also one of those game soundtracks that inspired others to become game composers, such as Alexander Brandon (who composed for Deus Ex, Unreal and more).

Beep video clip: In this clip, recorded March 2015 in San Francisco, game composer Alexander Brandon talks about the importance of Metroid to his career.

The Shadow of the Beast game bears some resemblance to Metroid, which is probably why I really liked the game and its music when it first came out for the Sega Genesis. You’ll notice we’re still in the era of “wall-to-wall” music: non-stop looping background music. The repetition of the really happy chip music of the time tended to grate on me (and most other people in the vicinity at the time), but the more moody, slightly prog-rock styling of Psygnosis games in particular were a wonderful change from that style. Music on the Genesis (or MegaDrive as it was known in Europe and Japan) was created using a combination of FM synthesis as well as the more primitive waveforms of a programmable sound generator (the chip type used up to that time). It was the newer digital FM synthesis chip, which allowed for more realistic instrument sounds that could be programmed in by the composer individually, which defined the Genesis sound.

Monkey Island 2 was the first game to use LucasFilm Games’ new (at the time) software engine, iMUSE, or Interactive Music Streaming Engine. iMUSE enabled the game’s parameters to control what was happening in the music—if a player was winning a battle, the music could jump to a different cue, and if a player was moving around a space, the music could adjust to the location. The infamous Woodtick scene, for instance, changes music depending on which building the player enters. Although the techniques had been used previously in games, iMUSE made it easier for composers to have control over how music played back in the game.

Beep video clip: In this clip, the three creators of the iMUSE engine—Clint Bajakian, Michael Land and Peter McConnell—discuss iMUSE and the difficulties of the Woodtick scene and interactive music as a whole. All interviews were recorded in San Francisco in March 2015.

Tomb Raider was the reason I bought a PlayStation. Not only did the game have a female protagonist, but the wall-to-wall music of the past was suddenly replaced by the game’s more careful attention to ambience and using music more sparsely only to highlight key points in the narrative while showing a cinematic approach to the score. In an interview with Nathan McCree, he explained his choices about abandoning the wall-to-wall music in favor of ambience. In part, this shift was influenced by the switch to Redbook (CD-ROM)-based games, which allowed for higher production values, but reduced the amount of interactivity possible in the music. For a time, the concepts of interactive music dream of iMUSE was abandoned in favor of linear tracks, as composers adapted to the new format. McCree explained to me that he had no say in the implementation, which had been done on the last possible day before the game was shipped, and in some cases music was put into the wrong place or not as he had intended, but fans of the game never seemed to notice.

Beep video clip: Composer Nathan McCree talks about the change in the move to CD-ROM with the PlayStation and how that influenced the music and sound of Tomb Raider. The clip was recorded in London in February 2015.

I’m a fan of Ennio Morricone’s music for Sergio Leone’s “Spaghetti Westerns”, and so Wild Arms was a personal favorite not so much for the game, but for the heavily Morricone inspired music. The driving rhythms and some of the themes are close to being lifted directly from Morricone themes, but Naruke puts her own spin on them, and the nature of the role-playing game being set in a cross between the American West and medieval Europe meant a combination of Western themes and more traditional Japanese RPG musical elements.

Beep video clip: Michiko Naruke discusses Morricone’s influence on her music for the Wild Arms series. The clip was recorded in Tokyo in May 2015. My interpreter was Alwyn Spies.

No list of my favorite game music would be complete without some Koji Kondo somewhere. The composer for Super Mario Bros and the Zelda series is without doubt many people’s favorite. Along with frequently being listed in the Top Ten Games of all time, The Ocarina of Time’s music is also among many top 10 lists. The music is carefully integrated into the game, drawing on past themes established in earlier Zelda games, but adding new music and allowing player interaction with the music in the form of an ocarina. I’m convinced that the game inspired sales of ocarinas the world over––indeed, there are many Zelda-themed ocarinas for sale. For me, the theme-driven nature of Kondo’s music, combined with the careful integration of the music into the game itself, represents some of the best that game music has to offer.

I first wrote about Grim Fandango in my 2008 book Game Sound. At the time I researched and wrote the book, the game had been out for a few years, and hadn’t really had a huge following or notable public interest, but was near the top of my list straight away. The game has since become a cult favourite, particularly amongst music fans. Peter McConnell’s delightful orchestral music fits the game so well—a mixture of the dark underworld Día de los Muertos (Day of the Dead) and mariachi. To me, Peter McConnell is the Danny Elfman of the game music world—he writes playful, humorous music with highly memorable themes, and the Grim music is no exception.

Beep video clip: The music was re-recorded and re-orchestrated for the remake of the game in 2015, and I spoke with both Peter McConnell and the recording engineer, Jory Prum, about the work that went into the remake. They both talked about the difficulties with recovering the original files to remake the music. Jory’s interview was recorded in his studio in Fairfax, California, and Peter’s clip was recorded in San Francisco in March, 2015. Jory passed away suddenly in April of 2016, and thousands of people flooded the internet to express how much his work meant to them, despite his being very much a “behind the scenes” person in game audio.

The soundtrack for Harmony of Dissonance was much maligned by the press, largely for its 8-bit aesthetic, but also for its dense layers and morose themes. I’ve always enjoyed the Castlevania games and their music, but to me, the nature of the music as more complex, fitting the game well, and its 8-bit sound are precisely why I like this soundtrack. The music always heads in unexpected directions, and although it has fewer immediately recognizable themes or melodies, it offers a lot of sonic interest despite having being created on the Game Boy Advance’s crunchy speaker and four-tone (plus sample channel) synthesizer chip.

Yoko Shimomura tops my list of favorite game composers, and although her more famous work lies in Street Fighter 2 and the Kingdom Hearts series, it’s the MarioandLuigi role-playing games that I find most endearing. Shimomura had to take the well-known and well-loved Mario musical style and make it her own for the games, a difficult task, but she manages to give each game its own characteristic style and memorable themes. It’s worth noting that playing games on the Nintendo DS are always better heard with headphones rather than the limited speakers present in the DS.

Beep video clip: Yoko Shimomura talks about writing the music for the series and balancing adaptations of the original Mario tunes with her own style. The interview was recorded in Tokyo in May 2015, and my interpreter was Alwyn Spies.

BioShock was the first game music that left me absolutely speechless. I first became aware of the music when I heard Schyman give a talk about it at the Game Developers’ Conference in 2007. BioShock’s dramatic orchestration and sound design blew away everyone’s expectations of what game music was, and what it could be, inspired by 20th Century modernism. I’m also a big fan of Schyman’s other music, which ranges from Bernard Hermann-esque Destroy All Humans to the beautiful Dante’s Inferno.

Beep video clip: Here, Garry Schyman talks about the direction he received from audio director Emily Ridgway for the BioShock game. The interview was recorded in Los Angeles in October, 2014.

Another soundtrack to blow away everyone’s expectations of game music was Peggle 2. A casual game created by PopCap, the first game Peggle originally established the theme of playing Beethoven’s Ode to Joy upon each level’s success in a game that combines elements of pachinko and pinball. For Peggle 2, Whitmore plays on this idea, bringing classical themes with a modern twist to each character, or “master” that the player chooses. My favorite, though, abandons classical music and aims for a bizarre, chaotic dubstep track for character Jimmy Lightning. More remarkable than the music itself is the careful integration of the music into the game—with each peg being a note that harmonizes with the theme for the level.

Beep video clip: Guy Whitmore tells us all about the musical choices in selecting and creating the music for Peggle 2. The interview was recorded in Vancouver in August 2015.

Author’s Info:
Karen Collins is Associate Professor in the Digital Arts Communication program at the University of Waterloo, Canada. She has published five books on game sound, including Game Sound (MIT Press 2008), From Pac-Man to Pop Music (Ashgate 2008), The Oxford Handbook of Interactive Audio (Oxford 2013), Playing With Sound (MIT Press 2013) and the 2-volume Beep Book (Ehtonal 2016). She is also the director of the award-winning game audio documentary Beep: A Documentary History of Game Sound (Beepmovie.com).