Technological innovation in the video games industry is a rich area of research that has barely been explored as of yet. 1. Gamers are always clamoring for novelty and a remedy to the oft-decried “sequelitis” that “plagues” the industry, while game publishers and platform holders secretly plan a next-gen platform to capture the ever-shifting market. In this light, the importance of graphics cannot be understated, as it is usually taken for granted in game historiography that “[g]ame graphics were, and to a large extent still are, the main criteria by which advancing video game technology is benchmarked” (Wolf, 2003, p.53). This formulation, however, needs to be expanded and broken down if we want to truly capture the reasons for success and innovation in the games industry. One key aspect to be factored into the equation is that gamers are sophisticated and literate enough to look beyond the mere graphics “coating”, and seek new gameplay opportunities.

The strongest evidence for this rests in gamers’ criticism of the “bullshot” phenomenon. See http://www.penny-arcade.com/comic/2005/09/12

To extricate the complex interlocking of graphics, technology and innovation will require us to articulate the interdependent uses and discourses surrounding the notion of graphics in games. Working around Kline, Dyer-Witheford & De Peuter’s (2003) model of the game industry as the interaction of three circuits—technology, marketing and culture—we will make terminological and conceptual distinctions that will help clarify the roles played by graphics, innovation, technologies and aesthetics in games. Although we agree with Andrew Hutchison, who “explicitly highlights the important co-dependence [of] game aesthetics [as] the combination of the audio-visual rendering aspects and gameplay and narrative/fictional aspects of a game experience” (2008), our approach is to take this statement as a starting point and to deconstruct this co-dependence in order to analytically identify the properties of each half and understand how and when they can form a whole.

Distinguishing Functional and Aesthetic Innovation
Studying innovation in video games is a tricky proposition because it threatens to confuse distinct sets of issues 2. As Ian Bogost argued, the design of video games can be understood as a practice that straddles the functional and aesthetic dimensions: “Video games are software, but they are not meant to serve the same function as spreadsheets. They are not tools that provide a specific and solitary end, but experiences that spark ideas and proffer sensations.” (2008, p.1) Conceivably, innovation can occur on both of these levels. But this does not mean these two types of innovation are of the same kind.

Functional innovation is a somewhat straightforward matter: a game franchise may automate tiresome processes (by auto-saving or auto-mapping a gamer’s progress, providing a fast travel option, or automatically managing supplies efficiently unless the gamer wants to give customized orders), add more simulational complexity (such as line changes and stamina meters in sports games), or offer new modes of play (for instance,thePractice Mode in Killer Instinct). Functional innovation is often thought of as teleologic, but in truth has no such pre-established, absolute direction to follow. It advances through reiteration, each new game largely repeating its precursors’ successes while pitching a couple of new ideas to “revise” the set “schema,” in the words of art historian E. H. Gombrich’s schema and correction theory (1960). Even on the functional level, then, a certain kind of game culture is established.

Aesthetic innovation may at first glance seem like either an oxymoron or a tautology. If we postulate that the aesthetic phenomenon is linked to originality and uniqueness, then any aesthetic component of a game is always by default an innovation; conversely, by definition no aesthetic proposition can be inscribed in a straight teleologic line with an earlier proposition because it would then fall under the functional dimension. Yet in any given game design, form follows both function and the cultural criterion of a satisfying media experience that stands between a wealth of existing artifacts and a horizon of promises yet to be actualized. A new game is thus both a new idea to be explored through an original experience, and a reassessment of past explorations of related experiences. As aesthetically unique as it may appear, no game springs forth from a designer’s mind untouched by the larger gaming culture: the historical context is an unavoidable part of the equation. Aesthetic innovation, then, can be thought of as Hans R. Jauss’ aesthetic variation, which is the degree to which a given work differs from our expectations and manages to surprise us by positioning itself in the margins, or in another space entirely, from the horizon of expectations (1982). Functional innovation can be seen as a small step or a leap forward along a trajectory; aesthetic innovation is a small step or a leap sideways, in another direction.Circumscribing Technological Innovation
The nature of video games as technological constructs (and subjected to Moore’s law that processors double in power every two years) makes any investigation of innovation seem inherently technology-driven. And technology can and does influence a number of innovations: auto-mapping, for example, requires additional data storage. Hardware advances in game console generations provide ever more computational power, more buttons on game controllers, higher graphical resolutions, etc. But many innovations cannot be charted up to technology. Killer Instinct’s Practice Mode, for instance, is in fact much easier to implement than its standard fighting mode, as it consists of letting an opponent stand still waiting to be beaten up forever, with no artificial intelligence, damage calculation or timer rules to be dealt with. Hence both functional and aesthetic innovations hinge on genre and media conventions, which are socio-cultural habits largely independent from questions of technology.

Technology is only one term in the broader equation of game innovation, and it often functions as a facilitating agent, rather than a necessary cause, for many innovations. A technological innovation opens a field of possibilities in the technological circuit. The possible must be understood here in the philosophical tradition of the actual and the virtual. For Gilles Deleuze (1966), the virtual is opposed to the actual (rather than the real): it represents an open field that contains everything needed for an event or a thing to actually take form, but it is already real insofar as the real always holds, in itself, a part of virtuality, of differentiation. By contrast, the possible is a realm that is conceptualized in some form as independent from the real; a possibility is a set of definite preconditions for existence that have already been met, so that the only thing left is for it to be realized.

Applying these concepts to game innovation and technology, we would claim that technological innovation may carve out a part of the virtual and move it into the domain of the possible. This was the case with the ray casting technique employed by id Software for Wolfenstein 3D (1992), which simulates tridimensionality out of 2D bitmap sprites. Their methods for doing so could have been actualized earlier, as the principles behind them stem from the virtualities of programming, visual rendering and data treatment. When they started licensing their engines as technologies, the subsequent game developers who worked on them did not operate from the virtual, but from the possibilities which this engine allowed them. (They could, of course, add unexpected features to the engine from the unactualized virtualities of reality, just as id had done before them).

Technological innovation thus acts as a pole of attraction for game developers by breaking down the infinity of the virtual and delimiting a set of possibles from which they can easily work. This intersects with what Nelson & Winter (1982) have identified as a technological trajectory, a natural way for technologies to evolve based on the exploitation of latent economies and optimization (such as increasing hard drive sizes, faster processing, more dedicated graphical memory, etc.). Importantly, the trajectory develops in accordance with the larger technological regime, as Marsili’s summary of the research on innovation and technological regimes show:

A “technological regime” (Nelson & Winter 1982, Winter 1984) or “technological paradigm” (Dosi, 1982) defines the nature of technology according to a knowledge based theory of production (Rosenberg, 1976). Innovation is viewed as a problem-solving activity drawing upon knowledge bases that are stored in routines (Nelson & Winter, 1982). Accordingly, the technology is represented as a technological paradigm defining “a pattern of solution to selected technological problems based on selected principles derived from natural sciences and selected material technologies” (Dosi, 1982). In a similar way, a technological regime defines the particular knowledge environment where firm problem-solving activities take place (Winter, 1984) (1999, p.3).

The successive techniques and technologies used to materialize a given game idea, which partially depends on the graphical regime, are to be considered as forming a technological regime: Quake’s (1996) full-3D implementation of virtual environments and actors is a new way of solving the problem of providing a first-person shooting experience, just as Doom’s (1992) binary space partitioning was an answer to Wolfenstein 3D’s ray casting, itself an answer to Maze War’s (1973) step-based approach to 3D space (Arsenault, 2009), etc.Videogame Innovation and Technological Novelty
Returning to Wolf’s claim that game graphics serve as a benchmarking tool for new technologies (Wolf, 2003), we must add a crucial dimension to the statement. If graphics act as a conceptual interface linking consumers with the underlying, invisible technologies, we must also integrate separately the usages that aremade of these technologies. This means that graphics, in and of themselves, have an indirect and limited impact on a game or console’s success. 16-bit graphics were not enough to bring success to the TurboGrafx-16 in America because many of its early games did not exploit the new graphical capabilities of the console to expand the range of possible game experiences. The separation of technologies and usages allows us to relativize the classic video game marketing claims, which have historically heavily emphasized graphical fidelity, with ever more on-screen colors and background layers, higher resolutions, sprite sizes and polygonal counts, more advanced shading effects, etc. These are all accounted for as technological trajectories, but innovation does not always rely on technological advances. This is why Nelson & Winter (1982) distinguish the technological trajectory from the trajectory of innovation: an innovative product invites reiterations and incremental refinements, which can develop into its own trajectory regardless of technological progress or stagnation. Isabelle Raynauld has shown how a new technology’s appearance always constitutes a promise to consumers as well (2003); in the case of video game graphical technologies, that promise could be said to imply more than just “prettier” graphics, and rather promise new play experiences through new modes of representation.

In other words, the technological trajectory must be coupled with an interesting trajectory of innovation, that is, a renewing of game forms and possibilities of action for players. Nowhere is this more evident than during the launch of a new game console, where the launch title games become the privileged vessels of all three circuits of marketing, technology and culture, as they are tasked with demonstrating the possibilities of the hardware, keeping alive the promises of the new technology, and regulating the horizon of expectations of gamers. This was the case with the Super NES’ special Mode 7 graphics, a form of planar projection that could render a 2D bird’s eye view image in pseudo-3D by foreshortening the pixels up to a horizon line. Two of the SNES’ launch titles illustrated this convergence of technological and innovation trajectories, albeit differently. Pilotwings (1990) showcased the potential for Mode 7 to bring about new types of gameplay and opened up a novel trajectory of innovation, while F-Zero (1990), though quite content with providing a classic racing game experience, took that innovation trajectory to a new level of visual details and smooth scrolling animation. This dual discourse from Nintendo (the platform holder) managed to attract both kinds of game developers: those favoring conservative refinements along the existing innovation trajectories, and those more adventurous developers that wanted to push new innovation trajectories.

Framing innovation as a facilitating agent and pole of attraction for game developers allows us to simultaneously treat technology with the importance it is due, but also to envision innovation outside of technology. There is legitimate cause for a relativistic approach of its importance in our understanding of the medium. This is precisely where graphical regimes are helpful to us, as they can account for continuities and ruptures in visual forms of gameplay that transcend technology as a material imperative. In other words, we believe that the essential feature of new graphical technologies is to cement new graphical regimes, as in innovative ways of viewing and—more importantly—of playing. The term “cementing” is not chosen lightly. If we are to postulate an essential continuity of forms that is independent from particular technologies (at least to some degree), then we must replace all images of newness and metaphors of appearance, emergence and birth by metaphors of cementing and coalescence. In this view, a technology seldom introduces newness that springs out of a materialistic “big bang” that creates matter out of nothingness, but rather articulates or reshapes some primal matter and elements that were already present.The Synergistic Forms between Graphics and Gameplay: Graphical Regimes
The graphical regime is to be understood as the junction point between gameplay and graphics: it is defined as the imaging of gameplay and the gameplay of the image, independently of the technological graphical capabilities or limitations. As such, it serves to describe the range of affordances that the game creators open or close for the player as a result of visual configurations. For instance, even though Starcraft 2 (2012)is powered by real-time polygonal 3D graphics, its creators did not allow the player to freely move the virtual camera anywhere they wanted, staying true to the graphical regime of the top-down view that had characterized its classic predecessor. The same conservatism transpired through Donkey Kong Country (1994)and Killer Instinct’s integration of cutting-edge pre-rendered 3D modeling and animation technology into classic 2D fighting and platforming gameplay. A graphical technology may not translate into new modes or affordances of gameplay if it is not accompanied by a corresponding change in graphical regime. To further clarify, the graphical regime is a qualitative descriptor of video game artifacts.

The first task for any new concept is to interrogate the medium anew. In our specific case, we have moved away from a technologically-driven view of video game history and instead envision it according to the ways in which the imagery can be mobilized to enhance or transform gameplay and, reversely, the ways in which the game allows for interactivity with the visual elements of play experiences. Can the player alter the image’s framing, point of view, and visibility of distance or layers? How much and how often is he or she in control of the virtual camera? Are the user’s interactions with the image a crucial aspect in the game’s structure, or more of a secondary addition to meaningful play?

Aside from acting as descriptive statements, graphical regimes can help to highlight complex aesthetic effects, such as the Scarecrow’s nightmare sequences in Batman: Arkham Asylum (2009), where gameplay is reduced from the usual 3D exploration to a 2D side-scrolling view. In the context of this action game, the brutal reduction in the gamer’s control over the camera positioning quite literally puts the player under the villain’s graphical regime (an ongoing metaphor throughout the entire narrative).

Keeping this interrogative stance, it can be very instructive to consider the phenomenon of video game remakes. What kind of added value can be gained from enhancing a classic game’s visual characteristics? The Playstation Portable (PSP) release of the PC Engine’s Castlevania: Rondo of Blood (1993)can provide us with an example. The decline of its original platform has significantly reduced the game’s accessibility, long desired by fans of the series. The resulting offer to this demand was Dracula X Chronicles (2007), a polygonal 3D version of Rondo of Blood that ran contrary to the visual strategy taken by the 1997 Symphony of the Night (also included in the package as unlockable content). In terms of graphical regime, nothing is changed: the player’s relationship and stance adopted toward the game space is bound to the classical sidescroller, allowing no action to alter anything on the Z-axis. The same graphical regime characterizes Jordan Mechner’s Karateka (1984) and Prince of Persia (1989), independently of the perspectivist graphics they feature: though the ground is pictured with depth cues, the player still moves along a single horizontal line 3. Graphically, these appear to be pseudo-3D spaces, but this depth is not implemented in gameplay, unlike in Double Dragon (1987)and other beat ‘em ups.

So far, we have presented the graphical regime as an analytical tool that allows us to link together games that use different technologies or techniques to achieve a similar way of playing and viewing, so to speak, from the point of view of an analyst or gamer. As a metaphor for political control, it always implies a game creator somehow constricting a gamer with imperial authority. Understanding the deployment of graphical regimes then requires us to focus on the pole of creation as much as reception. To this end, we would like to propose a new distinction into the model of relationships between innovation, technology and graphics, from the perspective of a game’s creators: the concept and process of mise-en-image.Graphics as a Design Preoccupation: Defining the Process of Mise-en-image
Looking at the situation from the point of view of game creators requires us to historically situate the rhetorical importance of graphics, which is always relative to the state of affairs of the industry at a given moment. While Kline et al.’s model can be used as a flat sheet mapping of the industrial arena, in actuality the birth of an individual video game artifact always occurs within a certain hierarchical configuration of the circuits, in a constant dynamic of initiatives and adaptative responses. Nevertheless, what matters for videogame creators (in spite of the historically numerous marketing efforts to give credits to graphics alone) is the way in which a given interactive pattern of input and feedback is visualized by the interacting player. In certain cases, the designer may start with a gameplay concept, and then struggle to implement it through a corresponding visualization concept: here, a particular model of what “playing a game”, or of what “a game of such-and-such kind” should be acts as the starting point, which means that it is the cultural circuit that takes the initiative, while the technology and marketing must adapt and respond to sustain this initiative. The creative effort to build such a relationship can be accurately synthesized as mise-en-image, akin to the mise-en-scène by which a director struggles to implement a dramatic script through a corresponding visualization for the camera or the stage. Of course, this process can start with an initial choice of favored visual pattern, but what really matters is that in both cases, vision and gameplay must be articulated according to aesthetic and technical considerations. This articulation is an irreducible preoccupation of game imagery. In our understanding, the choice of a graphical style of representation is separated from those of gameplay and vision, even though the three aspects are intertwined in the play experience as a whole. A short quasi-caricatural table of features will clearly illustrate the differences between what we term graphical style and vision, which follow the same split between the dimensions of functionality and aesthetics that we traced at the beginning of this paper. Graphical style is what we commonly mean by visual aesthetics, while vision refers to the functional aspects of graphics:

While an innovation in the technological circuit can open new possibilities on one or more of these creative processes, the medium’s history also shows that many games have expanded the possibilities of interaction beyond what their technology allowed at face value. Consider, in this light, the already mentioned cases of the beat ‘em up subgenre of action games exemplified by Double Dragon, which offered a playfield with navigable depth even though actions were performed on a single line on the horizontal x-axis, or the ray casting technique which projected a tridimensional perspectivist space out of 2D bitmap graphics in Wolfenstein 3D. That determined individuals can push forward new game experiences even before their facilitation by new technology suggests a continuation of the ‘hacker’ culture famously responsible for the birth of the 1961 Spacewar!. But even for spectacular technical innovations, the question remains as to their actual effect on gameplay. As the mise-en-image is a process that ties representation to interaction, it is always a way to construct both game space itself and the point of view, which is crucial to the graphical regime’s influence on visual feedback. As Michael Nitsche pointed out: “One has to explore the interaction and the media that present it. Any concentration on either presentation or functionality but not both would destroy the holistic principle of spatial experience” (2008, p.8).

In other words, our vision cannot be reduced to simply mechanistic considerations. Gameplay is not an activity that follows reductionist, abstracted choice-and-payoff grids from game theory, but is the actualization of an experience predetermined to some degree by the game’s designer(s). Thinking in terms of game mechanics can only inform us about the gameplay or simulational logic dimension of games, but we must not discard the other components that shape the user experience as a whole. A robot might play Doom in the same way whether it is looking at it through the map screen or the first-person point of view 4, but then a robot would play Doom without any screen connected to the computer anyway. This goes along with Juul’s statement that “games that are formally equivalent can be experienced completely differently” (2005, p.52).

Steve Swink’s concept of “game feel” also provides a good framework to account for the complexities of the play experience, and relativizes the part played by graphics by telling us that “the point is to convey the physical properties of objects through their motion and interaction. Any effect that enhances the impression that the game world has its own self consistent physics is fair game” (Swink, 2007, p.4). Even visual polish, according to Swink, does not depend on graphical enhancement, but has in fact more to do with the coherence of the various technical choices that are made to tie a given game’s imagery to corresponding rhythms and contexts of gameplay, a distinction championed by our chosen term mise-en-image.

Of course, if we imagine a game like Star Fox (1993) on the Super Nintendo without polygonal graphics—perhaps with the then-paradigmatic Mode 7 foreshortened scrolling spaces and 2D sprites—we dramatically alter the ride that the game offers. Indeed, it would probably be more akin to HAL Laboratory’s 1991 release Hyperzone. Tridimensional real time rendering not only brought a heightened precision for spatial simulation on the technical dimension of graphics, but also transformed possibilities for visual “polish” on an aesthetic dimension.

Hyperzone

Star Fox

Star Fox remains a good example here, albeit in a negative form, since the Super FX chip’s features famously premiered by the cartridge did not include a lot of visual refinement. What would Star Fox be, as an overall gaming experience, with particle effects, texture mapping, and dynamic lighting? An argument could be made that Star Fox 64 (1997) is already a significantly different experience, even as it reiterates most of its 16 bit predecessor’s graphical regime and gameplay mechanics. Nevertheless, the concept of graphical regime invites us to treat the SNES and the Nintendo 64 titles as a continuity of forms and to claim that Hyperzone differs more from them than them between themselves, again relativizing the importance of material platforms and hardware.

Graphical style, of course, has its part to play. As much as we argue to limit its potential role as a component in the confusing golden lamb of “graphics” in videogame terminology, we must acknowledge that it is always a part of any gaming experience. This is complicated by the fact that it is not impossible to find examples of games where the graphical aesthetics (the graphical style outside any functional considerations) are in direct connection with their proposed gameplay aesthetics. In Frédérick Raynal’s 1992 Alone in the Dark, the objects available for interaction are visually highlighted as they are polygonal objects—like the protagonist and creatures—in an environment that is entirely pre-rendered with a markedly different visual style.

Alone in The Dark

Such contrasts are also of prime importance when playing Mirror’s Edge (2008), a first person parkour action game where the usable objects are highlighted with a bright red over the monochromatic white of the environment. Here, the choices regarding the sensory stimuli of the screen’s surface work in synergy with the mise-en-image to indirectly influence the pacing of gameplay by explicitly distinguishing a plane of interaction possibilitiesfrom a plane of non manipulable décor for the player, giving him a clear line to follow. Graphical resolution can also become a central gameplay preoccupation if we were to imagine two different video game adaptations of the Where’s Waldo? books, one in 256 x 322 pixels and the other in 1920 x 1080; surely the resolution here would transcend mere questions of style and render the search significantly easier or harder.Theory Going 3D: Beyond Unidimensional Gameplay and Graphics
Our investigation of the relations between innovation, technology, graphics and gameplay can open new areas of inquiry that have yet to be charted out. For example, a significant problem with any discussion concerning the videogame image lies in the inherent hybridity of the visual flux that games present us: imaginary diegetic spaces, themselves often a complex composite of real time and pre-rendered polygons, 2D graphic overlays, video sequences and/or still photographs, are often presented as coexistent with non-diegetic game menus, interface items and abstract or iconic symbols representing more complex diegetic elements. How can we circumscribe the mise-en-image, i.e. the interaction of gameplay and image, in a game like Final Fantasy Tactics (1997), where an important part of playing the game happens within menus rather than in the spatial projection of the fictional world? As much as we separate the different aspects of games and recognize them as multidimensional artifacts, we also need to move away from global, totalizing descriptive statements that attempt to circumscribe given games in their totality, for the good reason that our games are not only multidimensional (a multiplicity of levels which we could conceivably chart out in simple 2D graphs), but these dimensions are proteiform and multilayered, such that we must also account for their inherent hybridity or dynamically shifting expressions.

These considerations invite us to stop treating gameplay as the sole or exclusive focus of scholarly efforts to arrive at an essential ontological heart of “gameness,” isolated from other aspects. Even though gameplay might be conceived as the heart of games or even of game studies, a heart is still organically linked to other components of the body. In the same way, we need to analyze gameplay as a relational entity linked to the other aspects of video games, just as we have studied the gameplay/image symbiotic unit here. A future study could investigate the relationship between gameplay, vision and control. It would be interesting to study games like Super Paper Mario (2007) and Metroid: Other M (2010), where the player is tasked with actively shifting between different graphical regimes, in order to trace lines of continuity and innovation along this axis. When Capcom’s 2001 Ace Attorney series, originally released on the Game Boy Advance, was remade in 2005 for the Nintendo DS, the dual display screens of the DS allowed a more immediate access to in game data, which is a central aspect of these games. As Wired journalist Chris Kohler wrote:

the quickie ports of these games to the Nintendo DS just a few years later might have been seen as a cheap cash-in were it not for the fact the DS’ array of innovative features were perfect for the genre. I can’t imagine playing these games without using the touch controls to investigate rooms and flip through menus, or without checking my case evidence on a separate screen while reading a witness’ testimony (2011).

The notion of graphical regimes permits a new look at video game history and an appropriate theoretical framework for accurately describing and analyzing the contributions of agents in the technological and cultural circuits while avoiding the exuberant discourses on innovation from the marketing circuit.

See Arsenault, Côté, Larochelle and Lebel’s paper Graphical Technologies, Innovation and Aesthetics in the Video Game Industry: A Case Study of the Shift from 2D to 3D Graphics in the 1990s, also in this issue, for more on the study of innovation in the games industry ▲

See Arsenault, 2009, for a discussion of game innovation in the context of game genres. ▲

It is worth noting, in passing, that Karateka is a rather rare example of a 1D game on the gameplay level: the player progresses forward or walks back, without being able to move along the Z-axis from the foreground to the background, nor jumping on the Y-axis. ▲

See Nitsche, 2005: “Doom (id Software Ltd., 1993), the seminal First Person Shooter (FPS) provides a vectorized 2D map overview. The view is not merely representational as players stay in control of the avatar and can explore the world further” (p. 2). ▲