Search

In what I imagine is a congregation of prominent game journalists draped in flowing togas and discussing politics in some ancient stone forum somewhere in the Mediterranean, N’Gai Croal of Newsweek has presented the first part of Shawn Elliot’s Symposium on Game Reviews. While it’s actually being conducted via email, and all the participants are fully clothed, it’s interesting nonetheless. Everyone has put their bullshit filters aside and slipped on their thick leather debating mitts. Below is a quick summary of the first exchange (on review scores), but I encourage you all to go and read it in full — especially for John Davison’s recounting of CGW’s attempt at removing review scores, and the fan reaction.

Shawn Elliott, 2K Boston: How much is on our minds before we begin playing any given game for review purposes? Will we imagine a range of probable scores that a heavily marketed, highly budgeted, and hugely anticipated game will get? What when the game is branded “budget” or is the work of a lesser-known, less-storied studio? If so, how closely have actual scores correlated with our assumptions?

Kieron Gillen, Rock, Paper, Shotgun: I remember the attitude being crystallized by a comment I saw ages ago on Kotaku which stuck with me, when they linked to a B-game someone had 9/10ed: “It can’t be any good, as I haven’t heard of it”. It’s an ugly, but common, tautology.

Leigh Alexander, Gamasutra/Sexy Videogameland/Variety: There’s a line, I think, between making a prejudgment, and bringing with you a context within which to make an evaluation. Games are an industry and a culture, not a fragmented, compartmentalized list of disparate products, and rather than pretend we have no early opinions, I wonder if it’s not beneficial to be prepared to bring that context—which also applies, perhaps to being aware of budgets, of team sizes, of other challenges?

Shawn Elliott, 2K Boston: I’d argue that our preconceptions are active when we decide which games we want to review. That’s not to suggest that, when given the choice, all critics go straight for the gravy (I’ve often volunteered to review games that I imagined would be interesting but not the best available). But what, if not a preconception of some sort, drives these decisions?

N’Gai Croal, Level Up/Newsweek: A reviewer helps consumers decide whether or not they should buy a game; a critic helps players think about a game that they’ve played–in its entirety or in part–and that is the end of the spectrum where I believe my writing lies.

Kieron Gillen, Rock, Paper, Shotgun: You can mention the hyped intention and mention whether it measures up—but that’s not what you’re rating. Marketing doesn’t necessarily understand their games and what’s interesting about it. And occasionally a game is fascinating despite what their creators were trying—Jim Rossignol loving the deeply buggy unpatched release of Boiling Point for its sheer constant surreality comes to mind as an extreme example of that.

Stephen Totilo, MTV News: The question we’re answering is whether those who review games pick a number before writing a word. Kieron says the ideal reviewer would not; he and Leigh agree it’s hard not to pick a figure already. Shawn’s acknowledging the humanity of having preconceived notions but dodging his own question about whether that made him start with a number. But I guess it’s hard in some ways to pick a figure at all when it’s so unclear what the point of it is.

Robert Ashley, freelancer: I have no methodology for choosing a review score. I certainly don’t think about it much. Your gut feeling (after either beating the game or the game beating you) is more accurate than whatever you might come up with after careful consideration. This is how the rest of the gaming community arrives at an opinion–and probably why so many people feel that critics are out of touch. When you sit at your computer, running down all the plusses and the minuses–technical issues, story concerns, lovable roughness, annoying roughness–you can end up talking yourself into a score that doesn’t really represent your true reaction. You can’t explain the magical pixie dust that made the empirically bad game good. You can’t explain the soullessness and sterility that made the empirically good game bad. You let your stupid logical brain take the wheel and explain yourself into a lie.

Jeff Gerstmann, Giant Bomb: I don’t really think too much about scores when I’m playing a game. I attempt to go in feeling cautiously optimistic about the game in question, and as I’m playing, I think about text, and things in the game that need to be specifically called out. I start to think about the best way to mention those moments, and the best way to call out its flaws. At some point, all that text swirling around in my head starts to sound like a range of scores, so maybe around halfway through playing a game I start thinking a little more about the score. But it isn’t until after the review is written that the score is actually assigned. The score is meant to sum up the text. If I’ve just written a review full of harsh criticisms, well, then that sounds like a pretty low score. Assigning a score and then attempting to justify it with text puts the cart before the horse.

Shawn Elliott, 2K Boston: Even worse is when the paragraphs that constitute a template are themselves composed of yet more methods of avoiding actual analysis. I mock the overuse of words such as compelling not because there is anything wrong with the words themselves but rather with the way that they’re used to replace real explanation. We know that any guy in the game store can say he likes or doesn’t like a game’s graphics or story. We recognize that it’s our responsibility as paid writers to say something more than “I like it” or “it’s good.” Replacing “like” and “good” with “compelling” isn’t even trying.

John Davison, What They Play: If nothing else, review scores serve as the starting point of a discussion for readers. As Jeff says, they serve as a shorthand for those that have no interest in digging deeper than a fundamental thumbs up or thumbs down gauge of quality. I think we can all safely assume this, but back in my time at Ziff we experimented sufficiently that we got absolute, empirical proof.