Alpha Protocol’s Arbitrary Problems

lpha Protocol was released last year to mixed reviews. On Metacritic, the game scored between 63 and 72 points, depending on platform, and mixed reviews are poor reviews in the world of Metacritic. The Xbox reviews range from 10 to 86, the PC reviews from 42 to 90. In fact, if we use Metacritic as our metric, Alpha Protocol is one of the very worst RPGs released in the past decade. It is Obsidian Entertainment’s worst game, beating the poorly regarded Neverwinter Nights 2 expansion Storm of Zehir by one point. It is worse than anything Bethesda or BioWare have developed (or have ever developed, excluding DLC), worse than The Witcher, worse than two of Troika’s three published games – the list goes on. Two Worlds, a notoriously unpolished game, is the only major western RPG I can think of with a notably lower score, and even then it’s close, and even then the sequel to Two Worlds surpasses Alpha Protocol.

I hesitate to focus on Metacritic scores – as a critical tool, Metacritic offers only vague assistance – but as I played through Alpha Protocol, I simply could not rationalize the reviews I read. For the life of me, I couldn’t understand why Alpha Protocol became a critical failure. It’s not that the game is some incredibly transcendent experience; it’s that many other RPGs aren’t. As I played through the game, I was struck by how, well, usual it was for a western RPG.

The typical western RPG review goes something like this: the game is buggy and unpolished, the gameplay is clumsy at times, but the story is dynamic and engaging and the RPG elements provide an enjoyable variety of gameplay. Positive reviews brush off the bugs and emphasize the story, while negative reviews do the opposite. This review format is incredibly common and examples are easy to find, and Alpha Protocol‘s reviews follow this format to a tee.

To their credit, most reviewers aren’t substantially wrong about Alpha Protocol. The game is indeed buggy, the combat clumsy, the enemy AI questionable. In its best moments, Alpha Protocol is a competent if unimpressive cover-based shooter with heavy RPG elements. The game’s spy genre trappings – the hacking, the lockpicking, the stealth – are fun, but don’t begin to approach the depths of proper stealth games like Thief or Metal Gear Solid. The story, a labyrinthine espionage plot, managed to keep me engaged, and you are often asked to make significant decisions about the lives of others and various machinations of the plot. (Again, to their credit, many reviewers noted the game’s abundance of plot choices.) Overall, I enjoyed the game, but I understand if others find it underwhelming or even boring. I’m not even interested in defending the game, and it’s more than reasonable to consider it inferior to many other RPGs. But it beggars belief that the game is considered one of the worst in recent memory. I think Alpha Protocol‘s reception reveals a deep flaw in the current critical landscape: number scores are often partially, if not totally, arbitrary.

Alpha Protocol’s ambitions are commendable, and if you’re a role-playing fanatic, you’ll enjoy investigating its intricacies. It’s unfortunate that its various ingredients are so undercooked. The flaky cover system, the mediocre production values, the fundamental blemishes gone unchecked–these elements add up quickly and drag the experience down … The elaborate storytelling and character progression are impressive. It’s too bad that the gawky, glitchy gameplay can’t rise to the same standard.

We see all the familiar language. Alpha Protocol is an “ambitious” game with noteworthy “storytelling,” unfortunately hampered by its “glitchy gameplay.” The GameSpot review ultimately award Alpha Protocol a score of 6.0 out of 10, “fair” according to GameSpot’s rating guide but a poor rating in most people’s eyes. (Anything below 75% is “mixed” on Metacrtic.) Now look at GameSpot’s review of Fallout: New Vegas, another game released by Obsidian Entertainment:

Unfortunately, Fallout: New Vegas isn’t technically capable of supporting these high ambitions. Simply put, it frequently breaks in some of the most phenomenal ways. You can’t mention any given aspect of its design without also mentioning a related bug–and the more you explore and the more you do, the more the game buckles under its own weight … Role-playing veterans expect glitches in games this complex, but this one far exceeds tolerable limits for these kinds of issues.

High ambitions but glitchy, suitable for roleplaying veterans – virtually the same language. This could perhaps be explained by the fact that the same writer wrote both reviews, but this language is far from unique in RPG criticism. The major difference between these two games, however, is that New Vegas was awarded a 7.5 out of 10. Despite leveling many of the same criticisms at these games, despite “phenomenal” technical issues, despite very nearly calling New Vegas outright “intolerable,” the GameSpot reviewer concluded, “And yet as busted as it is, Fallout: New Vegas is periodically awesome and consistently compelling.” Why? I’m not sure. Neither review satisfactorily articulates why New Vegas is a significantly better game than Alpha Protocol, why New Vegas is 1.5 units of quality better than Alpha Protocol, despite both games demonstrating many of the same weaknesses.

GameInformer’s reviews, another high profile source of reviews and the official magazine of GameStop, just don’t make sense. Taken by itself, the Alpha Protocol review seems to justify the game’s 6.5 rating: the main character’s “lack of charisma,” the “generic” plot, a “jumbled mess of mechanics.” But again, compare these to the GameInformer review of New Vegas. That writer comments: “None of the missions or battles stand out as memorable,” the game “lacks polish” and is “more of the same,” the gunplay is “flawed and inaccurate,” and:

Mid-battle a foe may suddenly plummet through the game world or get stuck on a rock, making for an easy kill. The pathing for AI characters wandering the wastes often takes them into the side of buildings or parked vehicles. When enemies die, some of them will float a good five feet in the air. I even ran into a reoccurring bug where my gun wouldn’t stop firing after a loading sequence. To top it off, I broke the last boss by hiding on a rock. He just stood there as I unloaded 300 bullets into him.

The difference? New Vegas received an 8.5, a full two points more. “A sequel that retains almost everything that was great from its predecessor” – except, apparently, anything memorable.

Even Eurogamer, which wrote a sympathetic 7/10 review of Alpha Protocol, offers some bafflingly contradictory reviews. The writer recommends the game and, despite addressing its faults, is sure to highlight the game’s strong points. “If you accept Alpha Protocol as a project made by a plucky team working to a faintly cruel budget, it’s surprisingly hard not to get behind it.” The writer concludes, “Obsidian can’t really compete with the bigger boys in the RPG field,” which is a bizarre statement for two reasons. First, Obsidian is run by some of the premiere figures in the history of RPGs, including Feargus Urquhart, Chris Avellone, and Josh Sawyer, whose collective pedigrees would put many other RPG designers to shame. Second, Eurogamer went on to give New Vegas a 9/10, calling it “fantastic” and “effortlessly, shamelessly entertaining” – an odd thing to say about a game by a studio that can’t compete with the big boys.

I’ll stop here, but I could go on – not just with Alpha Protocol, and not just with Obsidian. I understand and appreciate that game reviewers are not some homogenous body of writers, and there will be a wide variety of opinions on games. I could be accused, perhaps with some validity, of cherry-picking quotes from these reviews. Certainly, there are reviews out there that are well-thought out and well-articulated. But GameSpot, GameInformer, and Eurogamer are hardly outliers, and these reviews represent a fundamental problem with assigning numerical scores to games: what, really, is the difference between a 6 and a 7.5? Between a 6.5 and an 8.5? In the examples above, there doesn’t seem to be any difference at all – at least none we can explain. Both games share many of the same strengths and many of the same weaknesses: poor AI, clumsy action, a lack of polish; and yet New Vegas is mysteriously a better game.

Alpha Protocol and New Vegas are, of course, very different games. I am not arguing that they deserve exactly the same score, or that one isn’t better than the other. Plenty of writers have written intelligently about New Vegas, and plenty of writers are capable of writing intelligently about Alpha Protocol (though it seems to have slipped by unnoticed). But when you look at the existing literature, the numbers appear divorced from the actual content. You could replace GameSpot’s 6 with a 5 or a 7 and the Alpha Protocol review would still make total sense.

What makes these reviews unfortunate is that they can kill a video game. Alpha Protocol was a commercial failure, and Sega, the game’s publisher, squashed the possibility of a sequel. I don’t want to act as if Alpha Protocol‘s failure was some abortion of justice, nor will I claim that everybody in the world was wrong to ignore the game. Maybe I’m simply in the minority of people who enjoy the game. I’ll take many reviewers at their word that they just didn’t like the game very much.

It’s just a shame that their words don’t make much sense. When I read these reviews, especially in the context of other reviews, I can’t uncover any coherent narrative. I’m still not sure why Alpha Protocol is supposed to be worse than New Vegas. I’m still not sure why Alpha Protocol is somehow one of the worst RPGs to be released in the past 10 years. Cynically, I wonder if this is simply a matter of the game with less name recognition falling by the wayside. All I know is that the Alpha Protocol franchise is dead. The game’s failure can be attributed to the vagaries of the marketplace, perhaps to poor marketing, perhaps to the game’s troubled development – but also to the curse of the middling Metacritic score. Maybe Alpha Protocol deserved to fail, but it also deserved a coherent explanation. Reviewers owe that much to the games they banish to the bargain bin. Obsidian Entertainment survives, but not every studio will be so lucky.

7 Responses to Alpha Protocol’s Arbitrary Problems

I still haven’t actually gotten a chance to play Alpha Protocol, although I just bought it cheap off Amazon. I’ve been interested in it ever since seeing it at PAX East last year, but was wary because the game was still unplayable to media just two months before release.

I think it’s curious about the overwhelmingly positive reception for New Vegas. That’s a game I’ve played extensively and love because it’s Fallout, of a sort, but objectively is not a great game. It’s absolutely broken, even after two rounds of patching.

I don’t want to believe the Fallout name is what carried New Vegas’ scores, but I know it certainly didn’t hurt it.

“I don’t want to believe the Fallout name is what carried New Vegas’ scores, but I know it certainly didn’t hurt it.”

Yeah, I hesitate to accuse reviewers of anything. At best, it’s just conjecture on my part, and even though I may disagree with them, I believe most game reviewers are just writing what they think. But it’s hard to shake the feeling that Fallout got some special treatment here because, well, it’s Fallout.

For another interesting comparison, you might want to compare reviews of Fallout 3 and Fallout New Vegas. Both games run on the same buggy engine, yet only reviews for New Vegas were critical of such flaws. Plenty of quests in F3 were severely buggy at release, yet reviews typically ignored or glossed over them and handed out 90%+ reviews. It’s the same lack of consistency that the article discusses.

I’d argue that the publisher’s advertising budget is more likely to affect the reviews that games get, than their actual merits.

I think comparing 3 and New Vegas’s critical response is a bit different because I think New Vegas should be held more accountable. In the two years between the games you’d expect there to be significant improvements to make GameBryo more stable, and in my own experience New Vegas was significantly more unstable than 3, which was admittedly also rather broken.

Plus, on a personal level, I think 3 is a superior game through and through. I compared the two at the site I write for, not in terms of critical reception, but what I felt was the fundamental difference design wise.

The reviews say both games were compelling but buggy. And your question is “Does the buggy outweigh the compelling or vice versa?” And then the reviewers answer your question with a numeric score. Clearly most reviewers thought the compelling outweighed the buggy more in New Vegas than in Alpha Protocol.

Maybe they could have more explicitly talked about whether it was more bugs dragging Alpha Protocol down or not as much awesome dragging it up in spite of the bugs, but they weren’t comparing AP to a specific game the way you are. So the best they can do is “It has problems, it has good stuff, and I think it balances out to a 6.5″ or whatever. And they did the same with New Vegas.

Sure reviewers aren’t always consistent in how they assign numbers and there are problems with the whole idea of rating games on a numerical scale, but the thing you’re complaining about here is what the numbers ideally are good for.

You raise a good point. I don’t wholesale object to the idea of a numeric rating. Like you said, they provide a sort of snapshot of what the review means. And in fairness, there are plenty of reviews where the number is satisfactorily explained.

But at the same time, in many of these reviews, the number is simply not explained by the text. I have to take it on the reviewer’s word that the number makes sense, when I don’t think that many reviewers are consistent enough for me to do that.

I believe the major differences between New Vegas and Alpha Protocol isn’t in the ammount of bugs but rather overall execution. New Vegas was essentially, Fallout 3 in a new setting and as such, worked as well as Fallout 3. It was only hindered by its glitches

Alpha Protocol however was far more inconsistent. When it was good it was phenomenal and easily one of the most unique fun, immersive, and unique RPGs that I’ve ever played. Unfortunately, when it was bad, it was abysmal and those flaws shone throughout half of the game. The AI ranges from flatout stupid to omnipotent, making stealth borderline impossible, with very few exceptions, not to mention how terrible the aiming is until you dump a lot of points into it.

Furthermore it is especially painful to watch something so immersive succumb to video game logic. I allow myself a rather high suspension of disbelief in games trying to tell a story or offer an experience and even I was annoyed to find that when I’m pumping entire ammo clips into the enemy, I have to look at a large depleting boss meter.

All of these problems, in addition to game breaking glitches that have caused me to completely reload previous saves, have tarnished the greatness that Alpha Protocol could have been. Don’t get me wrong, I really wish that they made a sequel to it if only to fix all of the problems of the game. I would love to see this executed properly because playing it as the wasted potential that it is is painful.

I don’t understand why Obsidian feels the need to skip beta testing like this. Between these 2 games and KotOR 2, they’re earning a reputation for being sloppy. This game was delayed for 8 months 1 week before its original release date in October of 2009. If this is supposed to be polish, I shudder to think of what the game was supposed to be like before hand.

Great Post by the way. I just discovered the site last night and I never get sick of reading gamers that judge based on what a game’s merits are rather than what they thought it should have been.