ImRage: Why You Can’t Trust Big Review Sites

If there’s one thing that’s downright poor in the world of gaming journalism it’s the way the games are rated. You log onto the most popular gaming websites like IGN or Gamespot and you see swaths of games rated 7, 8 or 9 and, less commonly, a 6 or a 5. Only the worst of dysfunctional unplayable muck get scores below 5 and if 5 is supposed to be an average, then how is it even possible that so many games are above average? (It doesn’t even make intuitive sense if you think about it!) I’m not going to go into the details of why I think rating games based on a number out of 10 is a poor system because many others have tackled that particular issue before me. Instead, this week, I’m going to talk about something a little closer to home: why you can’t trust the big-name rating sites out there.

Picture this: You go the store, you pick Syndicate off the shelf and you pop onto your phone to check your favourite review site for a score. Maybe Gamespot: 7.5/10. Seventy-Five Percent? If we’re going with a standard rating scale (like the ones used to grade you in school or university) then 75% is excellent; at 75% you should be expecting the cream of the crop. I mean if a movie got rated 75% on Rotten Tomatoes I doubt you’d be skeptical about going to watch it. Too bad that with Syndicate, this isn’t the case. For anyone with reasonable quality standards, Syndicate is decent but unimpressive. It’s not something you rush home to play; it’s entirely missable and it’s doubtful that it will be remembered outside of 2012. In fact, many readers of this column probably haven’t even heard of it and if you haven’t, don’t bother looking it up. If you want a good FPS, there are so many better options and, frankly, 75% gives you the impression of Four-Star Cuisine when what you’re getting is McDonald’s.

CLICK TO EXPAND: 18/22 of these entries are 7 out of 10 or higher. Granted there are 3 repeats here but does it make sense that almost 80% of games released in the last two weeks (15/19) are rated 70% or higher? Scroll down the list on GameSpot if you need more convincing…

You see, the problem with video games is that as much as we want to compare them to movies, music and books, they are not these things and they never will be. A movie costs under R50 and even if you go see something mediocre with your friends, it’s not a complete waste if you have a good night out and find something to laugh about; at worst, you’ve lost two hours and the price of a take-out meal. A couple of songs on iTunes are hardly that pricey either and you have the option of previewing them fully on YouTube before you buy them. A book is maybe a little different. If you buy something you don’t like, it’s R100 down the drain, but book reviews tend to be of a much higher quality and most people don’t seem to read much these days anyway.

Video gaming on the other hand, is a huge investment. First you need to put a couple of thousands into a gaming console and TV set or a decent PC and then each game you buy costs in the region of R500 to R700. Even if you make use of the second-hand market and trade-ins, it’s a hobby that costs several orders of magnitude higher than most and if you’re putting that much money into it, you really should be having a hell of a time. If you’re not, then you might as well just go and spend money on things you’ll enjoy more; like travelling or going out with your friends. There are few feelings worse than paying R600 for a game that you really don’t enjoy and then forcing yourself to play through it because you don’t want your money to go to waste; trust me, I’ve been there.

Even if you keep it simple or make use of second-hand markets and trade-ins, gaming is a very expensive hobby.

That’s why I feel that gaming journalists have more responsibility than any other media reviewers in making sure that they correctly evaluate games. Few people are rich enough to afford buying crap games and, especially in South Africa where the prices are high and the incomes are low, gaming is a huge luxury. I guess that’s why it saddens me that gaming journalists do the worst job out of anyone; they’re essentially an extension of the advertising. In fact, that’s pretty much their function. We’ve all heard the story about the GameSpot editor who got fired for rating Kane & Lynch 6/10 (click here) and when Eidos offered to let magazines review Batman: Arkham Asylum early if they scored it 9/10 or higher (click here). It would be nice if these were just isolated occurrences but if you take one look at the image I posted up top, then the inflated review scores should tell you that a lot more than a few games are getting higher scores than they deserve.

And if you think about it for a second, it’s not all that far-fetched, it’s simply a case of misaligned incentives. Reviewing websites make their money primarily off of advertising and, as such, getting website hits are very much on the top of their priority list. One of the best ways to do this, of course, is to review as many games as possible and to review them early so that they can get attention ahead of other sites. Video game developers, on the other hand, make their money by selling as many games as possible and if a popular site is going to rate them highly and recommend them to consumers then this fits right into their plans. What you have then is a market where reviewer sites are trying to please developers so they can get the games first and developers forming relationships with popular sites where they provide games and the sites rate them highly.

If you’ve studied markets with conflicts of interest, like the pharmaceutical industry (click here and here), then you know that this is a very real issue in the world and what this results in is a market where developers and the suppliers look out for each other’s interests and nobody looks out for the interests of the consumers. For your information, that’s us by the way. What’s scary about this, is how much power developers actually have in the industry. Few will complain when mediocre games like FF13 are rated 82% on Metacritic (any game in which you have to play through 20 hours of crap to get to the ‘good parts’ is mediocre) but when Medal of Honour: Warfighter was rated badly recently the developers actually publicly complained about it (click here)! That’s why it’s been positive in the last year to see gamers rally against companies like Capcom and their abhorrent DLC practices. It’s also been encouraging recently to see that gamers are starting to become more and more aware of how untrustworthy the big name review sites can be and that needs to continue.

This baby represents your innocence towards all the big review sites that are trying to take advantage of you.

It surely must be a rude awakening to many to realize that big name review sites aren’t your friends but, in the long run, it really will benefit you as a consumer. Gaming is costly as a hobby and it requires roughly R600 and 8 hours to enjoy a single game; both of which are valuable resources. So the next time you pass a Medal of Honor game on the shelf and it’s rated 9/10, you should think very carefully about whose recommendations you’re going to trust. Maybe ask your friends or watch Zero Punctuation… that guy usually gets it right.

I think thats what i like about you guys cause you don’t give a game a number at the end, as it was explained on a podcast many months ago and thanks for that btw. Just read a few reviews of Far Cry 3 and one place actually went and gave it a 10/10 and i remember one reviewer told me he would never ever give a game a perefect score cause there will always be that *one thing* And i can respect him for that. And while i don’t always agree with the review i will always take you guys word for it and its helped me many times in not picking up a game which i would later on regret buying so thanks :)

http://www.facebook.com/profile.php?id=602797306 Imran Amien

I’m actually quite glad that you DON’T always take our word for it :P. Game Reviews aren’t definitive scores, just opinions :P. Main purpose is to assist you in the buying process, point out the gems and keep you away from junk. End of the day, it’s entirely your decision. We only ask that you make informed ones.

Glad it’s helping :P

http://egamer.co.za Dean Oberholzer

Yeah, you shouldn’t base something solely on a review. And ours aren’t perfect either. No one has perfect reviews.

I said in the Black Ops article, it’s just a guideline, really. You should always try the game yourself if you can, before you buy. Even if it’s a demo. That will give you extra help on whether or not it’s worth it. That also won’t be definitive, as one can easily take the best part for the demo to try and sell it to you.

Alessandro Barbosa

Zero Puntuation has good points, but the whole point of that show is the cynical nature of it. I don’t think you should base decisions off of what Ben Yahtzee says all the time, because he’s also extremely nitpicky. Again, not a bad thing, but the whole reason his show is known is because it’s overly cynical and different. Go watch his recent Hitman: Absolution review and tell me if it’s an accurate description of an otherwise amazing game. :P

AG_Sonday

I watch it for the humour mostly and sometimes to see if he’s spot on with what I say in my review such as with inFamous 2 :P but after a while you learn to read his style and know when he’s talking about a good. His Absolution review rang of a game he enjoyed more than most.

I enjoy the overt cynicism because then those few games that he truly enjoys stand head and shoulders above the rest which is why I employ a far milder discerning taste to my gaming :P

http://www.facebook.com/profile.php?id=602797306 Imran Amien

I feel like he has become far more cynical in recent years. Wasn’t expecting him to trash Dishonored. But he hardly ever hates on a game without a strong reason. If a game ever gets his stamp of approval or even a half-hearted recommendation (like Hitman did), then you know it’s top-drawer stuff…