Frankly I find posting about – and thus somewhat contributing to the sensationalism – this almost as unsavoury as the news story itself, but I suppose it’s the sort of thing I guess we should all be aware of, if only to shake our heads, tut loudly and make doomy prophecies about the world going to hell in a handbasket.

There didn’t appear to be anything in the way of pre-release reviews for Telltale’s Jurassic Park game (I should note that they kindly set across code for RPS today, however), but somehow there were a couple of very, very positive user reviews on Metacritic. Stuff like “a mix between Heavy Rain and LA Noire”, “lovingly-crafted” and “if Steven Spielberg decided to direct Heavy Rain” and other eyebrowing-raisingly effusive endorsements in this vein. Which rather suggests they played a different game entirely to the one I did. Doing a little digging, Gamespot identified that the posters of these gushy comments did, in fact, work for Telltale.

Telltale has since responded, essentially with the Adam Werrity defence – these guys were doing their own thing rather than under orders, but the adventure game dev felt there was no reason they shouldn’t effuse about their game if they so wished.

“Telltale Games do not censor or muzzle its employees in what they post on the internet. However, it is being communicated internally that anyone who posts in an industry forum will acknowledge that they are a Telltale employee.”

And here’s the bit that makes me do the sharp intake of breath thing:

“In this instance, two people who were proud of the game they worked on, posted positively on Metacritic under recognizable online forum and XBLA account names.”

Right. So it was all basically out in the open, and nothing at all like pretending to be genuine purchasers of the game? If you say so, I guess.

For me, it’s all just a bit silly and tawdry rather than shocking, and speaks to the overinflated importance Metacritic has somehow achieved within the games industry. This kind of thing is, I am quite sure, hardly uncommon. I’m sure all the chaps who work there are lovely, but a review aggregation site having such enormous status to publishers and developers is sickness that’s harming the way videogames are made and their developers rewarded.

“Between the reviewers’ constant lionizing of Telltale Games, complete sentences, proper punctuation, and paucity of spelling errors, we began to suspect that the user reviews were not the product of actual players, but of Telltale representatives.”

Really? All the genuinely great games have good user scores.
I find the user score usually is true to how a review scale should really be used, which is:
1-3 = bad
4-5 = average
6 = good
7-8 = great
9 = amazing

Put it this way… some “professional” review sites are well known to boost scores because they want to keep getting early review copies, advertising revenue, various other things which basically boil down to bribery.

Users generally call out bad products, though can be heavily trolled/astroturfing.

Why in god’s name would you listen to what some random asshole you don’t know thinks about a game? Also have you really looked at user score’s? I mean really looked. Go now and take a look at any game. Saint’s Row 3 has like 9 scores that are a 10. Yes that’s very realistic. Or how many 0’s are there for AssCreed Revelations or MW3? I get that you might not like the game but a 0? Maybe as a total the average will level it out to what resembles a realistic score but don’t give me this crap idea about how the user ratings are more reflective of the quality of the game.

I’m gonna take issue with you about the nature of review scores. That’s one way of doing it, sure but not the only way. It relies on comparison with all other games around it otherwise there’s no meaningful defintion of average, and that’s a comparison that’s almost impossible to judge. Are you comparing AAA games, games from major publishers, indies, Kongregate, Facebook?

Personally I find a more useful way to use scoring, and happily one that corresponds to the way most reviewers score games, is as a rough measure of “How much does a game get right?” So I would expect most commercial games to get at least 70% or so. It’s still a subjective judgement of course, now the comparison is to platonic ideal for a game of that genre under current technology but I still think that’s easier to get a sense of than the theoretical average of all games. And it also leads to a happier life with less grumbling about review scores.

The best way is still, of course, “Read the damn words and make up your own mind.”

Someone writing a well structured review covering both positive and negative aspects of the game in their opinion? maybe.

I’ve read plenty of reviews by people who bring up stuff that actually sounds interesting, while another professional review simply skips over it… It’s not exactly like most people are on first-name terms with the “professionals” either.

User scores can be good for determining whether there is something odd going on, when user and critic averages are far apart, that needs further investigation. Related, they can also be useful as a guide for when an obscure niche game that gets average reviews because most reviewers didn’t really get it is good or average.

I rarely find the text of their reviews particularly illuminating however.

My issue with Metacritic user reviews is that a lot of them seem unduly influenced by one issue that usually has little to do with overall game quality. Look, a game is not a 0 just because it has DRM or you like a rival game better (BF3/MW3, I’m looking at you). A game is not a 10 just because you liked the previous game in the series.

User reviews can be useful and cover stuff you want to know that the critics didn’t mention, but aggregating the user scores is lunacy. It’s the age-old survey problem of anyone driven to comment on a product is only likely to do so because they have an overwhelming urge to gush about it or lambast it. Hence all the 0s and 10s. And then devs doing things like this and forums agreeing to spam review games the other way just throws it out entirely. Much better if they took an RPS approach and just used their words, and not numbers. It’s easy to pick out who is over-gushing/bashing just for the sake of it

“Someone writing a well structured review covering both positive and negative aspects of the game in their opinion? maybe.”

The problem is most sites don’t offer enough space to even approach that. I’m pretty sure even the professional critics would hit issues if asked to review a game in the space of a single tweet. Although that would be an interesting experiment …

“I’ve read plenty of reviews by people who bring up stuff that actually sounds interesting, while another professional review simply skips over it… It’s not exactly like most people are on first-name terms with the “professionals” either.”

No, but I don’t need to be. A good professional review will usually justify their statements by explaining why they happen to like or dislike something. Most reader reviews simply tell you something sucks and treat it as objective fact. Which is a crucial difference; I don’t agree with every Wot I Think RPS does, but they break down their criticism enough that I can decide for myself whether I’m likely to have the same reaction.

The other problem of course is there’s a high chance that a reader review is by someone who bought and enjoyed Battlefield or Modern Warfare, which instantly disqualifies them from having a valid opinion on anything. Ever. At least as far as I’m concerned :P

You can tell within 10 lines if a user review is worth paying attention to. Most of the extremely low ones are not as they’re usually filled with bizarre pet hates poorly expressed ‘the musci? well ive heard worst but this is making MY EAERS BLED!!! 0/10. Witch brings me to the gamplay. there are none!!! 0/10′

Which is why most ranking sites use a simple like/dislike option, because over the average it works out to be a more accurate representation to any scale (where some people only vote in binary max/min and others vote somewhat properly).

Like IMDB I vote properly, but only so I can have a nice list that I can show others – if I wanted to have the most impact on the score of a film I’d only ever vote 9’s or 2’s (because I suspect 10’s and 1’s are weighted to actually be worth less, unless the score is already very near to them).

i reley dont wan to say this, but i have to now.
this game is so esey. i mean, all you do is hit the spacebar. thats it! how is this an acsun adventre anyway? you cant contrail anything but what it says on the screen! what if i didnt want to go tat way? what apout quests? there is no way you can lose to the boss at the end! this game is crap! its not even an acsun adventre at all! i mean look at it! in what way is this supposed to be an acsun adventre if you can do quests and stuff? all you do is press one butten the entier time! explain to me! the athore coments al totol lies! is it supposed to be wire dudes? i dont even know how this damn game got the daily 3rd prize, or a rating of 4.26!
pepole think this review is worthles.
go ahead! say it! i dont care! im just trying to make a point here!
blam this piece of crap!!!!

P.S the only reson im giving this a 1 is beacuase the runnning where pretty good. but thats it!”

I am not a fan of user reviews exactly because most of them are written by people who are extremely passionate about things I don’t care about. They are too busy proselytizing or whinging about things that don’t matter for me to get anything useful out of their opinions. This sort of personalization of the game experience tends to be rarer among professional reviewers. I want a calm, reasonable, fact-laden retelling of the reviewer’s experience with the game. What I get is this mess:

“This RPG sucks because action controls”

“WTF 2 MENE WURDZ LOL”

“This is best game ever made because it has one or two features I find deeply and personally compelling, therefore, everyone else should agree with me.”

“This game gets a 0 because I couldn’t install it.”

I could go on and on, but we’ve read it all before. Kudos to RPS for having a higher standard.

The user reviews on Metacritic are completely useless. Anyone who thinks they have any value should start checking out specific games the very day the game comes out. Almost half of the user reviews posted appear within the first few hours of their ability to post being available. No, but I’m sure those hundreds of people telling me they legitimately went to the store to buy Modern Warfare 3 and played dozens of hours of it already so that they could say the game was worthy of a “0”, or vice versa some piece of garbage like Bodycount really earned that 8 within minutes of these games being legally available, are people to be trusted.

Metacritic scores are just as bad as itunes and Amazon scores were before they forced it so only users who actually bought the product could rate them (and they still aren’t that great, to be honest).

I don’t agree with metacritic user score holding any real value. While usually they seem to be honest, I know examples of games which were/are much better than average user score suggests. I think it is akin to Amazon scores, where users give one star for one quality (like a hated DRM scheme), and don’t bother with objectivity. Not knowing whether this is such a case makes the whole idea of user score completely bogus.

One example is Two Worlds 2, which has relatively good critic score (8+) and comparably low users score (about 6). In this case the users reviews are mostly useless, just like their score. This is why I prefer review, without scoring. I make exception only for good, weighed (or at least broad and detailed) scoring systems, but English-language gaming sites are devoid of such.

“Complete sentences, proper punctuation, and paucity of spelling errors”? The fuck??? So these things are now giveaways of a corporate plant rather than something everyone should aim for?
Does it mean that you now need to write your reviews like a caveman lest you be mistaken for a shill???

When I first heard Telltale and JP, I was so damn excited. I pictured a point and click adventure on an island full of dinosaurs out to eat you. The moment I saw the first gameplay snippet and realized it was all QTE’s, the game ceased to exist for me. They could have done so much with this IP. It’s an absolute shame they turned the whole thing into a mini game. It really says something that Trespasser is still the best JP game ever made. Sad.

That’s near heartbreaking. I’ve been waiting for a truly awesome Jurassic Park game, and (once more) I have to deal with the crippling realization that this may never happen. At least I have Operation Genesis to fall back on: sorry to say that by the halfway point, Trespasser’s wonky controls felt like a bigger obstacle than the dinos.

…Well, right. Don’t think anyone’s arguing that killing the dinosaurs is crucial to their enjoyment. Hell, my favorite game is the one where you manage the park’s rides and gift shops, ignoring all the messages about man’s hubris in exchange for truckloads of money. The consensus, near as I can tell, seems to be this game would simply be more satisfying if you were able to directly control where a character goes to flee a pack of raptors, rather than tapping WASD to perform a series of escape animations.

Are you seriously making a stink about the already-known-to-anyone-with-a-brain, useless as shit user reviews? How shocking that people would abuse a system that is abused improperly on a regular basis.

Are YOU seriously suggesting that when a games company is caught red handed in a misleading/deceptive practice, even if it is an “open secret” that companies do it, that journalists shouldn’t call them out on it?

His point is that cheating on Metacritic user scores is like cheating to win your family board game. It’s completely lame, but no one should care enough that the news would end up spread on countless internet board gaming news sites.

Thank you TigerMike. No Vesuvius I do not work for Telltale, but why am I not surprised I would be accused of such? How dare I make light of the sanctity of Metacritic user reviews! To be honest yes I consider this a complete non-issue, but mainly because I’m not the kind of asshole who reads user reviews on Metacritic.

@Vesuvius: But to be fair, we’re clearly not talking about “a gaming company” here – we’re talking about the actions of a couple employees. It’s pretty obvious they weren’t operating under company orders, as if they were, it wouldn’t have been done like this. I imagine the employees are actually getting some flack at work at this point for their “reviews.”

@phlebas I believe you’re confusing Metacritic’s professional reviews (which when weighted determine the Metacritic score) and Metacritic’s user reviews. This article is about the user reviews, not the Metacritic score.

Well, I’m thinking more along the lines of (once again, in the space of a week) “employees acting on their own with no direction from Telltale” (should we start adding a “nudge nudge, wink wink” here?) started posting snippets of emails and forum posts out of context on reddit (posts I gathered were deleted from Telltale forums when the fellow was trying to get the damage on his Jeep taken care of.)

Someone seriously needs to learn how to manage their employees, if indeed they aren’t giving the old “nudge nudge wink wink” to their repeated actions.

And for the record, Telltale was responsible for providing covered transportation for the jeep as specified in the agreement, and didn’t record the damage to the jeep upon arrival in Seattle, so they should pay for repairs.

Ugh, the Telltale guys seem like they’re getting more and more incompetent lately, making bad decision after bad decision. I’ve still liked what I’ve played, which isn’t much. There was an enjoyable game under all the sloppy design and misspelling in Puzzle Agent 2 (f u, I love the Fargo+Twin Peaks+creepy Annable vibe), and I thought there was an enjoyable game under all the problems of Jurassic Park, too. But I also have nothing against QTEs and am admittedly a Jurassic Park fanboy since childhood, so anything and everything related fills me with a warm fuzzy feeling.

Felt like playing a Goosebumps book or a Saturday morning cartoon, IMO. Some of the writing was solid, as were some of the characters. Stress: Some. A lot of it was cheesy as hell and I don’t agree with any of the reviews saying the game is loyal to the film, because there’s way too much Telltale Goofiness™ under all the dialogue and plot points.

Anyway, pretty dumb and obvious mistake by those guys. Again. I kinda made that assumption and shrugged when I saw the first user reviews on MC. I still hope they continue making more JP games with the ability to learn from their mistakes, e.g., expand the non-QTE stuff, e.g., finish the product before release.

C’mon. We know this has been going on since year dot. Just like the posters on game forums who slate the game and praise another (Rift is shite, come play WoW etc…) when it’s quite obvious they’re just doing a bit of trash talking and advertising.

Still, it’s nice to see someone calling these people out and shaming them. I gave up taking notice of review scores a long time ago when, even if there was no foul play involved, it certainly looked that way.

Leaving positive reviews/comments on games that the poster is involved with happens more often than you’d think. Even without touching on the ethical aspect of it, it really is a stupid thing to do, considering the potential shitstorm if it gets found out, but the chance of discovery is low enough for many to take that risk.

User scores are something I pay absolutely no attention to, because half of them don’t have a score representative of the review. Look at Modern Warfare 3 for example, OK maybe it’s bad. Or at least not their cup of tea, but I sincerely doubt it deserves all the ‘0’ scores it got. People will give scores like in order to make a statement or, in some cases, because they’re stupid and didn’t move the metascore slider properly.

User reviews on the other hand can be pretty helpful. As with any review, it all depends on the content and the quality of the writing. It takes hard work to cunningly conceal bias, so it’s usually pretty easy to spot the fanboys or the haters who aren’t being fair, and get a good idea of the game from someone you can be confident is giving an honest opinion.

Until of course you get stuff like this. Telltale’s justification is pretty sketchy at best and pretty much implies that it is a practice within their company that they approve of. Anyone with half a brain can tell you that the absolute last people who should be reviewing a piece of media are the people who worked on it.

That really depends on who the intended audience is. For a smaller developer for example who’s writing games aimed at people with identical tastes for themselves, simply explaining what they think works and what they aren’t happy with is incredibly likely to be an accurate appraisal of how their target audience is going to feel about the game. Of course it’s useless if you aren’t already in that target audience, which is why such things need to be declared beforehand.

The problem with user reviews is anonymity, which is why they shouldn’t be trusted in the first place. There’s nothing stopping a company hiring a bunch of people to go talk up their game on metacritic or the like, and there’s no easy way of identifying when this is the case (and that’s before you consider indirect methods. Some smaller devs offer discounts or similar to people who go out and spread the word which is exactly the same thing).

“simply explaining what they think works and what they aren’t happy with is incredibly likely to be an accurate appraisal of how their target audience is going to feel about the game”

Whilst you do have a good point, I think this kind of thing is fraught with problems. It’s incredibly difficult to be unbiased about your own work. In my experience people tend to err towards being over-precious about what they do or over-critical. So although I agree that a developer writing about what they were setting out to do and areas where they felt they were successful/unsuccessful can be very useful, it shouldn’t be considered a fair ‘review’ of a game, which is better left to the audience that plays it.

User reviews can actually be a lot more useful and way more honest than you give them credit for. Users tend to go to extremes with scores though. Just take them as positive or negative reviews. If you saw metacritic drop the score and give users 2 options, thumbs up and thumbs down, it would be more useful than most review sites.

Someone earlier attempted to redefine the ‘score scale’ but they got it horrendously wrong (as do most sites).

Goodreads – a book review site – is one of few places which have this right – their system is

1 Star – Didn’t like it
2 Stars – It was OK
3 Stars – I liked it
4 Stars – Really liked it
5 Stars – It was amazing

Note that the KEY thing is “I liked it” or “I didn’t like it” – not aggrandising nonsense like “it was bad” or “it was terrible” – or utter drivel like “I hated it” (it’s a game – love and hate don’t apply),

All you can ask is what someone felt about it – don’t ask them to infer that their views would transfer to someone else – don’t ask them to speak for everyone because most people can’t do that.

More sites need to switch to this approach IMO – stop asking people to become reviewers, just ask for THEIR take on it.

“Someone earlier attempted to redefine the ‘score scale’ but they got it horrendously wrong (as do most sites).”

Yes. Let’s all bow down to your heroic system that gets it ‘right’. Except it’s just another 5-star system with some pointlessly (89%) daft reasoning as why using words like love/hate are ‘drivel’. Semantic Silliness rating: 10 out of 9.

The REAL issue – in my opinion – is that most reviews (or ‘takes’) seldom provide a picture of the reviewer’s overall gaming persuasions. So the context of the score is related only to that single game.

It’s the difference between basing a like or dislike off a film’s single movie frame instead of a full motion sequence.

Anyway – back off to read Crash to sumptuate over the next Spectrum cassette releases.

Why take issue with ‘It was terrible’ when your preferred system includes ‘it was amazing’? Surely they are two sides of the same coin?

Surely ‘it was OK’ should be the mid-point of any scale? Otherwise you’re not making any distinction between fundamentally bad games/books/etc. and those that are deeply flawed but have some redeeming qualities.

Is this really so bad? They weren’t pretending to be anyone else. There aren’t any restrictions on who can post user-scores on metacritic.

Honestly I don’t think their opinion on the game is any more or less valid than that of some guy who hasn’t even played it but has a beef with the developer, or people rating it down on account of DRM. Or some fanboy that loves the series.

Because of reasons like that metacritic user scores are often pointless, and only matter in so far as there are enough honest reviewers to eliminate a lot of the noise. Two extra scores won’t matter.

I’d love to see a fair metacritic, where people involved in the game couldn’t post scores AND people who haven’t played the game can’t post scores either. But since the second is a far bigger problem, and impossible to work around, I can’t get bothered about the first either.

I think there’s only an issue here if Telltale was aware of this and/or encouraging their staff to do it.

For all I know, the ‘Telltale staff’ involved were 20-something year-old junior programmers who got a bit too excited about the first game they’d ever worked on. Equally, they could be senior marketing managers who thought they could sneak this under the radar and sell a few extra copies.

I just don’t think we should immediately jump to the conclusion that it was a company policy endorsed by anyone other than the people who did it.

This is a complete outrage, and it’s only natural for a site of Rock Paper Shotgun’s calibre to call this unethical conduct into question. After all, RPS represents the absolute pinnacle of video gaming journalism, like Leigh Alexander meets Heavy Rain. Lovingly crafted words for the most selective of audiences, RPS is the Citizen Kane of blogs. And, like the finest of wines, it will only improve with age. I would wholeheartedly recommend the site to anyone.

While I agree that it is unprofessional of any developer to give a review score to their own game, I think you need some perspective before you gather your torches and pitchforks against Telltale.

This practice is commonplace on Metacritic and I’m surprised you don’t run more articles like this pointing out just how absurd, skewed, and abused the Metacritic system is.

For example:

When the Witcher 2 released, their user review section was immediately flooded by several dozen user reviews of 0, almost as soon as the section became active (before anyone could have had reasonable time to actually play a legitimate copy), by Bioware employees. They made no effort to hide this, in fact using names such as BiowareEmployee and BiowareQA and so forth. And no one ran any articles chastising them for essentially trolling their competition.

The same thing happened between EA and Activision in the battle between Modern Warfare 3 and Battlefield 3.

My point is, while this practice should definitely be called out and frowned upon, I would just appreciate if you did so to more companies than singling out Telltale alone, like they are the only company in history who does this or has gotten caught doing this. As this article stands, it reeks of personal bias, because it seems like you are attacking Telltale for a practice that other companies regularly execute while ignoring those other companies for whatever reason.

I work in publishing, where Amazon reviews are pretty important. At a few points in my career, I have had people asking me to go and post 5 star reviews on Amazon for books we’d published. I’ve always refused, because I find reviews on websites useful – to some extent; I don’t put a lot of stock in user reviews, but I still read them and find them interesting. I try to apply the old rule, what if everyone did this all the time? Well, a marginally useful thing would become completely useless as it drowns in spam.

On the other hand, I’ve had books that I’ve published get trashed on Amazon, on the first day of sale, for things like ‘Amazon hasn’t delivered it on time.” A couple of times I’ve gone on and astroturfed purely out of a sense of redressing an unfair balance. Amazon’s supposed to weed out unfair, off-topic reviews but often they sit there throughout the whole peak period for a new book’s sales and give it a one-star rating. It’s infuriating, and I hate myself for putting my thumb on the scales even in what seems to me to be a good cause.

It’s also actually illegal, as far as I know, to do that, so although I understand the temptation, it’s not something I think anyone should ever do.

Ugh, people who use product reviews to criticise the delivery service should be fired from a large cannon into a vat of custard filled with genetically modified piranhas capable of surviving in custard.

Other reviewers who should be subject to punishment:

1) Reviewers who criticise a revised version of a product for an issue which existed in a previous version and is now corrected. This was especially frustrating when WD produced a batch of hard drives with a design flaw. Some users took it upon themselves to post warnings in the review section, but didn’t bother confining those to the affected models.

2) Reviewers who give zero star reviews for an issue that they clearly didn’t even bother to try and address.

Trying to justify it just compounds it. It might well have been down to the actions of individuals, but now the company has effectively condoned it. I like Telltale. But this Universal deal hasn’t really worked out for them, has it?

I find the OVERALL averages of user reviews far more accurate than the ‘professional’ reviews.

Honestly, look at how many ‘9’ professionally reviewed games are about ‘6’ by user scores. It seems to me that users are far more likely to give average games average scores. IE. unlike certain large publications that give just about anything a 7 or higher.