ON REVIEWING AND STANDARDS

Solitayre's review criteria and thoughts on why they are important.

Reviewing is a subjective process; it is thought and opinion, so this article and everything in it is by definition going to be biased by my personal beliefs. Some of you aren’t going to like what I’m about to say.

Some of you have very specific goals in mind when you create a game. Some of you are trying to tell a great story, some of you are trying to create fun puzzles, some are trying to create awesome and epic battles, and some are trying to show off your art or music. That’s all well and good; game-making can be a perfectly legitimate artistic endeavor and I do not intend to diminish that. But you have to understand not everyone is going to see it that way. People who download your game are looking for a fun game experience, and they’re going to have certain expectations of your project.

Some of you like to break down the components of a game in terms of their weight or importance and plug them into some sort of ridiculous math equation. {Gameplay = Story + (graphics/ music^2)} This is asinine for a couple of reasons. The first reason is it doesn’t even really make a lot of coherent sense. The second, and more important reason, is trying to imply that some part of an RPG is more important than any other is pretty ridiculous.

For you to have a fully realized game, none of these aspects can be lacking. Sorry gameplay people, whine about how story is a waste of time all you want, in an RPG there’s an expectation that there’s going to be some decent writing somewhere. I don’t understand this movement to sort of hack out parts of a game the creator doesn’t like with some kind of meat cleaver and expect the game to be better for it. It’s like a house of cards. You can’t take cards away and call the resulting mess a complete project. It might still be fun in some ways, but I think most people would agree that something was missing. The same goes for story people. If you’re going to tell a story but neglect your “gameplay”, why include it at all? If you’re going to completely forgo the RPG mechanics in pursuit of your storytelling, the resulting story might be pretty good, but you aren’t really making a “game” anymore. You’re kind of making something else.

Now, I don’t want anyone to misunderstand. I am not saying your story needs to be enthralling, but it needs to make sense. Your graphics don’t need to be gorgeous but they should be consistent and functional. Your gameplay doesn’t need to be stellar but it better not be boring.

So while I’ll judge your game on the individual aspects that make up your project, I’m also going to judge it as a whole. And if your intentions are clear I’ll even try to look at it from your perspective, as you intended it. But the bottom line is I am going to tell you what’s good and what’s bad, what’s wrong with it, even if it wasn’t a part of your project you cared about. Because it’s still part of your project and that part of it still reflected on the overall game as a whole.

Now, how can you minimize this? If you are not good at a certain aspect of a game, how can you make it so it’s not a problem? By playing down those aspects. If you’re not good at writing, don’t dump lots of long, badly written conversations on me. If you’re not good at making puzzles, don’t put a lot of puzzles! If you’re bad at balancing combat, don’t throw me into a fight every four steps. If a problem doesn’t bother me much, I’m not going to get hung up over it. If I don’t notice that there’s a problem at all, then it’s a non-issue.

Now, supposing you see a review you don’t agree with. It could be written by me or someone else, it could be for your game or someone else’s project. It is very important to remember that reviews are one person’s opinion. Getting a bad review is not the end of the world. It does not mean no one in the world likes your game. It also does not mean you should flip out and get defensive. If someone took the time to actually review your game it means that had a vested interest in it. If they didn’t care they’d simply not say anything at all. You should be thankful that someone, anyone, took the time to comment about your game and try to help you improve. Disregarding a review or claiming a reviewer has suspect motives just because they gave a negative review is not only offensive and disrespectful but ultimately a disservice to yourself and your fans.

By request, I am including my general reviewing guidelines here so that people know generally how I will look at a game and rate it.

My reviews will usually start with a short backstory about my understanding of the project and why I chose to review it. I will then provide a brief synopsis of the game’s plot to let the player know basically what they can expect going in. From here I rate things by the following categories.

Balance: This is my general impression of how battles in the game were handled, including any aspects of the game that tie into this including optional or additional systems or character abilities. I will judge by how mechanically interesting they were, if they were fun, if they were of an appropriate level of challenge and if everything generally worked the way it was supposed to.

Level Design: This is where I will talk about the general layout of the game and all the things it encompasses. Things like graphics will be discussed here, though I will generally not mention them unless they are very good or extraordinarily bad. Things like town design, dungeon design, puzzles, including aesthetics, will be included here, and if I have any concerns about the controls or interface, this is generally where I’ll mention them.

Characters: I will discuss my thoughts on the writing, dialogue, characters and their subsequent development. If I have any in-depth questions or thoughts on a character I will go into it, including any things I liked. If I have worries that a character seemed under-developed or seemed weak, it generally means I wanted to know more so shouldn’t be construed as necessarily a bad thing.

Story: I will discuss narrative structure here, including the overall plot, and any storytelling methods used. Any unconventional storytelling methods aside from basic, chronological linearity will probably be mentioned, and I will also discuss the nature of side quests or plot branches, if any. If I notice any stylistic influences, such as a famous work or author, I will usually give those a nod here as well.

Music and Sound: I will talk about how any musical choices used in the project added or detracted from the overall mood or atmosphere of the game, as well as pointing out any times I felt sound was being used inappropriately. Having “okay” music generally won’t garner much of a response from me, but having really great music will really impress me. I will generally favor music I haven’t heard before over popular commercial rips.

Finally, the general scale I use to “rate” projects is as follows. My ratings tend to be fairly arbitrary at times but this is a general indicator of what I thought.

5.0- A near perfect game. It must do nearly everything right as well as go above and beyond expectations in every way. I do not expect I will ever give this score, but you never know...

4.5- An exemplary game. Not only does it do almost everything right but goes the extra mile in terms of content, programming, or some other area that sets it above other games. Bugs are all but non-existent.

4.0- An excellent game I recommend people play. It has few flaws and those it has generally don't detract from the overall experience. Almost no bugs.

3.5- An above-average game. It does a lot of things right but unfortunately has a few flaws that drag it down. Very few bugs.

3.0- An average game. It has a mix of good qualities and poor design choices. Not everyone may like it. There may be bugs but they do not detract from the experience.

2.5- Still pretty average, but here the design flaws are very noticeable. One or more aspects of the game are severely lacking which detracts from the experience. There may be multiple obvious bugs.

2.0- A below average game, it has a lot of faults that overshadow its good points. There may still be a good game inside...somewhere, but it's not there yet. It is very hard to get a lower score than this from me.

1.5- A poor game. The game is severely lacking in multiple areas that lead to a bad game experience on my part.

1.0- A very poor game. A game with a score is generally a sign I did not think the author was trying or that they didn't care. The game is disjointed, mechanically broken and may be near unplayable. Even this is not a 0.5 from me, oh no, only a very special kind of bad gets a 0.5...

0.5- A blight on humanity. The game was offensive to me on a primal level. It has no redeeming qualities. It sends me into apoplexies of rage. I want to set it on fire. I think less of people who like it.

Posts

0.5- A blight on humanity. The game was offensive to me on a primal level. It has no redeeming qualities. It sends me into apoplexies of rage. I want to set it on fire. I think less of people who like it.

Now, supposing you see a review you don't agree with. It could be written by me or someone else, it could be for your game or someone else's project. It is very important to remember that reviews are one person's opinion. Getting a bad review is not the end of the world. It does not mean no one in the world likes your game. It also does not mean you should flip out and get defensive. If someone took the time to actually review your game it means that had a vested interest in it. If they didn't care they'd simply not say anything at all. You should be thankful that someone, anyone, took the time to comment about your game and try to help you improve.

"That's right you filthy mongrels! How dare you expect someone to actually LIKE your game. You should be GRATEFUL to anyone took the time to explain to your retarded ass how terrible it was! If someone savages your game, wipe that blood off of your face, look up meekly and say..."please sir, may I have another"?"-Drill Sergeant Max McGee

I'm sorry but I think you've hit on possibly the exact reason this community's amateur mentality holds it back. Because people spend hundreds of hours working on games and then they feel like other people are doing a favor by playing them. Besides fanfiction writers (I just read that Shadowtext article about the arrogance of storytelling) we have to be the only people in the world who work so hard on something and then are grateful (or at least expected to be) when someone plays it. Everyone else in the world who makes anything has at least some regard for the flip-side of the coin, which is that the creators are doing a huge service for everyone else, by making something that entertains them. Imagine how much better the creators would be doing if they had as much confidence as they do humility. Oh god, actually...you know what. I changed my mind. Don't think about that sentence, it's way too easy to invert. Because if I can use what I've seen as any kind of an example, we're all swinging violently from self-obsessed egomania to self-effacing insecurity all the time.

I hope my off-the-cuff comments lead directly to another thought provoking article. I can't imagine I didn't influence this one, however indirectly.

"That's right you filthy mongrels! How dare you expect someone to actually LIKE your game. You should be GRATEFUL to anyone took the time to explain to your retarded ass how terrible it was! If someone savages your game, wipe that blood off of your face, look up meekly and say..."please sir, may I have another"?"

we have to be the only people in the world who work so hard on something and then are grateful (or at least expected to be) when someone plays it.

No. This is something that extends well beyond the RPGmaker community. Amateur authors are competing with a great deal of other things for people's time, and I'm not only talking about main stream entertainment.

Everyone else in the world who makes anything has at least some regard for the flip-side of the coin,

Everyone possesses this quality. The RPGmaker community is not made up of an identical legion of automatons hell bent hurting your feelings. The world isn't that simple, Legion.

which is that the creators are doing a huge service for everyone else, by making something that entertains them. Imagine how much better the creators would be doing if they had as much confidence as they do humility.

Confidence comes with time and practice. The confidence you preach of is a false confidence; a confidence born of ignorance (Yay - Mass Effect quote). You don't gain this confidence when people are reluctant to tell you what's wrong with your game; you don't move forward when you're not challenged.

Oh god, actually...you know what. I changed my mind. Don't think about that sentence, it's way too easy
to invert. Because if I can use what I've seen as any kind of an example, we're all swinging violently from self-obsessed egomania to self-effacing insecurity all the time.

I hope my off-the-cuff comments lead directly to another thought provoking article. I can't imagine I didn't influence this one, however indirectly.

No, that's just you.

I've been sitting here for ten minutes at a loss for words. I can't decide how to close this up. My mind goes around paces around your post trying to figure out just what the hell your problem is. You take issue with criticism? Do you think people should be nicer? You want to build confidence in the community?

If you want to build confidence in the community, a good start would be advocating for higher standards, encouraging collaborative projects like Ill Will, maybe not throwing a giant fucking hissy fit when someone says something you don't like about your game.

True confidence comes with practice. It comes with tenacity and diligence; there are no shortcuts.

You do have a good point, Max. As much as a person can improve by welcoming constructive criticism, I think you'll agree that the best works require the confidence that comes with believing people will enjoy that work when you create it. However, I don't think Soli's ideas are wrong. We are a community of amateurs, and we don't have the luxury that professional writers in professional mindsets have; professional criticism. A number of a game's downloads doesn't offer nearly the same level of assurance of your product's quality as, say, sales numbers (which in itself doesn't have to correlate with the product's quality, as I imagine many creators would consider getting their message out to as many people as possible a greater victory).

In context to this article, I see no reason why we SHOULDN'T be grateful for the amount of feedback provided in a review, positive or negative. I think it's the creator's responsibility to size up the overall response to their creation. One person's opinion shouldn't affect their confidence (which I'd venture to say people either have or don't have, regardless of taking the advice of an article like this), especially if they receive another review or comments with conflicting viewpoints. The combination of these things, and to a lesser extent overall downloads, helps us determine whether or not the confidence we have in our content is enough to entertain people.

I guess to put it another way, a creator should start with confidence when they create. After a release, remaining over-confident will make one oblivious to their shortcomings. On the other hand, relying too much on everyone's feedback will dilute the integrity of the creation, where the ideas the creator wanted to convey become lost. This is inherent. I think what Soli wants to address more specifically is that there is no benefit to lashing out against criticism or even discarding it outright. The greatest benefit to all types of criticism is to weigh the strengths and weaknesses outlined in them against other comments and criticisms received, and then through your own confidence. In the end, you may dismiss a comment and conclude, "they just didn't get it," but it does more good than harm to first consider why that might not be the case.

I guess to put it another way, a creator should start with confidence when they create. After a release, remaining over-confident will make one oblivious to their shortcomings. On the other hand, relying too much on everyone's feedback will dilute the integrity of the creation, where the ideas the creator wanted to convey become lost. This is inherent. I think what Soli wants to address more specifically is that there is no benefit to lashing out against criticism or even discarding it outright. The greatest benefit to all types of criticism is to weigh the strengths and weaknesses outlined in them against other comments and criticisms received, and then through your own confidence. In the end, you may dismiss a comment and conclude, "they just didn't get it," but it does more good than harm to first consider why that might not be the case.

You bring up a good point; it's important to remember that you don't HAVE to do what everyone says. Nobody reasonable advocates this.

Paragraph #1:
I feel that the gap between professional and amateur criticism is much smaller than the gap between professional and amateur art. This is not a counterpoint just an observation.

Paragraph #2:
Yes, in theory. Of course the system could be improved so that review scores and feedback level and download numbers better correlated with game quality. But the two caveats are that game quality is not an absolute and that this is an ongoing process. Again, just an observation.

Paragraph #3:
Absolutely! On one level I am grateful for every piece of feedback I get that isn't overtly hurtful or dismissive. And I have to say that any positives from any criticism I've received that actual had COMPLAINTS about the work in question were much more inspiring to me than any kind of unadulterated, sycophantic praise which generally seems like it came from people without discriminating standards. The goal is to impress the toughest critics.

On the flip-side of course I think we should all be grateful for every minute of entertainment we get from other people's games- even the ones that are mediocre or merely "okay"- and the hours of work behind those minutes of entertainment. I suppose what I advocate above all else is respect for the creator in 99% of cases.

But I don't believe I actually disagree with anything you said, S.F. LaValle.

Getting a bad review is not the end of the world. It does not mean no one in the world likes your game. It also does not mean you should flip out and get defensive. If someone took the time to actually review your game it means that had a vested interest in it. If they didn't care they'd simply not say anything at all. You should be thankful that someone, anyone, took the time to comment about your game and try to help you improve.

The greatest benefit to all types of criticism is to weigh the strengths and weaknesses outlined in them against other comments and criticisms received, and then through your own confidence.

It is rare for a game to get one review in the first place, and only those games that are publicised significantly (for bad or for good) end up with more than one review. I've seen terrible games end up with lots of reviews because they are terrible and vice-versa. It is rare your "average" game ends up with several reviews unless it is publicised quite well/it is a featured game/something else like this.

What this means is that comparing the thoughts of a single reviewer with other reviewers becomes quite difficult; this leads the developer to either disregarding any single, negative reviews completely or taking them too personally - both of which are damaging. (Of course, this is a generalisation. Not everyone hates reviewers).

For instance, if the only review Sore Losers had was the one written by Darken I would be pretty annoyed about it as I wouldn't have any other opinions to draw upon. I'd have no choice but to either accept all the criticisms or decide that I am right and ignore them. However, it isn't the only review, so the negative comments in that review can be compared with general opinion from other reviews and I can come to a balanced idea of what is good/bad about the game. This is how reviews should work, there should be a good number of comments available for the developer to draw upon and form an opinion from!.

An example of reviewing being inadequate would be the game linked to earlier on, Razor. This game has two reviews, one of which is quite positive and the other of which is quite negative. It is hard for the developer to take all the negative points seriously when there is someone else who has reviewed the game quite positively and vice-versa. Therefore, depending on the personality of the developer, they are either going to take all the negative points seriously and ignore the good points raised or they are going to disregard all the bad points on the basis of one good review (both of these options are silly). It is very hard to balance one with the other and most developers (especially in a community were we have to work so hard just to get anything half-finished out) could be forgiven for taking the much easier, "everything is fine, look at that one review! The other guy is an idiot" option.

A further example of reviews gone wrong would be Master of the Wind. For a long time, the only review that game had was the one I had written, a review that was quite negative. If the developer had taken this personally I wouldn't have been surprised as that game has a strong fan-base and this suggests it is better than my review makes it out to be - and yet it was still the only review. When your game is rated at 2 stars for so long despite having a good fan-base it must be extremely annoying and hard to swallow. Now more reviews are available it is easy to see that the majority of people who play the game enjoy it and I just happen to be in the minority who don't - more reviews make my criticisms easier to swallow as they can be taken in the context of a majority who enjoy the game.

In short, for someone to take reviews less personally there needs to be a high enough volume of reviews available that one negative review can't get to the developer too easily. Or, on the other hand, enough reviews that the criticisms are consistent and the developer can't take them personally as they are part of a consensus (or everyone loves the game, but why would that be a problem?). Simply telling people to take reviews less seriously is naive and idiotic; it is hard to take something less seriously if it is the only feedback you are getting for something you have most likely worked extremely hard on over a long period of time.

And, no, I don't think reviewers should sugar-coat anything (anyone who has read my harsher reviews would know that). What I think is that more people need to put out reviews for the games they've played so that developers have a nice range of points to choose from and don't end up taking a single review too personally.

EDIT: This is a decent enough review guide, but I don't think people should follow a guide when they write a review. They should just write down their thoughts, however they choose to categorise them, and then give a rating at the end. Nothing to it.

EDIT2: The most important thing about reviewing is that your review standards as consistent, by the way.