Build a Better Review System, Templates might help -- but they just as easily hinder Options

[ZarroTsu]

Wanting to write reviews, I decided I'd use the strategy discussed by one Yatzee Croshaw here.

The problem is that, because I don't give an "overall rating", this kind of review is rejected from the forum.

What reason is there for there to be an "overall rating"? Isn't it suppose to be what I think of the game? Giving it an overall rating because that fits your format is fine. But forcing me to follow this strategy isn't asking for what I think of the game; it's asking me for what you WANT me to think of the game. And that defeats the purpose of writing in general.

Moreover, their presence from a reader perspective is solely to ignore the rest of the review and judge the game on a given number. Some here might say they don't do this, but often times even they are guilty of it. Even I do it. But if these numbers don't exist, then it compels me to actually read the review. So...


Generic forum discussion question for the reasoning impaired:
Why do "Overall Ratings" exist? Are they there to help, or are they there to defeat the purpose of writing a review when you could just write a number?



This post has been edited by ZarroTsu: Oct 4 2012, 02:51 PM

[X-M-O]

I'd love to hear more discussion on this topic, as ZarroTsu mentions we currently have only allowed community reviews to be approved if they have an "overall rating".
It was originally considered a requirement based on the idea that members prefer having a conclusion based on a frame of reference that most members are able to easily relate to (ex: 1 to 10 rating).

ZarroTsu has a few questions that make sense, but what solutions are there?
Well, here are several that could replace the "overall rating" requirement:
1 - No rating requirement. No frame of reference.
2 - A subject-based rating requirement (at least one aspect of the game would be required to be given a 1 to 10 rating, ex: Gameplay would be given a 1 out of 10 rating or Story would be given a rating).
3 - More than one subject-based rating requirement (same as above, but would require more than one aspect of the game to be given a 1 to 10 rating).

Any other ideas?


I suspect that not requiring a review has potential problems, for example most members don't spend more than a few seconds reading a topic and will simply not read the review if it doesn't supply a rating for the game it reviews.
Also, without a frame of reference such as a 1 to 10 rating, a member may be unable to decide if the review gave an overall positive or negative perspective. Sometimes it's obvious, other times it's 50/50 and you really aren't sure where the opinion truly lies.
Of course, these might or might not be concerns for many members, but we won't know unless you tell us!
So join in and help discuss the issue!
(Unless of course you want us to decide everything without you and we won't be as willing to listen to complaints if you never contribute to the conversation. In other words, if the topic matters to you, then speak up!)

[ZarroTsu]

Also, without a frame of reference such as a 1 to 10 rating, a member may be unable to decide if the review gave an overall positive or negative perspective. Sometimes it's obvious, other times it's 50/50 and you really aren't sure where the opinion truly lies.

If they were smart though, they'd take the review as a presentation of the ideas present in the game, and decide if they want to play it from there. If the review danced around too much to convey what it needs to be considered a review, then it may as well not be accepted.

[Shaddow]

Also, without a frame of reference such as a 1 to 10 rating, a member may be unable to decide if the review gave an overall positive or negative perspective. Sometimes it's obvious, other times it's 50/50 and you really aren't sure where the opinion truly lies.

If they were smart though, they'd take the review as a presentation of the ideas present in the game, and decide if they want to play it from there. If the review danced around too much to convey what it needs to be considered a review, then it may as well not be accepted.

This may be true, but a lot of people who come on here are just kids, or do not really want to read all the information. Sure they SHOULD be putting thought into reading the review and what it says about the game, but not everyone wants to spend that time and it is not our place to judge them for not doing so. That's why I think a numbered review makes sense.

Now don't get me wrong, I love that we need to put details into the actual review, it shows WHY we rated it what we did. I also think that the number system is not enough information. We should take a note from formal gaming magazines and put a rating system on each section, as well as a summary of the reasoning. I.E.:

Sound 7/10 - While the sound was interesting and well placed, it wasn't anything that hasn't been heard before, a lot of the RTP was used and it could have been more unique.

I hope this makes sense.

[The Welsh Paddy]

I don't see why an 'Overall Rating' should matter too much, but... It would only take a few seconds to give one, so what's the big deal about not giving an overall rating?

Say you give the story a 8/10, graphics 5/10, sound 6/10 and gameplay a 3/10, then the overall rating would be a 5.5/10. It's easy to find out the average of all the ratings you've given per category.

If yer not sure how to figure out the average of a sum, then you need to add up each rating and then divide it between the amount of ratings... if you get what I mean. In the example I gave, it's 8 + 5 + 6 + 3 divided by 4, because there are four numbers.

This post has been edited by The Welsh Paddy: Oct 4 2012, 11:24 PM

[X-M-O]

I don't see why an 'Overall Rating' should matter too much, but... It would only take a few seconds to give one, so what's the big deal about not giving an overall rating?

Say you give the story a 8/10, graphics 5/10, sound 6/10 and gameplay a 3/10, then the overall rating would be a 5.5/10. It's easy to find out the average of all the ratings you've given per category.

If yer not sure how to figure out the average of a sum, then you need to add up each rating and then divide it between the amount of ratings... if you get what I mean. In the example I gave, it's 8 + 5 + 6 + 3 divided by 4, because there are four numbers.

At current, we haven't had any reviews give ratings to each category/aspect of the game (such as story/graphics/etc.), we've only had them give an overall rating without giving individual ratings.
So what you're suggesting is basically number 3 from the list I mentioned above:

3 - More than one subject-based rating requirement (same as above, but would require more than one aspect of the game to be given a 1 to 10 rating).

And that makes sense, I just wanted to clarify. =]


Also, I think Shaddoval is implying we have a mixture of overall rating and #3.
Am I correct in assuming that?

[Shaddow]

Yes! Exactly that.

[X-M-O]

All right, just trying to clarify. ^^

It's completely reasonable to require a "grade" or "score" for multiple design categories/aspects of a game when reviewing the game, and I'm not at all opposed to making that a requirement.

I also agree that we cannot be presumptuous and assume that members will (or should) read every bit of text within a review, especially if they are just trying to confirm that the reviewed game is worth trying out. Most members hardly spend time going through all that just to play a game, and will likely download it and start playing by the time they've read most reviews. As this is primarily the case (and the average member does not read more than a few lines of text anyway) I am not convinced that we should be without the overall rating requirement (however I am convinced that adding ratings for different aspects of the game would be beneficial). =]

[KD648]

If I understand correctly, and if you're like me, the issue you take with being forced to provide with in overall score is that it allows, and in fact encourages, the reader to make a snap judgement about the game. Especially when the overall score is an amalgamation of separate, smaller scores. As an example, on metacritic the Elder Scrolls V: Skyrim recieved a 94, whereas the original Bioshock sits at 96. Therefore, Bioshock is better than Skyrim. We all know that a game is more than just the sum of its parts, but I also think none of us want to pretend that everyone is going to read every bit of every review.

An in between, for me, is to offer scores for different aspects of the game, but than rather than have an "overall score", rate the game on a "recommended" scale. Just say upfront whether or not you think they should play it, and then write a sentence about why. As an example, I'm going to fake review the two games I just said.

Bioshock (2007)
Gameplay: 10 - Tight FPS action mixed with RPG elements are balanced, fun, and frantic.
Story: 9 - Great writing and atmosphere suppliment a pretty straightforward story.
Sound: 8 - Ambient sounds are incredibly well done, but the minimalistic music is forgettable.
Graphics: 10 - Rapture is stunning and terrifying, all at the same time.

Rating: Recommended - Bioshock doesn't revolutionize the genre, but great atmosphere and polish put it far above the rest of the FPS crowd.

The Elder Scrolls V: Skyrim (2012)
Gameplay: 8 - Combat feels much better than it did in past iterations of the series, but specific instances like riding a horse still feel awkward.
Story: 8 - The lore behind Tamriel is as solid as ever, but the plot is really just a well-written excuse to explore the sandbox.
Sound: 9 - The music is epic, and the voice acting is highly improved... with a few exceptions.
Graphics: 9 - Skyrim is georgeous, but animations still feel a bit awkward. Play in first person.

Rating: Highly Reccomended - Technical bugs included, Skyrim is the best 300+ hour long sandbox RPG on the market and an experience unlike any other.

This way people can still add up points if they so choose (Bioshock is the winner obviously), but you still have a shortcut that explains why a game may be more or less than the sum of its parts.

P.S. Bioshock is awesome and one of the best games I've ever played, so the "Reccomended" vs "Highly Reccomended" thing is just to illustrate a point.

This post has been edited by KD648: Oct 5 2012, 12:38 PM

[Shaddow]

I actually like your idea even better than my own, it makes sense and still covers everything I wanted to cover with my idea. Not to mention it works for those who make snap judgements as well, as these reviews really need to appeal to all types.

[ZarroTsu]

Say you give the story a 8/10, graphics 5/10, sound 6/10 and gameplay a 3/10, then the overall rating would be a 5.5/10. It's easy to find out the average of all the ratings you've given per category.

If yer not sure how to figure out the average of a sum, then you need to add up each rating and then divide it between the amount of ratings... if you get what I mean. In the example I gave, it's 8 + 5 + 6 + 3 divided by 4, because there are four numbers.

To quote Yatzee's conclusive paragraph (I did link this article in the opening post),

You know, I have a long-standing grudge against the concept of awarding review scores to games, because I think it represents everything that's wrong about videogame reporting by treating every given game like some kind of kitchen appliance whose chopping blades have been slightly rearranged since the last generation and are now therefore precisely 1 point more efficient at dicing sweet potatoes. But if I did finally knuckle under to those bean counters at Metacritic, this is exactly how I'd give scores to games. Three separate marks out of ten for Context, Challenge and Gratification.

None of this buggering about with graphics or sound or anything else as consequential as the color of the wallpaper in an operating theatre.

Of course what I would definitely not do then is combine the three scores into some kind of "overall" value, because that's totally fucking meaningless. That'd be like having a meal where the main course was tasty but the dessert was disgusting, so you give it a final review of TASTGUSTING.

[amerk]

The problem isn't just the scoring, but in how you review each game. You've reviewed 3 games, and all of them you've rated terrible at 1/10 star. There needs to be justification for that. You mean to tell me there is not one good quality in any of those 3 games? A 1-star rating means it's unreadable, unplayable, horribly designed, full of bugs and glitches.

But I get the feeling that the reviews you wrote (and have not been approved) are nothing more than an attempt to flame the developers, especially when you start spouting off insults. Reviews that do nothing more than incite flame will obviously not get approved, regardless if there is a rating attached to it or not.

Had you made some of these comments in the regular forums, chances are high you would have been suspended or banned.

[Jonnie19]

As Amerk has said, these "Reviews" aren't really reviews they are just an excuse for you to slate the developers. Reviews are meant to help the developers. There is a difference between harsh critique and just pure slating. Sometimes it can be a fine line, however it's not the case in all three of your reviews. You do not give any ideas on how the games could be improved without this it is just flaming, which of course isn't acceptable in any format.

It takes very little effort to add in constructive points. and from a developers point of view, gives a developer a feeling that they are actually going somewhere.

[ZarroTsu]

I reviewed them against the standard of the RPG Maker game, Exit Fate, if I'm to give a standard. To be frank, there's nothing stopping any game made in RPG Maker from being just as good an an officiated console release. So maybe I'm choosing some especially bad, in my eyes, games for me to review. And, comparatively, I really and honestly found neither that I reviewed very good. The 'Overall score' I added to the end was more vague summary than anything, and was only really there to pass your 'requirement' in that case.

Both these games, Zandir and Nexus, come from completely opposite writing standards. Zandir was written quite well, where as Nexus looked much like a poorly done fan translation of some obscure Asian game (maybe it was, I'm not one to read topic background). But neither game really stood to hold my attention, because neither game seemed to understand how to establish a story for the sake of grabbing the player's attention.

I suppose I was thinking things more on a standard of 'actual game' as opposed to 'constructive criticism', but the fact stands that both these games were considered "Complete", and were placed in that designated forum. Maybe I'm wrong, but that seems like an invitation to hold high expectations for what I might hope to see out of the game -- at the very least an engaging story that keeps me wanting to play longer than the first boss.

And I don't expect perfection, I just expect it to BE there. A proper framing setup to let me know what kind of game it will be, and how dedicated the maker was. In either case, they very much danced around these things and made them hard to take seriously, causing everything to smash face-first into the ground when the main characters acted not only moronic, but UNLIKABLY moronic. Within what would be the first chapter of a novel, I was given no pretense to the setting, and characters I could not bring myself to like. Perhaps at that point, I should have simply denied a review -- but there's no reason not to bring these things to light.

In either review, while decidedly negative after my negative experience with the game, I did pose several serious questions that the author of the game could do well to ask themselves. Why ARE Golems hanging out at an abandoned, burning building in Nexus? Why IS this building underground? I could forgo answering this question with the obvious "hidden mansion of wizard cultists", which I'm throwing out there without playing any more of the game than I did in-stream (and am probably entirely correct), but what was to stop the protagonist from acknowledging this himself rather than just focusing all his attention on a goal without stopping to investigate anything CAUSING that goal to BE a goal. Why DO the bats run away unprovoked during random encounters, as the very FIRST enemy in Zendir? What possible reason would there be to punish the player for seemingly no reason, just because I missed a minor plot detail that the game failed to convey coherently?

While these are more questions that I could pose in my review, the firstmost of either series is a question enough. Maybe I came across as condescending in a way some users may not understand how to deal with -- but it's something they could stand to learn as well. To look on a review boasting a severe number of negatives, and understand precisely why these are negatives. To correct them, not because they were asked to correct them, but because they were asked why they weren't correct.

So maybe I should be looking for games that understand how to draw player attention to the game in the game itself from the get-go, or maybe I'm looking at things way more strictly than I should. I said before the final score was there just to appease you, but I suppose 1/10 was far too low, when an "F" is anywhere from 0 to 50 on a scale of 100. I'll consider reevaluating this if the "Overall score" stays a requirement (otherwise I can remove it outright), and I'll also work on toning down my language on the Zendir review.

Nexus, however, REALLY got me angry with it, so I won't be resubmitting that review. For reference of when and how, you can see the exact moment I got thoroughly ticked off in the blind stream of it, within the last 10 minutes of the video.

[Shaddow]

I'm glad that you posted what you truly thought about the game, this CAN help a developer. One thing I did notice is that you state you don't have enough room in a review to point out all the things you found wrong. Do not forget, if you truly feel these need to be fixed that you can pm the author or even post it on their game topic.

I know that personally, I do not always notice when I make a mistake, or something that makes sense in my mind may not make sense to others. Yes they should have gotten a beta tester to review the game, but that is not always an option. THAT is why this forum exists, so we can play and help each other grow as developers.

Personally, I appreciate that you are even doing reviews, and I wish that more of our members would contribute in that way. I also am appreciative that you realize you are being a bit harsh and are willing to tone it down some. I also understand why you dislike our format, as it can be rather restrictive, but that is why we are having this topic in the first place, so we can find a better method.

I guess the bottom line I'm trying to say is, that we do not dislike what you are trying to do, but rather how you are going about it. I'm sure if someone were to attack your reviews in a harsh manner, assuming there was cause, you would not appreciate it either. I'm not saying to sugar coat things, because sometimes you have to be blunt. There is a difference, though, between being blunt and just being mean.

[Jonnie19]

As Shadowvaal said, we do appreciate the effort you are putting into reviewing games from this site, and we do invite you to do more reviews, because it's currently rather quiet.
As a developer and reviewer myself, I'd rather have someone being blunt rather than sugarcoated. Sometimes however there is a fine line between being blunt and flaming. You also should ensure that you at least read the topic post. In the case of Nexus if you had read the topic post, you'd have noticed that English isn't his Native Language.
Getting a balance when reviewing a game can be hard. Try not to focus just on what you feel are terrible games. There are some brilliant games around, Try reviewing them too...It will mean you don't end up having your reviews ignored, because some readers will think you are just wanting to review bad games, and will not actually like to get reviews by you, whereas a balanced but honest review means people will come to you for reviews!

A good reviewer on this site is actually Amerk. His review of Spirit Timer was fair but honest. I messed up with quite a few things, however he handled it with stating the BIG game breaking issues without saying that it was a complete disappointment of a game, and that he hated me for making a game breaking bug. Obviously this is a different style, but using better language helped me to work on my game and develop it better and further, because of his honesty

I personally love your style of reviewing, it has a great structure. It's in depth and honest/blunt and I'm looking forward to seeing more.

[amerk]

To echo what others have said, I try to find the good and bad of all my reviews. It's a hard balance to do, but rather than take my own experience I also try to decide if it really is a bad game, or if it's just something I personally wouldn't like. Not so much recently, but I've played plenty of games that I don't like because it's not my thing (real-time strategy, as an example).

But when it comes to reviewing, I try to put myself into the mind of the casual game player and take my own likes and dislikes out of it. I also try to find a few good things as well, just to enforce some possible potential. It helps to keep in mind that most of the developers in fan communities probably won't have the experience in commercial development, and most will use the RTP (at least in terms of graphics). However, I also pay attention to how they are used, and if any attempt was made to go outside of those graphics. There are some games available that really do shine above others, and if that's what you are aiming for, you may want to look at those.

I don't want to point names, but since Jonnie19 opened himself up already, I'll pick on him. Was the game bad? No, not by any means. But if you look at the score, you may think it's a bad game. This is one of the reasons we are in discussion about changing this up (we'll announce the changes once we get approval). The game is unique and has plenty of potential, and luckily it's in demo stage, however, it also has plenty of mistakes that prevented me from going much higher in my review. But there's a lot more there (currently as a concept) that will make me give it one more chance when it's finished.

In the case of Crysalis, I haven't played it much myself, nor do I know who made the game. My guess it was made more with a casual audience in mind versus the professional. Since VXA attracts more hobbyists than commercial-like, they probably wanted a game that the casual maker can open and study from, which is why it doesn't showcase VXA to the max. Without having played the game, I think it was more of a tutorial type game to teach the basics of using the maker. A good follow up would be for them to make a more hardcore game that stretches the system to the max for advanced training.

Now, some games are so bad that there's really nothing to sugarcoat. Typically, I tend to steer away from those games because I know my overall review would be very negative. But somebody has to review them, if not for any saving grace on the developer's part, then to inform the audience why it's bad and why it should be avoided. Keep in mind, though, that you are attacking the game, not the developer, so avoid insults that may be considered flaming. Most of the time, the reviewer uses a more humorous approach for these kind of games. Look at some of the "Let's Play" reviews on youtube. I can't believe some of the games they are willing to play. Instead of lashing out at the developers, though, they make it fun and enjoyable by doing voice makeovers, and cracking jokes at the expense of some of the scenes or designs.

It's a bit of a trick reviewing a bad game in a typical post without coming across as insulting, but it can be done. Rob_Riv used to do these kind of reviews a lot on his/her own personal blog. Screenshots and funny humorous captions helped.

Now one final point, more to Jonnie19's comment, I understand if English is not a person's native language, however, that shouldn't excuse bad writing and grammar. It'd be up to them to hire a proofreader to help them along before displaying their game, if for no other reason than to avoid a comment against their writing mechanics. I have not yet played Nexus (it's in a to-do list with a long list of games going as far back as 2009), but if it's as bad as I've heard people say then chances are high I won't care to play much past the first hour.

But again, even if I don't bother finishing a game (bad grammar can be a deal breaker for me), I have to consider if the game play, mapping, and balance were at least subpar.