If you are in need of another judge, I'd forgo competing (although I'd still write an entry just for fun).

I'd say I'm decently qualified... I've been around the community for a while, I like to think I'm pretty helpful and knowledgeable. I've been making games for 9 years and games in Java for 5. I've been a professional game developer for 3 months. I'm 22 and I've got good English skills.

If you are in need of another judge, I'd forgo competing (although I'd still write an entry just for fun).

I'd say I'm decently qualified... I've been around the community for a while, I like to think I'm pretty helpful and knowledgeable. I've been making games for 9 years and games in Java for 5. I've been a professional game developer for 3 months. I'm 22 and I've got good English skills.

Erm. Yeah. So if you need another judge let me know.

Looks great! I think you'll make a good judge.

However, since I've been away from the community for some time now, I'm really out of touch with who's who and what And since I will be participating myself, I will request that some other members in here vouch for the new judges to ensure that judges are not biased towards me.

I believe ChrisM and darkfrog need no vouching as they were judges in the previous competition.

Yeah that's certainly a good idea. People from JGO will probably spend more time being impressed with technical accomplishment, while non-programmers will probably be looking more for a fun/pretty game, which should probably give everyone a more balanced chance to excel.

ChrisM was nice enough to ask one of his friend to join the judging panel.

The new judge is Mark DeLoura.

He is the former head of Dev Relations at Sony and Nintendo, is the creator of the Game Programming Gems series of books and is an advisory board member of the Game Developers Conference.

Nice to have someone with his background on the judging panel! Let's not disappoint him!

The panel is now fully manned!

Yeah! Mark is absolutely fantastic and we are lucky to have him judge this year. Hmmm...perhaps I should talk to him about a 4K game entry into the IGF. Or, perhaps we do a 4K wall of games at JavaOne this year......

as a suggestion why don't we open also for public vote. We could have 2 judgments, one public and one private. If the winners are different we could start a new judgment

I think that makes sense. There could be a vote like in Ludum Dare where all the different game authors can vote on each other, and then the other will be determined by the judges as usual. Could be pretty interesting to do it that way.

I've thought about user generated reviewing/rating. I'd appreciate ideas about how this could be implemented without trivializing the judging process.

Also. I wanted to notify everyone in here that I'm enlisting as a judge. I was originally going to stay out of judging this year, as I was going to submit games, but since I didn't then I'd like to judge. In any case I have a gut feeling that we'll need a backup judge for the competition. If nobody objects to this I'll consider it as OK.

I've thought about user generated reviewing/rating. I'd appreciate ideas about how this could be implemented without trivializing the judging process.

Maybe just keep them separate. So there is a winner by judges (which is the real winner), but also and a winner by user ratings.

Users can already leave feedback on the games on this Forum or on your Java4K site, however, a system where a user could give it a score from 1 to 10 or something like that would be nice, just like you can with youtube videos.

Also. I wanted to notify everyone in here that I'm enlisting as a judge. I was originally going to stay out of judging this year, as I was going to submit games, but since I didn't then I'd like to judge. In any case I have a gut feeling that we'll need a backup judge for the competition. If nobody objects to this I'll consider it as OK.

Cool! Disappointed we didn't get to see your game(s), but you'll redeem yourself by being a judge!

I've thought about user generated reviewing/rating. I'd appreciate ideas about how this could be implemented without trivializing the judging process.

Also. I wanted to notify everyone in here that I'm enlisting as a judge. I was originally going to stay out of judging this year, as I was going to submit games, but since I didn't then I'd like to judge. In any case I have a gut feeling that we'll need a backup judge for the competition. If nobody objects to this I'll consider it as OK.

The only way to make user ratings useful is to somehow make them play and rate every game. The last time user ratings were used to determine the winner most users only played the first 10 or 20 games from the alphabetical list. My game started with a "T" so it got only a couple of plays from the all the user jusdges. That is why I am against community voting because most users have trouble getting through all the games. I know I had trouble the year they did user votes.

The only way to make user ratings useful is to somehow make them play and rate every game. The last time user ratings were used to determine the winner most users only played the first 10 or 20 games from the alphabetical list. My game started with a "T" so it got only a couple of plays from the all the user jusdges. That is why I am against community voting because most users have trouble getting through all the games. I know I had trouble the year they did user votes.

That's exactly one of the reason I haven't implemented community/user voting. I can remember how disappointed I was with the results (2007 I believe), because half the games got 0 score, and no reason was given. With the judging process all games get a fair shake, all games get a review by multiple judges, all games get played, and all games get a score. With user voting what will stop a user from simply voting for 2-3 games and ignore the rest?

If we were to add the option of user voting then that option would simply supplement the judging process, never substitute it.

If we were to add the option of user voting then that option would simply supplement the judging process, never substitute it.

I don't think anyone was suggesting otherwise. The impression I got was that some people wanted to be more Web 2.0-ish rather than to replace the judges.

The solution to the issue of users not rating games is fairly simple: scale each user's rating by a function of the number of games they've played, and normalise them. The function should probably be of games on the site as a whole rather than games in a given year. I suggest something along the lines of

1

f(x) = x > 20 ? 1 : x < 4 ? 0 : Math.pow(0.9, 20 - x)

For normalisation, don't display any scores (show a placeholder "Insufficient ratings") until you have N people with more than 20 ratings. Then take the mean of their mean ratings as the mean of the normalised rating, and the mean of the variance of their ratings as the variance of the normalised rating. Possibly re-evaluate these once a month.

Do a Youtube to the extent that if someone has rated a game then you show them their rating rather than the overall one. That way they won't notice that their first rating had weight 0. If a game has fewer than 10 ratings show the placeholder.

Note: I'm not a statistician - I have an A/S in statistics and nothing more. Someone who really knows what they're talking about can probably suggest improvements to this scheme. In particular, it needs a workaround to the fact that each person's distribution will probably be non-normal - if ratings are 1 to 5, then I expect 2 to be less used than 1 or 3.

The judges list was actually the thing that pushed me over the edge in deciding to enter this competition, so I'd like to see that stay the same.

However, I wouldn't mind seeing a user judging system if it was kept separate. Perhaps ratings could be weighted by the number of games the rater rated? You could just do a weighted average with %reviewed as the weight. For example, if there were 10 games, and your game got a 5/5 from somebody who rated all of them, a 4/5 from somebody who rated half of them, and a 1/5 from somebody who just rated yours, you'd get a user rating score of (5*1+4*0.5+1*0.1)/(1+0.5+0.1)=4.4375.

Since the raters would all know this is how it works, it'd also be motivation to rate more games, as it makes your vote count more.

I don't think it'd motivate people to dole out random ones. Your motivation in sending out extra reviews would be that you want your voice on one game to be heard more, so applying random reviews is counter to your goal: it muddies the whole scoring system, so your review matters less.

If somebody came in and gave one game a rational, fair-minded view, and wanted to increase what they have to say, the natural course of action would be to go do that to other games.

But if they hated your game, they'd probably just go say they love the rest of the games to broaden that impact. Or vice-versa. So that's definitely a weakness in the system.

java-gaming.org is not responsible for the content posted by its members, including references to external websites,
and other references that may or may not have a relation with our primarily
gaming and game production oriented community.
inquiries and complaints can be sent via email to the info‑account of the
company managing the website of java‑gaming.org