I doubt they will announce today. They only asked me for my activation keys on the 14th and as of this morning, none have been used. So my two game entries have not been evaluated by any of the judges as far as I can tell.

I doubt they will announce today. They only asked me for my activation keys on the 14th and as of this morning, none have been used. So my two game entries have not been evaluated by any of the judges as far as I can tell.

I'm just wondering - did the rules state anywhere that the judges would *definitely* play the games before evaluating them? Maybe it's going to be done on quality of website?

Maybe sun was overly optimistic about how many games could be reviewed in a week by 5 people (who all had to play each game). I'm going to use my rough estimate of the number of entries which I'm sure is pretty much wrong, to determine the scientific judging thing (similar to the scientific santa, only less interesting and much simpler math cause big math hurts my brain).

We assume about 250 games, just because I assume 1/3 of the people who signed up submitted and a lot of those people submitted 3 games...

Sun employees probably work 8 hour days, but they are heavily dedicated so they actually work an extra 2 hours just for fun... well, ok, maybe they only do that on weeks when their job is to play games all day... now, we yank out an hour for lunch and 30 minutes for 15 minute breaks, and then we round it down to the nearest hour to avoid any real math... so sun employees work 8 hour days (40 hour weeks, even though I'm pretty sure athomas worked a lot more than an 8 hour day and probably lost some weekend to the games contest near the deadline). Without allowing sun employees to take a bathroom break that's about 10 minutes per game to play a game, write down their judging scores based on the 4? categories in the agreement.... If every game works right away and loads super fast then.... hopefully my math is wrong cause 10 minutes hardly seems like enough time to do anything.

Which is, of course, an incredibly stupid way to run such a contest, for the reasons you highlighted (I'm not saying your reasoning is stupid, just that it's stupid for someone to try running one like that, but many do anyway).

From the position of extensive experience and wisdom on this I can vouch that this is the REAL reason for having "finals" and "semi finals" (and you all cynically thought it was so that you could do two extra rounds of press releases and triple the publicity! Well, OK, it's for that too ) .

What you normally do is:

Organizers create a 2D matrix with one row for each category and one column for each of "terrible, poor, normal, good, excellent". Each row is further subdivided into 4-8 rows which are just individual aspects of that category

Each box in the matrix has a textual description of what a "typical game" that scored that (terrible, poor, etc) in that category would be like

Round 1, each judge just has to print out one sheet for each game, and tick one box in each row

Games get e.g. 1 point for poor, 2 for normal, 3 for good, and 5 for excellent - or balanced in some other way depending upon what you are trying to judge for (overall quality, or outstanding specialisms?)

The top 20-30 scoring games go through to round 2

Round 2 progresses more or less as DanK describes, since now there are only a few games to judge. However, if you want the judges to spend more time on each game, but they are still strapped for time, you go for the IGF model, whereby each game only gets judged by a subset of the judges in this round (also, IIRC, used by the GDC organizers when deciding which conference submissions to accept...). If there are 30 in this round, and 5 judges, each judge will review perhaps 18 games, so that each game has been reviewed by three judges. These scores are used for deciding who goes to the final.

Note: because every judge has seen every game even if only for a few minutes, at Round 2 if a judge was surprised a particular game they did NOT review didn't get through to the final, they can bring it up. Usually each judge gets to choose up to one game they didn't review to "bump" into the final.

Final is usually 5-10 games, which the judges have a 1-hour or 2-hour meeting to decide on final positions. If the organizers are particularly good at their role and know what they're doing, often you can get judges to decide in half an hour (...and we often had to do this with more than 15 judges; with less than ten it's probably much easier) - but only if they're all in the meeting together (doing this by email, for instance, - or with a few people missing - is a waste of time).

Obviously, you can be lazy as organizer and put in less effort, and you get inferior judging. Or you can beg/cajole/bribe the judges to spend more of their time on it. We tended to find that the process above was an excellent way to make use of everyone's time without wastage (although we still had occasional problems with judges "cheating" and not declaring vested interests )

If the Sun guys didn't do something at least as sensible as this, i.e. made it take much longer or got much less out of their judges, then they have only themselves to blame, since they know there are people around here like me who have run big competitions before with bigger prizes than this. I have no doubt that they did something as better - they did, after all, have the IGF template available to copy , and I'm sure Mark DeLoura is an old hand at this sort of thing. It would have been nice to know though (my opinions on the importance of transparency in such things are well known around here )

I had assumed from the letter that they had simply changed the date that the contest results were announced myself, I don't think not being notified today of winning is an indication of anything other than sun probably didn't plan things out nearly as well as you'd think a company of that size would have, there have been plenty of posts where people described the contest rules as poorly thought out (and the clarification post supports that). Anyway, I figure if sun did notify winners today in accordance with the contest rules that they'd probably have asked those people to keep it under wraps, or possibly even made it a contractual obligation, there is a form that has to be sent back to sun by the winners, I remember there was a week time period for it but I don't remember any other details.

The rules outlined a judging system that rated games based on 4 things, and gameplay was one, so I'd assume that the plan was to at least try to play each game, if not I think they'd have required screenshots with the entry so they'd have some criteria for passing a game over unreviewed.

My scientific judging post isn't intended to be particularly accurate, or really taken too seriously, but it does outline that the judges are probably very busy, and probably traumatized for life from playing games...

java-gaming.org is not responsible for the content posted by its members, including references to external websites,
and other references that may or may not have a relation with our primarily
gaming and game production oriented community.
inquiries and complaints can be sent via email to the info‑account of the
company managing the website of java‑gaming.org