Gp2x Community Game Contest!

[Magnulus]

Okay, so if I understand this correctly, it will be like thus:

If there are fifteen finalists, and none of them have a judge attached, there will be a scale from 0-14. If one of the judges DOES have a game, it's 0-13 for everyone. If there are two, 0-12. three, 0-11. Am I getting this right? So if all but one judge are also finalists in the competition, there will actually only be a single point to give out?

No, I'm being silly, probably. What you meant was that if one judge is also a finalist, it goes to 0-13, but this stays consistent as long as there are no judges that have more than two games, right? If there's a judge with TWO games, the scale will move to 0-12. Am I getting it right NOW?

Next issue: Ok, so we have a pile of ratings. How are they combined to produce a winner? Just add up the scores and highest total wins?

That was the idea, yeah.

 

[Dzz]

Okay, so if I understand this correctly, it will be like thus:

If there are fifteen finalists, and none of them have a judge attached, there will be a scale from 0-14. If one of the judges DOES have a game, it's 0-13 for everyone. If there are two, 0-12. three, 0-11. Am I getting this right? So if all but one judge are also finalists in the competition, there will actually only be a single point to give out?

No, I'm being silly, probably. What you meant was that if one judge is also a finalist, it goes to 0-13, but this stays consistent as long as there are no judges that have more than two games, right? If there's a judge with TWO games, the scale will move to 0-12. Am I getting it right NOW?

Your second paragraph matches my understanding. No judge can score his own entry but all judges need to give out the same number of scores.

I think it would be better to just say "entrants cannot be judges", but if we don't want to do that, this scheme should work (though it has a significant penalty for entries made by judges).

 

[X-Code]

A judge can't judge his / her own work over the work of others in an objective manner, this means that a judge may not be able to judge him / herself but could judge others unfairly in order to reach his / her goal... I do think that judges should not be entrants, and I doubt that this will suppose a problem, after all , people can be asked if they want to be judges first :lol:

 

[PSyMastR]

How about, a judge cant submit entries?

 

[X-Code]

I've been saying that for quite some time, it is the best solution B) , a judge can't submit " OR BE RELATED " to any entries :lol: .

 

[PSyMastR]

Seems fair. Ok, if Magnus agrees, then its set.

 

[Magnulus]

I've been saying that for quite some time, it is the best solution B) , a judge can't submit " OR BE RELATED " to any entries :lol: .

In a community like this, that's near impossible. I have contact with people who are developing games for the GP2X both on this forum and off it, and I'm pretty damn sure a lot of people know MORE than me. I also have personal biases toward some of the the developers I don't necessarily KNOW here. So you can't really remove all bias in any way other than if you recruited judges solely from outside the GP2X community.

However, you CAN'T judge everybody else down with the proposed system of judging. There's simply no way, since you can't give more or less than one of each point. I really don't want to exclude entrants from judging, but I suppose that's the only fair way of dealing with it.

So, fine. No judging entrants. Agreed.
*thumps the table with a hammer*

 

[Dzz]

I've been saying that for quite some time, it is the best solution B) , a judge can't submit " OR BE RELATED " to any entries :lol: .

In a community like this, that's near impossible. I have contact with people who are developing games for the GP2X both on this forum and off it, and I'm pretty damn sure a lot of people know MORE than me. I also have personal biases toward some of the the developers I don't necessarily KNOW here. So you can't really remove all bias in any way other than if you recruited judges solely from outside the GP2X community.

However, you CAN'T judge everybody else down with the proposed system of judging. There's simply no way, since you can't give more or less than one of each point. I really don't want to exclude entrants from judging, but I suppose that's the only fair way of dealing with it.

So, fine. No judging entrants. Agreed.
*thumps the table with a hammer*

Good work! I think you've got everything nailed down.

 

[Magnulus]

Thanks a lot! Psy has updated a bit of stuff on the page, like the rules and such. I will compile our discussion on judging today or tomorrow, and Psy will update the page with that as well, and after that, we'll see how you like the rules and everything else.

Hmm... there was something else I was thinking about that we needed to clear up, but I can't remember what it was.

 

[X-Code]

As all seems excellent up to now, I would suggest that everyone with a spine and a bank account gets some money donated! :lol:

Don't make me go around chasing you with a couple of my nasty friends, capisce ? B)

 

[Magnulus]

Don't make me go around chasing you with a couple of my nasty friends, capisce ? B)

I'm reminded of the fantastic Monty Python joke...

"Nice military base you've got here... It would be a shame if anything were to... happen to it."

 

[PSyMastR]

Don't make me go around chasing you with a couple of my nasty friends, capisce ? B)

I'm reminded of the fantastic Monty Python joke...

"Nice military base you've got here... It would be a shame if anything were to... happen to it."

LOL. I have season 2 on DVD. That skit was so funny.

 

[Magnulus]

I've written up a work-through of the judging which lies on my own computer at home. I'll send it to Psy tomorrow to put it on the website so you can have a look and a talk about it. What you'll notice is that I've decided to accept applications for the judging panel. This is due to the new judging system that also requires a few more judges. Fifteen, preferably.

Don't send any applications just yet, though. I'm right now working on a sample review to use as a sort of template for applicants and future judges. It's sort of important that the reviewers are capable of both rational judging and of sticking to "protocol", all in the interest of the quality and efficiency of the judging process. When this sample is available, you're free to apply.

We're finally approaching the last stages of the laying down of the rules and the approaches of the competition. I couldn't ever have come this far without the help of, of course, PsymasTR. But I also want to thank everyone who's contributed to this thread, especially Dzz. Your level-headed scepticism and questioning have been instrumental in making the competition what it's becoming.

Thank you.

 

[X-Code]

I hate bumping a topic ( this should be a sticky now :lol: )

 

[tshroom]

Yes. make this sticky.

 

[BradN]

If you want a suggestion for how numeric scoring could work, I think the point system mentioned above could easily be tweaked to allow for non-self-judging.

Let's say there are 5 entries; one by a judge. Four people obviously get to vote all entries 1-5, but let's switch it so instead of points, it's percents. They vote 0%, 25%, 50%, 75%, 100%. The fifth judge (who has a game entry), only votes for the other four entries, with percents of 0%, 33%, 66%, 100%. The winner has the highest average percent.

Does this seem sensible enough? The judges could still vote numerically (1 through whatever number), but the person who tabulates the results just has to make a spreadsheet or some such to apply the correct percents. Make sure it's clear to the judges whether 1's or 15's are "high".

 

[Magnulus]

Personally... I think we'll stick with the proposed plan. It's the simplest way of doing it, and whenever I've "tried" (unwittingly) to complicate it, we've basically simplified it back down again. Simplicity is key to avoid confusion.

Since I've not seen Psy online yet, here's the compilation:

JUDGING

A panel of fifteen judges will be assembled, all of whom are excluded from entering their games in the contest. If you wish to join the panel of judges, please submit a written review of at least one game that has been released for the GP2X. The review should follow [[this formula]].

Once the deadline (december 11th) has passed, the judges will each give a single vote to fifteen of the entries, and the fifteen entries with the most accumulated points will go on to the final round of judging.

A week after the deadline, all entries (including the ones that didn't make it to the finals) will be released for download, and the community will be given the chance to rate the entries themselves, deciding a winner of the special Community Favourite award, the prize of which is dependant on what support we will get from sponsors.

In the final round, the judges will range the games from the best to the worst in four categories[[*]], giving the best game a score of 14, and the worst game a score of 0. Each judge can only give each score once in each category, meaning that no two games can be rated the same by the same judge in the same category. This follows the style of judging from the Eurovision song contest, but with the added twist of separate categories.
The judges will also be required to write short reviews of each game according to the [[formula]] mentioned above.

Once all judges have submitted their ratings and reviews, these will be added together to decide the winner. The final score of the games will be the sum of all four categories.

The winners will be announced on the 12th of January, 2007.

* The categories are as follows:
Playability
Innovation/ Originality
Use of Graphics
Use of Sound


PS: The double brackets signify future links.

 

[Dzz]

Looks good! What if you can't find 15 people willing to do all that? What if some judges sign up but don't finish?

 

[Magnulus]

Looks good! What if you can't find 15 people willing to do all that? What if some judges sign up but don't finish?

Then there will be fewer judges, I guess. Not much more to do about that.

 

[Megagun]

I don't think finding 15 judges would be THAT hard, at least not if not everyone is thinking about submitting a game..

Myself, I'd join if I'm not going to submit a game, which I currently don't know if I'm going to, because I have no idea if I would be able to get the required motivation and knowledge needed for the job.. :/