Ideas For A Gp32X.com Summer Competition

[Magic Sam]

@ iprice: I understand your fears

I agree, judges can't own all the machines...

EDIT: removed all my bad suggestions, what about Jan-Nik's idea, further down the page ?

Bye, Magic Sam

 

[glezmen]

i'd suggest to select the currently active consoles (Wiz and Caanoo), they are almost identical, and they are comparable with each other

 

[Magic Sam]

Hi glezmen

i'd suggest to select the currently active consoles (Wiz and Caanoo), they are almost identical, and they are comparable with each other

This is intended to be a gp32x.de Community Competition, reducing it to only 2 "active" platforms would be a bit sad don't you think ?
Bye, Magic Sam

 

[glezmen]

Hi glezmen

i'd suggest to select the currently active consoles (Wiz and Caanoo), they are almost identical, and they are comparable with each other

This is intended to be a gp32x.de Community Competition, reducing it to only 2 "active" platforms would be a bit sad don't you think ?
Bye, Magic Sam

that's true, but see iprice's comment (i wanted to reply to that post, but i forgot to click on 'reply')
you can't really compare a gp32 game with a caanoo one (i'm speaking about two different games)

a judge most probably has either the gp32 or the caanoo, not both, so he/she will try only one of the games, the other judge who has no gp32 will try only the caanoo game. if the first gives higher score to all the games, and the latter gives lower scores (because he/she has different opinion about the weight of the gameplay/design elements, etc), the competition is not balanced

the sad part is, there's no good solution for the problem
(if all the judges had all the consoles, that would be awesome, but i really doubt that's possible)

 

[Jan-Nik]

What about this: Every game has to run on either 3 handhelds or on 1 handheld and Windows (so that the judges can still rate the game)?

 

[glezmen]

What about this: Every game has to run on either 3 handhelds or on 1 handheld and Windows (so that the judges can still rate the game)?

I second that (but why windows? )

having Ginge on the Caanoo would be a huge help, so the developer can build the game for GP2X too

 

[Magic Sam]

What about this: Every game has to run on either 3 handhelds or on 1 handheld and Windows (so that the judges can still rate the game)?

I second that (but why windows? )

having Ginge on the Caanoo would be a huge help, so the developer can build the game for GP2X too

+1 Entries have to run on at least 1 community handheld (obviously) and their sources must compile on desktop computers (Windows, Linux, MacOS, etc...) for judging purposes.

@ Jan-Nik: I have updated my first post with your idea I've also received your PM, please post the URL here when it works

Bye, Magic Sam

 

[Iorgy77]

A possible solution to the multi-platform judging dramas is to allow judges to only judge one platform. If I ever did another contest I would do this. It stops judges from unfairly favouring one console over another. Also the more judges you have the more a fair average result can be obtained, if one person out of 6 for example gives 9 points out of 100 and everybody else is giving lets say 60-70 it doesn't throw the scores out of whack like it did in the PACC2010 compo where some platforms had as little as two judges.

 

[iprice]

There are a few problems to having a pc/Mac etc. version of the game to compensate for not owning a particular console -

1 - Pc controls are so much more precise than any of the handheld machines. An example of this is my Balloonacy game. On pc it's piss easy with the cursor keys. On Wiz, it's a real challenge. Sure you can play it, but the experience is VERY different.

2. What about touchscreen games? I've developed several apps for touchscreen (all of which can be emulated with a mouse), but they never feel the same. The gameplay just feels better with your finger physically controlling the action on-screen, rather than via a mouse.

3. Graphics. A handheld game will use perfect pixel imagery. A pc/Mac etc. can either do the same (in a tiny, near-no-playable window), or scale the graphics up. This can have a negative affect on the user's perception of the graphics (tiny or blocky - or even worse, smoothed with some awful shader).


With so many potential formats, this really is a major problem. Having a different set of judges for each machine is the better option, but again, what if someone sends in the same app for two or more devices? Would you limit it to only one entry of any particular game? this means that you could still develop for GP2X and Wiz, but the games would have to be completely different.

 

[Iorgy77]

With so many potential formats, this really is a major problem. Having a different set of judges for each machine is the better option, but again, what if someone sends in the same app for two or more devices? Would you limit it to only one entry of any particular game? this means that you could still develop for GP2X and Wiz, but the games would have to be completely different.

Well in the PACC the scores from more than one platform never combined on the same entry, so each was judged against the criteria alone and the highest scoring of the two or three versions would have a shot at winning. I awarded bonus points for it because I wanted to encourage multiple platforms so the whole community could benefit from new software.

 

[iprice]

With so many potential formats, this really is a major problem. Having a different set of judges for each machine is the better option, but again, what if someone sends in the same app for two or more devices? Would you limit it to only one entry of any particular game? this means that you could still develop for GP2X and Wiz, but the games would have to be completely different.

Well in the PACC the scores from more than one platform never combined on the same entry, so each was judged against the criteria alone and the highest scoring of the two or three versions would have a shot at winning. I awarded bonus points for it because I wanted to encourage multiple platforms so the whole community could benefit from new software.

That's where some of the trouble came from - some multi-format games got very different scores, because they were reviewed by different judges, even though they were identical. This should not be allowed to happen.

 

[Iorgy77]

The reason the scores were so different was due to a lack of judges as explained above. I am only making suggestions here so take it with a grain of salt. There was no trouble due to things being multi-platform as far as I am concerned and I got to see the every judges individual score for every entry. The "trouble" in my opinion was due to a lack of consistent scoring amongst the varied judges and this will happen whether things are multi platform or not. The only way to keep things consistent and fair in my opinion is to have more judges, an equal number for each platform and use an average score method to achieve the final result for each entry.

 

[Jan-Nik]

Maybe judges shouldn't give points but a ranking?

 

[iprice]

Lorgy: I think it's difficult to see things from each's perspective as you were at one end of the spectrum and I (and others) were at the other end. I'm a Manager over at reRroRemakes.com and judged most of their competitions, some of which were huge (with £thousands in prizes), and this issue never came up once as we ensured it didn't. Multiple entries of the same game were not allowed.

In no way do I hold you responsible or accountable for inaccuracies/problems in judging, but it's better to prevent a problem, than attempt to cure one after the fact.

Preventing the same game being entered in two formats would remove this problem completely - irrelevent to the number of judges there were (and remember, you had difficulty getting reliable judges in the first place IIRC - it took way too long to get the results). What makes you think that there will be 5 or six people wanting to judge a comp like this nowadays? There seem to be far fewer people here now than ever before (for various reasons)

However, even refusing the same entry on separate formats causes issues - the coder would have to think tactically and decide which format his/her game would have the most chance of winning on?

All: It might be a good idea now to put out feelers for who would like to judge (and which formats they can judge on), who might participate. If no potential judge's own say, a Caanoo, then that format can be removed from the competition immediately.

Timescale-wise, anything under two months is too short, anything over three months is too long. Start date, I'd say Jan 1st - New Year, New game comp.

Anyway, I've said my piece(s) - I'll leave it upto others to voice their opinions now

 

[Iorgy77]

It's Iorgy by the way capital with a capital i, as in Yiorgy or Georgy.

There is a pattern in the results for the PACC, all the wiz entries suffered a brutal judge who gave almost every wiz entry a really low score and at the same time we had a dingoo judge who was ultra generous. If I remember correctly these guys didn't judge any other platforms. So the results would have been the same even if some entries weren't multi-platform (besides bonus points) we would have just missed some entries.

Nether dimensional runner actually placed second and third in points as one shining example of consistency but only received second place winnings as stated in the rules you only qualify for one prize per the same multi-platform entry.

I understand you have a different point of view, I just cannot see what it is in regards to this particular issue. Please explain how entries being multi platform has any effect on the results because in the nicest possible way I just don't understand how its a problem at all. All it did in the case of PACC is illuminate inconsistencies in the judges scoring.

I am not saying it should be done either way for this competition, but it could help me come up with a better scoring system in future if I do organise another compo.

(on topic) I would also like to volunteer to judge this competition's entries if nobody objects, to whatever system the organisers come up with. I have all the systems from the PACC as well as a pandora.

edit: suggestion: perhaps a seperate prize pool for each platform?

 

[foxblock]

Well, there are multiple problems to tackle here and you probably can't get it completely right, so here is what I think:

Then main problem is that you simply cannot judge in an ordinal (or even arbitrary) scale, mostly because everyone has a personal opinion about something, one likes shooter games, the other despises them.
Additionally you need some sort of scale in a competition as you want to rank the entries, which is the point of a competition.
Probably the best way to counter this (if you want to keep the ordinal scale) is to write some more or less strict judging guidelines (what should be judged in which category with examples). The downside of this is, for one that it takes a lot of time and effort and also that some people won't like getting told how to judge.
So I also opt for Jan-Nik's idea, just have every judge make a ranking of the games (or maybe just a top 5) and then award points according to this ranking (first gets 5 points, second 4, third...). Then you accumulate these points and have a winner.
This method actually is totally subjective and does not try to put a subjective view in an objective, ordinal scale - you make it more or less objective by accumulating the votes and declaring an overall winner.

I think every judge should play every game in the competition to get the best results (the more judges the less you have the problems described by Iorgy77), but this probably is not possible due to platform limitations and differences.
Crazy ide: Maybe we could ask EvilDragon or Craig to get sent some devices to do the judging on and send them back afterwards (I am pretty sure they both have used devices which cannot be sold again for some reason) - maybe even someone discovers a new "love" and even buys the thing afterwards...

I think cross-platform development and working in teams should be encouraged, but awarding bonus points is always a double-edged sword - some people then just go for these without much good content and score better than other, better entries.
Maybe there should be a small bonus price for the highest ranked game working on the most platforms and the highest ranked game developed by a team (and you can win these alongside the normal prices).

In my opinion there also should not be multiple price pools for every platform, this just helps to split the community even further, which this competition actually should try to counter.
I know it will be hard to implement, but Jan-Nik's idea for judging might also help in this regard.

foxblock out