[Release] Compo4all MAME - ladder/scoreboard competition for classic arcade games :)

[skeezix]

It was posted to forum since it was/is an unstable test version, to test launchging from the then-current version of Tourny Hub. It didn't address any issues (none were known at the time? ;0)

I'll do a couple quick fixes to it soon and update the repo edition for a new baseline.

(Will probably add to it to only sync invoked game, and not look for other possible games; or perhaps sync at leat ones recently changes, not just all of them.) Should help cut down on cycle in/out of game time..)

jeff

 

[Kumaki]

I'll wait to see if it works for other people before updating mine.

Updates tend to scare me a bit.  If the new version has bugs, then often you can't go back to the older version that worked.

 

[foxblock]

[...]

Okay, thanks for your input on how to handle scores.

I agree that end of the session is probably the way to go, although I see an advantage of sending scores as they are achieved: save, not lost on a crash without caching. Sending every score (even the ones not being highscores) makes sense, since we have leaderboards by months and only sending new highscores would make that more or less irrelevant compared to the all time scores. On the other hand you send a lot of superfluous scores (starting a new level on Super Hexagon will usually result in 10 new scores being uploaded as you adapt to the new difficulty level).

So eventually I am tending towards giving the user the option to select which scores he wants to upload (press of a button on the game-over/score screen, Press X to upload highscore to C4A).

A queueing system in sc would be awesome, so one can pile up scores during a playthrough and upload them on shutdown (less stress on the server). Most games will probably have this eventually, so why not implement it in the client.

Additionally this would allow for scores achieved offline to be sent at a later date (important).

Finally, I vouch for focus on encryption and security in one of the next updates. We are a small community, but if this projects wants to grow (possibly to other handhelds, there is no reason to keep it Pandora exclusive), there needs to be a protection against cheaters. (Right now I could easily upload arbitrary scores for any game without even trying...).

While I trust everyone involved and people can report malicious scores, putting some work in automated protection will pay off in the end (I realize this is on the todo list, but I rather have it sooner than later).

Thanks for your work so far, skeezix, hope you get some sleep some time

Cheers.

 

[skeezix]

Thats the rub though.. how to protect things, when you're giving the client away openly (some people have requested the code as well, so they know its not full of explosives

So..

Usually you bind it tight to the app, so that theres some control; or you have a life daemon that monitors things for anti-cheat. Etc etc.. all pretty painful, much harder to integrate, more annoying to user, etc.

But here we have. . just dropi a 'sc', invoke it from shell or your code, good to go. Great! We could encrypt end to end all we want, but theres _no way_ to prevent people from launching it by hand....

The main defence I've leveraged is that, at least we know 'who' submitted the scores; if theres a rash of oen guy blamming away, fine, we can nuke out all those submissions; drag, but no big deal. But there are ways to abuse it.. what if someone just puts in subtle done scores, that look legit? slowly getting better each few days? ... no way to really detect that.

So I find it all very depressing

jeff

 

[pmprog]

Why not make sc into a linked library then? And games require a password to submit scores. This will take out the more casual cheats, leaving only those who would decompile your game, or have a TCP socket recorder.

The linked library could also allow the developer to download scores for the game in case they wanted to include and display them in the game too

As the library expands, it would also allow the developer to access any newer features of C4A that may get added in (friends lists, saves/replays/custom level file sharing etc.)

It would require some extra dependancies for the developer, but using something like libcurl is available on all platforms

 

[_jr_]

I think it is not any more fair to make it cheating difficult for the normal user but still have the experienced hacker cheat totally undetectable. It just gives a false sense of security (just like the useless doping checks).


IMHO the only solution is to not take the competition that seriously and simply accept that every single one may be cheating. But I know from experience how difficult that is.

 

[_wb_]

I want to release my games as Free Software, which means I'm also distributing their source code. That pretty much rules out obfuscation as a technique to avoid cheating.

There is, even from a theoretical perspective, no way in which full cheat protection can work, unless you have full control over all software and hardware between the gamer and the highscore server. With encryption it may be possible to have full cheat protection even while using a regular internet connection between the gamer and the server, but then you still need to fully control all software and hardware on the client side, which is highly undesirable.

We could do partial cheat prevention (making it harder to cheat by adding more obfuscation and complications), but I'm not really in favor of that, for the following reasons:

- it makes it harder for game devs to add C4A support to their game.

- if cheating becomes nontrivial, it becomes a challenge/game of its own. Now it's trivial so it's no fun to cheat. But if it becomes nontrivial, then it becomes a kind of "puzzle" that may be interesting for some of us to try to "solve".

- we're starting a weapons race, meaning backwards-compatibility will become harder to assure since protocols may get revised regularly to fix security holes, breaking existing C4A support in games.

 

[foxblock]

To summarize:

Problem: You can simply call sc with an arbitrary score for any game and submit that.

Solution: Encrypt the score and the game, give out an "api key" to every game, so the encryption of one game does not match one of the others. Simple symmetrical encryption where one key is stored on the C4A server (safe) and one in the game (potentially unsafe), but since it's a different key for each game, obtaining one does not break the whole system.

New problem: Can't do that with fully open source software, since the key would need to be distributed, too if you want to stick to your principles. This would mean, no game with C4A support could be "Free Software" (bad).

Another problem: Starting an arms race you cannot win (just like with captchas)...

Personally I think a weak encryption is better than none, since it stops the "weekend hackers", who just want to be famous by submitting a number 1 score, even if it is taken down the next day.

I don't think many people will take the "challenge" of cracking such a weak and small system too seriously.

Another approach would involve the community and enable players to easily report scores (with a team of moderators to review those and remove cheaters).

But how to detect cheated scores? I could just take the no1 highscore, add 10 points and nobody could tell I cheated...

I like how "Audiorace" handled this, for every highscore it would also upload a slop-view of the track, so you could compare the slopes and easily detect music tracks that have been altered to benefit higher scores.

A system like this could be implemented in many games. Project4 for example has a "replay" feature, so for every highscore I could upload the corresponding replay and players could watch that. For other games you could upload some stats, a screenshot of the score screen, etc.

Still not fully secure, but it's better integrated in the games, less arbitrary and probably just as safe as some crazy encryption. The server would need to host additional data (so developers need to keep it small), but players could actually benefit from that (see above replay example).

I think the encryption method is simple to implement (encrypting scores could be handled by a small library supplemental to sc, very little work for the game developer) and effective for the most part, the community method is probably more effective, but requires much more work on both ends.

In the end you cannot stop cheaters and trying to fully eliminate them is futile, but adding some sort of barrier is necessary in my eyes (just like a locked bicycle is much less likely stolen than an unlocked one, no matter how cheap the lock).

Involving the community (by adding a report function on the website) would be good, too, since it streamlines the process of highlighting and eliminating cheated scores.

 

[skeezix]

yeah, the one idea I had when I started all this, was top also uplaod videos of the play (or input recordings); higher complexity, higher burden for compliant games.. right now you just drop sc in and go, easy; but if you want to change your app to record inputs, and be able to 'play them back' etc, its quite a bit more effort.

Changing sc to a shared lib is of course trivial (and how it was done originally; originally it was all embedded in MAME entirely, with live ram updates imn real time etc), but I realized quickly that..

i) I want it open source anyway, so someone could just cheat by modding a MAME anyway

ii) modding MAME to encrypt it and all sortd of protection is not in the spirit of mame and probabyl violates the mame license somewhere (didn't bother to look)

iii) its all an illusion; theres a lot of ways to defeat that anyway

ie: Even including sc type functionality as a lib, well, day zero some idiot would turn it back into a command line tool and poof, nothing was achieved

The only 'real' way to do it is taking control of the whole process.. authors uploading their apps to the system which then signs them and posts them to app download locations/stores for re-use; this getting you all sortd sof signing abilities and proofing etc, but very inconvenient, and very burdensome to all involved.

Hence my laissez-faire approach .. don't sweat it; but I worry when we go wider with PC and r-pi etc, we'll get totally screwed up. I dunno

 

[foxblock]

Hence my laissez-faire approach .. don't sweat it; but I worry when we go wider with PC and r-pi etc, we'll get totally screwed up. I dunno

Yeah that's what I worry about, too. Right now cheating is not a problem.

Although I agree, any kind of encryption is not going to hold up and honestly I have not thought about license complications too much.

 

[_wb_]

Replay file "certificates" are a way to make it harder to cheat, especially if you have automatic replay file verification (i.e. a game engine without rendering to screen that checks whether the claimed score corresponds to the one obtained from playing back the recorded inputs). Of course it still allows "slow-mo" cheating, as well as "save state" cheating and "post-processing the playback file" cheating, but at least it requires the cheater to come up with a valid replay file.

There are numerous downsides to this approach though. It is a big implementation effort, especially if you want to make things robust (you don't want the verification server to misbehave on malformed playback files, avoid DoS attacks while still checking potentially long replay files, etc.). Those replay files can be relatively large, even with compression (e.g. a 60fps game like NubNub which uses the nubs as input would need at least 8 bytes per frame (2 bytes per axis per nub) before compression, or about 30KB per minute of gameplay; compression will not help that much on this type of data). That means uploading scores takes much more time and bandwidth, and the server needs more space (unless it deletes the certificates after verification). Also the game needs to be fully deterministic - usually not that hard, just store the random seed in the replay file if you use random numbers, but it's easy to introduce determinism bugs - or else the replay files need to store full states, which is much more information than just the inputs.

Microbes has replay files, and it would be easy to make a replay file verifier. But since the score in Microbes is not based on playing one level, but rather it's a cumulative score based on how many levels you have beat (and at which difficulty setting you beat it). So that would mean that in this case, the certificate would consist of ALL replay files that contribute to the score, which could be a very large amount of data, and I would need to store all those replay files in appdata (now a replay is only saved if you want it to be saved). Doesn't seem very practical to me.

 

[porg]

I will have fun with C4A also if it is technically totally insecured. (If you agree with this statement, please "like" the post).

I have a strong certainty, that I compete against honorable men and woman within the OP community. Waiting for a product for many years in an otherwise very professional-corporate/consumeristic/marketized world, already drags certain characters. Of course, sociologically OP is likely a heterogenous group as most others, but nevertheless, I have had mostly fine encounters here so far.

Furthermore: Competition can also be of relative rather than absolute nature: I try to be match myself with the know persons A, B, C, and my pride is relative to that and am not disappointed to be far behind X, Y, Z (in the top ranges), whom I don't know, and/or whose credibility I cannot assess.

Out of this another feature idea arrises: In the client the ability to filter highscores for: <user1>, <user2>, … <userN>. Than you get a personalized/peer competition as opposed to a global competition. Would require that C4A sync all records (also the lower ranks), or that your peers make it into the topX, if only the topX are synced.

 

[skeezix]

porg -- yep, that feature is planned

jeff

 

[hipnotoad]

Anyone else experience the bug where c4a fails to set the nubs back into "mouse" mode?

 

[Kumaki]

Furthermore: Competition can also be of relative rather than absolute nature: I try to be match myself with the know persons A, B, C, and my pride is relative to that and am not disappointed to be far behind X, Y, Z (in the top ranges), whom I don't know, and/or whose credibility I cannot assess.
Out of this another feature idea arrises: In the client the ability to filter highscores for: <user1>, <user2>, … <userN>. Than you get a personalized/peer competition as opposed to a global competition. Would require that C4A sync all records (also the lower ranks), or that your peers make it into the topX, if only the topX are synced.

Now THIS is an idea I like.

Thus three or four friends could get together on their own competition...

Person A - I could whip your butt on Game X

Persons B, C, D and E - no way!!

All - why don't we find out?

And there you go...set up the competition just among those five participants.  Pretty cool.

This way, people could post up a challenge right here on this Board and see who takes them up on it.

Maybe they could then establish their own prizes...five bucks for signing up, winner takes the whole pot...thus, five bucks to sign up...winner takes the entire $25 in the above scenario.

And maybe OP itself...could sponsor a contest or two, you never know.

Could be a way to get publicity

 

[levi]

@hypnotoad


Hmm, sounds like a crash. Did you exit it normally?

 

[porg]

Anyone else experience the bug where c4a fails to set the nubs back into "mouse" mode?

Yes, and it is even more annoying, as the alternative (keyboard input) also has its shortcomings:

Changing keyboard focus beyond the fold does not change the viewport: Left/right switches focus between game list (left) and showcase (right). Fine! Up/down within the game list cycles focus through the game list. But as soon as you put focus to a game beyond the fold, the viewport does not change accordingly (to bring the focused element into the visible area).

But luckily skeezix said he will fix 'em

 

[porg]

@kumaki: As soon as it this'd be about money, I'd not trust the honor system anymore 

But as I never play for money, just for fun/challenge/honor, I am ok with an unsecured system.

 

[Kumaki]

@kumaki: As soon as it this'd be about money, I'd not trust the honor system anymore 

But as I never play for money, just for fun/challenge/honor, I am ok with an unsecured system.

I'd agree with you...if it was anyone could jump in.

What I am talking about is a "gentleman's agreement" among people who trust one another.

I only brought up that possibility because some here are clamoring for prizes all the time...how about setting up your own little Tournament, with people you trust not to cheat...and have each player put money into a pool that goes to the winner?

I would not go for a Tournament like that unless it included only people I trust not to cheat.  I'm assuming you would be okay, too, if it was only among people you trusted.

and from what I am reading, it sounds like this sort of capability is coming.  Unless I'm misunderstanding what I am reading?

Me, I don't need prizes, for me, bragging rights are sufficient.  But this talk about setting up individual Tournaments among certain players...opened up the possibility to suggest - for those who want prizes so much...a way in which they COULD compete for prizes...all among themselves, see?

 

[porg]

^ I agree.