ScummVM dat file test (verifies game files)

[DrasticNerd]

Greetings all,

I'm uploading a custom dat file for anyone to test and provide feedback on.

I'm a hobbyist script monkey and one of my interests is validating game files (aka rom naming). I built my own tool that automates custom merges, deletions, renames & edits to dat files. The attached file is a sample of what the tool can output.


How to use:

1. Download the attached file
2. Extract the 7z using your favourite decompression program
3. Import the xml file to your favourite rom manager program
4. Place a COPY of your ScummVM games into the folder created by your program for the imported XML (NB only use a copy. This dat only includes a small selection of games emulated by ScummVM.)
5. Run your rom manager program on this folder (if your rom manager can not compartmentalise by processing a single folder, stop now and create a temporary/portable copy of your rom manager program and use that instead.)

All feedback welcome.

Spoiler: Answers to expected questions (For beginners to rom naming)

Q. What does this do?
A. Validates the files for your games. (Any missing files should be marked by your rom program. Any unwanted files are deleted or moved elsewhere - depending on what rom manager you are using and what settings you have set.)

Q. This is pointless because ScummVM already has built-in validation of ScummVM games!
A. UPDATED: Yes and no. ScummVM does a partial checksum of the game files. However AFAIK it doesn't check for (or remove) any bad/unwanted files (like a rom manager does). Also a rom manager ensures that all your folders are named correctly (including the version/region in this case)

Q. This is pointless because RetroArch already has built-in validation of ScummVM games!
A. You are mistaken. This only checks for the existence of certain filenames. It does NOT do a checksum hash for all files for the game.

Q. Where do I download a rom management program?
A. A few examples:
ClrMamePro
RomCenter
Romulus

Q. Rom managers for Linux? Mac?
A. I don't know of any rom managers for these systems. Try using WINE for Linux - Romulus reportedly works on Linux. UPDATE: @PowerGod reports that AdvanceScan works on linux

Q. I don't have any ScummVM games. Where can I download some for testing?
A. Some ScummVM games are available for purchase from online stores.
Broken Sword 2.5 is a free fan game. You can download it from here
You can freely download ex-commercial titles from here (My dat does include: Beneath a Steel Sky, Flight of the Amazon Queen and Broken Sword 2.5. )

Q. I ran the rom manager on your dat and it doesn't detect my game!
A. The dat file is only a test sample. Your game may be a title that was not included in my sample. If the title was included then possibly the version you have is a demo version or a non-English title.

Q. I ran the rom manager on your dat and it claims my game is missing files.
A. UPDATE: It happens to me on some games too.

• Broken Sword - If you are missing 5x txt files from your retail copy, you can download these from ScummVM website (look for subtitles zip on https://www.scummvm.org/games/ )
• Escape from Monkey Island - If you are missing the update.exe from your retail copy, you can download this from I recall downloading this exe but I can't see it on the ScummVM website? Nevermind, the game is playable without it and isn't as if there is any more updates for it anyway.
• Anything else - I'm afraid I can't help as I didn't create these checksums myself.

Q. I ran your xml through my rom manager and IT DELETED ALL MY SCUMM GAMES! (This is extremely unlikely to happen but I'm including this hypothetical situation JIC)
A. I'm sorry for your loss. I did however pre-warn you in the steps to only run this on a COPY of your games.

Q. This is fun! Can you do this for games other than just ScummVM?
A. Yes, yes, it is and you certainly can! There are dat files for most emulation needs and most of these dat files are publicly available. There is a lot of useful information out there so I won't try to redo it here. If this is all new to you and you need a hand getting started, let me know and I will try and help you out.

Spoiler: Answers to expected questions (For those who have some experience with rom naming)

Q. Why did you choose Gruby as the source?
A. I ignored TDC and TGOD dats in favour of Gruby for several reasons. The main reason is that TDC & TGOD are dats for intended for DOS. While there is some overlap with ScummVM, not all ScummVM games are DOS and it would be a pain pulling out only the ScummVM games from these dats. (The other reasons are lack of checksums, much longer running time and more scene-oriented focus for TDC)

Q. Which rom manager do you use?
A. I tried several in the past but I settled on Romulus a few years back.
I wouldn't recommend Romulus if you are mainly interested in MAME roms and/or if you are worried about file deletion. (I only say that because today Romulus deleted a few games on me. The first time it has ever done this so I can't be certain that it wasn't some other factor to blame.) UPDATE: As far as I can make out, there was some folder permission errors that caused the problem. The program tried to move unrecognised files to a separate folder before deletion (like it is supposed to) but due to insufficient permissions, they couldn't be moved before deletion.

 

[PowerGod]

AdvanceScan works also on linux

Advance Projects

Play MAME on your TV Arcade Monitor or Fixed Frequency Monitor

www.advancemame.it

 

[DrasticNerd]

Welp good thing I wasn't expecting a lot of feedback!

1. Since my OP, I have updated the dat file for my own use. IDK if anybody would be interested in a copy of this. (Pro = Updated to latest Gruby. Con = I have removed titles to focus on the games I play. I have also moved all non-point&click games to a seperate dat file. e.g. Ultima games, MI4, Last Express etc.)(Last Express because it isn't recognised by ScummVM )
2. maybe there isn't much love for ScummVM these days. Would niche dat file conversions for translated roms be of more interest?

 

[DrasticNerd]

I've attached an archive containing 3 different rom dats. GameBoy Color translated roms only.

Sources I used for my conversions:

• RetroArch (I don't recommend these as RA rdb files rarely get updated for translations and romhacks.)
• Retroplay
• SmokeMonster

 

[DrasticNerd]

I've attached an archive containing 3 different rom dats. GameBoy Color translated roms only.

WARNING! Do not download this attachment. Delete it if you already have.

I recently discovered a problem while auditing translated roms. After some investigating, I discovered that a small number of roms had different CRC checksums to their source data.

I have solved the problem by changing my scripts so that CRC32s are treated as string type variables at all times.

 

[levi]

Yes, I guess you're using a duck-typed language. In many of those if a string contains a series of 0-9 digits alone it may be treated as a number. CRCs that had hexadecimal a-f in them wouldn't be affected, but if any came along only containing 0-9 digits then it will be considered a number, and unequal to a string.

 

[DrasticNerd]

if a string contains a series of 0-9 digits alone it may be treated as a number. CRCs that had hexadecimal a-f in them wouldn't be affected, but if any came along only containing 0-9 digits then it will be considered a number, and unequal to a string.

You got it first go. (Y)

I didn't pick this up sooner as most CRC32 checksums were unaffected. Mea culpa. I have changed my scripts and I haven't detected any issues since.

I can upload a corrected dat for ScummVM if anybody wants one...

 

[DrasticNerd]

IDK if the ScummVM sample dat I uploaded a few months back had any CRC32s screwed up. But just in case, I have replaced the attachment with a version that was created after I fixed my scripts.

 

[bugmenot]

I love the idea of this! It has been too long since this was needed. I'd head over to the ScummVM forums and post there for better exposure.