Command line help needed (All files within foldertree)

[DrasticNerd]

I don't see what's so scary about converting between a command line output or even relatively crusty html and xml. XML is just a text format with a tightly defined schema. You could write a program in perl, python, java almost any language if you wanted to do this repeatedly and relatively safely, or you could even do the conversion manually in a text editor for a one off.

ah I think you have misunderstood my comment. Converting between file types (XML, HTML etc.) isn't a problem.

The conversion I was referring to is converting the output from a generic checksum tool into a dat file.

Have you ever opened up a dat file in a text editor to look at its contents? If yes then maybe you will understand why it is going to be onerous to write this conversion code entirely from scratch (seeing as there is no foundation I can build off. Editing dat files for rom audit programs is too niche). It's not that I am incapable of writing this converter - I know that I have the skills to do it. It's the length of time that it will take to get it perfect.

If you are not familiar with dat files, you can see a small sample I attached to my thread here

 

[Djoga'Ro]

If you are not familiar with dat files, you can see a small sample I attached to my thread here

No dat file there - just an archive with an xml file.

 

[DrasticNerd]

No dat file there - just an archive with an xml file.

Thanks @Djoga'Ro That is the file I meant.

I use the term "dat file" to refer to any file designed to contain rom checksum information (i.e. for use in a rom audit program), regardless of the file extension of the file in question.

 

[levi]

Yeah, but regardless of the extension, it contains text that is valid XML. You've said converting to and from that isn't a problem already.

 

[Djoga'Ro]

The file extension is just part of the name - you can change it, it wouldn't change the content's format.
So the question is, will you know or can you recognise the format of every file you wish to manipulate and will there only be formats you'll know to handle. Unpacking and packing archives can be done with cli tools or via libraries, for a lot of archive formats that is, not all of course.

 

[DrasticNerd]

Yeah, but regardless of the extension, it contains text that is valid XML. You've said converting to and from that isn't a problem already.

Sorry. I am having difficulty explaining.

It isn't that difficult to convert a text-based file to another text-based file type. That is if the contents don't need to adhere to a strict structure.

My problem is that I will need to convert a report output by a generic hash tool into a XML dat file. From previous experience (converting a different file format into a XML dat file) I know that it will be ardous.

(Which is why I was hoping that DatUtil would be the answer. Only to hit a roadblock when it appears that the tool can not search below one level of subfolders! Can not understand why it was ever programmed that way.)

So the question is, will you know or can you recognise the format of every file you wish to manipulate and will there only be formats you'll know to handle. Unpacking and packing archives can be done with cli tools or via libraries, for a lot of archive formats that is, not all of course.

Not sure what you mean by manipulating files? I have no intention of unpacking any files.

I point the tool at the game folders and run it to record the checksums (and filename + paths) of every file inside each game folder.

Neither the tool (nor I) know what types of files will be inside each game folder and it actually doesn't matter. I only need to record their existence, location and checksums. The hard part will be converting the contents of the generic report into a dat file.

 

[Djoga'Ro]

It's been a while, but iirc you mentioned something like you could have output files lik eyou need them almost, just one kind of hash missing. In that sense, manipulate := add the missing values.

 

[FBnil]

If the tool can not look into sub-sub directories, why not create lots of symbolic links to fool the tool that the files are there?

So original dir:

• C:\ExampleOfFolderContainingGames\Game1\Imafile.dll = DETECTED
• C:\ExampleOfFolderContainingGames\Game1\SubfolderOfGame1\Imafiletoo.dll = NOT DETECTED

After transforming:

• C:\ExampleOfFolderContainingGames\Game1\Imafile.dll = DETECTED
• C:\ExampleOfFolderContainingGames\Game1\SubfolderOfGame1___Imafiletoo.dll = DETECTED
• C:\ExampleOfFolderContainingGames\Game1\SubfolderOfGame1\Imafiletoo.dll = NOT DETECTED

Windows has junctions which you can set, and after the replacing all / with ___ (and then in your xml you can replace them back to / )

How can you find and replace text in a file using the Windows command-line environment?

I am writing a batch file script using Windows command-line environment and want to change each occurrence of some text in a file (ex. "FOO") with another (ex. "BAR"). What is the simplest way to d...

stackoverflow.com

and then to clean up, search for all junctions and delete these (just dir /s /b |findstr ___ and delete all files in that output; make sure you have no files with triple underscore BEFORE you create the junctions)

How to view all the Symbolic links, junction points, hard links in a Folder using dir?

The command dir /a displays a list of all the files and folders in a given location. But it displays the type <JUNCTION> for Junction Points as well as Symbolic Links to Folders. Is there any

superuser.com

Junction - Sysinternals

Create Win2K NTFS junction points.

docs.microsoft.com

 

[DrasticNerd]

If the tool can not look into sub-sub directories, why not create lots of symbolic links to fool the tool that the files are there?

Thanks FBnil. That is quite an original approach to the problem I have. (That is not sarcastic. I am sincerely impressed with the way you have thought outside the box)


The catch is - and the fault is mine, not you - is that I'm, ah, highly allergic to symbolic links!

(I first heard about sym links + junction links a few years back. I read up on them and proceeded to go wild with creating them all over my PC. It ended up with me accidently deleting lots of my own files when trying to remove some of the links. Since then I have refused to have anything to do with them!)

 

[DrasticNerd]

It's been a while, but iirc you mentioned something like you could have output files lik eyou need them almost, just one kind of hash missing. In that sense, manipulate := add the missing values.

Would that it were that simple!

Off the top of my head the only one I can think of that has a missing hash is QuickSFV. (It does all subfolders but it only hashes MD5)

As such, it is inferior to HashMyFiles (Which can do all subfolders, can do CRC32 + MD5 or SHA for that matter)

Ideally I would prefer to work with DatUtil as it does all of the right output - not just hashes but recognising each folder in the first layer as being unique games and the "rom dat" compatibility (which is arguably the most important aspect as creating rom dats is my primary goal for the projects I have in mind)

However, as far as I can make out, DatUtil can not process more than one level of subfolders. (I find it hard to take in how such a design flaw could have been introduced. Unless the creator thought this was a feature - as it was originally designed for MAME roms which don't have a complex foldertree structure AFAIK)

As the program was last updated around 10 years or so ago and as the source provided on the website is, I'm informed, incomplete, I don't see a way to get DatUtil working.

Which leaves me with running HMF and writing code to convert its output into "rom dat compatible" output. I have done similar things before - for instance, I have converted SmokeMonsters lists into rom dat files (for an example, see my GBC translated rom dats in this thread here ) but the source data was tab delimited text and it already had all files grouped for each game - which a generic hash tool can not do.


...I'm now thinking of shelving this project for the foreseeable future. (Too little benefit for the amount of effort involved + I'm now working on other interesting hobby projects...)




Afterthought - if anybody reading this in future has the full source code to DatUtil then should you ever decide to release an updated version (no depth restriction), PLMK TY!

Afterthought 2 - Or if anybody decides to create their own alternative to DatUtil then I would be highly interested in testing your release!

 

[PowerGod]

I still can't understand why you find this so difficult to obtain, but anyway it's also not clear to me in what way the subfolders should be included in the list...
as an example, this is a piece of one of the xml you posted:

<datafile>
    <machine name="Arle no Bouken - Mahou no Jewel (Japan) (SGB Enhanced) (GB Compatible) [T-Eng Jazz]">
        <description>English translation by Jazz version (2.1)</description>
        <rom name="Arle no Bouken - Mahou no Jewel (Japan) (SGB Enhanced).gbc" size="1048576" crc="9a14a37a" md5="d2bb101d6b203d8b29cc0db5944eb244"/>
    </machine>

Now, let's say that "machine name" is the main folder of the game, and "rom name" is one of the files in the first folder, how should the subfolders and relative files be included in this format ?
Should every subfolder become a "machine name" ? Or maybe the "rom name" will become "subfolder+'\'+name" ? Or you don't even want subfolder names reported here, but just the entire list of files/subfiles like if they where all in the main folder ?

 

[FBnil]

Same here: I still do not understand your structure. I failed to produce "extra games" by giving it subdirectories. It just fails to see a game there. Can you show me a structure of such zipfile or directory structure?
And do you need the directories in the rom name?
If we use the -a (append to output), then a flatfile will be created, that with little work (skipping all datafile/?xml/!DOCTYPE tags and parsing all game data, then prepending and post pending that simple xml header around it) and you get a valid xml. But the games do not reflect where they are. This can be then edited in a second-pass to be added, like so (working code for now with some perl, will try to CMD it):


I've created a directory ./roms/bombermans/bomberman_world/ and ./roms/bombermans/neobomber/ then just gave the datutil 2 rounds, with one directory each time, which then produced:

        <game name="bomberman_world">
                <description>bomberman_world</description>
                <rom name="bombermans/bombrman/bbm-p0.65" size="131072" crc="0a20afcc"/>
                <rom name="bombermans/bombrman/bbm-p1.62" size="131072" crc="982bd166"/>
        </game>
        <game name="neobomber">
                <description>neobomber</description>
                <rom name="bombermans/neobomber/093-c1.bin" size="4194304" crc="d1f328f8"/>
                <rom name="bombermans/neobomber/093-c2.bin" size="4194304" crc="82c49540"/>
                <rom name="bombermans/neobomber/093-c3.bin" size="524288" crc="e37578c5"/>
                <rom name="bombermans/neobomber/093-c4.bin" size="524288" crc="59826783"/>
                <rom name="bombermans/neobomber/093-m1.bin" size="131072" crc="e81e780b"/>
                <rom name="bombermans/neobomber/093-p1.bin" size="1048576" crc="a1a71d0d"/>
                <rom name="bombermans/neobomber/093-s1.bin" size="131072" crc="4b3fa119"/>
                <rom name="bombermans/neobomber/093-v1.bin" size="4194304" crc="02abd4b0"/>
                <rom name="bombermans/neobomber/093-v2.bin" size="2097152" crc="a92b8b3d"/>
        </game>

is that what you mean? Else, if you just want a flatfile without the location directory structure in the rom name, that second pass does not even to be done. Please explain more clearly what structure you will need.

 

[PowerGod]

bump because I'm curious

 

[DrasticNerd]

I still can't understand why you find this so difficult to obtain, but anyway it's also not clear to me in what way the subfolders should be included in the list...
as an example, this is a piece of one of the xml you posted:

<datafile>
    <machine name="Arle no Bouken - Mahou no Jewel (Japan) (SGB Enhanced) (GB Compatible) [T-Eng Jazz]">
        <description>English translation by Jazz version (2.1)</description>
        <rom name="Arle no Bouken - Mahou no Jewel (Japan) (SGB Enhanced).gbc" size="1048576" crc="9a14a37a" md5="d2bb101d6b203d8b29cc0db5944eb244"/>
    </machine>

Now, let's say that "machine name" is the main folder of the game, and "rom name" is one of the files in the first folder, how should the subfolders and relative files be included in this format ?
Should every subfolder become a "machine name" ? Or maybe the "rom name" will become "subfolder+'\'+name" ? Or you don't even want subfolder names reported here, but just the entire list of files/subfiles like if they where all in the main folder ?

bump because I'm curious

Well....OK

1. No. There can only be one machine name per individual game. Each game must have a machine name (1:1 relationship)
2. The subfolders themselves are not included as separate items in the output (Each file must be checksummed and you can't checksum a folder) Rather, they are necessary for specifying where files should be placed in the game folder tree.
3. If I have understood you correctly then you are right. The "rom name" would show relative paths for that particular game/rom structure. So if a file was in a subfolder of the game, its name property would be "Subfolder1\Subfolder2\example.ext"
4. No. It is essential that the relative paths are reconstructed otherwise the games would be unplayable.

What makes it confusing is the term rom is normally applied to the individual game. In the dat, "rom" refers to each individual file in the game folder tree. (Each game has one machine item and one or more rom items)

 

[DrasticNerd]

Same here: I still do not understand your structure. I failed to produce "extra games" by giving it subdirectories. It just fails to see a game there. Can you show me a structure of such zipfile or directory structure?
And do you need the directories in the rom name?
If we use the -a (append to output), then a flatfile will be created, that with little work (skipping all datafile/?xml/!DOCTYPE tags and parsing all game data, then prepending and post pending that simple xml header around it) and you get a valid xml. But the games do not reflect where they are. This can be then edited in a second-pass to be added, like so (working code for now with some perl, will try to CMD it):


I've created a directory ./roms/bombermans/bomberman_world/ and ./roms/bombermans/neobomber/ then just gave the datutil 2 rounds, with one directory each time, which then produced:

        <game name="bomberman_world">
                <description>bomberman_world</description>
                <rom name="bombermans/bombrman/bbm-p0.65" size="131072" crc="0a20afcc"/>
                <rom name="bombermans/bombrman/bbm-p1.62" size="131072" crc="982bd166"/>
        </game>
        <game name="neobomber">
                <description>neobomber</description>
                <rom name="bombermans/neobomber/093-c1.bin" size="4194304" crc="d1f328f8"/>
                <rom name="bombermans/neobomber/093-c2.bin" size="4194304" crc="82c49540"/>
                <rom name="bombermans/neobomber/093-c3.bin" size="524288" crc="e37578c5"/>
                <rom name="bombermans/neobomber/093-c4.bin" size="524288" crc="59826783"/>
                <rom name="bombermans/neobomber/093-m1.bin" size="131072" crc="e81e780b"/>
                <rom name="bombermans/neobomber/093-p1.bin" size="1048576" crc="a1a71d0d"/>
                <rom name="bombermans/neobomber/093-s1.bin" size="131072" crc="4b3fa119"/>
                <rom name="bombermans/neobomber/093-v1.bin" size="4194304" crc="02abd4b0"/>
                <rom name="bombermans/neobomber/093-v2.bin" size="2097152" crc="a92b8b3d"/>
        </game>

is that what you mean? Else, if you just want a flatfile without the location directory structure in the rom name, that second pass does not even to be done. Please explain more clearly what structure you will need.

UPDATE: I was looking back through some older tests and found I had reached the same result back in August. I theorise that you can get full folder tree but only with CRC32 checksumming.


Interesting. The output you pasted appears to be almost what I'm looking for.

1. It doesn't output a 2nd checksum (Essential to have either SHA1 and/or MD5 for each rom item)
2. Why did you need to run datutil 2 rounds?
3. (Probably not an issue) while the test appears to have more than one folder depth for each game, the files are all placed in the 2nd folder level. To replicate the real world conditions, the files will appear in the game folder root as well as within subfolders (multiple levels deep)

 

[PowerGod]

I put this thing here for now, I made it in like 20 minutes, but must be considered ALPHA VERSION, it doesn't work well on some kind of structures.

I used it in Cygwin, but should work also on plain Linux, as the utilities "cksum" and "md5sum" seems present in many distros.

Spoiler: dattree.sh

#!/bin/bash

if [ -d "$1" ]; then (

        export parapath=$1

        export searchpath=${parapath/%\//}/

        #export tmppath="~/script-hashfiles/"
        export tmppath="/tmp/"
        export tmpcrc="chksum.tmp"
        export tmpmd5="allsum.tmp"
        export tmpopa="oldpath.tmp"

        export xmllst="xmllist.xml"

        # Get all filename tree with added CRC32, size, and sort it by path/name
        find $searchpath | \
                sort | \
                xargs -n1 -I{} cksum "{}" | \
                sed 's/^\([0-9]\+ [0-9]\+ \)\(.*\/\)\(.*\)/\2 <===> \1\2\3/' | \
                sort | \
                sed 's/.* <===> //' > $tmppath$tmpcrc

        # Adds md5sum to the files
        rm $tmppath$tmpmd5
        while read line; do (
                export md5hash=$(md5sum "/${line##* /}");
                echo "${md5hash%% *} $line" >> $tmppath$tmpmd5
                );
        done < $tmppath$tmpcrc

        # Create XML formatted structure
        rm $tmppath$tmpopa
        rm $tmppath$xmllst
        export subfolder="";
        echo $subfolder > $tmppath$tmpopa
        while read line; do (
                export oldpath=$(cat $tmppath$tmpopa)
                export md5hash=${line%% *};
                export crc32=${line#* };
                export crc32=${crc32%% *};
                export size=${line#* * };
                export size=${size%% *};
                export fullpath=${line#* * * };
                export filename=${line##*/};
                export subfolder=${fullpath%/*}/;
                export gamefolder=${subfolder##*$searchpath};
                #export subfolder=${fullpath##* /};
                export foldeep=$(echo "$gamefolder" | grep -o "/" | wc -l)
                #if [ ! "$oldpath" == "$subfolder" ] && [ $foldeep = 1 ] ; then (echo "<machine name=\" "$gamefolder"\">" >> $tmppath$xmllst); fi
                if [ ! "$oldpath" == "$subfolder" ] && [ $foldeep = 1 ] ; then (echo "<machine name=\" "$gamefolder"\">" >> $tmppath$xmllst); fi
                #echo " <rom name=\"$filename\" size=\"$size\" crc=\"$crc32\" md5=\"$md5hash\">" >> $tmppath$xmllst
                echo "  <rom name=\"${fullpath##*$searchpath}\" size=\"$size\" crc=\"$(printf '%x\n' $crc32)\" md5=\"$md5hash\">" >> $tmppath$xmllst
                echo $subfolder > $tmppath$tmpopa
                );
        done < $tmppath$tmpmd5
); fi

It works like this:

./dattree.sh /FULL_PATH_OF_THE_GAME_FOLDER/Games/

It generates a file called "/tmp/xmllist.xml"

Spoiler: /tmp/xmllist.xml

<machine name=" captive1/">
    <rom name="captive1/CAPPO.EXE" size="73379" crc="4f8fc33d" md5="d2c11cc8db849752bac7ceb99355ec3f">
    <rom name="captive1/FILEPLAY.EXE" size="1706" crc="c0b3a03" md5="e668301d2bc1fa57d657a389c3576ea4">
    <rom name="captive1/CAPTIVE.BAT" size="46" crc="87795407" md5="4be1b4c3dc1722ac4dc42394255ed8f5">
    <rom name="captive1/INSTALL2.BAT" size="243" crc="956c53a4" md5="566315d11044679bdba4aacc04ee9b1d">
    <rom name="captive1/CAPTIVE1.SAV" size="15090" crc="b1f75cc4" md5="fb8507e386fdbfebd1ed8b48f211bd0b">
    <rom name="captive1/CAPTIVE9.SAV" size="15090" crc="b1f75cc4" md5="fb8507e386fdbfebd1ed8b48f211bd0b">
    <rom name="captive1/CAPTIVE6.SAV" size="15090" crc="b762bc21" md5="26710ff2949907a5b19b97785f869421">
    <rom name="captive1/INSTALL.BAT" size="319" crc="bef9175f" md5="356d02bdafed3ed1f07fc4ded5257cd2">
    <rom name="captive1/CAPTIVE0.SAV" size="15090" crc="d2d54d15" md5="95787ec59e5cdebc1bc74e5ec4bd4ef4">
    <rom name="captive1/GM.EXE" size="10541" crc="51d3339" md5="92f75305259dd6e55f23aa9e102beef9">
    <rom name="captive1/INTRO.EXE" size="893" crc="37fac3d6" md5="1b3a475ecfce6b80dd805484e702c215">
    <rom name="captive1/ANIMS/TEST12.ANM" size="44416" crc="3f103866" md5="5202b527f8e7ec7039ee618c883f6494">
    <rom name="captive1/ANIMS/TEST15.ANM" size="7900" crc="55b1eabb" md5="8e46805be93d2d1b02818b39573ec9ed">
    <rom name="captive1/ANIMS/TEST11.ANM" size="2089" crc="67d980c7" md5="d2e1fa5da713539655ca566eb5e627d0">
    <rom name="captive1/ANIMS/TEST4.ANM" size="58486" crc="84cacebf" md5="01975be47abc6887e0420e3c4d3bfc6b">
    <rom name="captive1/ANIMS/TEST2.ANM" size="53799" crc="d71d30e" md5="df0d137b98d01d51b88e4da639715740">
    <rom name="captive1/ANIMS/TEST1.ANM" size="42993" crc="9db4f070" md5="03f4384afd3f0004f6614f91f5f50343">
    <rom name="captive1/ANIMS/TEST10.ANM" size="112137" crc="a598e431" md5="9ef94f03d833776e5c9397aff17e46a0">
    <rom name="captive1/ANIMS/TEST6.ANM" size="43007" crc="b093a41e" md5="a98c65265685d4120c26768e5332407f">
    <rom name="captive1/ANIMS/TEST9.ANM" size="213917" crc="c374e21e" md5="5c7c84b2eb42e3d95c801761dc613177">
    <rom name="captive1/ANIMS/TEST8.ANM" size="93741" crc="cc0af389" md5="3f6c15c106292fd2020323e19eb50172">
    <rom name="captive1/ANIMS/TEST0.ANM" size="2472" crc="cf395f92" md5="1303e58e7c464406dd5097a6105fddef">
    <rom name="captive1/ANIMS/TEST7.ANM" size="7577" crc="d4e57b40" md5="5d774f949e255de1f5efc16cb256d02a">
    <rom name="captive1/ANIMS/TEST14.ANM" size="50976" crc="d6f32460" md5="84dcf3af83471f274e2372446312be77">
    <rom name="captive1/ANIMS/TEST3.ANM" size="52306" crc="e8f1c880" md5="0206c9e13a9ed5bd979b86962f191fda">
    <rom name="captive1/ANIMS/TEST13.ANM" size="130335" crc="298aa52b" md5="74a7eb4957108fe37ec30755b455dc54">
    <rom name="captive1/ANIMS/TEST5.ANM" size="50109" crc="321838f9" md5="7c3716957f0eb561e4ab58829e409bb0">
    <rom name="captive1/CAPICS/WALLB.PL5" size="40000" crc="3c6726be" md5="c2091c916b5e1e363f26925eb35a38b4">
    <rom name="captive1/CAPICS/ALIEN4.PL5" size="40000" crc="3d3fa902" md5="26861c0528d90e1b3e0b051a3de2f23e">
    <rom name="captive1/CAPICS/ICONS.PL5" size="40000" crc="42b4b6a7" md5="c6a4b5eeab5a7a5eb429efbd2d0f14e1">
    <rom name="captive1/CAPICS/ALIEN5.PL5" size="40000" crc="4743eaeb" md5="4e6f924ff8f007bae3a62a71bb7f71e8">
    <rom name="captive1/CAPICS/OBJECTS.PL5" size="40000" crc="4ba88243" md5="a178f53ce39258dde9d0933b6dfa42cb">
    <rom name="captive1/CAPICS/ANIM3.PL5" size="40000" crc="506f800a" md5="043377a8fa41c1b23b818f9493d7d4c5">
    <rom name="captive1/CAPICS/KEYBOARD.PL5" size="40000" crc="c475d3c" md5="b522dbabf29e690c6e88b9bd5444c531">
    <rom name="captive1/CAPICS/ANIM1.PL5" size="40000" crc="831e5c64" md5="680b82c2c12efd74a33ca0df52e9c90d">
    <rom name="captive1/CAPICS/ALIEN2.PL5" size="40000" crc="8ad160b6" md5="717c9344745906c4dcf22be46a006fad">
    <rom name="captive1/CAPICS/ROOFS.PL5" size="40000" crc="a16dccb5" md5="d25df8850b1299106782ae546cc5d836">
    <rom name="captive1/CAPICS/WALLE.PL5" size="40000" crc="a2272273" md5="0d0f3d116b3c9270cfbd847db20304d5">
    <rom name="captive1/CAPICS/ALIEN1.PL5" size="40000" crc="a6d785b7" md5="88833beae09d8893641be91ee253c6c0">
    <rom name="captive1/CAPICS/SHOP1.PL5" size="40000" crc="af2e5532" md5="a85fd4edc52e076a8c5beabd97ee70e2">
    <rom name="captive1/CAPICS/ANIM2.PL5" size="40000" crc="b0bf6ae2" md5="f7093d6a252b9cc396c8ef2bd3fad023">
    <rom name="captive1/CAPICS/ALIEN3.PL5" size="40000" crc="b4e4a163" md5="1a79c34e928535c9c0f4e46cc9801f62">
    <rom name="captive1/CAPICS/GAMESCRN.PL5" size="40000" crc="b59fb3ff" md5="072e4b79ebc0cc065eade74aad6b1bb1">
    <rom name="captive1/CAPICS/SHOP2.PL5" size="40000" crc="b754d0f2" md5="d8b3172ddd1309463ec33f1d77f542bf">
    <rom name="captive1/CAPICS/WALLD.PL5" size="40000" crc="c78d6f70" md5="dc4e38451c793afbd421acc7becc24df">
    <rom name="captive1/CAPICS/WALLC.PL5" size="40000" crc="d0a0111c" md5="20b59ed0785ae4954a9812003ebf66b3">
    <rom name="captive1/CAPICS/DOORS2.PL5" size="40000" crc="f7d5346c" md5="beccbb531fc35196ad94335a0ba63a7f">
    <rom name="captive1/CAPICS/ALIEN6.PL5" size="40000" crc="f8152db2" md5="b7387563e9a346a058fb1bf5a8c4f1ab">
    <rom name="captive1/CAPICS/DOORS1.PL5" size="40000" crc="fcaab8df" md5="5689f2209d2dede0b15cf2b95f328e89">
    <rom name="captive1/CAPICS/WALLA.PL5" size="40000" crc="56f121b" md5="0d34280b434628bf656a9fa610776e68">
    <rom name="captive1/SOUND/BATTB.MID" size="2376" crc="3cb195fa" md5="95035466090b569dab9fc79a0236c46e">
    <rom name="captive1/SOUND/BATTA.MID" size="2376" crc="3cd8902d" md5="1bcd5114fc9b8edddffabb2f90e28adb">
    <rom name="captive1/SOUND/W6.MID" size="2213" crc="3e312db8" md5="18795c922fa940e2094444e84c8af526">
    <rom name="captive1/SOUND/W3.MID" size="2213" crc="46c774cc" md5="c4d8306ca99843b64c1e7dbddfa31513">
    <rom name="captive1/SOUND/BATT4.MID" size="2376" crc="4a0e0ff7" md5="da39dab4754179cb1811afb2f6839973">
    <rom name="captive1/SOUND/VCBASE2.MID" size="3192" crc="4b906c57" md5="856a07a4550428ed6bb810a0943fb575">
    <rom name="captive1/SOUND/W4.MID" size="744" crc="51e0d393" md5="e734bf7ad5e3a637debcc32d3e4d97c1">
    <rom name="captive1/SOUND/BATT5.MID" size="2376" crc="532ec216" md5="9f2551aa6e99850ed5a0b5515bcbf440">
    <rom name="captive1/SOUND/FCBASE5.MID" size="1598" crc="54726642" md5="adf30772780ad37ea849c76faa411f4e">
    <rom name="captive1/SOUND/WA.MID" size="744" crc="59f77e8f" md5="c25df77316cb594c9f69fbec1401a80c">
    <rom name="captive1/SOUND/FCBASE3.MID" size="1598" crc="5ee36965" md5="4fb82bd6eda2f695f0e940ab50aa9bce">
    <rom name="captive1/SOUND/LONGNT5.MID" size="378" crc="5f3c3511" md5="7025db3f1984492a937a2f66b03bd4e8">
    <rom name="captive1/SOUND/VCBASEB.MID" size="3192" crc="62ecbdd0" md5="17b350dc72e09ad2774fa48295dfebea">
    <rom name="captive1/SOUND/LONGNT4.MID" size="378" crc="64e154b4" md5="248e4efbc11965d4473ae7477288e125">
    <rom name="captive1/SOUND/VCBASE5.MID" size="3192" crc="69f69edf" md5="5d53c19150b3f42d379ada9184c75841">
    <rom name="captive1/SOUND/NEW_BEP.BIN" size="1757" crc="6d143361" md5="6ed63ec03677e92e095f58deb648518e">
    <rom name="captive1/SOUND/SB15.CTV" size="3910" crc="797ba48b" md5="a847fc9f9078e1961a7c9ae9cf9bb705">
    <rom name="captive1/SOUND/FCBASE8.MID" size="1598" crc="cd7a9f0" md5="041283057b32c18e70d2fc61a1ad0ab1">
    <rom name="captive1/SOUND/LONGNT1.MID" size="426" crc="825a99bb" md5="e41d1d814d8055d8c7570c304ef14b9a">
    <rom name="captive1/SOUND/VCBASE3.MID" size="3192" crc="85860294" md5="8634e6040e3600083c600d55f08d60ad">
    <rom name="captive1/SOUND/VCBASE7.MID" size="3192" crc="8833e1d4" md5="cdcec8dac23b529de7206d43a6abd113">
    <rom name="captive1/SOUND/W9.MID" size="2213" crc="dbde0b4" md5="63ae2fabece9c8fed6ddb2977f6586e5">
    <rom name="captive1/SOUND/W5.MID" size="1292" crc="8c825223" md5="f50096afc28bdbcc6544e8d122a5bac2">
    <rom name="captive1/SOUND/LONGNT6.MID" size="378" crc="e70eac8" md5="77b3808dcef1568e567e8ec8be70dcb7">
    <rom name="captive1/SOUND/ESCAPED.MID" size="6759" crc="945203ac" md5="f98ed190ec4cd65d594ebe07db3b4752">
    <rom name="captive1/SOUND/W1.MID" size="744" crc="99c292e6" md5="108d7ee7085b0966e2e8d14b18643ea8">
    <rom name="captive1/SOUND/FCBASE1.MID" size="1598" crc="9ae29c19" md5="69f1ade062dce1d4965da1fdcfab57a8">
    <rom name="captive1/SOUND/VCBASE9.MID" size="3192" crc="a4376f33" md5="cded4c5187be9bd04aa0def95ba23849">
    <rom name="captive1/SOUND/FCBASE2.MID" size="1598" crc="a6e6853f" md5="9da5c3cac519e32ce283e6292d398dff">
    <rom name="captive1/SOUND/VCBASE1.MID" size="3192" crc="a8c0a98a" md5="3775e72be1a263e13edb73e0ec62a22f">
    <rom name="captive1/SOUND/W2.MID" size="1292" crc="aadbb706" md5="8b595498bdb6c45d462219c4a62edeca">
    <rom name="captive1/SOUND/CAP_A.BIN" size="5426" crc="af39726f" md5="ac50f7cc9ba7ab9a3471cd7005103a80">
    <rom name="captive1/SOUND/BATT3.MID" size="2747" crc="b181203c" md5="390bac1fbb7c32410467250cdc25faa7">
    <rom name="captive1/SOUND/LONGNT2.MID" size="378" crc="b54fade4" md5="a5140b854a4416419a1a22bd66c73a61">
    <rom name="captive1/SOUND/LONGNT3.MID" size="378" crc="b7096f30" md5="2d32492316846093a2bf38096f279b2c">
    <rom name="captive1/SOUND/HOLOMAP.MID" size="8723" crc="b761cb59" md5="e24fd474e892e276f92868069a49abb8">
    <rom name="captive1/SOUND/SBPRO.CTV" size="5124" crc="b7d60fd4" md5="1008a24b0c6846e359b6453ed4f71b14">
    <rom name="captive1/SOUND/RUNNING.MID" size="4134" crc="b86cf25d" md5="0d1049eab9cf4e4171850ff463ebb691">
    <rom name="captive1/SOUND/VCBASEA.MID" size="3197" crc="b9fe251e" md5="3b5b1a6fd4a039791f99fb81796b0f77">
    <rom name="captive1/SOUND/BATT2.MID" size="2368" crc="be141131" md5="3090819bf3da827c8b6a80b237ff06bb">
    <rom name="captive1/SOUND/LONGNT8.MID" size="378" crc="c37dbba3" md5="c80c0e8d01ee50f20bdaa5313f6c987f">
    <rom name="captive1/SOUND/FCBASEB.MID" size="1598" crc="c5a02b91" md5="69289dc5ac880b58dc62fa1ce2619f5b">
    <rom name="captive1/SOUND/FINAL2.MID" size="23604" crc="c755bf6c" md5="0923e6345d50b5d0718346a4c31f002c">
    <rom name="captive1/SOUND/COMPROOM.MID" size="1454" crc="c7e78ea9" md5="6c2bc2141b07df83a50af16bf6092723">
    <rom name="captive1/SOUND/_SBNEW.DAT" size="58049" crc="c8734acd" md5="a195be0cf102a340f6ab6a8daa37dfda">
    <rom name="captive1/SOUND/TRAPPED.MID" size="2298" crc="cbe5be0d" md5="c6eb3f44c3caefff3a1866f270b63310">
    <rom name="captive1/SOUND/LONGNTA.MID" size="378" crc="d55f743a" md5="ca69e75686a5e6342aa16c41480c32f1">
    <rom name="captive1/SOUND/FCBASEA.MID" size="1598" crc="d7320f53" md5="152e09bea40dbba46891ee9735f258c6">
    <rom name="captive1/SOUND/BATT9.MID" size="2376" crc="d76c072c" md5="02e49b6c518b646a06df0dcf72a6d082">
    <rom name="captive1/SOUND/FCBASE9.MID" size="1598" crc="ddc8b5be" md5="68dc15121636198733fedb9a98f8d340">
    <rom name="captive1/SOUND/VCBASE4.MID" size="3192" crc="de7443ed" md5="4e5230ef6b5dcd972f3d6d59f672c898">
    <rom name="captive1/SOUND/VCBASE6.MID" size="3192" crc="e33eb3ec" md5="0b49da7b1d61badb5b0947e726640727">
    <rom name="captive1/SOUND/BATT7.MID" size="2376" crc="ea1b57a0" md5="f57bb0d647b30453eeed88cdbb821451">
    <rom name="captive1/SOUND/CAP_R.BIN" size="3959" crc="ecc42cbc" md5="ec4273c1c32b892d0d6eb05bbbf5a58c">
    <rom name="captive1/SOUND/VCBASE8.MID" size="3192" crc="17b2339f" md5="962a16f1d6df5dd768d8da78b729d217">
    <rom name="captive1/SOUND/LONGNTB.MID" size="378" crc="ee82159f" md5="e04d25cab4511d31744c23b0e25a2696">
    <rom name="captive1/SOUND/W8.MID" size="1284" crc="f1634865" md5="e498902b185e25327a93ea1fabf288fe">
    <rom name="captive1/SOUND/NEW_ADL.BIN" size="6094" crc="f4f471f9" md5="5ba4590445566c084a10d8bbaaa9feaa">
    <rom name="captive1/SOUND/SHOPKEEP.MID" size="5306" crc="f63924d0" md5="aec4e04a70d7bdece03188ed3d567492">
    <rom name="captive1/SOUND/LONGNT9.MID" size="378" crc="f8a0da06" md5="cb039450fdb87372812dceacc2d25b73">
    <rom name="captive1/SOUND/NEW_ROL.BIN" size="1584" crc="19abea48" md5="8ab516d8fe41dc3957542c8fb14e960f">
    <rom name="captive1/SOUND/LONGNT7.MID" size="378" crc="2a39687" md5="454bf490c628099f248eaf80bddb670c">
    <rom name="captive1/SOUND/FCBASE6.MID" size="1598" crc="225afe48" md5="fe01fdc246d2af7ef9c99af04392611b">
    <rom name="captive1/SOUND/MAIN2.MID" size="14585" crc="23866b33" md5="050743d698dae82a3974a315f1abb3e0">
    <rom name="captive1/SOUND/BATT1.MID" size="2368" crc="668d53" md5="a8215ba58589474a3fcc927d2d8dfdb9">
    <rom name="captive1/SOUND/SB20.CTV" size="3910" crc="285956d3" md5="9ff1b8d079b746e0a46341ff45fcaca5">
    <rom name="captive1/SOUND/W7.MID" size="744" crc="296a3ded" md5="d94120135e9be98b85ac17b361933c13">
    <rom name="captive1/SOUND/WB.MID" size="10553" crc="2aff5d2a" md5="dd0c9f717a91ad9a83016148cb14da23">
    <rom name="captive1/SOUND/BATT8.MID" size="2376" crc="2c0660a7" md5="5ff83b88c742349f70e158913814701a">
    <rom name="captive1/SOUND/FCBASE7.MID" size="1598" crc="2cbc97c7" md5="40347132e6527c3b86e50a28c10a35d6">
    <rom name="captive1/SOUND/FCBASE4.MID" size="1598" crc="31cc3988" md5="aee0093ff8281ecce16a132c6e246936">
    <rom name="captive1/SOUND/BATT6.MID" size="2376" crc="3284ab40" md5="3f6b647549dd02aa8c17c4f4de2cf185">
    <rom name="captive1/SOUND/GENBASE.MID" size="3507" crc="3a491284" md5="9b10119284c3e181151eb3e9f855bdf0">
<machine name=" Chamber/">
    <rom name="Chamber/KULT1.PXI" size="140537" crc="3c6c776c" md5="5e53b765c4cd557627946628d9da42d3">
    <rom name="Chamber/PUZZL.BIN" size="45671" crc="6171cfb" md5="eb161d329debd3429cba51748c11d9d4">
    <rom name="Chamber/FIXIN.EXE" size="9001" crc="42d3289f" md5="00773e736752b3de03516cb7bc8d48b7">
    <rom name="Chamber/DIALE.BIN" size="9636" crc="4d99623c" md5="16fd83c6f8aa821a54d8e7f2eea201b8">
    <rom name="Chamber/BRUIT1.EFF" size="718" crc="51ac8a7a" md5="7b662d04238296e2e86f2e3dcd8dc395">
    <rom name="Chamber/FOND.BIN" size="4476" crc="574d0bca" md5="0bd98a7443ccf6cfdef29ecc41d9f2a6">
    <rom name="Chamber/BRUIT5.EFF" size="547" crc="5aef701d" md5="b6d1ad77b2e283cf5c3aed6259c7b7f6">
    <rom name="Chamber/BRUIT6.EFF" size="472" crc="6577e489" md5="137701084e50d0bae7a58ee17a964f00">
    <rom name="Chamber/PERS1.BIN" size="14294" crc="6d0bdad4" md5="0b20505d6f0da49214b5e76529a265d2">
    <rom name="Chamber/RESIDENT.EFF" size="1721" crc="70966b8a" md5="9dc1e462259ad891243f681cbd7d3e0d">
    <rom name="Chamber/PRESEGA.EGA" size="32000" crc="728d3cef" md5="b0a6474fa95276255f8eb85c12aab031">
    <rom name="Chamber/PUZZL.EGA" size="37902" crc="7df3ab68" md5="318a880fb8960e66e2f13f2baa2ce04f">
    <rom name="Chamber/MOTSE.BIN" size="1082" crc="82658b00" md5="e833bf3f307ce3e046778c0aa34919b8">
    <rom name="Chamber/VEPCE.BIN" size="1345" crc="89b8b0ab" md5="fb104b51a5327eccc1376ac85c0c7a1f">
    <rom name="Chamber/BRUIT3.EFF" size="1032" crc="925e6f1c" md5="39d6c2d54999d9e35498a7de6a08797c">
    <rom name="Chamber/SPRIT.EGA" size="34274" crc="958f6595" md5="b4be1e2fa7e06a2e05a451b5287b3334">
    <rom name="Chamber/BRUIT2.EFF" size="409" crc="980d5c0a" md5="fab2f4b3f4eefe0b3d09e8ddd65030dd">
    <rom name="Chamber/PERS2.BIN" size="10587" crc="126681e3" md5="60d32e562a844450750a1e54b48c65ca">
    <rom name="Chamber/BRUIT0.EFF" size="2839" crc="12cbabca" md5="07a3cd1457798b0c9516a84612d8863f">
    <rom name="Chamber/BRUIT7.EFF" size="531" crc="be6cdb0d" md5="af8192abb2252e4565c94994ed83fd3d">
    <rom name="Chamber/BRUIT4.EFF" size="302" crc="ce2bc511" md5="d4b59fc911385391f33d3657745dedc3">
    <rom name="Chamber/DESCE.BIN" size="10515" crc="158bdf95" md5="e308e5ed37fb623b2b67cff9600a87ab">
    <rom name="Chamber/PERSO.EGA" size="38274" crc="e123a3ac" md5="994ae435c4c6310691db997df76fd972">
    <rom name="Chamber/LOADER.EXE" size="7275" crc="e2505d34" md5="9486ca71e876fd01a17befb5a4360642">
    <rom name="Chamber/FOND.EGA" size="32000" crc="eeb34603" md5="1c7f0b5eb7fa3c63afa5d79c4a2455ea">
    <rom name="Chamber/CLEAR.BIN" size="10303" crc="fea44c41" md5="80f486a3cff8b3608e44342988816632">
    <rom name="Chamber/PUZZ1.EGA" size="42784" crc="2711bae6" md5="c35af4292807a658f3b24fa0287f4065">
    <rom name="Chamber/KULT2.PXI" size="142658" crc="2acce592" md5="2f5cea29072e8c16bf4724a0d0a2201d">
    <rom name="Chamber/CONFIG.TAT" size="314" crc="32683e10" md5="8b2621fb0a98de99fc124b6ca502f858">
    <rom name="Chamber/PRESCGA.BIN" size="5105" crc="52c2df7" md5="664a75a95340bde200a2752f54ba7aa7">
    <rom name="Chamber/SPRIT.BIN" size="23811" crc="34eb275b" md5="84a1ea56c1aafeca307cc6851594074a">
    <rom name="Chamber/INTRO.ADD" size="3348" crc="381f32eb" md5="f9da37326f40df6206e1bd4069207226">
<machine name=" DN2INS/">
    <rom name="DN2INS/DISK4.EXE" size="1042748" crc="477de5e6" md5="52dded2118b5fd8a9fdfe8827104557e">
    <rom name="DN2INS/DISK3.EXE" size="1454978" crc="737f3c42" md5="dfec8c09db135c4130d456b5de8bf8a8">
    <rom name="DN2INS/INSTENG.DIP" size="2147" crc="84692667" md5="f052c5feb8af5608ba4916c7b3e3fbe9">
    <rom name="DN2INS/INSTFRE.DIP" size="2861" crc="8b90c3f9" md5="12fa87c88543118e0b2dcfce4a3e31ce">
    <rom name="DN2INS/INSTGER.DIP" size="2750" crc="ab3a0adf" md5="c3ae3fc8da1a03acd05b96fd06a65edb">
    <rom name="DN2INS/DISK2.EXE" size="1439492" crc="d1c01e33" md5="f63e9feb0f74cd5ad87c0863badd4eb6">
    <rom name="DN2INS/INSTALL.EXE" size="33966" crc="161bab46" md5="470bbcbce83dbff23f351f8f45af89e2">
    <rom name="DN2INS/DISK1.EXE" size="1402039" crc="2599e665" md5="d0ba9f40d2dc3b779df836b3d63415fa">
<machine name=" drally/">
    <rom name="drally/FILE_ID.DIZ" size="493" crc="60a2237" md5="13a3190c6f81b5db20d08568bcbdf3e4">
    <rom name="drally/TR9.BPA" size="427555" crc="41d49868" md5="d1c52e5f38042b35917bd0a7cd5e241b">
    <rom name="drally/DR.SW7" size="2195" crc="525b266b" md5="1b8137a635cc8596698700fe4fb42ed1">
    <rom name="drally/MUSICS.BPA" size="2010456" crc="593046a5" md5="6d06d256ec80e4cdc280d020498aa9b0">
    <rom name="drally/TR4.BPA" size="260466" crc="6073deab" md5="1bec191b8b8652ca0c93b0bc30ae831f">
    <rom name="drally/RALLY.EXE" size="374230" crc="6af1be62" md5="d16fb4f939ce2d473d55c88a18b1d8c3">
    <rom name="drally/ENGINE.BPA" size="387193" crc="80117b2e" md5="d9e69a33ca77c9c8508a0d2c3ef232a6">
    <rom name="drally/IBFILES.BPA" size="85989" crc="8f62681f" md5="6e6b3e635299174bb55868530a516712">
    <rom name="drally/TR7.BPA" size="417431" crc="9e81c651" md5="fbe984a255d9ce78ef833127c9c72fc1">
    <rom name="drally/ORDER.FRM" size="5714" crc="a59abae2" md5="063d3bfda9c014b6395c1aa952ad2f8b">
    <rom name="drally/DRHELP.EXE" size="50293" crc="c19e46e1" md5="427b7f6cd59beb30fd538c780da200d9">
    <rom name="drally/LICENSE.TXT" size="6752" crc="c6c7f563" md5="8adcda2649cd05fa59d80d28609dcfb4">
    <rom name="drally/INSTALL.EXE" size="142318" crc="ce058fa7" md5="f0e56350a7a5dea7d312a21e192aa1c2">
    <rom name="drally/DR.CFG" size="2942" crc="de822ae0" md5="44af1d6a588ef530229f55f485746000">
    <rom name="drally/SETUP.EXE" size="34227" crc="17d05cf5" md5="871fd06b004416b82317e89e6a0dee06">
    <rom name="drally/DRSW11.SHR" size="6842403" crc="f6022cbe" md5="8076f44c28dfc800a9911e9922fb8022">
    <rom name="drally/RALLY.ICO" size="2238" crc="fd3017a1" md5="1b76affbcaae6c5f7d253e3a598e06c7">
    <rom name="drally/MENU.BPA" size="3198740" crc="1fb7ab47" md5="45f53ed7f20cbd0b6db41fa9d70b58f2">

 

[DrasticNerd]

I put this thing here for now, I made it in like 20 minutes, but must be considered ALPHA VERSION, it doesn't work well on some kind of structures.

I used it in Cygwin, but should work also on plain Linux, as the utilities "cksum" and "md5sum" seems present in many distros.

Whoa. That's extremely generous of you to help out like that. Especially as I had placed this project on indefinite hold (per my post back on 13 Oct)

If I can run this, the output will contain almost everything * I was looking for. (That's not a problem. I can easily script the remaning 1% by adapting existing code to modify the output file)

Question - How do I get this to run on Windows 64 bit? Do I need to download Cygwin? Any particular version? Will I need download any dependencies to get the "cksum" and "md5sum" functions?




* "Almost" = explanation below provided for the curious. The example output shown would not be usable as a rom dat as-is for the following reasons:

1. Xml header needs to be set
2. Description is missing (Each game needs a Description. 1:1 relationship) Since we have no context other than the files/folders themselves, I would reuse the machine name for the description.
3. Machine name has to equal the game foldername. The additions the script has made will need to be removed (i.e. space character prefix and path separator suffix)
4. rom name uses paths relative to the game level folder. (i.e. "captive1/ANIMS/TEST12.ANM" is including the game level folder)

None of these are game breaking. As I said, if I can get this script to run, I can easily tweak my existing code to handle the above 4.

 

[PowerGod]

Question - How do I get this to run on Windows 64 bit? Do I need to download Cygwin? Any particular version? Will I need download any dependencies to get the "cksum" and "md5sum" functions?

You can install the last version of Cygwin 64, the installer is also the package manager, so to add packages or update things you just run it again.
The requested packages to add are these:
- coreutil (that should contain both "cksum" and "md5sum")
- findutils
- sed
- grep
- bash

If you never used Cygwin before you should know that inside its environment your drives are mounted like this: "C:\folder\file.txt" becomes "/cygdrive/c/folder/file.txt".

 

[DrasticNerd]

You can install the last version of Cygwin 64, the installer is also the package manager, so to add packages or update things you just run it again.
The requested packages to add are these:
- coreutil (that should contain both "cksum" and "md5sum")
- findutils
- sed
- grep
- bash

If you never used Cygwin before you should know that inside its environment your drives are mounted like this: "C:\folder\file.txt" becomes "/cygdrive/c/folder/file.txt".

Thanks. I will give this a shot in a week or two.