1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Dismiss Notice

[WIP] Advance Projects and ROM management guide

Discussion in 'General Discussions' started by PowerGod, Mar 28, 2016.

  1. PowerGod

    PowerGod Advanced Member

    Joined:
    Jun 20, 2011
    Messages:
    2,446
    == WORK IN PROGRESS ==

    I'm trying to create a complete guide, easy to read but very explanatory, to describe how to use and manage ROM SETS for MAME-like emulators, but also I want to point out why you have to do this type of things.

    I never found a guide like this, I just found little guides here and there for the main graphical rom-managers, just to get you run, but them weren't telling "why" you had to do that, what if you select another option ?! Or what are those lists needed by them ?

    All the informations I found about the background of this practice were scattered around many forums.

    Also I never found an "easy" to follow guide for the Advance Projects, and this is the main point I want to cover.

    So, the main points to cover are:
    • Advance Projects GUIDE (possibly both for Windows and Linux)
      • AdvanceSCAN/AdvanceDIFF/AdvanceCOMP: to generate romsets and many tricks to optimize the work
      • AdvanceMAME: just some info to set the most useful configurations (then point to the included docs)
      • AdvanceMESS: this needs a better explanation than the previous one
      • AdvanceMENU: this will need a great part of the guide, I want to explain how to use it as a single interface for many emulators (also not related to Advance Projects)
    • Comparison with the other rom-managers out there, and maybe little guides for them too (but links to already existent guides can be a good start)

    And also:
    • Easy to understand (the order of the arguments can be important)
    • Not a wall of text (I'm trying to use spoilers for this, to separate the arguments)


    This is still at an unfinished and a very messy state, I did a mashup about Windows and Linux instructions because I use Cygwin, but I don't have a Linux box... but I want to separate the two type of instructions, and I can't find on Windows cool substitutes for the things I can do with a GNU terminal...


    I'll appreciate all the info/corrections/tricks you can give to me








    --- The guide ---

    I'll use some termns here that maybe not everyone will understand, so I here there's a little "glossary"
    MAME - Multi Arcade Machine Emulator, what to tell, the original project that started all the other ones. Maybe all the Arcade machines seemed all the same, but them were very different inside from one to another... this emulator wants to be able to run everything Arcade related.

    MESS - Multi Emulator Super System, the only difference from MAME is that it deals with home consoles :)

    SET (or ROMSET) - It's a collection of files, in an archive (mostly .zip/.rar/.7z, or just a folder), that contains a digital copy of the contents of a physical support (for example a cartridge). Obviously the main content of this supports were games :)

    ROM - It's a file contained in a SET, and it's a digital copy of one of the components of a physical support.

    SAMPLE - Audio file used by some games. These are not usually contained within SETs because many games can use the same samples.

    CHD - It's a file containing a copy of a physical hard drive, it is required by some SETs to be run. Their size can be HUGE for some games, many gigabytes.

    PARENT and CLONE - Many SETS can contain some of the same ROMS, in these cases, one of the SETS is called "PARENT" and contains all its required ROMS, while the other SETS are called "CLONES", and contains only their unique ROMS, accessing the needed others from the PARENT SET.
    (More info)
    Actually there are 3 main ways to manage the sets:

    Non merged SETS
    Both the parent an clones sets contains all their needed roms. (Clones don't need a parent to be run)

    Split SETS (or Differential Merging)
    This is how AdvanceSCAN works.
    The parent SET contains all it's needed ROMS, while the clones just contains their unique roms (them need a parent)

    Merged SETS
    This is not supported by many emulators.
    All the parents and clones roms are together forming a single set
    Also, a little reading to explain why it can be chaotic dealing with this kind of emulators, and how useful is to be able to manage ROMs
    Every version of a MAME-like emulator could need a different SETs/ROMs format, different CHDs images and different samples, it's as simple as that, you must accept the fact that if you update your emulator, your games could not work anymore.

    This is not because the programmers are evil, but because when something new is discovered on the original cartridges they try to use it, optimize it, and sometimes, a new copy of it in another format from the original support is required, making the older SETs/ROMs obsolete.

    Most of the time we can deal with this situation using managing softwares like "clrmamepro", "RomCenter" or the advertised "AdvanceSCAN".

    A manager will just "ask" to an emulator how it wants the SET/ROM files, how it wants them to be called and how the contained files must be to be run correctly, and after that it tries to follow all the instructions to modify the SET/ROM files in the requested way.

    This operation can preserve many games to be still playable for different versions of an emulator, but there are situations where there's no solution, for example when "missing files" happens...
    If an emulator reports a SET to be not playable because of "missing files", then the only solution is to find out a newer SET (or the missing ROMs), copied from the original cartridge in a different way than before. (Or stay with the older version of the emulator.)

    A "missing file" can also be a CHD, this is a file mostly separated from the SET, so maybe in this case it could have just been put in the wrong spot: mainly the .chd files must be uncompressed and put inside a folder with the same name of the SET zipped file (but this requirement may differs from an emulator to another).

    gamename.zip
    gamename
    |-> gamename.chd

    It's also true anyway that CHD files sometimes must be copied again from the original support, the same as ROMs...


    Q: What are those things ?
    A: A bunch of (multiplatform) programs to manage and play games (and other kind of software) made for different systems, like arcade machines and home consoles.
    There's a graphic interface, two emulators and a rom manager, all made to work together or standalone.

    Q: Isn't this MAME ?
    A: Yes and no, the two emulators are a fork of the MAME and MESS projects, I will not explain all the differences, let's just say that these are less user-friendly but mainly faster.
    (If you ever used "PanMAME" on the Pandora, you already know them, because it is mainly "AdvanceMENU + AdvanceMAME" )

    Q: What about the interface ?
    A: All the programs are command-line driven, but AdvanceMENU is a very cool graphic interface that can manage more than one emulator, also the ones not "Advance" like the original MAME, or mainly anything that can start from command-line.


    I'm on Windows now, so I'll have to leave away the compiling part, because you don't need it here, you can download directly the working executables from the official site
    (I'll add even Linux instructions in some future... anyway the configuration on Linux is almost the same)

    http://www.advancemame.it/download

    This guide will include also a limited support for others emulators like MAME, MAMEUI, FB ALPHA (and maybe more to come...)


    AdvMENU-mameUI64.png


    The following instructions are in the order needed to make everything run, also I will consider that every program from the Advance suite BLAH BLAH BLAH TO FINISH !!!


    AdvanceSCAN
    • First start and configuration
      - Extract the content of the archive in a folder

      - Open the file "advscan.rc" with a text editor

      - Modify the contents of the file as required, in this example I will consider that checked ROM files will be put in "c:\arcade\roms", samples in "c:\arcade\samples" and CHDs in "c:\arcade\chd".
      Also, to not modify the original rom files, I'll use a "read-only" starting directory "c:\arcade\newroms", where I consider there are all the original rom files.
      (Samples and CHDs in their folders will be directly managed, so if you want to maintain your original files intact you should also copy them somewhere else before the scan)
      (Lines starting with # are comments)

      Code:
      # starting ROM paths (the files here will not be modified, but just copied to the rom folder, it can be useful to add more roms to the collection without modifying the original ones)
      rom_import c:\arcade\newroms
      # ROM paths where files will be copied/fixed
      rom c:\arcade\roms
      # ROM path where new archives will be created (should be the same as the rom path, or at least added there to not recreate the new archives everytime)
      rom_new c:\arcade\roms
      # ROM path where unknown files will be moved (it will be read again at every scan, to get back files that can become usable with a different version of the emulator)
      rom_unknown c:\arcade\roms_unknown
      # SAMPLES path where files will be checked/corrected
      sample c:\arcade\sample
      # SAMPLES path where unknown/wrong files will be moved
      sample_unknown c:\arcade\sample_unknown
      # CHD paths
      disk c:\arcade\chd
      # CHD path where unknown/not working files will be moved
      disk_unknown c:\arcade\chd_unknown
      
    • Obtaining a list of supported ROMSETS
      To get those lists you will need to use a console:
      - For Windows, cmd console (press WIN+R ,and type cmd)
      - For Linux, every terminal should work

      AdvanceMAME
      - Go to the advmame folder
      Code:
      advmame -listxml > romlist.lst
      
      MAME
      - Go to the mame folder
      Code:
      mame -listxml > romlist.lst
      
      MAMEUI
      - Go to the mameui folder
      Code:
      mameui -listxml > romlist.lst
      
      FB Alpha
      - Go to the FBA folder
      Code:
      fba -listinfo > romlist_tmp.lst
      
      WARNING
      In this particular case, the list is not AdvanceSCAN compatible, so it must be edited a bit before the use.

      Modify with a text editor
      Replace every one of these (without the quotes)

      Code:
      "<datafile>"   TO  "<mame>"
      "</datafile>"  TO  "</mame>"
      "<game name"   TO  "<machine name"
      "</game>"      TO  "</machine>"
      

      Modify with a *nix terminal
      Note that the "romlist_tmp.lst" is used as a source file, while the new created file is "romlist.lst"
      Code:
      sed 's#\(</*\)game\(\s*\)#\1machine\2#' romlist_tmp.lst | sed 's#\(</*\)datafile\(>\)#\1mame\2#' > romlist.lst
      
    • Managing ROMSETS, CHDS and SAMPLES
      To use the scanner you will need to open a console:
      - For Windows, cmd console (press WIN+R ,and type cmd)
      - For Linux, every terminal should work

      - Go to the advscan folder

      - Start the ROMSETS scan/fix with
      Code:
      advscan -R -t -v -f working < romlist.lst"
      




      - CHD scan works only with uncompressed files, and must be all together in the same directory (the scanner can't find them in subdirs)
      Code:
      advscan -K -v -c MAME169b_CHD.rc -f working < romlist.lst
      
      -- OPTIONAL --
      - If you want to put all the CHDs back in their respective folder, use this command from the CHD directory, it works only on *nix environments (like Cygwin on Windows)
      (just change "romlist.lst" to the right list. It could give a lot of error messages about files not found, but that's normal)
      Code:
      sed 's#>#\>\n#g' romlist.lst | grep "machine name\|disk name" | tr -d '\n\r' | sed 's#<machine name="#\n<machine name="#g' | grep "disk name" | sed 's#\(machine name=\|disk name=\)"\(\w*\)[^"]*[^>]*#\1 \2#g' | sed 's#>\s*##'  |sed 's#<machine name=\s*\(\w*\)#mkdir \1; cp -t \1/ #' | sed 's#<disk name=\s*\(\w*\)>#\1.chd#g' | sed 's#$#;#' | sed 's#\s\+# #g' | grep -v "ATTLIST" | bash
      
      - Because the previous command created directories even for the CHDs that were not physically present (but only on the list), we can delete every empty folder, just launching this in the CHD folder:
      Code:
      find . -empty -type d -delete
      
    • Merging all your ROMSET directories into one
      Let's say you have multiple folders with roms sets, and many of them have the same names, but somehow have different contents...
      How do you know what is better to keep and what you can delete ? How do you find what folder was for what emulator ?

      Answer: don't mind about all those things !! ;)

      Just move ALL those files in the same directory, without overwriting them, just give new names to the ones already present, and then just launch AdvanceSCAN in that folder using the latest MAME list.

      It will rename everything in the right way, keeping just the right roms while removing dupes, and putting all the not needed things to the trash (removing when possible the dupes even there)

      Now you have just 1 folder with everything you need, and you can keep it to be used as a "master", to regenerate all the rom sets for all the emulators you use ;)
    • Cleaning the trash (rom_unknown) folder
      AdvSCAN puts in the trash only a copy of every known ROM, but it can't do this for unknown ones, so in this case you can have a lot of duplicates of files that can keep space.
      While those files are actually unknown for the used data list, them could be useful for another emulator, and AdvSCAN will automatically take them back from the trash,
      but it needs just one copy (doesn't mind the name or the location), so the other duplicates can be deleted.

      Here there's a solution (only for *nix systems) to extract the archives, remove the duplicate files and recompress everything
      (fdupes, 7z, grep and sed are needed for this)

      WARNING !! Use the following commands only after you are already in the right folder

      - to extract all the zips in different folders, also deleting the original zips
      Code:
      find . -type f | grep -i ".zip$" | sed 's/\.[Zz][Ii][Pp]$//' | xargs -I{} 7z e -r -tzip {}.zip -o{}/ && rm -v *.zip
      
      - to remove all the duplicate of ROMs and also empty directories
      Code:
      fdupes -f -r . | sed 's#\s#\\ #g' | xargs -r rm -v ; find . -type d -empty -delete
      
      - recompress all the folders in separated zips
      Code:
      find . -type d | grep -v "^.$" | xargs -I{} 7z a -r -tzip {}.zip {}/* && find . -type d | grep -v "^.$" | xargs -r rm -r -v
      
    • Some hints:
      WARNING: Using different lists from different emulators to regenerate always the same ROMSET folder, can make disappear some of the ROM files, but using the "rom_import" path will prevent this !!

      Also, you can change the rom, rom_new and rom_unknown destination paths for every different list you use, so to keep different working compilations
    AdvanceMAME

    • - Extract the content of the archive in a folder and launch "advmame.exe", it will give an error but will also create some needed configuration files.

      - Open the file "advmame.rc" with a text editor and search for the line that starts with "dir_rom"

      - Substitute the last "rom" in "dir_rom rom" with the path containing your ROM files. Example: "dir_rom c:\mame\roms"

      - Copy the snapshot images in the "snap" folder (if it's not there, create it)

      - Set the images pixellated as the old times :)
      #display_resizeeffect auto
      display_resizeeffect none
    AdvanceMESS

    • This emulator works in a different way compared to the others, it doesn't run the games directly, but must use external engines, that in this case are called ROMS, to launch the games (cartridges/tapes/etc..) that are called IMAGES.

      - Extract the content of the archive in a folder and launch "advmess.exe", it will give an error but will also create some needed configuration files.

      - Open the file "advmess.rc" with a text editor and search for the line that starts with "dir_rom"

      - Substitute the last "rom" in "dir_rom rom" with the path containing your ROM files (that in this case are actually the "consoles/machines" emulated, NOT the games). Example: "dir_rom c:\mess\roms"

      - Do the same for the line that starts with "dir_image", but put the path containing your games. Example: "dir_rom c:\mess\images"

      Now, AdvMESS needs the games to be separated by emulator, so the final structure of the folder should be like this:

      c:\mess\images\gamegear\game1.zip
      c:\mess\images\gamegear\game2.zip
      c:\mess\images\gamegear\game3.zip
      c:\mess\images\sms\game1.zip
      c:\mess\images\sms\game2.zip
      c:\mess\images\genesis\game1.zip
      c:\mess\images\genesis\game2.zip
      c:\mess\images\genesis\game3.zip
      c:\mess\images\genesis\game4.zip
      etc...

    AdvanceMENU
    • First start and configuration
      - Extract the content of the archive in a folder and launch "advmenu.exe", it will give an error but will also create some needed configuration files.

      - Open the file "advmenu.rc" with a text editor

      - Add the lines for the emulators you want to use, with the correct path to the executables, for example:

      Code:
      emulator "advmame" advmame "C:\advancemame\advmame.exe" ""
      
    • Manage also others emulators
      AdvanceMENU is already capable to use many emulators similar to AdvanceMAME, and that means that it can find their configuration and already knows what parameters to use for running them.
      There is also a way to add emulators unknown to it, but you'll have to add also some instructions about using them.

      Other examples
      emulator "advmess" advmess "C:\advancemess\advmess.exe" ""
      emulator "sdlmame" sdlmame "mame" ""

      MAMEUI
      - Start MAMEUI at least one time, to let it create the "mame.ini" file (it will be in the INI folder).
      - With MAMEUI closed, edit that file and change the line "inipath ini" to "inipath ."
      - Modify the relevant paths (rompath/samplepath/snapshot_directory) if needed, from the MAMEUI interface "Options->Directories" menu, or directly editing the "mame.ini" file (with MAMEUI closed)

      - Now that the emulator is ready, add the following line to the "advmenu.rc" file, just change "C:\MameUI\Mameui.exe" to the right path
      Code:
      emulator "MAMEUI" mame "C:\MameUI\Mameui.exe" ""
      

      MAME
      - Create the "mame.ini" file launching the emulator with this command:
      Code:
      mame -createconfig
      
      - Modify the relevant paths (rompath/samplepath/snapshot_directory) if needed, directly editing the "mame.ini" file

      - Now that the emulator is ready, add the following line to the "advmenu.rc" file, just change "C:\Mame\Mame.exe" to the right path
      Code:
      emulator "mame" mame "C:\Mame\Mame.exe" ""
      

      FB Alpha
      - Start FBA to make it create all it's needed files

      - Now the emulator is ready (AdvanceMENU will not use it's configuration)

      - Add the following lines in the "advmenu.rc" file, just change "c:\FBA\fba.exe" to the right path, "1024x768x32" to the resolution you like, and also, in this case, change the "c:\FBA\roms" path to the one you use
      Code:
      emulator "FB Alpha" generic "C:\FBA\fbai.exe" "%s -r 1024x768x32"
      emulator_roms "FB Alpha" "c:\FBA\roms"
      
      WARNING
      This has still some issues and I need to have a better look if it's possible to solve them...

      ISSUE: Fullscreen doesn't work and gives an error
      SOLUTION: Check carefully the resolution and the color deep set in the file above, all of them must be supported by both your monitor and video driver

      ISSUE: Fullscreen resolution is fixed, and doesn't adapt to the game proportions...
      SOLUTION: Start FBA and set the option "Video->Stretch->Full Stretch", after exiting, this option will be always used by default at every start

      ISSUE: The snapshots needs to be added
      SOLUTION: I'm working on it...

      ISSUE: Game info in the interface are missing
      SOLUTION: I'm working on it...
    • advmenu.rc - Useful configurations
      - Change the value of "display_size" to make the interface better fit your resolution. (For a FullHD display I found that "display_size 1280" works well)

      - To put the menu in fullscreen, search "device_video_output" and change "auto" to "fullscreen" (or comment the original line and add the new one):
      Code:
      #device_video_output auto
      device_video_output fullscreen
      
      - To use MOUSE and/or JOYSTICK to select games, you can search for "device_joystick" or "device_mouse" and edit those lines like this:
      Code:
      device_mouse auto
      device_joystick auto
      
      also you can define what does every buttons editing the values of "event_assign" lines, in the example the UP direction is assigned to both joystick button 2 and keyboard UP arrow key:
      Code:
      event_assign up 2_pad or up
      


    MAME TIPS
    • Using the MOUSE as a LIGHTGUN
      - Edit the "mame.ini" of the interested emulator, searching for the line "lightgun_device keyboard", and change it like this:
      Code:
      #lightgun_device           keyboard
      lightgun_device           mouse
      - Also some games require that the line "mouse" is set to "1"
      Code:
      mouse          1
      
      - Now start the game that require a LightGun and press TAB, go to Input (this Machine) and then assign mouse button(s) to fire key(s), mainly called P1 Button 1, etc...
      (Sometimes mouse input is not recognized well while assigning the key, so if clicking doesn't end the procedure, you could try to click and press ESC to end it manually, and at that point it should have assigned it.)

      - Configure mouse movement speed pressing TAB and going to Analog Controls, then modify the values of Lightgun X Sensitivity and Lightgun Y Sensitivity (100 is a good value for me)

      Special calibration required by some games like "Zombie Raid" (Where MAME cursor is not aligned with the game cursor)

      - At the start of the game press the SERVICE button (F2) and then the SOFT RESET (F3), you should see a test menu.
      In this case you use "Trigger/Reload" to move the menu cursor and "Player 1 start" to select.
      Select the "Gun Adjust Mode", "Gun Adjust", and then put the cursor in the lower right as pointed, then press the "Trigger", point to the upper right and press the "Trigger", repeat with Player 2 controls.
      Now disable the SERVICE mode (F2) and then SOFT RESET (F3) the game, it will now run properly calibrated.
      With "Zombie Raid" this procedure must be done everytime... seems like the game doesn't save the hardware configuration.
      Workaround to save the cabration for "Zombie Raid"
      - After the calibration is complete and you are ready to play, create a SAVE STATE (SHIFT+F7) and select a save slot (for example "1")
      - Now, everytime the game is launched, pressing the LOAD STATE (F7) and the slot number ("1" as the previous example) will make it already calibrated.
    • Assign more then one key to a function
      - From the "TAB" menu during a gameplay you can modify the assigned input key, but mainly the new key will substitute the last one... to solve this issue and ADD another key to the same input, you can modify it entering again the same value it had, and then modify it again using the new button, that this time will be added to the old one.
     
    Last edited: Sep 29, 2016
    levi, Linux-SWAT and Alpibrine28 like this.
  2. xnopasaranx

    xnopasaranx Superbad!

    Joined:
    Jan 31, 2006
    Messages:
    2,960
    super cool! now we need this as a pdf to have on the pandora. I was recently thinking, that there are so many guides and how-tos hidden in this forum, now that the pyra is coming maybe we should compile the most important tweaks, tuts and info bits into one big compendium to have on the pandora itself, before they disappear from the net.
     
  3. PowerGod

    PowerGod Advanced Member

    Joined:
    Jun 20, 2011
    Messages:
    2,446
    I'll leave the "super ultimate guide of everything" as the last thing to do, I want to advertise the Advance tools first, and I must admit that making an easy to follow guide for command line tools is not so easy in the end... but I'm still working on it.

    Anyway, if you have some useful links to other guides, I can at least add those for now, so to have a little "index"
     
  4. Alpibrine28

    Alpibrine28 .............................!

    Joined:
    Apr 11, 2015
    Messages:
    574
    Location:
    Script Kiddie's World
    If I could only understand what are you talking about... :p
     
  5. PowerGod

    PowerGod Advanced Member

    Joined:
    Jun 20, 2011
    Messages:
    2,446
    I suppose this is a joke... but if it's not, I'm interested to know your opinion, cause I want to put here as many explanations as possible
     
  6. Alpibrine28

    Alpibrine28 .............................!

    Joined:
    Apr 11, 2015
    Messages:
    574
    Location:
    Script Kiddie's World
    Yes, it is. This thread is about managing rom sets in MAME-like emulators, right? You gave the explanation at the beginning. I am just trying to point out that Pyra-Handheld Boards is full of people that tries to make their thread long but readable and explanatory. Thumbs up for a well-designed thread.
     
  7. PowerGod

    PowerGod Advanced Member

    Joined:
    Jun 20, 2011
    Messages:
    2,446
    damn, seems like advmenu doesn't support MESS :( but I will find a workaround ! :-||
     

Share This Page

Loading...