Dos Box

[M-HT]

If we switch to normal ARM could it be more stable than using the thumb version?

I actually first wrote the backend using arm instructions, but I didn't get it 100% working and together with already mentioned reasons I decided to write the backend using thumb instructions. But if someone wants to take a look at it, be my guest.

Source here

I think the incentive here should be to overhaul DOSBox's recompiler structure, since it's quite limiting (I know what you mean, I've looked through it). This could benefit the PSP version too if suitably adapted for, but if it was anything more complex than it is I'm not sure we would have seen a PSP recompiler in the first place.

And sorry, but I'm not overhauling anything. I don't actually work on other people's emulators, so I can't really do much (can only start a few myself >_>)

Writing a new (or improving the existing) DOSBox recompiler would bring more effective recompiled code, but since I can't get even the simple backend to work, it would have to be done by someone else. Somebody volunteering?

Could you describe at all what the process of static recompilation was like for you, what you recompiled and how well it worked, how much you had to manually change, etc..

I spent a long time writing the recompiler and than I spent at least as much time to find and fix all bugs.
You can check for yourself how well it worked here: Albion in GP2x archive
The recompiler is not 100% automatic, so it needs some manual input. It doesn't recompile all x86 instructions - only those that are needed for Albion (no segment registers, flat memory access, ...).

 

[ldesnogu]

The recompiler is not 100% automatic, so it needs some manual input. It doesn't recompile all x86 instructions - only those that are needed for Albion (no segment registers, flat memory access, ...).

That's exactly how I converted the Atari ST game to C: pick up Hatari emulator, strip the UAE core it used from all instructions not used by the game, generate C and then manually remove useless stuff such as flag setting (a real pain but I was too lazy to implement data flow optimization ). And before starting I had of course a fully disassembled source file to identify code (for this part I used a more automated 68k disassembler I wrote that does some control flow analysis).

Not sure I would redo it in the same way. I would surely do some data flow analysis

 

[Pickle]

If I find some time (and some hardware), I will take a look at the ARM backend

You could start at my attempt to write the ARM backend (using thumb instruction set).
I am out of ideas why it doesn't work.

Source here

PS:
If you comment the definition of DRC_FLAGS_INVALIDATION_DCODE (line 32) it seems to work at the beginning, but still fails at a later point (instead of failing at the beginning).


M-HT could you explain a little more about what doesnt work or whats happens? For example does it just crash for no reason? Do you even get to the prompt?

 

[M-HT]

M-HT could you explain a little more about what doesnt work or whats happens? For example does it just crash for no reason? Do you even get to the prompt?

I tested it on one game (Tyrian) - the game gets to menu, but while loading level (or after it) the game crashes with runtime error 200 (if memory serves me right) - this doesn't happen with simple core.
Of course it might be error in the dynamic recompiler and not my backend, but I tried the recompiler on x86 and it worked there.

 

[Exophase]

M-HT could you explain a little more about what doesnt work or whats happens? For example does it just crash for no reason? Do you even get to the prompt?

I tested it on one game (Tyrian) - the game gets to menu, but while loading level (or after it) the game crashes with runtime error 200 (if memory serves me right) - this doesn't happen with simple core.
Of course it might be error in the dynamic recompiler and not my backend, but I tried the recompiler on x86 and it worked there.


Are you handling cache clears? Does DOSBox have provisions for these? This is what has to be done just in case you aren't:

- If it uses one big translation cache that linearly grows end to end as new blocks are added then you have to invalidate the icache line at the start of each newly recompiled block because it could have already been in icache.
- For each newly recompiled block you have to write back (not invalidate, but it won't really matter if you do) every line in dcache that it spans, otherwise it won't be in memory when the CPU prepares to load it into icache.
- If you flush the translation cache for whatever reason (like if it gets too large or because of self modifying code) then you have to invalidate those lines in icache. If given the option it's probably better to just invalidate all of icache.

On GP2X the function to clear cache does both icache and dcache, so doing step 2 would double for step 1.

 

[Pickle]

Are you handling cache clears? Does DOSBox have provisions for these? This is what has to be done just in case you aren't:

Im not much of an expert on this, but there appears to be cache clearing in cache.h.

 

[M-HT]

Are you handling cache clears? Does DOSBox have provisions for these?

I use this code after writing cache block. I have no idea if this is correct or not.
CODE

{
register unsigned long _beg __asm ("a1") = (unsigned long)(block->cache.start); // block start
register unsigned long _end __asm ("a2") = (unsigned long)(cache.pos); // block end
register unsigned long _flg __asm ("a3") = 0;
__asm __volatile ("swi 0x9f0002 @ sys_cacheflush"
: /* no outputs */
: /* no inputs */
: "a1");
}

Im not much of an expert on this, but there appears to be cache clearing in cache.h.

Which part are you refering to ? Because I didn't find anything about cache clearing in DOSBox.

 

[Pickle]

Which part are you refering to ? Because I didn't find anything about cache clearing in DOSBox.


Dont forget i didnt claim to know what I was talking about, but there is a call called CodePageHandlerDynRec that has member functions that appear to clear the cache blocks?

Edit: Is that clearing asm code you just posted new or is in another file? I dont see it the backend header file.

 

[ldesnogu]

Im not much of an expert on this, but there appears to be cache clearing in cache.h.

Which part are you refering to ? Because I didn't find anything about cache clearing in DOSBox.

Neither did I

 

[Pickle]

Im not much of an expert on this, but there appears to be cache clearing in cache.h.

Which part are you refering to ? Because I didn't find anything about cache clearing in DOSBox.

Neither did I
Ok now i feel like the village idiot... :unsure:

 

[ldesnogu]

Ok now i feel like the village idiot... :unsure:

You should not, at least I did not try to make you look idiot or whatever.

Now I wonder how it can work on the PSP... Or perhaps the code pages are mapped uncached which would not be good for performance.

 

[Pickle]

Ok now i feel like the village idiot... :unsure:

You should not, at least I did not try to make you look idiot or whatever.

Now I wonder how it can work on the PSP... Or perhaps the code pages are mapped uncached which would not be good for performance.

it was more of joke, I dont take things that seriously( although I do wish I knew more of what im looking at.)
I do remember one of the dosbox guys mentioning special RAM banks in the PSP, I think it was in the thread where I was trying to get dosbox to run in the first place and you suggested the stuff the PORTING doc. It may be that its not related.

 

[mrdave]

you can get warcraft 2 for psx, which can be emulated by a psx emu

 

[M-HT]

Dont forget i didnt claim to know what I was talking about, but there is a call called CodePageHandlerDynRec that has member functions that appear to clear the cache blocks?

It was probably method InvalidateRange, but this this method (and the whole class) deals with memory that DOSBos uses to store recompiled code (= cache) and not the physical cache in processor.

Is that clearing asm code you just posted new or is in another file? I dont see it the backend header file.

It's new - I added it into function cache_closeblock (in cache.h) when I thought the problems in recompiler were caused by not clearing the cache.

 

[Pickle]

It's new - I added it into function cache_closeblock (in cache.h) when I thought the problems in recompiler were caused by not clearing the cache.


Have you had any chances to test it?

 

[M-HT]

It's new - I added it into function cache_closeblock (in cache.h) when I thought the problems in recompiler were caused by not clearing the cache.


Have you had any chances to test it?

I added and tested it before posting my recompiler backends - meaning, if it does what I think it should be doing then the problems with recompiler are not related to cache clearing (or not cache clearing).

 

[Exophase]

It's new - I added it into function cache_closeblock (in cache.h) when I thought the problems in recompiler were caused by not clearing the cache.


Have you had any chances to test it?

I added and tested it before posting my recompiler backends - meaning, if it does what I think it should be doing then the problems with recompiler are not related to cache clearing (or not cache clearing).

You covered the first two conditions I mentioned, but not the third. The third one is not routine, but it could still happen - especially with DOS games that overlay code blocks.

Anyway, the best course of action here is to run the debugger against a PC version, which I assume it would be synchronized against (no real reason why it shouldn't be). This is also assuming it has a debugger which can single step recompiled code... I doubt they'd have made it very far without one though.

Since you say the problem is after a menu is entered, you can either hard code input triggers at certain points or find a new PC after that region and put a breakpoint at it. You can find viable new PCs by making the recompiler print out which blocks it's compiling. Then just break to them and work backwards until the output of the PC version and the GP2X one are the same, then step forward until they aren't. If there's multi-step (skip N instructions) then that could make things easier, you could binary search out the divergence.

Wherever it diverges you can look at the disassembly and then see what looks wrong. And you could clear cache at each step to narrow that out if you suspect it.

EDIT: Also, someone really ought to put the cache handling in DOSBox proper (and the x86 versions can just do nothing)... probably as macros...

 

[M-HT]

You covered the first two conditions I mentioned, but not the third. The third one is not routine, but it could still happen - especially with DOS games that overlay code blocks.

Ok, I tried also clearing the cache after clearing block of translation cache, but the result is the same.

Anyway, the best course of action here is to run the debugger against a PC version, which I assume it would be synchronized against (no real reason why it shouldn't be). This is also assuming it has a debugger which can single step recompiled code... I doubt they'd have made it very far without one though.

Since you say the problem is after a menu is entered, you can either hard code input triggers at certain points or find a new PC after that region and put a breakpoint at it. You can find viable new PCs by making the recompiler print out which blocks it's compiling. Then just break to them and work backwards until the output of the PC version and the GP2X one are the same, then step forward until they aren't. If there's multi-step (skip N instructions) then that could make things easier, you could binary search out the divergence.

Wherever it diverges you can look at the disassembly and then see what looks wrong. And you could clear cache at each step to narrow that out if you suspect it.

Good idea, but I will probably never be bored enough to do something like this.

 

[Pickle]

Good idea, but I will probably never be bored enough to do something like this.


That stinks....seems like your close.

 

[Exophase]

Good idea, but I will probably never be bored enough to do something like this.

Oh well, guess if you're not man enough to see the job through then poor Pickle will have to pick all of it up instead.