8086 (ibm Pc) Static Recompiler For Gp2x

[pupnik]

i have plenty of low level x86 cough. hm.

*Cough* dynrec... dosbox... *cough*

I spent the past few days trying everything available to tweak some more performance out of dosbox 0.72+cvs

Results have been fairly disappointing. Here's how wing commander spends its time (running on TI OMAP2420 - arm core)

samples % app name symbol name
144808 31.9365 dosbox CPU_Core_Simple_Run()
43815 9.6631 dosbox mem_writew(unsigned int, unsigned short)
39148 8.6338 dosbox OPL2::YM3812UpdateOne(int, short*, int)
33403 7.3668 no-vmlinux (no symbols)
32406 7.1469 dosbox mem_readw(unsigned int)
24525 5.4088 dosbox VGA_ChainedVGA_Handler::writew(unsigned int, unsigned int)

All games profiled show core taking 25-35% of the time, so speeding this up with dynrec would be a dream come true. Please don't let this idea die.

The second most time-consuming portion is the fmOPL synth - whether opl2, dualopl2 or opl3 - an assembly optimized version would be a huge boost.

 

[sand_man]

nice bump.
i actually thought something had come out of this static recomplier.
sorely disappointed.

 

[Exophase]

I tried replying to this like 3 times now and somehow never seemed to finish what I was saying.. let me try again.

I'd say that the core CPU emulation is actually using closer to 50%, if you include the memory reading/writing. I say this because a recompiler should be able to help with this too. I'd like to know where the rest is going. And what's spending those cycles in Linux - are they syscalls? I wonder which.

One of the interesting things about the original PCs is that I don't think there's anything mapped to the address space that isn't just memory, that is, nothing that has side-effects. Just RAM and video RAM and whatnot. If this is taken into consideration then faster/simpler emulation could be done. I would think that DOSBox wouldn't ever emulate like this, if only to allow provisions for other memory mapped devices to be allocated there.

Wing Commander's minimum requirements are a 12MHz x86 with 640KB of RAM. What isn't clear is what kind of x86 they're referring to (8086? 286?).. but a 12MHz 8086 PC should be very feasible for emulation.

I think it'd be interesting if someone wrote a new emulator that just did simple PCs, but was geared towards top performance. I'm not sure how much demand there'd be for one though.

 

[slaanesh]

Interest in a dosbox would be high for me if the performance was there.
If it's not really going with enough oomph, then it's little more than a tech demo. Nice, but of no long term use.

I think a nice benchmark would be being able to run the old Microprose games like Railroad Tycoon and Civilization (with touch screen). Okay you don't need much FPS, but you still want reasonable speed as the computer sometimes took ages to make it's moves at later stages of the game.

Mmmmm.... Handheld PC Civilization. Railroad Tycoon. Fantasy General (okay that's an SSI but I loved this game too).

[Yes I know, play the Amiga version or ST version, but swapping those virtual floppies around is a pain the ass.]

 

[Blah]

IIRC, the old Pascal games were interpreter + bytecode, not actually (or at least not completely) native code.

 

[WhiteFalcon]

Whatever happened to the so promisingly looking project?

 

[Ze_ro]

[Yes I know, play the Amiga version or ST version, but swapping those virtual floppies around is a pain the ass.]

Does UAE4ALL support hard drives? Because you could theoretically set up WHDLoad within the emulator and avoid disk changes entirely... It's been a while since I've tried Amiga stuff on the GP2X though, so not sure what kind of speed this would get.

It still amazes me that no one has written an interpreter for the SSI Gold Box games yet... Maybe there are too many hard-coded rules (Dragonlance vs. Forgotten Realms vs. Buck Rogers) to make it work properly or something? I used to love these games when I was a kid... I wish I had the time to look into a project like this.

--Zero

 

[namco]

So, from the looks of things this is a x86 to ARM layer to achieve the same performance from the original x86 code, much like how DOSEmu is written?

 

[Exophase]

So, from the looks of things this is a x86 to ARM layer to achieve the same performance from the original x86 code, much like how DOSEmu is written?

No. DOSEmu uses virtualization (using the x86's virtual 8086 mode which was made specifically for this purpose). Static recompilation is an attempt to translate an executable written for one platform into one that can be ran on another platform. In this case (and in any other case I've ever heard of, really) some of the work had to be done by hand because it couldn't be fully automated. Even if it were fully automated (which is infeasible for the general case) it might still be much slower than the original would have been on "similar" hardware, whatever that is.

 

[JakeK]

So what happened to it this? I was getting excited reading it then looked at the date of posting and it was 2006! So I thought "surely something has been done" but nothing!

Dang.

 

[Lint]

Ok, just some comments:
Protected mode emulation could be possible by doing high level emulation of a DPMI, there are some games that accept running exclusively on ring 3 (lowest protection) so we could run this as plain linux executables, as long as the program itself don't try to mess with cr0, lmsw, etc... All these things are (should) be handled by the dos extender, and normally the dos extender obeys the dpmi, BUT anyway I don't think that this is a way to go since we're barely able to run 4.77mhz (or perhaps 8mhz) 8086

What program handle the dos/bios ints? Are those recompiled too? Makes me wonder.... Is it dosbox or will it ever be that?

 

[hilarycheng]

I am busy @ my work and no time to finish it at all. The major problem was the game will modify the memory itself. You have to translate the game one by one without general rules. The only way to do it is using Dynrec Code.

I have studied the QEmu. It seems its coding style is much easy to write it and implement for other platforms.

 

[BadAstronaut]

I am busy @ my work and no time to finish it at all.

So does this mean the project is officially dead? Or is there someone else who is interested in taking it over?

Between this project potentially being over, and DOSBox on GP2X not really able to run a hell of a lot at the moment, am I right in thinking that we may never see much in the way of improvements in running DOS games on the GP2X?

 

[Jaguarandine]

I am busy @ my work and no time to finish it at all. The major problem was the game will modify the memory itself. You have to translate the game one by one without general rules. The only way to do it is using Dynrec Code.

I have studied the QEmu. It seems its coding style is much easy to write it and implement for other platforms.

Hmm, another vote for Qemu. I think I see an interesting trend here
It'd be nice if someone took a serious look at it though.

Zodttd is unfortunetly distracted by the iPhone now. Maybe he'll make Simcity for the F-200 also

 

[hilarycheng]

I am busy @ my work and no time to finish it at all. The major problem was the game will modify the memory itself. You have to translate the game one by one without general rules. The only way to do it is using Dynrec Code.

I have studied the QEmu. It seems its coding style is much easy to write it and implement for other platforms.

Hmm, another vote for Qemu. I think I see an interesting trend here
It'd be nice if someone took a serious look at it though.

Zodttd is unfortunetly distracted by the iPhone now. Maybe he'll make Simcity for the F-200 also


I think QEmu make dynrec much more easy to implement it. I am not voting of direct of QEmu. It is too complicated for GP2X to drive it.

 

[Jaguarandine]

What do you think of DOSEMU running on top of QEMU? Sounds good to me.

Coincidentally, I'm actually learning 8086 assembly in school right now, but I wouldn't be of much help on this. Maybe in a few months

 

[hlidegp2x]

its what does apps did before DLL's came along. they loaded, some/all/parts of external executables into memory, relocated/linked and called them and unloaded them.

in fact, all modules which can be loaded in run-time and/or may be relocated to another address range : it is a situation a little similar to self-modified code, indeed.

QUOTE
you static recompile chunks into a buffer, execute chunks, etc. when the recompiler hits things like hardware register mods and screen addresses, interrupts, doscalls, dos low mem manipulation, you let dosbox handle that then you return to static recompiled code.

except for the static compiler can "optimize" this part by calling directly the function of the emulated hardware, which DOSBox probably cannot

QUOTE
its a lot harder, as you suddenly have more memory addressing modes (paged, segmented, flat, voodoo), access control (read/write/execute), protection levels (with x86 0,1,2,3), floating point included (som 386 had it, some didnt same with 486slc , etc etc etc)... lets throw in mmx, 3dnow, sse1, sse2 etc, a20 for protected mode.. mapping hi+low irq's (0-8, 70-78)... call gates, task gates, amd fastcall, intel fastcall...

lets not forget sound chips, gfx chips etc, bios calls, undocumented dos memory poking. single step interrupts, etc

oh... i guess you are speaking about some old 386 games which uses a DOS extender. But honestly, would you expect for a game running in protected mode to handle those kernel tasks ? I doubt it. Your best option is to rewrite this DOS extender (ususally the same for most 386 DOS games) as a library that static recompiler will link with the recompiled game code (and not recompile all the kernel stuff !)