GP2X Proof Of Concept: Emulation Without Emulating The Cpu

[dmantione]

Hi!

I've written my program on the GP2X.

Emulators like GpSp for the Gameboy Advance also emulate the cpu. My question is "why?!", because the Gp2x has 2 ARM processors on board, so it seems nonsense to emulate an ARM processor.

So, I started doing some coding, seeing wether I could run a GBA rom on the native processor. I took a ROM from Francesco Lombardi's FPC4GBA project and tried to get it running on the Gp2x.

This now works. Don't expect an emulator, because the current program is barely worth such a description, and will probably not run anything besides this ROM (the only thing it emulates, in a primitive way, is GBA video mode 4). No BIOS, or any other hardware is emulated. But it runs the ROM, directly on the ARM920. The ARM940 runs the GBA mode 4 emulation.

The potential could be efficient (more than) full speed emulation with low battery drain, but likely less accurate emulation is possible (i.e. forget about clock cycle perfect emulations).

You can download the result here:

http://www.freepascal.org/~daniel/gbaemu_concept.zip

It contains a fpcLogo.gba ROM, which will run on your GBA or emulator.

Unzip it on your SD and run it, it runs on your Gp2x. There is no way to exit the emulation, the only way to stop it is to kill it. Powercycling your gp2x will of course work too.

I have source code available too:

http://www.freepascal.org/~daniel/gbaemu_concept_src.zip

I am encountering a few problems though:
- Sometimes the emulation is fast, often it is slow, the ARM940 seems to work very slow in such a case. I have no idea what is going on here, but probably I am not setting up the ARM940 correctly so it has poor performance, depending on the initial state when the emulator start.
- Running some programs before my emulator may cause my 940 code not to start at all. Again, my 940 initialization is probably incomplete.

Are there any ARM940 guru's on these boards that possibly might have an idea?

 

[Squidge]

Carry on with your emulator - you'll soon figure out where emulators such as GpSp emulate the processor rather than executing the code directly Believe us, I know there's at least 3 people on this board that have written a gba emulator running directly on the processor, and all three of us have come across the same problem in one way or another. The only real solution we have come up with is Dynarec, which is the best of both worlds. You run the code directly, and you still have almost full control over the executing program.

You might want to look up gba emus on the gp32 as well. The same thing was tried there for some college project - It ran a number of pd roms very well.

As for your 940 problems, grab the minimal sdk. It contains arm940 routines. I have some arm940 examples, but minimal sdk is more easily downloadable (my examples are probably hidden in some ancient thread on this board).

 

[Exophase]

Nice proof of concept, but before you write off recompiling emulators like gpSP and call them "nonsense" you should know that virtualization won't work for most real GBA games.

Most GBA games run code from IWRAM, as soon as this code is loaded into IWRAM on the emulator it'll get stuck in dcache and the icache fetch will fail.

I'll listen to your criticism of recompilation if you think of a way to combat this (I can't fathom anything). You'll note that the emulator GPAdvance, which attempted the same thing and finally concluded that cache of IWRAM has to be disabled. With the memory speeds on GP2X this is not acceptable, as compared to a recompiler.

What I will commend you for is demonstrating that a simple file mmap works and that Linux will happily let you mmap the GBA virtual address space without rejecting it. I'd like to know, however, what's actually going on behind the scenes with this since you're mapping to a device node.

The only concern I have with this method is whether or not it'll handle mirroring correctly. If it does then it'd be nice to finally incorporate it into some real emus.

 

[Squidge]

Hey, thanks for the heads up Exophase, I didn't realise he was using mmap (I noticed he said it was written in pascal and kind of abandoned the download at that point). Maybe that'll be a better idea than getting rid of linux to attack the mmu tables directly.

 

[Exophase]

One other thing: you aren't intercepting any memory accesses correct? Every region you've mapped is marked as read/write/exec (don't be fooled, there is no actual exec permissions on ARM920T, in case you've thought about using it to get around the problem I mentioned).. so you must be polling the IO registers once per frame to render.

I don't think this will generalize very well for the rest of GBA emulation. For instance, you need to trigger DMA as it happens: imagine if a game does several DMAs in a row (very likely) - if you're periodically polling the registers then you'll miss all but the last. Some things also need to be protected from being overwritten, like the line count register (yes, games do this >_>)

Of course this can be fixed by making the region write only, as well as ROM, BIOS, and the open regions need to be captured too. The question is, can a signal handler be installed to handle this properly? I do imagine that it can be caught, but can it be recovered from successfully so that the memory access is properly emulated? And how much overhead will it be?

 

[Hooka]

Is it only me getting dead links?

 

[Blah]

Dead links here, too.

I really wanted to see that code

 

[dmantione]

Dead links here, too.

I really wanted to see that code

Sorry! Server has a problem, it'll be back soon. In the meantime I have uploaded it here:

Binary:

http://www.hu.freepascal.org/~daniel/gbaemu_concept.zip

Source:

http://www.hu.freepascal.org/~daniel/gbaemu_concept_src.zip

 

[dmantione]

Edit: Somehow UBB

tags are not recognized in this post, switched to manual quoting.

>Nice proof of concept, but before you write off recompiling emulators like gpSP and call them "nonsense"
> you should know that virtualization won't work for most real GBA games.

> Most GBA games run code from IWRAM, as soon as this code is loaded into IWRAM on the emulator it'll
> get stuck in dcache and the icache fetch will fail.

> I'll listen to your criticism of recompilation if you think of a way to combat this (I can't fathom
> anything). You'll note that the emulator GPAdvance, which attempted the same thing and finally
> concluded that cache of IWRAM has to be disabled. With the memory speeds on GP2X this is not
> acceptable, as compared to a recompiler.

I haven't really thought out all details yet, so indeed there are some challenges ahead

How about a scheme like this:
- For each page, we keep a state
- State A is data RAM, we use mprotect to set it (PROT_READ or PROT_WRITE), cache enabled for that page.
- State B is code RAM, we use mprotect to set it (PROT_READ or PROT_EXEC), cache enabled for that page.
- State C is self modifying code RAM we use mprotect to set it (PROT_READ or PROT_WRITE or PROT_EXEC), cache is disabled for that page
- By default all pages are in state A
- In case of an execute, someone has apparently copied code into it. Catch the signal and put the page in state B
- In case of a subsequent write to this page, catch the signal and put the page in state C.

This way you could limit the disabling of the cache to only the code that really modifies itself, while keeping the rest of IWRAM fast. I'm not sure what the speed effect would be on a page in state C, if it is only for code I expect the penalty to be accepted, because processors generally fetch code in chunks of 32 bytes or so. However, if there are also variables in such a page a load/store to such a page would get a large penalty.

> What I will commend you for is demonstrating that a simple file mmap works and that Linux will happily
> let you mmap the GBA virtual address space without rejecting it. I'd like to know, however, what's
> actually going on behind the scenes with this since you're mapping to a device node.

Mmap rules You have to be a bit carefull though, because Linux uses anymous mmap to map in shared libraries and the heap is also allocated using anonymous mmap. You have to allocate things in the right order, otherwise there can be anonymous mmaps in the places you need GBA memory.

What happens if you mmap some special devices (like /dev/mem and /dev/fbdev) is that Linux uses the MMU to map physical memory into your address space. This functionality is used by XFree86 for example to get access to the video ram.

> The only concern I have with this method is whether or not it'll handle mirroring correctly. If it does
> then it'd be nice to finally incorporate it into some real emus.

What do you mean with mirroring?

 

[Exophase]

I haven't really thought out all details yet, so indeed there are some challenges ahead

How about a scheme like this:
- For each page, we keep a state
- State A is data RAM, we use mprotect to set it (PROT_READ or PROT_WRITE), cache enabled for that page.
- State B is code RAM, we use mprotect to set it (PROT_READ or PROT_EXEC), cache enabled for that page.
- State C is self modifying code RAM we use mprotect to set it (PROT_READ or PROT_WRITE or PROT_EXEC), cache is disabled for that page
- By default all pages are in state A
- In case of an execute, someone has apparently copied code into it. Catch the signal and put the page in state B
- In case of a subsequent write to this page, catch the signal and put the page in state C.

Like I mentioned later on in the thread, there isn't actually exec protection on ARM920T's MMU. I've checked several times and checked again today. You won't be able to turn off execution and catch it so you won't be able to determine when code in IWRAM is first executed without catching every read or write (remember, that's where the stack is). Most games load code into IWRAM because it's much faster there. Any that don't will have so fewer virtual cycles that they're probably much less of a concern anyway.

Mmap rules You have to be a bit carefull though, because Linux uses anymous mmap to map in shared libraries and the heap is also allocated using anonymous mmap. You have to allocate things in the right order, otherwise there can be anonymous mmaps in the places you need GBA memory.

If these anonymous mmaps happen before the program is executed then what can you even do to prevent it? How can ordering help? If not, why would they happen later on?

What happens if you mmap some special devices (like /dev/mem and /dev/fbdev) is that Linux uses the MMU to map physical memory into your address space. This functionality is used by XFree86 for example to get access to the video ram.

Okay, so that device is not ever arbitrated once it hits the page tables.

What do you mean with mirroring?

The GBA address space is heavily mirrored, for instance IWRAM is not only at 0x3000000 but 0x3008000, 0x3010000, 0x3018000, etc until I/O starts. VRAM has a more fragmented mirroring because it's not a region that's a power of two in magnitude. Some things actually do rely on this mirroring (the BIOS does, but I guess that's irrelevant since you must HLE it with this approach anyway.. I have seen games do it, but I haven't looked for it much specifically so it's hard to tell).

So you have to be able to map different spaces to the same location. I've heard before that there can problems with this. I don't recall if the ARM920T MMU hardware allows it or not - even if it does, Linux might throw up.

 

[dmantione]

One other thing: you aren't intercepting any memory accesses correct? Every region you've mapped is marked as read/write/exec (don't be fooled, there is no actual exec permissions on ARM920T, in case you've thought about using it to get around the problem I mentioned).. so you must be polling the IO registers once per frame to render.

I don't think this will generalize very well for the rest of GBA emulation. For instance, you need to trigger DMA as it happens: imagine if a game does several DMAs in a row (very likely) - if you're periodically polling the registers then you'll miss all but the last. Some things also need to be protected from being overwritten, like the line count register (yes, games do this >_>)

Of course this can be fixed by making the region write only, as well as ROM, BIOS, and the open regions need to be captured too. The question is, can a signal handler be installed to handle this properly? I do imagine that it can be caught, but can it be recovered from successfully so that the memory access is properly emulated? And how much overhead will it be?

Installing a signal handler is no problem, you can intercept and recover correctly. I was planning to emulate the BIOS this way. We are lucky that GBA and Linux SWI's don't overlap, we can just intercept the signal (otherwise you would have had to intercept syscalls by a helper process performing ptrace).

It will have quite a bit of overhead though if you get a signal for each register write, it depends how efficient the ARM is on page faults (luckily probably more efficient than x86).

 

[dmantione]

Like I mentioned later on in the thread, there isn't actually exec protection on ARM920T's MMU. I've checked several times and checked again today. You won't be able to turn off execution and catch it so you won't be able to determine when code in IWRAM is first executed without catching every read or write (remember, that's where the stack is). Most games load code into IWRAM because it's much faster there. Any that don't will have so fewer virtual cycles that they're probably much less of a concern anyway.

Ok, that's indeed a big issue then.

If these anonymous mmaps happen before the program is executed then what can you even do to prevent it? How can ordering help? If not, why would they happen later on?

If they happen later on, Linux will simply use another free address for it. So, the only thing you have to care about is doing things in the right order. I don't have to care about shared libraries, because I don't use any, and do the GBA mapping before any heap allocations take place.

The GBA address space is heavily mirrored, for instance IWRAM is not only at 0x3000000 but 0x3008000, 0x3010000, 0x3018000, etc until I/O starts. VRAM has a more fragmented mirroring because it's not a region that's a power of two in magnitude. Some things actually do rely on this mirroring (the BIOS does, but I guess that's irrelevant since you must HLE it with this approach anyway.. I have seen games do it, but I haven't looked for it much specifically so it's hard to tell).

So you have to be able to map different spaces to the same location. I've heard before that there can problems with this. I don't recall if the ARM920T MMU hardware allows it or not - even if it does, Linux might throw up.

Under normal circumstances mmapping a region twice should be no problem; there is no limitation in Linux. Hardware limitations could spoil things, but I don't expect them to be there, thing would cause big trouble for processes accessing the gp2x registers for example, I'm sure some kernel driver has mapped them, yet there is no problem for processes to map it too.

 

[Exophase]

If they happen later on, Linux will simply use another free address for it. So, the only thing you have to care about is doing things in the right order. I don't have to care about shared libraries, because I don't use any, and do the GBA mapping before any heap allocations take place.

Okay, so that means the heap isn't allocated until the first sbrk (or equivalent). That's good to know. Then my other question for you is, what is the default address mapping for the program that we can expect? That is, where will Linux put the text, data, bss, etc segments, and the stack. And where does the kernel itself reside (is it at the upper 1GB?)

Under normal circumstances mmapping a region twice should be no problem; there is no limitation in Linux. Hardware limitations could spoil things, but I don't expect them to be there, thing would cause big trouble for processes accessing the gp2x registers for example, I'm sure some kernel driver has mapped them, yet there is no problem for processes to map it too.

That's also good, however bear in mind the limitation can still exist because of how cache works (if virtual tags are used). It wouldn't affect memory mapped registers because those wouldn't be cacheable, for obvious reasons.

By the way, if you have AIM/MSN/Yahoo/ICQ and want to talk I'm open for it. Or you can join #gp2xdev on efnet.

 

[dmantione]

Okay, so that means the heap isn't allocated until the first sbrk (or equivalent). That's good to know. Then my other question for you is, what is the default address mapping for the program that we can expect? That is, where will Linux put the text, data, bss, etc segments, and the stack. And where does the kernel itself reside (is it at the upper 1GB?)

The code/data/bss segments start at $8000. This is a good location as there is nothing in the way here. If necessary it is possible to use a different location by using a linker script, but would be rather complicated. The stack is at the end of the user mode address space below $c0000000, again a good location.

Here is a memory map of the program in action:

CODE

00008000-00011000 r-xp 00000000 3c:01 41 /tmp/mnt/sd/gbaemu.gpe
00019000-0001a000 rw-p 00009000 3c:01 41 /tmp/mnt/sd/gbaemu.gpe
0001a000-0001b000 rwxp 00000000 00:00 0
02000000-02040000 rwxs 00000000 00:04 1204 /dev/zero (deleted)
03000000-03008000 rwxs 00000000 00:04 1205 /dev/zero (deleted)
04000000-04008000 rw-s 03080000 00:06 10 /dev/mem
05000000-05001000 rw-s 030a8000 00:06 10 /dev/mem
06000000-06018000 rwxs 03090000 00:06 10 /dev/mem
07000000-07001000 rwxs 00000000 00:04 1206 /dev/zero (deleted)
08000000-08080000 rwxp 00000000 3c:01 40 /tmp/mnt/sd/fpcLogo.gba
40000000-40010000 rw-s c0000000 00:06 10 /dev/mem
40010000-40090000 rwxs 03000000 00:06 10 /dev/mem
bffff000-c0000000 rwxp 00000000 00:00 0



The kernel is indeed in the upper 1GB starting at $c0000000.

That's also good, however bear in mind the limitation can still exist because of how cache works (if virtual tags are used). It wouldn't affect memory mapped registers because those wouldn't be cacheable, for obvious reasons.

By the way, if you have AIM/MSN/Yahoo/ICQ and want to talk I'm open for it. Or you can join #gp2xdev on efnet.

Okay, I'm on irc!

 

[Squidge]

I'm interested in if you setup a page for "no access", how you can use a signal trap to determine what address was attempted to write to, and with what value. At the moment I take over the arm abort handler, but that is kinda linux-unfriendly. If it's not much slower to do a linux friendly version, then I'm all ears

 

[dmantione]

I'm interested in if you setup a page for "no access", how you can use a signal trap to determine what address was attempted to write to, and with what value. At the moment I take over the arm abort handler, but that is kinda linux-unfriendly. If it's not much slower to do a linux friendly version, then I'm all ears

This quite simple: You install a signal handler and find all relevant data on the stack. There is a good example of this process in the FPC system unit take a look at the procedure InstallSignals in it:

http://svn.freepascal.org/svn/fpc/trunk/rtl/linux/system.pp

When your signal handler is called, there will be structure on the stack with all register contents, see this file for its details:

http://svn.freepascal.org/svn/fpc/trunk/rt...arm/sighndh.inc

In the file:

http://svn.freepascal.org/svn/fpc/trunk/rt...arm/sighndh.inc

... the procedure SignalToRunerror is the signal handler, and uses this data structure to determine the value of the IP register at the moment of the error. You can modify the registers, and when returning from the signal handler the program will continue with the new values in the registers.

I believe it is reasonably efficient. The kernel catches the CPU exception, builds the stack data structure and transfers control to the signal handler, on x86 the page fault itself consumes more cycles than the code that transfers control to the signal handler.

The server might go down again, it has had a disk crash and we are still busy getting everything right on the new disk. Should it be down, this source archive contains the files mentioned:

ftp://ftp.freepascal.org/pub/fpc/dist/sou...0.source.tar.gz