GP2X Gba Emu And Static Translation


Epicenter posted on Sep 29 2006 at 09:41 PM said:
He's definitely breaking some rules with that link and his sig is quite large (if he got rid of the giant goddamn Link picture it'd be fine). But that's a hell of a pile of ROMs, I'll give him credit for that. I'm going to be setting a mass-downloader on some sections of this site tonight, that's for sure. Wait, you didn't hear that. I didn't say anything. I was never here.

EDIT: What the fuck? Ecco and Gunstar Heroes are completely omitted from his Genesis section. That's like making a sandwich and forgetting the bread. Wow.

just because he has a goodset or two doesn't make him impressive. =)
 
Last edited by a moderator:
I would absolutely love to be proven wrong.
And I would absolutely love to prove you wrong. ^_^

Sorry everybody for the lack of updates recently, I don't have access to the computer labs at my university anymore (was doing an IT course, gonna change to comp sci next year).

About the house, we've finally got an appartment to live in (21st floor, wooo! I'm gonna have fun tossing moldy fruit out the window! :p) but our stuff hasn't arrived yet.

I knew things would smell fishier than normal with the lack of activity, but I can't really do much about it, can I?
 
TKF15H posted on Oct 1 2006 at 05:48 PM said:
I knew things would smell fishier than normal with the lack of activity, but I can't really do much about it, can I?
Glad to hear your getting back into the groove. Prob' feels real good to get settled in. Well, atleast as much as you can until your stuff is delivered.
 
Last edited by a moderator:
/me wants to code his own emulator so badly
lol
Yeah, no i'm too stupid to code an emulator yet :\
What your going to be doing is udderly amazing (in my eyes anyways) so take your time. As you can see, your work will be incredibly appreciated by alot of people :p Wish I could help tho... *goes to continue learning to make a chip-8 emulator.*
 
Even if the instructions are translated, likely the rest of the hardware is still emulated, so it wouldn't necessarily be wrong calling it either.
 
Hello,
just dropped in to say I'm still alive and I finally got my computer back.
Just no internet connection yet. >_<
I've been working on IBA lately, got quite a bit done this week despite the
unpacking of our stuff that still hasn't finished.
As for IBA's status: It currently Seg faults, but it does so really really quickly, so the optimizations are doing their job. ^_^ :p
 
This post is just what I'm thinking about concerning the emulator at the moment, what I'm currently
implementing, written down and shared in hopes of obtaining another point of view, even if it's just, "hah,
that's silly."

At first this was going to be a statically-recompiling emulator, but then plans changed when it was
observed that the cost did not justify the benefit in comparison to a Dynarec-based CPU core.
IBA is emulating an ARM processor on another ARM, so statically recompiling with GCC in the
middle wasn't worth it: Register allocation and the code in general would not be optimal (did a
test, results didn't look that good), CPSR flags have to be calculated (rather than using the
GP2X's), and it's rather cumbersome for the end-user (run in decompilation mode, exit,
compile, get back in again, rinse, lather, repeat). Since the whole thing is so experimental, one
should not be surprised with the changes. A GCC backend would probably do a better job with
a processor that has more registers than the ARM (PPC maybe?), but being so similar there is no
need for it and it ends up getting in the way.

A dynarec has been built for outputting blocks of code that supports all of the ARM's instructions
except, currently, the opcodes for co-processor communication. The code it outputs can be saved
to the SD so that ROM code does not have to be re-translated... so in a way, this emulator can still
be a static recompiler, it just does ASM->ASM instead of ASM->C->ASM. (If anybody goes
out and says, "I told you so" I'll pull out a large trout and deliver a few slaps before the day ends)

When the emulator starts, the processor attempts to execute whatever is at address 0x08000000,
which resides on the cartridge (GPROM). To know if the instruction at that address has been
previously compiled, a second table is needed. Due to the size of the GPROM (up to 32MB) a
table that keeps track of what's beed compiled and what hasn't needs to be as compact as possible.
Since the worst case is that all 32MB are filled with Thumb code, you could say that every short (16 bits)
needs one bit to indicate previous compilation. So, 32MBytes / 2Bytes = 16M Thumb instructions =
16Mbits are necessary for flagging each instruction = 2MBytes for the GPROM
Compilation Bit Map (GPROMcbm from now on).
Now that there's a table that tells the emulator that it has executed that instruction before, it needs
to know where the translated block is. We can't afford to have a 32bit pointer for each instruction
(it would require 64MB plus the GPROM's 32MB totalling 96MB!) so we'll have to over-write
whatever the GPROM is storing there with a pointer to the code's actual location. In the case of
ARM code, this is not a problem as the instructions are the same size as a pointer... but what about
thumb mode? A pointer can't be stored in 16bits! Well, where you can't store an absolute address, a
relative one will have to do.
Let's say the letters on the left are GBA instructions and the ones on the right are the translated GP2X
equivalent:
Code:
0  A   E
1		F
2		G
3   B   H
4   C   I
5		 J
With 16 bits, if a jump to C is necessary, you can't simply store the pointer to I. Therefore, instead of
a pointer, an offset is used. This gives us 65536 instructions that are possible jump targets, which isn't
enough. The solution (?):
Target = (R15 - 0x08000000) + GPROM[ R15 ] + CacheBlockPtr;
When A is compiled, a zero is put in its place, and the GPROMcbm is updated.
Then B is compiled, a (0+2) two is put in its place, and the GPROMcbm is updated again.
Finally C is compiled, (2+0) two is put in its place, and the GPROMcbm... you'll never guess... is
updated yet once more.
The number that is put in place of the old opcode is the number of "extra" instructions needed up till
now. This gives us far more than 65536 instructions due to the fact that most opcodes are translated
with a 1:1 proportion.

Ok, enough of the GPROM, let's move on to the rest...

The GBA has the following address-space regions:
ROM
Internal RAM
External RAM
IORAM
Palette RAM
Video RAM
Tile RAM
GPROM

I am going to assume code can be executed on the ROM, IRAM, ERAM, and GPROM and will
never be executed on the IORAM, Palette RAM, VRAM, or the Tile RAM.
Executable memory that can't be written to (ROM, GPROM) does not need over-write checking, and
so can be optimized and simplified in comparisson to the RAM.

Let's have a look at Interal / External RAM. These two banks are rather small - 32KB and 256KB - so
there's a lot more room to work with, and I'll need it. Since they're writable, the emulator needs to know
if the block of code it has written is valid. This means that for each GBA instruction, there needs to be a
pointer to the equivalent GP2X code and a pointer to the cDynarec object it came from.
The cDynarec class has a Key and CurrKey. When a cache block is compiled, the sum of each GBA
instruction is stored in Key and CurrKey. That way I hope to have a signature of each block. Each
time a STR/STM instruction is executed on an address that contains compiled code, the old value is
subtracted from the CurrKey and the new one is added. When code generated by cDynarec has to
be executed, it checks if Key equals CurrKey, and if it isn't, the CurrKey is searched for in a binary
tree. If it isn't found, a new block has to be generated, and the current one stored in the tree for later.
If it is found, the old block replaces the current one.

.... I hope that makes sense >_<

As for the other RAMs, I'd like to know... is it reasonable to assume that if a GBA instruction accesses
the IORAM (for example) it is unlikely to access another region? Because if it is, I'd like to do the
MMU search once and then make the generated code access that area directly.
 
/me slaps sand man around a bit with a large trout

It wasn't exactly "I told you so", but it was along those lines... :p
 
I don't know what much of that means, but it makes you seem very smart and capable! You should know that everyone's rooting for you. I can't wait to see where this goes.
 
Sorry if I misread anything in what you said, you'll have to beat me over the head with it if I did. But from what I grasped:

Overwriting code with metadata is liable to get you into a lot of trouble, for the following reasons:

1) diminishes your ability to recompile that block again, necessary after self modifying code or when the translation buffer fills up,
2) (more important) games actually read the code. For instance the SWI handler will read the SWI's comment field, from the actual instruction where the lr register points to.

Also, it seems that having those two tables, ie "what has been compiled and what hasn't" and "where have things been compiled to" is redundant. If the code isn't in the second table then it hasn't been compiled yet. I take it you're using the former for SMC markers too, but you only need to mark code in RAM (since code in ROM, of course, cannot be changed)

The typical approach is to use a hash table. Branch addresses hash pretty well.

For GBA emulation I use a hash table for the ROM, and direct tables for the RAM areas (those tables are shared with SMC markers). Rather than play games with the pointers to make them fit in 16bit areas it's a lot easier to just have them index into a second table, 16bit large. You're probably not going to get more than 64k branch locations total during the game's execution. I KNOW I'm not going to for RAM only (which is all they're used for here).

Oh, code is sometimes executed from VRAM. But not often. I haven't found any real games that do it yet (I know Nintendo's NES emulator does)
 
Thanks for the support guys! :)

Exophase posted on Oct 29 2006 at 05:36 AM said:
Sorry if I misread anything in what you said, you'll have to beat me over the head with it if I did. But from what I grasped:

Overwriting code with metadata is liable to get you into a lot of trouble, for the following reasons:

1) diminishes your ability to recompile that block again, necessary after self modifying code or when the translation buffer fills up,
2) (more important) games actually read the code. For instance the SWI handler will read the SWI's comment field, from the actual instruction where the lr register points to.
/me beats Exophase over the head with a large trout ^_^
For #1, code overwriting would never happen on the GamePackROM, so self-modifying code isn't something to be concerned with. If the translation buffer fills up, I'd like to dump a block of code that hasn't been executed in a while onto the SD (if it's needed again, reload). I'm not at that stage yet (filling up the buffer, all work is currently being done on a small open-source demo) so I don't know if storing things on the SD is a bad idea. Any comments on this?

As for #2, the SWI handler will be taken care of, the problem would be LDR/LDM instructions, though I don't know why they'd be used on code that has been executed on a ROM (code that has to be copied to RAM for faster execution is unlikely to have been executed previously on a ROM.... right?). If it is necessary, I guess I could mmap the original file and make them read from there. Anybody have any idea if this is viable? An LDR/LDM trying to read code is unlikely, so it shouldn't be much of a speed hit, but I don't know if it's actually possible to have a 32MB file mmapped with most of the RAM being used up (I'm pretty new to GP2X development, I'm not used to the tiny space :p).

Also, it seems that having those two tables, ie "what has been compiled and what hasn't" and "where have things been compiled to" is redundant. If the code isn't in the second table then it hasn't been compiled yet. I take it you're using the former for SMC markers too, but you only need to mark code in RAM (since code in ROM, of course, cannot be changed)
the "what has been compiled and what hasn't" table is just for the ROMs, the other table is just for the RAMs.

The typical approach is to use a hash table. Branch addresses hash pretty well.
Yeah, I actually thought about using one, but now I can no longer remember why I gave up on that idea. Probably because they'd work fine for immediate jumps but would't help with jumps that use a pointer register. (Or it could be I'm just thinking inside the wrong box here...)

Rather than play games with the pointers to make them fit in 16bit areas it's a lot easier to just have them index into a second table, 16bit large.
Don't you mean 32bit? Well, that's what I'm doing with the RAM, but the GPROM is the tricky one, can't spare having such a big table with a 32MB bank on a device that has 64MB RAM. :p

Oh, code is sometimes executed from VRAM. But not often. I haven't found any real games that do it yet (I know Nintendo's NES emulator does)
Eh?? Any idea why they'd do such a thing? o_O
 
Last edited by a moderator:
TKF15H posted on Oct 29 2006 at 12:01 AM said:
/me slaps sand man around a bit with a large trout

It wasn't exactly "I told you so", but it was along those lines... :p

Sorry. I really didn't intend for it to sound that way.
 
Last edited by a moderator:
Back
Top