sephiroth111
Member
- Joined
- Mar 2, 2006
- Messages
- 139
Epicenter posted on Sep 29 2006 at 09:41 PM said:
just because he has a goodset or two doesn't make him impressive. =)
Last edited by a moderator:
GP2X Gba Emu And Static Translation
- Thread starter A_SN
- Start date
just because he has a goodset or two doesn't make him impressive. =)
- Joined
- Dec 8, 2005
- Messages
- 6,603
- Age
- 40
- Location
- UK
- Website
- pokeparadox.itch.io
- WEBSITE
- https://github.com/pokeparadox
- YOUTUBE
- pokeparadox
Ah... memories... 
And I would absolutely love to prove you wrong. ^_^
Sorry everybody for the lack of updates recently, I don't have access to the computer labs at my university anymore (was doing an IT course, gonna change to comp sci next year).
About the house, we've finally got an appartment to live in (21st floor, wooo! I'm gonna have fun tossing moldy fruit out the window!
) but our stuff hasn't arrived yet.I knew things would smell fishier than normal with the lack of activity, but I can't really do much about it, can I?
Epicenter
Well-Known Member
Don't mind the ungrateful assholes, some of us appreciate what you are doing.
Vollgasasi
Member
- Joined
- Aug 14, 2003
- Messages
- 347
tkf15h, take your time. some people just need a lesson in patience that's all.
that's all.
wipeout2000
Member
Glad to hear things are going better for you
/me wants to code his own emulator so badly
lol
Yeah, no i'm too stupid to code an emulator yet :\
What your going to be doing is udderly amazing (in my eyes anyways) so take your time. As you can see, your work will be incredibly appreciated by alot of people Wish I could help tho... *goes to continue learning to make a chip-8 emulator.*
lol
Yeah, no i'm too stupid to code an emulator yet :\
What your going to be doing is udderly amazing (in my eyes anyways) so take your time. As you can see, your work will be incredibly appreciated by alot of people
Wish I could help tho... *goes to continue learning to make a chip-8 emulator.*Hello,
just dropped in to say I'm still alive and I finally got my computer back.
Just no internet connection yet. >_<
I've been working on IBA lately, got quite a bit done this week despite the
unpacking of our stuff that still hasn't finished.
As for IBA's status: It currently Seg faults, but it does so really really quickly, so the optimizations are doing their job. ^_^
just dropped in to say I'm still alive and I finally got my computer back.
Just no internet connection yet. >_<
I've been working on IBA lately, got quite a bit done this week despite the
unpacking of our stuff that still hasn't finished.
As for IBA's status: It currently Seg faults, but it does so really really quickly, so the optimizations are doing their job. ^_^
Vollgasasi
Member
- Joined
- Aug 14, 2003
- Messages
- 347
great to hear your still there! take your time buddy!
This post is just what I'm thinking about concerning the emulator at the moment, what I'm currently
implementing, written down and shared in hopes of obtaining another point of view, even if it's just, "hah,
that's silly."
At first this was going to be a statically-recompiling emulator, but then plans changed when it was
observed that the cost did not justify the benefit in comparison to a Dynarec-based CPU core.
IBA is emulating an ARM processor on another ARM, so statically recompiling with GCC in the
middle wasn't worth it: Register allocation and the code in general would not be optimal (did a
test, results didn't look that good), CPSR flags have to be calculated (rather than using the
GP2X's), and it's rather cumbersome for the end-user (run in decompilation mode, exit,
compile, get back in again, rinse, lather, repeat). Since the whole thing is so experimental, one
should not be surprised with the changes. A GCC backend would probably do a better job with
a processor that has more registers than the ARM (PPC maybe?), but being so similar there is no
need for it and it ends up getting in the way.
A dynarec has been built for outputting blocks of code that supports all of the ARM's instructions
except, currently, the opcodes for co-processor communication. The code it outputs can be saved
to the SD so that ROM code does not have to be re-translated... so in a way, this emulator can still
be a static recompiler, it just does ASM->ASM instead of ASM->C->ASM. (If anybody goes
out and says, "I told you so" I'll pull out a large trout and deliver a few slaps before the day ends)
When the emulator starts, the processor attempts to execute whatever is at address 0x08000000,
which resides on the cartridge (GPROM). To know if the instruction at that address has been
previously compiled, a second table is needed. Due to the size of the GPROM (up to 32MB) a
table that keeps track of what's beed compiled and what hasn't needs to be as compact as possible.
Since the worst case is that all 32MB are filled with Thumb code, you could say that every short (16 bits)
needs one bit to indicate previous compilation. So, 32MBytes / 2Bytes = 16M Thumb instructions =
16Mbits are necessary for flagging each instruction = 2MBytes for the GPROM
Compilation Bit Map (GPROMcbm from now on).
Now that there's a table that tells the emulator that it has executed that instruction before, it needs
to know where the translated block is. We can't afford to have a 32bit pointer for each instruction
(it would require 64MB plus the GPROM's 32MB totalling 96MB!) so we'll have to over-write
whatever the GPROM is storing there with a pointer to the code's actual location. In the case of
ARM code, this is not a problem as the instructions are the same size as a pointer... but what about
thumb mode? A pointer can't be stored in 16bits! Well, where you can't store an absolute address, a
relative one will have to do.
Let's say the letters on the left are GBA instructions and the ones on the right are the translated GP2X
equivalent:
With 16 bits, if a jump to C is necessary, you can't simply store the pointer to I. Therefore, instead of
a pointer, an offset is used. This gives us 65536 instructions that are possible jump targets, which isn't
enough. The solution (?):
Target = (R15 - 0x08000000) + GPROM[ R15 ] + CacheBlockPtr;
When A is compiled, a zero is put in its place, and the GPROMcbm is updated.
Then B is compiled, a (0+2) two is put in its place, and the GPROMcbm is updated again.
Finally C is compiled, (2+0) two is put in its place, and the GPROMcbm... you'll never guess... is
updated yet once more.
The number that is put in place of the old opcode is the number of "extra" instructions needed up till
now. This gives us far more than 65536 instructions due to the fact that most opcodes are translated
with a 1:1 proportion.
Ok, enough of the GPROM, let's move on to the rest...
The GBA has the following address-space regions:
ROM
Internal RAM
External RAM
IORAM
Palette RAM
Video RAM
Tile RAM
GPROM
I am going to assume code can be executed on the ROM, IRAM, ERAM, and GPROM and will
never be executed on the IORAM, Palette RAM, VRAM, or the Tile RAM.
Executable memory that can't be written to (ROM, GPROM) does not need over-write checking, and
so can be optimized and simplified in comparisson to the RAM.
Let's have a look at Interal / External RAM. These two banks are rather small - 32KB and 256KB - so
there's a lot more room to work with, and I'll need it. Since they're writable, the emulator needs to know
if the block of code it has written is valid. This means that for each GBA instruction, there needs to be a
pointer to the equivalent GP2X code and a pointer to the cDynarec object it came from.
The cDynarec class has a Key and CurrKey. When a cache block is compiled, the sum of each GBA
instruction is stored in Key and CurrKey. That way I hope to have a signature of each block. Each
time a STR/STM instruction is executed on an address that contains compiled code, the old value is
subtracted from the CurrKey and the new one is added. When code generated by cDynarec has to
be executed, it checks if Key equals CurrKey, and if it isn't, the CurrKey is searched for in a binary
tree. If it isn't found, a new block has to be generated, and the current one stored in the tree for later.
If it is found, the old block replaces the current one.
.... I hope that makes sense >_<
As for the other RAMs, I'd like to know... is it reasonable to assume that if a GBA instruction accesses
the IORAM (for example) it is unlikely to access another region? Because if it is, I'd like to do the
MMU search once and then make the generated code access that area directly.
implementing, written down and shared in hopes of obtaining another point of view, even if it's just, "hah,
that's silly."
At first this was going to be a statically-recompiling emulator, but then plans changed when it was
observed that the cost did not justify the benefit in comparison to a Dynarec-based CPU core.
IBA is emulating an ARM processor on another ARM, so statically recompiling with GCC in the
middle wasn't worth it: Register allocation and the code in general would not be optimal (did a
test, results didn't look that good), CPSR flags have to be calculated (rather than using the
GP2X's), and it's rather cumbersome for the end-user (run in decompilation mode, exit,
compile, get back in again, rinse, lather, repeat). Since the whole thing is so experimental, one
should not be surprised with the changes. A GCC backend would probably do a better job with
a processor that has more registers than the ARM (PPC maybe?), but being so similar there is no
need for it and it ends up getting in the way.
A dynarec has been built for outputting blocks of code that supports all of the ARM's instructions
except, currently, the opcodes for co-processor communication. The code it outputs can be saved
to the SD so that ROM code does not have to be re-translated... so in a way, this emulator can still
be a static recompiler, it just does ASM->ASM instead of ASM->C->ASM. (If anybody goes
out and says, "I told you so" I'll pull out a large trout and deliver a few slaps before the day ends)
When the emulator starts, the processor attempts to execute whatever is at address 0x08000000,
which resides on the cartridge (GPROM). To know if the instruction at that address has been
previously compiled, a second table is needed. Due to the size of the GPROM (up to 32MB) a
table that keeps track of what's beed compiled and what hasn't needs to be as compact as possible.
Since the worst case is that all 32MB are filled with Thumb code, you could say that every short (16 bits)
needs one bit to indicate previous compilation. So, 32MBytes / 2Bytes = 16M Thumb instructions =
16Mbits are necessary for flagging each instruction = 2MBytes for the GPROM
Compilation Bit Map (GPROMcbm from now on).
Now that there's a table that tells the emulator that it has executed that instruction before, it needs
to know where the translated block is. We can't afford to have a 32bit pointer for each instruction
(it would require 64MB plus the GPROM's 32MB totalling 96MB!) so we'll have to over-write
whatever the GPROM is storing there with a pointer to the code's actual location. In the case of
ARM code, this is not a problem as the instructions are the same size as a pointer... but what about
thumb mode? A pointer can't be stored in 16bits! Well, where you can't store an absolute address, a
relative one will have to do.
Let's say the letters on the left are GBA instructions and the ones on the right are the translated GP2X
equivalent:
Code:
0 A E
1 F
2 G
3 B H
4 C I
5 Ja pointer, an offset is used. This gives us 65536 instructions that are possible jump targets, which isn't
enough. The solution (?):
Target = (R15 - 0x08000000) + GPROM[ R15 ] + CacheBlockPtr;
When A is compiled, a zero is put in its place, and the GPROMcbm is updated.
Then B is compiled, a (0+2) two is put in its place, and the GPROMcbm is updated again.
Finally C is compiled, (2+0) two is put in its place, and the GPROMcbm... you'll never guess... is
updated yet once more.
The number that is put in place of the old opcode is the number of "extra" instructions needed up till
now. This gives us far more than 65536 instructions due to the fact that most opcodes are translated
with a 1:1 proportion.
Ok, enough of the GPROM, let's move on to the rest...
The GBA has the following address-space regions:
ROM
Internal RAM
External RAM
IORAM
Palette RAM
Video RAM
Tile RAM
GPROM
I am going to assume code can be executed on the ROM, IRAM, ERAM, and GPROM and will
never be executed on the IORAM, Palette RAM, VRAM, or the Tile RAM.
Executable memory that can't be written to (ROM, GPROM) does not need over-write checking, and
so can be optimized and simplified in comparisson to the RAM.
Let's have a look at Interal / External RAM. These two banks are rather small - 32KB and 256KB - so
there's a lot more room to work with, and I'll need it. Since they're writable, the emulator needs to know
if the block of code it has written is valid. This means that for each GBA instruction, there needs to be a
pointer to the equivalent GP2X code and a pointer to the cDynarec object it came from.
The cDynarec class has a Key and CurrKey. When a cache block is compiled, the sum of each GBA
instruction is stored in Key and CurrKey. That way I hope to have a signature of each block. Each
time a STR/STM instruction is executed on an address that contains compiled code, the old value is
subtracted from the CurrKey and the new one is added. When code generated by cDynarec has to
be executed, it checks if Key equals CurrKey, and if it isn't, the CurrKey is searched for in a binary
tree. If it isn't found, a new block has to be generated, and the current one stored in the tree for later.
If it is found, the old block replaces the current one.
.... I hope that makes sense >_<
As for the other RAMs, I'd like to know... is it reasonable to assume that if a GBA instruction accesses
the IORAM (for example) it is unlikely to access another region? Because if it is, I'd like to do the
MMU search once and then make the generated code access that area directly.
- Joined
- Dec 8, 2005
- Messages
- 6,603
- Age
- 40
- Location
- UK
- Website
- pokeparadox.itch.io
- WEBSITE
- https://github.com/pokeparadox
- YOUTUBE
- pokeparadox
Well please keep us updated now you've finally got to a position to get the ball rolling.
Exophase
Nothing good will ever come of Exophase.
Sorry if I misread anything in what you said, you'll have to beat me over the head with it if I did. But from what I grasped:
Overwriting code with metadata is liable to get you into a lot of trouble, for the following reasons:
1) diminishes your ability to recompile that block again, necessary after self modifying code or when the translation buffer fills up,
2) (more important) games actually read the code. For instance the SWI handler will read the SWI's comment field, from the actual instruction where the lr register points to.
Also, it seems that having those two tables, ie "what has been compiled and what hasn't" and "where have things been compiled to" is redundant. If the code isn't in the second table then it hasn't been compiled yet. I take it you're using the former for SMC markers too, but you only need to mark code in RAM (since code in ROM, of course, cannot be changed)
The typical approach is to use a hash table. Branch addresses hash pretty well.
For GBA emulation I use a hash table for the ROM, and direct tables for the RAM areas (those tables are shared with SMC markers). Rather than play games with the pointers to make them fit in 16bit areas it's a lot easier to just have them index into a second table, 16bit large. You're probably not going to get more than 64k branch locations total during the game's execution. I KNOW I'm not going to for RAM only (which is all they're used for here).
Oh, code is sometimes executed from VRAM. But not often. I haven't found any real games that do it yet (I know Nintendo's NES emulator does)
Overwriting code with metadata is liable to get you into a lot of trouble, for the following reasons:
1) diminishes your ability to recompile that block again, necessary after self modifying code or when the translation buffer fills up,
2) (more important) games actually read the code. For instance the SWI handler will read the SWI's comment field, from the actual instruction where the lr register points to.
Also, it seems that having those two tables, ie "what has been compiled and what hasn't" and "where have things been compiled to" is redundant. If the code isn't in the second table then it hasn't been compiled yet. I take it you're using the former for SMC markers too, but you only need to mark code in RAM (since code in ROM, of course, cannot be changed)
The typical approach is to use a hash table. Branch addresses hash pretty well.
For GBA emulation I use a hash table for the ROM, and direct tables for the RAM areas (those tables are shared with SMC markers). Rather than play games with the pointers to make them fit in 16bit areas it's a lot easier to just have them index into a second table, 16bit large. You're probably not going to get more than 64k branch locations total during the game's execution. I KNOW I'm not going to for RAM only (which is all they're used for here).
Oh, code is sometimes executed from VRAM. But not often. I haven't found any real games that do it yet (I know Nintendo's NES emulator does)
Thanks for the support guys!
For #1, code overwriting would never happen on the GamePackROM, so self-modifying code isn't something to be concerned with. If the translation buffer fills up, I'd like to dump a block of code that hasn't been executed in a while onto the SD (if it's needed again, reload). I'm not at that stage yet (filling up the buffer, all work is currently being done on a small open-source demo) so I don't know if storing things on the SD is a bad idea. Any comments on this?
As for #2, the SWI handler will be taken care of, the problem would be LDR/LDM instructions, though I don't know why they'd be used on code that has been executed on a ROM (code that has to be copied to RAM for faster execution is unlikely to have been executed previously on a ROM.... right?). If it is necessary, I guess I could mmap the original file and make them read from there. Anybody have any idea if this is viable? An LDR/LDM trying to read code is unlikely, so it shouldn't be much of a speed hit, but I don't know if it's actually possible to have a 32MB file mmapped with most of the RAM being used up (I'm pretty new to GP2X development, I'm not used to the tiny space).
/me beats Exophase over the head with a large trout ^_^Exophase posted on Oct 29 2006 at 05:36 AM said:
For #1, code overwriting would never happen on the GamePackROM, so self-modifying code isn't something to be concerned with. If the translation buffer fills up, I'd like to dump a block of code that hasn't been executed in a while onto the SD (if it's needed again, reload). I'm not at that stage yet (filling up the buffer, all work is currently being done on a small open-source demo) so I don't know if storing things on the SD is a bad idea. Any comments on this?
As for #2, the SWI handler will be taken care of, the problem would be LDR/LDM instructions, though I don't know why they'd be used on code that has been executed on a ROM (code that has to be copied to RAM for faster execution is unlikely to have been executed previously on a ROM.... right?). If it is necessary, I guess I could mmap the original file and make them read from there. Anybody have any idea if this is viable? An LDR/LDM trying to read code is unlikely, so it shouldn't be much of a speed hit, but I don't know if it's actually possible to have a 32MB file mmapped with most of the RAM being used up (I'm pretty new to GP2X development, I'm not used to the tiny space
).the "what has been compiled and what hasn't" table is just for the ROMs, the other table is just for the RAMs.
Yeah, I actually thought about using one, but now I can no longer remember why I gave up on that idea. Probably because they'd work fine for immediate jumps but would't help with jumps that use a pointer register. (Or it could be I'm just thinking inside the wrong box here...)
Don't you mean 32bit? Well, that's what I'm doing with the RAM, but the GPROM is the tricky one, can't spare having such a big table with a 32MB bank on a device that has 64MB RAM.
Eh?? Any idea why they'd do such a thing?
Last edited by a moderator:
TKF15H posted on Oct 29 2006 at 12:01 AM said:/me slaps sand man around a bit with a large trout
It wasn't exactly "I told you so", but it was along those lines...
Sorry. I really didn't intend for it to sound that way.
Last edited by a moderator: