GP2X Gba Emu And Static Translation


A_SN

Active Member
Joined
Jun 8, 2006
Messages
899
Might sound like a dumb question, but I was wondering if statically translating GBA roms would help for emulation.

Someone on IRC told me it was not possible because ROMs code was self-modifying. Is it true and if so how does it work? And is there any way static translation of ROMs can help emulation of the GBA on the GP2X?
 
Well, code that executes from ROM can't be self modifying (barring some insane cache tricks that I hope nobody would ever use) - that's the nature of ROM, but code loaded into RAM could be. It might be necessary to also have a normal interpreter for cases like this (or switch the whole arrangement to dynamic recompilation where sections of code are only recompiled the first time they're executed).
 
I started having a look at this today. ROMs aren't self-modifying, but they might be compressed. The GBA BIOS has a few decompression calls which means some games' code could be decompressed in RAM before execution. What I'm planning on is mixing a static compilation with VBA. I would get ARM code, churn out C code, compile in GCC, and link a slightly modified VBA.
 
A lot of gba roms extract or copy code to ram before execution simply because rom is 16-bit, whilst ram is 32-bit, so is quite a bit faster - you can use full arm code in ram, whilst you have to use thumb in rom or suffer the penalties of reading 32-bit instructions across a 16-bit pipe...
 
Since you're starting with ARM code anyway, maybe it would be better to just reprocess it to fix memory and hardware accesses, plus then the compiler wouldn't totally mangle stuff that was already hand optimized. Then again, you've got like a 10x speed difference anyway so I don't think it'll be a big problem. Whatever's easiest.

I bet some structures might be tricky to translate into C code though... I'm thinking like jump tables and such like that. Then again, you could just switch to VBA at that point so it might not be a big deal.
 
I'm pretty new to ARM coding, so I don't think I can generate code as fast as GCC's.
As for jumping around... whenever a jump is done to a point that can not be predicted at compile-time, I'm thinking of jumping to a large switch() which then jumps to the appropriate place. If that place is not C code, bail out to VBA.
This should be optimizable by GCC into a jump-table, so it's not too horribly bad.

Alternately, I could have an array of label pointers, but.... is there any way of getting a label pointer in GCC? I know you can use:
goto *label_ptr;
but how do I do this:
label_A:
label_ptr = label_A;
GCC doesn't like it. Searches on google don't say anything about it. Books don't even mention the fact that C/C++ have a Goto statement. :p
 
TKF15H posted on Jul 6 2006 at 05:36 PM said:
but how do I do this:
label_A:
label_ptr = label_A;
GCC doesn't like it. Searches on google don't say anything about it. Books don't even mention the fact that C/C++ have a Goto statement. :p

The one I learned C from many years ago (8? 9? heh.) included Goto but it was heavily discouraged because it 'turned your code into spaghetti'. It really does; and it throws any reason to organize things effectively out the window.

If there is a lot of compression in the ROMs it's PROBABLY just graphics, due to the obscene number of sprites in your average game. So .. I sort of doubt instructions would be compressed-- that wouldn't be the bulk of the program anyway. My only worry with this idea is that if the GBA data IS compressed it's probably a proprietary algorithm unless the GBA SDKs from Nintendo provided something the third parties all used. Historically (16-bit era and back) it's been a mostly proprietary thing.
 
Last edited by a moderator:
No, it's not proprietary. They use LZ, Huffman, and RLE. And judging by the teeny amount of RAM that GBA's have, I do find it likely that they'll compress code.

Regarding Goto being a bad practise, it's a tool like a chainsaw. If you don't know what you're doing, you'll end up with an ugly mess. Use it properly, it won't be so bad. Besides, since the code is going to be generated rather than written by hand, readability is not that important. Even so, I don't think it will uglify the code much.
 
I *quickly* skimmed through this thread, but is someone actually going to try this or did I miss something? ;)
 
Epicenter posted on Jul 6 2006 at 08:05 PM said:
TKF15H posted on Jul 6 2006 at 05:36 PM said:
but how do I do this:
label_A:
label_ptr = label_A;
GCC doesn't like it. Searches on google don't say anything about it. Books don't even mention the fact that C/C++ have a Goto statement. :p

The one I learned C from many years ago (8? 9? heh.) included Goto but it was heavily discouraged because it 'turned your code into spaghetti'. It really does; and it throws any reason to organize things effectively out the window.

If there is a lot of compression in the ROMs it's PROBABLY just graphics, due to the obscene number of sprites in your average game. So .. I sort of doubt instructions would be compressed-- that wouldn't be the bulk of the program anyway. My only worry with this idea is that if the GBA data IS compressed it's probably a proprietary algorithm unless the GBA SDKs from Nintendo provided something the third parties all used. Historically (16-bit era and back) it's been a mostly proprietary thing.

Okay, not starting a flamewar, just a comment. Yes, they teach that goto is bad, but the branch instruction(s) is one of the most important instructions in a processor. And goto is about as optimized as you can get. The subject of this thread includes static translation. If you want to do a static translation, goto's may very well be a huge part of the translation (Depending on how you approach it). Goto's are also a very good way to test a compiler, branching out of loops or switch statements for example. Want to see some spaghetti code? Check out gcc's test bench, there are things in there I didnt know you could do in C, almost all of which I would never try. Basically, yes goto's are hard to follow and should be avoided in general. for a static translation, depending on the approach, can be an integral part of the solution.

Another comment, remember that because big N neutered the processor on the GBA, thumb is the preferred mode for performance, so that makes the static translation twice as fun, you really have to know the arm and thumb instruction sets inside out.

You also will have to emulate the gba video which was hardware heavy (which is why it continues to survive despite having a crippled arm).

There were freeware tools for compressing your programs. And of course a freeware emulator (or ten), so I assume there is a free decompressor, that or do write some test programs for the vba to decompress your data and dump out the decompressed data to the pc. bottom line, yes it can be done, for free, how much effort?

The gp2x runs a little over 10 times faster than the gba mhz to mhz, thats 10 instructions to one, with the video overhead I dont think you will make it. Where you may get lucky is doing a virtual pc or vmware type thing where you let most of the instructions through to the processor which would be a one to one, then use the mmu and vector tables to trap memory accesses so that they can be emulated...Not knowing enough about the gp2x (yet) I dont know how hard that would be. I still think 10 to 1 is not enough for success on this job, esp with the slider video overhead.
 
Last edited by a moderator:
heh, thanks iignotus. :)

I finally found out how to get a pointer to a label. :)
In case anyone else is interested:
Code:
Labels as Values

You can get the address of a label defined in the current function (or a containing function) with the unary operator `&&'. The value has type void *. This value is a constant and can be used wherever a constant of that type is valid. For example:

void *ptr;
...
ptr = &&foo;

To use these values, you need to be able to jump to one. This is done with the computed goto statement(1), goto *exp;. For example,

goto *ptr;

Any expression of type void * is allowed.

One way of using these constants is in initializing a static array that will serve as a jump table:

static void *array[] = { &&foo, &&bar, &&hack };

Then you can select a label with indexing, like this:

goto *array[i];
 
Like the next GP2X user, I'd love a GBA emulator.

I love how Epicenter can post something universally known, like "Gotos make speghetti" and someone has to post a mile long thread about an exception to the rule. Any decent programmer knows there's exceptions to the rule! *chuckles*

Ah well.
 
The gp2x runs a little over 10 times faster than the gba mhz to mhz, thats 10 instructions to one, with the video overhead I dont think you will make it.

The VBA emulator re-compiled for the GP2X runs at 10% speed (or so it says).
Static recompilers are famed of multiplying speed by 25. So even if I made a recompiler that could do only half of that, it should be enough to get 100%.
I'm using software SDL for now, with no overclocking.
The second processor is not being used.
GCC will be re-compiling the GBA's code, with optimization it may actually generate code with a ratio of less than 1:1 in some cases, if we're lucky.
With so much to do, I don't think getting 100% is going to be such an amazingly difficult task.

Right now I'm working on generating a database that stores information on each instruction in a ROM. Data is to be stored in the DB while the game is run (rather slowly :p). A post-processor will get the database and generate a C file from it. GCC will then get the C file, make an object file which is then linked to VBA. Running the game again will be much faster as the interpretor will not be executed in the places that were run previously, and quite a few optimizations will have been made.
 
TKF15H posted on Jul 7 2006 at 10:04 AM said:
The gp2x runs a little over 10 times faster than the gba mhz to mhz, thats 10 instructions to one, with the video overhead I dont think you will make it.

The VBA emulator re-compiled for the GP2X runs at 10% speed (or so it says).
Static recompilers are famed of multiplying speed by 25. So even if I made a recompiler that could do only half of that, it should be enough to get 100%.
I'm using software SDL for now, with no overclocking.
The second processor is not being used.
GCC will be re-compiling the GBA's code, with optimization it may actually generate code with a ratio of less than 1:1 in some cases, if we're lucky.
With so much to do, I don't think getting 100% is going to be such an amazingly difficult task.

Right now I'm working on generating a database that stores information on each instruction in a ROM. Data is to be stored in the DB while the game is run (rather slowly :p). A post-processor will get the database and generate a C file from it. GCC will then get the C file, make an object file which is then linked to VBA. Running the game again will be much faster as the interpretor will not be executed in the places that were run previously, and quite a few optimizations will have been made.

I got stung before translating Asteroids to the GBA, even with a 10 to 1 it was quite slow because the vector graphics took more horsepower than the 6502 translation itself. Sure, esp with Asteroids, shortcuts are easy in the vector graphics engine, but I gave up because the commercial version came out.

You brought up an interesting idea though, if push comes to shove the coprocessor could handle the tile video hardware emulation...

It would be VERY interesting to see what ARM instruction sequences when disassembled, translated to C and then back to ARM instructions looks like. Unfortunately every memory access has to be filtered at a high level, and every few instructions most likely accesses memory. Anyway, it is starting to sound like a very interesting project. I wonder if a machine code to machine code translation might be better/easier than machine code to C to machine code.

I have an arm disassembler that I wrote that you can have/use if you like, no guarantees on how accurate it is, I used it in parallel with arms and gccs disassemblers so hopefully I worked out the kinks. (www.dwelch.com/ipod). I am sure I have a thumb disassembler, but dont know where it is, shouldnt be hard to re-create. Having the smarts to know whether to disassemble in arm or thumb, that is the trick. I would probably build a trigger list by running the rom on a known-good emulator (that I had the source to). In the same way that you might build a branch destination table for disassembling on a variable length instruction set architecture (most CISC, some RISC). Fortunately for ARM you dont necessarily need a branch destination table, you still have to determine instructions from embedded data. There again you can use a trigger list or the easiest (but least-legal) thing is to just carry the rom image around with the application.

A static re-compile takes hand holding rather than a generic program that can re-compile any rom. So every rom translated is going to take effort (vs simple emulation where one solution fits many if not all). Now what would also be an interesting thing is that some developers write programs in modules, for example this bit of code handles the game, builds a virtual display memory, the every so often a function is called to copy the display data from ram to the video memory. where I am headed is programs written this way will have large sections of code that only deal with system memory and do not access peripherals every few instructions. These programs would be the easiest to translate, either just do an instruction to instruction translation for these sections, then emulate or translate in C the sections that deal with the peripherals. When performance drives accessing the peripherals more often (as the case may be on the GBA), you will end up spending a lot of time with emulation overhead (even in a translation as calling a generic write memory and read memory function to trap peripheral accesses and modify the address for system memory is no different than emulation).

I dont know enough about the gp2x yet, much less the mmu, I am thinking that "All you have to do" is simply run the GBA rom as is, and all of the gp2x work is in handling mmu faults. Basically map the gba system memory to gp2x memory, and set the mmu to fault on every other address, then the mmu fault handler takes the address determines what peripheral it is and goes from there. What is nice is that the vba source would have to do all of this already when emulating memory reads and writes, so it is a matter of borrowing the bulk of the code from an existing emulator and just gluing it all together. This would give you a one size fits all solution (most if not all roms would run without any rom specific hand holding required). All of the non i/o code would run 1:1, you would not have to deal with thumb vs arm (except in your handler front end of course). No disassembly...And here again you have two processors, one could handle the peripheral emulation to get a foothold on the overhead.


This development wiki is already, what, 35 pages deep with developer related topics. Where would I start looking to find out what the state of the machine is in when GamePark starts execution of your application, what can and cant you do with the machine? (well actually there had better be a way to take over control of the whole machine, even if the user has to power cycle or the application has to force a cold boot). How is SDL implemented, how much does it rely on the underlying linux os, or is it independent of the os talking directly to the hardware?


Sorry to ramble on so long...
 
Last edited by a moderator:
I am thinking that "All you have to do" is simply run the GBA rom as is, and all of the gp2x work is in handling mmu faults.
I thought about that, but figured it would be more work than slapping an ARM->C translator in the middle of VisualBoyAdvance.
SDL does not work without linux so it would require re-writing almost all the code and figuring out the gp2x MMU (something I have no experience with). On the other hand, there's a possibility that, with Squidge's MMU hack, the emulator can have suficient control over the MMU without having to kick out Linux. Not sure, haven't experimented with it. Another problem is that executing the GBA ROM directly means no timing code.

Unfortunately every memory access has to be filtered at a high level, and every few instructions most likely accesses memory
True, but I am hoping to be able to do at least some of that filtering when generating code rather than while running it.

I wonder if a machine code to machine code translation might be better/easier than machine code to C to machine code.
It is easier on the programmer, but harder for the end-user. Since I am both, it's not a problem. ^_^
On a more serious tone, ARM->C->ARM permits more costly compile-time optimizations that are done only once. Besides, it's much easier to debug and is platform-independent. Dynarecs have to run around compiling and optimizing often and as quickly as possible.

I have an arm disassembler that I wrote that you can have/use if you like
I just got it, looks pretty well and cleanly written. I'll probably use it as a guide for a Dynarec I've been planning on (another project). :)

A static re-compile takes hand holding rather than a generic program that can re-compile any rom.
True, but I'm hoping the benefits will outweigh that. Once the hand-held part is done it will be the same as a normal game from the end-user's point of view so it's not all bad.
 
Daily update:
1) Today I started doing daily updates.
2) Currently re-writing VisualBoyAdvance/src/arm-new.h so that it outputs code rather than executing it. This will be quite a lot of work. Just finished OP_MOVS and all those before it.

example:
Original code:
Code:
#define OP_ADD \
	{\
	  reg[dest].I = reg[base].I + value;\
	}
#define OP_ADDS \
   {\
	 u32 lhs = reg[base].I;\
	 u32 rhs = value;\
	 u32 res = lhs + rhs;\
	 reg[dest].I = res;\
	 Z_FLAG = (res == 0) ? true : false;\
	 N_FLAG = NEG(res) ? true : false;\
	 ADDCARRY(lhs, rhs, res);\
	 ADDOVERFLOW(lhs, rhs, res);\
   }

My version:
Code:
#define OP_ADD \
  sprintf( TK::Code, "  {\n  reg[%d].I = reg[%d].I + %d;\n  }", dest, base, value );\
  tkdumpf( TK::Reg[ dest ], TK::Reg[ base ], TK::Code );

#define OP_ADDS \
  sprintf( TK::Code, "u32 lhs = reg[%d].I; u32 rhs = %d; u32 res = rhs + lhs; reg[%d].I = res;", base, value, dest );\
  sprintf( &TK::TOut[ 18 ][ 0 ], "Z_FLAG = (res == 0) ? true : false;" );\
  sprintf( &TK::TOut[ 16 ][ 0 ], "N_FLAG = NEG(res) ? true : false;" );\
  sprintf( &TK::TOut[ 17 ][ 0 ], "ADDCARRY(lhs, rhs, res);" );\
  sprintf( &TK::TOut[ 19 ][ 0 ], "ADDOVERFLOW(lhs, rhs, res);" );\
  TK::Last->reqscope = 1;\
  tkdumpf( TK::Reg[ dest ] | TK::Z | TK::N | TK::C | TK::V,\
		   TK::Reg[ base ], TK::Code );

tkdumpf adds that chunk of code to a linked list, where it will be stored untill the end of that basic block is found. When that happens, I intend to make it analize which tests are necessary and eliminating the ones that aren't. tkdumpf's parameters are Output registers (registers modified by this instruction), Input registers (registers this instruction needs to execute), code buffer.
 
TKF15H posted on Jul 9 2006 at 03:11 AM said:
Daily update:
1) Today I started doing daily updates.
2) Currently re-writing VisualBoyAdvance/src/arm-new.h so that it outputs code rather than executing it. This will be quite a lot of work. Just finished OP_MOVS and all those before it.

example:
Original code:
Code:
#define OP_ADD \
	{\
	  reg[dest].I = reg[base].I + value;\
	}
#define OP_ADDS \
   {\
	 u32 lhs = reg[base].I;\
	 u32 rhs = value;\
	 u32 res = lhs + rhs;\
	 reg[dest].I = res;\
	 Z_FLAG = (res == 0) ? true : false;\
	 N_FLAG = NEG(res) ? true : false;\
	 ADDCARRY(lhs, rhs, res);\
	 ADDOVERFLOW(lhs, rhs, res);\
   }

My version:
Code:
#define OP_ADD \
  sprintf( TK::Code, "  {\n  reg[%d].I = reg[%d].I + %d;\n  }", dest, base, value );\
  tkdumpf( TK::Reg[ dest ], TK::Reg[ base ], TK::Code );

#define OP_ADDS \
  sprintf( TK::Code, "u32 lhs = reg[%d].I; u32 rhs = %d; u32 res = rhs + lhs; reg[%d].I = res;", base, value, dest );\
  sprintf( &TK::TOut[ 18 ][ 0 ], "Z_FLAG = (res == 0) ? true : false;" );\
  sprintf( &TK::TOut[ 16 ][ 0 ], "N_FLAG = NEG(res) ? true : false;" );\
  sprintf( &TK::TOut[ 17 ][ 0 ], "ADDCARRY(lhs, rhs, res);" );\
  sprintf( &TK::TOut[ 19 ][ 0 ], "ADDOVERFLOW(lhs, rhs, res);" );\
  TK::Last->reqscope = 1;\
  tkdumpf( TK::Reg[ dest ] | TK::Z | TK::N | TK::C | TK::V,\
		   TK::Reg[ base ], TK::Code );

tkdumpf adds that chunk of code to a linked list, where it will be stored untill the end of that basic block is found. When that happens, I intend to make it analize which tests are necessary and eliminating the ones that aren't. tkdumpf's parameters are Output registers (registers modified by this instruction), Input registers (registers this instruction needs to execute), code buffer.

:D well i'm glad to see someone actively working on the problem. i'm not as knowledgable as most people participating to this topic in the domain of emulation, so tell me, which way did you choose, the ARM -> C -> ARM way ?
 
Last edited by a moderator:
which way did you choose, the ARM -> C -> ARM way ?
Yup. Which means you'll need GCC to compile games. Or you could get someone else to do it for you, provided you both have the exact same ROM.
 
Back
Top