- Joined
- Dec 8, 2005
- Messages
- 6,603
- Age
- 40
- Location
- UK
- Website
- pokeparadox.itch.io
- WEBSITE
- https://github.com/pokeparadox
- YOUTUBE
- pokeparadox
Noooo hats are bad! Take them off!
hats off to Zoddy
hats off to Zoddy
Dynarec Completed For Psx4all Project.
- Thread starter zodttd
- Start date
EvilAlex
Still Fresh
i hate hats, but i'l buy one with the name zodttd on it and wear it everyday of the week!
Shikaku
ROFL THE WORD PENIS IS HILARIOUS!
... You guys are silly...
Jarska333
What do you want?
Haha, you pissed of the mods!
Someone close this thread before it hits 50...
Someone close this thread before it hits 50...
Draken
Well-Known Member
What's that? We'll have no problems here.
Playable portable FF7 ftw.
Playable portable FF7 ftw.
Damume
Still Fresh
Forget FF7 ..... I want to play Power Shovel again
zodttd
Solving your premature emulation since the Tapwave
How'd we piss off the mods?
Some good news and bad news.
First the bad news...The ARM Dynarec just doesn't seem to perform well. It seems to be more of an issue of taking a processor with 32 registers, of which a good portion are for "general use", and then mapping them to the ARM's 16 registers, of which I have at most 8 to work with right now. I tried A LOT of techniques, including reg mapping, and constant known values to reduce the need to recompile ops. All in all, even with 32mb of translation cache, I land up with less speed than a highly optimized interpreter. It could very well be that I did a bad job on the ARM Dynarec, but with so much tried and failed, I'm putting the ARM Dynarec on hold...
The good news is, with little modification the interpreter is going really fast. With all the adjustments made lately to psx4all, after trying the "new" interpreter again, Wipeout is about 33% faster. I actually played an entire lap in Ridge Racer too. Sound also works now, and is enabled, but off by default. Sound slows some games down too much, and since we're still struggling for performance, it's more "fluff" than anything.
Now that I have more knowledge of ARM and MIPS in general, I believe I can increase the speed of the new interpreter even more.
One more feature being done is to have a "target FPS" setting changable during gameplay. It basically changes the timings to make things more suitable, and by using a target FPS it's more "readable".
Btw, for reference, the MIPS R3000A -> general MIPS32 dynarec has been a huge success, allowing many games on the V43 to perform fullspeed. I really wish there were more MIPS handhelds out there besides the PSP. It would make my life alot easier. Hopefully a Gamepark engineer is reading.
Once Unai let's me know what needs to be done on his end for a release, we can get a date set.
Oh and in other news, I'm still working out the details for doing an X86 -> ARM Dynarec for DOSBox. After psx4all, I'm wondering how feasible the dynarec will be, so by talking to DOSBox developers will answer that soon.
So all said, the dynarec was a downer, but there's still juice left in the interpreter it seems. Hopefully at a later time I can figure out what to do to make such a dynarec perform better.
Time to get more work done...
Oh yeah, if someone makes a ZodTTD ( or various variation ) hat I would really want one!
Some good news and bad news.
First the bad news...The ARM Dynarec just doesn't seem to perform well. It seems to be more of an issue of taking a processor with 32 registers, of which a good portion are for "general use", and then mapping them to the ARM's 16 registers, of which I have at most 8 to work with right now. I tried A LOT of techniques, including reg mapping, and constant known values to reduce the need to recompile ops. All in all, even with 32mb of translation cache, I land up with less speed than a highly optimized interpreter. It could very well be that I did a bad job on the ARM Dynarec, but with so much tried and failed, I'm putting the ARM Dynarec on hold...
The good news is, with little modification the interpreter is going really fast. With all the adjustments made lately to psx4all, after trying the "new" interpreter again, Wipeout is about 33% faster. I actually played an entire lap in Ridge Racer too.
Sound also works now, and is enabled, but off by default. Sound slows some games down too much, and since we're still struggling for performance, it's more "fluff" than anything.Now that I have more knowledge of ARM and MIPS in general, I believe I can increase the speed of the new interpreter even more.
One more feature being done is to have a "target FPS" setting changable during gameplay. It basically changes the timings to make things more suitable, and by using a target FPS it's more "readable".
Btw, for reference, the MIPS R3000A -> general MIPS32 dynarec has been a huge success, allowing many games on the V43 to perform fullspeed. I really wish there were more MIPS handhelds out there besides the PSP. It would make my life alot easier. Hopefully a Gamepark engineer is reading.
Once Unai let's me know what needs to be done on his end for a release, we can get a date set.
Oh and in other news, I'm still working out the details for doing an X86 -> ARM Dynarec for DOSBox. After psx4all, I'm wondering how feasible the dynarec will be, so by talking to DOSBox developers will answer that soon.
So all said, the dynarec was a downer, but there's still juice left in the interpreter it seems. Hopefully at a later time I can figure out what to do to make such a dynarec perform better.
Time to get more work done...
Oh yeah, if someone makes a ZodTTD ( or various variation
) hat I would really want one!
Julius
Well-Known Member
What about releasing the source code of the dynarec?
Maybe someone else can figure out how to speed it up.
Maybe someone else can figure out how to speed it up.
Jarska333
What do you want?
zodttd posted on Oct 3 2006 at 10:21 PM said:How'd we piss off the mods?
Some good news and bad news.
Shikaku had, for a while, "I pissed of the mods" as his signature...
And any news is good news.
Last edited by a moderator:
zodttd
Solving your premature emulation since the Tapwave
Cafepress here I come! I'm getting some Zod-wear. ;P
Shikaku
ROFL THE WORD PENIS IS HILARIOUS!
[vader]NNNNNOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!!!!!!!!!!![/vader]
But at least you tried... one day, maybe one day...
But at least you tried... one day, maybe one day...
X-Code
Beermaster
Hmmm... Yeah, ok
I'll get myself more beers for now
I'll get myself more beers for now
Jawbreaker4Fs
Still Fresh
- Joined
- May 30, 2006
- Messages
- 5
So does this mean a full speed PSX emulator on the GP2X isn't going to happen?
Shikaku
ROFL THE WORD PENIS IS HILARIOUS!
Jawbreaker4Fs posted on Oct 3 2006 at 03:54 PM said:
It wasn't ever going to happen anyway kid.
Last edited by a moderator:
Cervante
PS3 ID: Cervante1
Shikaku posted on Oct 3 2006 at 03:55 PM said:
Do I really need alot of speed to enjoy breath of fire tres?
Last edited by a moderator:
i learned a new word today threaded interpreter it sounds sweet
http://forums.ngemu.com/web-development-pr...troduction.html
hmm that guy M.I.K.e7 said
[/quote] Our next topic is the translation cache. But I won't go into detail how the generated code blocks are actually stored and freed, since I haven't given it too much thought yet. Although it is most likely that it's a good idea to allocate a large block of memory via the operating system and then make your own arena managament in that chunk for performance reasons.
The most interesting thing is how to remember which block is already translated and how to do that fast.
For 8-bit processors which normally have a 16-bit address range it would be easy to simply make an array of addresses and mark each single address when it is the start address of a recompiled block. But for those processors where dynamic recompilation is really interesting, this simple approach does not work, even if you take into account that most of the processors only permit aligned instructions. So other, less memory consuming methods have to be found.
When you come straight from computer science studies the most obvious solution would be a hash. That means a value is mapped through a special hash function (often containing a modulo operation to define the confines) onto a much smaller range array. The advantage is that in the best case you only perform the simply function and get the key to the memory location where you find the address of the recompiled code with one memory access (or a zero if it hasn't been recompiled yet). The problem is that the hash function is likely to generate the same key for different values, which happens more often when the hash array is too small and/or the hash function isn't that good. When this happens you get a so-called collision, which has to be solved somehow. The typical solution is to do a linked list of all values that collided, but that means that you need several lookups since you'll have to search the list in linear order until you find what you want.
So hashing is a possible solution, but not necessarily the best one.
Another problem that might have to be solved during the translation cache lookup is to detect self-modifying code. If the code behaves nicely you can skip that, but if self-modifying code occurs from time to time you'll have to detect it.
The typical computer science solution would likely be to run some kind of checksum (eg. CRC) over the original code, and regenerate the checksum every time the block is about to be executed. If the checksum has changed the block has to be translated again.
The problem here is that checksums can be fooled when several values (in the case instruction encodings) change but the change is not visable in the checksum. Also recalculating the checksum before every run of the code should hit on performance hard.
Since traditional solutions aren't perfect let's see what alternatives there are...
A drity trick that UltraHLE is said to be using works as follows:
Instead of using a data structure to memorize which blocks have been translated, the first instruction of the block is replaced by an illegal instruction that also contains the offset to the generated code - since MIPS has 32-bit instructions that's quite possible. So the emulator just takes a look at the start of the block to recognize if it has been translated already or if it has to be translated still. A side-effect is that code that modifies the first instruction of the block leads to the block being translated again automatically.
The disadvantage is that self-modifying code is only detected when the illegal instruction is replaced, and since you modify the original code you might run into problems when that block is actually just a sub-block of a larger block that might run into that illegal instruction. This could be handled of course, when the original instruction is stored somewhere, but it makes things a bit more complicated.
The elegant solution would be a paged translation map. When you don't know what to do in emulation, it often helps to take a look at how the hardware does things.
Most of the processors that are interesting candidates for dynamic recompilation organize their memory in pages, ie. the higher part of the address is the page number and the lower part is the page offset. The typical page size is 4K, which means that in a 32-bit address space you'd have 20-bit for the page number and 12-bit for the page offset. Even if you want to keep track of all possible pages (which isn't normally necessary) you'd need 1MB (= 20-bit address range) x 4 byte (size of an address on a 32-bit system) = 4MB, which might sound like much but actually keeps track of all pages in a 4GB address range. Now you still need 4KB x 4 byte = 16KB per page to have all the addresses, but you only need to keep track of pages that actually contain code, so that's far less than you might assume, and when you have a processor like MIPS where all instructions have to be aligned to 32-bit (ie. the start address of each instruction has the lower two bits cleared) it's only 4K.
When the emulation jumps to a certain address you first look at the page (by shifting the address right by 12 bit) to see if code in tat page has already been translated. If there is no translated code yet you allocate a new memory location that is large enough to hold all addresses for that page, enter a pointer to that area in the page number entry, and finally enter the address to the translated code in the location of the page offset at which the original code block starts.
The lookup needs just two memory accesses, one to find the find the location of the page directory via the page number and another to look up the address of the translated code via the page offset in the page directory.
I hope this doesn't sound too complicated, because it really isn't...
The paged transmap also allows for a solution to identify self-modifying code. Every time a write access is performed it is checked if the page number for that access indicates that code on that page has been translated, the cached code for that page is freed and the address in the page number entry cleared to force a recompilation of the code. This might sound crude, but in paged environments data and code are normally on different pages, so it is really likely that the code has been modified.
Since I was talking about Harvard architectures (ie. split integer and data caches) before, there is another way to detact self-modifying code in that case. Since those architectures have to flush their caches you can try to trace that (either some system call or the processor operation) and free the appropriate code that is no longer valid. According to an article (http://devworld.apple.com/technotes/pt/pt_39.html) the official 68K emulator for PowerMacs uses that solution.[/quote]
its like a freakin book on howto design a dynarec that thread
http://forums.ngemu.com/web-development-pr...troduction.html
hmm that guy M.I.K.e7 said
[/quote] Our next topic is the translation cache. But I won't go into detail how the generated code blocks are actually stored and freed, since I haven't given it too much thought yet. Although it is most likely that it's a good idea to allocate a large block of memory via the operating system and then make your own arena managament in that chunk for performance reasons.
The most interesting thing is how to remember which block is already translated and how to do that fast.
For 8-bit processors which normally have a 16-bit address range it would be easy to simply make an array of addresses and mark each single address when it is the start address of a recompiled block. But for those processors where dynamic recompilation is really interesting, this simple approach does not work, even if you take into account that most of the processors only permit aligned instructions. So other, less memory consuming methods have to be found.
When you come straight from computer science studies the most obvious solution would be a hash. That means a value is mapped through a special hash function (often containing a modulo operation to define the confines) onto a much smaller range array. The advantage is that in the best case you only perform the simply function and get the key to the memory location where you find the address of the recompiled code with one memory access (or a zero if it hasn't been recompiled yet). The problem is that the hash function is likely to generate the same key for different values, which happens more often when the hash array is too small and/or the hash function isn't that good. When this happens you get a so-called collision, which has to be solved somehow. The typical solution is to do a linked list of all values that collided, but that means that you need several lookups since you'll have to search the list in linear order until you find what you want.
So hashing is a possible solution, but not necessarily the best one.
Another problem that might have to be solved during the translation cache lookup is to detect self-modifying code. If the code behaves nicely you can skip that, but if self-modifying code occurs from time to time you'll have to detect it.
The typical computer science solution would likely be to run some kind of checksum (eg. CRC) over the original code, and regenerate the checksum every time the block is about to be executed. If the checksum has changed the block has to be translated again.
The problem here is that checksums can be fooled when several values (in the case instruction encodings) change but the change is not visable in the checksum. Also recalculating the checksum before every run of the code should hit on performance hard.
Since traditional solutions aren't perfect let's see what alternatives there are...
A drity trick that UltraHLE is said to be using works as follows:
Instead of using a data structure to memorize which blocks have been translated, the first instruction of the block is replaced by an illegal instruction that also contains the offset to the generated code - since MIPS has 32-bit instructions that's quite possible. So the emulator just takes a look at the start of the block to recognize if it has been translated already or if it has to be translated still. A side-effect is that code that modifies the first instruction of the block leads to the block being translated again automatically.
The disadvantage is that self-modifying code is only detected when the illegal instruction is replaced, and since you modify the original code you might run into problems when that block is actually just a sub-block of a larger block that might run into that illegal instruction. This could be handled of course, when the original instruction is stored somewhere, but it makes things a bit more complicated.
The elegant solution would be a paged translation map. When you don't know what to do in emulation, it often helps to take a look at how the hardware does things.
Most of the processors that are interesting candidates for dynamic recompilation organize their memory in pages, ie. the higher part of the address is the page number and the lower part is the page offset. The typical page size is 4K, which means that in a 32-bit address space you'd have 20-bit for the page number and 12-bit for the page offset. Even if you want to keep track of all possible pages (which isn't normally necessary) you'd need 1MB (= 20-bit address range) x 4 byte (size of an address on a 32-bit system) = 4MB, which might sound like much but actually keeps track of all pages in a 4GB address range. Now you still need 4KB x 4 byte = 16KB per page to have all the addresses, but you only need to keep track of pages that actually contain code, so that's far less than you might assume, and when you have a processor like MIPS where all instructions have to be aligned to 32-bit (ie. the start address of each instruction has the lower two bits cleared) it's only 4K.
When the emulation jumps to a certain address you first look at the page (by shifting the address right by 12 bit) to see if code in tat page has already been translated. If there is no translated code yet you allocate a new memory location that is large enough to hold all addresses for that page, enter a pointer to that area in the page number entry, and finally enter the address to the translated code in the location of the page offset at which the original code block starts.
The lookup needs just two memory accesses, one to find the find the location of the page directory via the page number and another to look up the address of the translated code via the page offset in the page directory.
I hope this doesn't sound too complicated, because it really isn't...
The paged transmap also allows for a solution to identify self-modifying code. Every time a write access is performed it is checked if the page number for that access indicates that code on that page has been translated, the cached code for that page is freed and the address in the page number entry cleared to force a recompilation of the code. This might sound crude, but in paged environments data and code are normally on different pages, so it is really likely that the code has been modified.
Since I was talking about Harvard architectures (ie. split integer and data caches) before, there is another way to detact self-modifying code in that case. Since those architectures have to flush their caches you can try to trace that (either some system call or the processor operation) and free the appropriate code that is no longer valid. According to an article (http://devworld.apple.com/technotes/pt/pt_39.html) the official 68K emulator for PowerMacs uses that solution.[/quote]
its like a freakin book on howto design a dynarec that thread
kevcal
Coding " Abduction;Retrieval "
Shame - doesn't look hopeful for a playable emulator then.. Still I guess you'll keep thinking about it..
Guess it was a long shot - hopefully you'll get some ideas in the future to improve the dynarec to work with less registers.
Either way it's been a good attempt!!!
Maybe a faster interpreter will get 2d games playable soon anyway.
Is the dynarec a standalone lib? If so, if you can, it may be of use to release the source-code for others to play around with? Up to you obviously!
Keep coding zodttd - you're very talented
I don't think we'll give up on it yet.
Kev
Still I guess you'll keep thinking about it..Guess it was a long shot - hopefully you'll get some ideas in the future to improve the dynarec to work with less registers.
Either way it's been a good attempt!!!
Maybe a faster interpreter will get 2d games playable soon anyway.
Is the dynarec a standalone lib? If so, if you can, it may be of use to release the source-code for others to play around with? Up to you obviously!
Keep coding zodttd - you're very talented
I don't think we'll give up on it yet.
Kev
