Release Pcsx-Rearmed

[ari64]

I was speaking about the way all the source file is coded : very long functions and no design pattern. I don't mean your code is unefficient but very hard to maintain or to be understandable by the others. I had to maintain similar codes in work which were coded by electronics engineers who coded similarly. It was always a nightmare to try to change something deeply and most time you need to rewrite from the scratch, so I understand Notaz's feeling about the difficulty to bring a change without breaking something. But take it easy, that you could code a working recompiler this way is pretty amazing indeed.

My question is what would you consider a good design?

For example, if you look at gpSP, it is a very similar layout, very long functions in a handful of source files. The original mupen64plus recompiler has many short functions. They are easy to find, but there is a lot of code duplication. PCSX is the same, lots of code duplication. Daedalus has many short functions in a dozen different source files, and I found it difficult to read because it is so scattered.

A major problem with cpu emulators is that you have a lot of functions that are similar but not exactly the same. Inexperienced developers tend to cut-and-paste a lot of duplicate code. I tried to factor out things into subroutines, although I still ended up with some bits of duplicate code that were hard to factor out. Strmnnrmn and Exophase primarily used macros.

Which approach do you consider best, and why?

Sure, just hlide and Exophase kind of implied it should be doable without too much effort and I wanted to show why I chose to fall back to interpreter this time.

It is doable. Move the stuff that sets the return address above the call to ds_assemble, but after the REG_PREFETCH bit. Then make sure it doesn't load the old value of r31. Under "// Load the delay slot registers if necessary" add
if(rt1==0||rs1[i+1]!=rt1) for the first case, and
if(rt1==0||rs2[i+1]!=rt1) for the second.
Then test it to make sure everything works correctly

You can do the constant propagation thing too if you want, but make sure you don't overwrite r31 with the original value if it's already been set with the result of the add instruction, which will happen in load_consts().

 

[tarwin]

@tarwin: My version of FF7 works like a charm without any special settings... maybe your image is faulty.

I've ripped the image myself, like i did with the g-police image, which runs.
But maybe you are right, i can't run them in epsxe either. ^^;

 

[hlidegp2x]

I was speaking about the way all the source file is coded : very long functions and no design pattern. I don't mean your code is unefficient but very hard to maintain or to be understandable by the others. I had to maintain similar codes in work which were coded by electronics engineers who coded similarly. It was always a nightmare to try to change something deeply and most time you need to rewrite from the scratch, so I understand Notaz's feeling about the difficulty to bring a change without breaking something. But take it easy, that you could code a working recompiler this way is pretty amazing indeed.

My question is what would you consider a good design?

For example, if you look at gpSP, it is a very similar layout, very long functions in a handful of source files. The original mupen64plus recompiler has many short functions. They are easy to find, but there is a lot of code duplication. PCSX is the same, lots of code duplication. Daedalus has many short functions in a dozen different source files, and I found it difficult to read because it is so scattered.

A major problem with cpu emulators is that you have a lot of functions that are similar but not exactly the same. Inexperienced developers tend to cut-and-paste a lot of duplicate code. I tried to factor out things into subroutines, although I still ended up with some bits of duplicate code that were hard to factor out. Strmnnrmn and Exophase primarily used macros.

Which approach do you consider best, and why?

In a company, that kind development in software leads to a very bad maintenance or evolution if the program author leaves the company, because he/she rarely documents his/her source or doesn't even create a document to help to understand the layout. I don't consider PCSX source as a good piece of code, unlike gPSP. Deadalus has some good ideas but was pretty hard to read for the reason you stated. What I noticed is that you wrote your long sequences of instructions in C as if you are writing an assembly code (over-control on what you write instead of trying to abstract some algorithms). Having macros helps because they name what they do instead of a long sequence of instructions and alleviates - doesn't remove - the source duplication. And sure if one use only C, it limits one's imagination about how abstract and factorize one's source. I personally think the best choice is when there is balance between efficiency and readability according to project purpose.

Anyway, it's your choice. I'm not saying you must change it and I don't think you need to justify your choice. It is not as if you coded your recompiler so it may be initially reusable in another project by other coders and for another MIPS processor than the one in N64.



By the way, I can see a lot of sequence of 'if' without 'else if', is there any reason to keep them this way ?

example:

 6352     if((source[i]&0x30000)==0) // BC1F
 6353     {
 6354       emit_mov2imm_compact(ba[i],addr,start+i*4+8,alt);
 6355       emit_testimm(s1l,0x800000);
 6356       emit_cmovne_reg(alt,addr);
 6357     }
 6358     if((source[i]&0x30000)==0x10000) // BC1T    <--- why no 'else' ?
 6359     {
 6360       emit_mov2imm_compact(ba[i],alt,start+i*4+8,addr);
 6361       emit_testimm(s1l,0x800000);
 6362       emit_cmovne_reg(alt,addr);
 6363     }
 6364     if((source[i]&0x30000)==0x20000) // BC1FL    <--- why no 'else' ?
 6365     {
 6366       emit_testimm(s1l,0x800000);
 6367       nottaken=(int)out;
 6368       emit_jne(0);
 6369     }
 6370     if((source[i]&0x30000)==0x30000) // BC1TL    <--- why no 'else' ?
 6371     {
 6372       emit_testimm(s1l,0x800000);
 6373       nottaken=(int)out;
 6374       emit_jeq(0);
 6375     }

 

[larry]

By the way, I can see a lot of sequence of 'if' without 'else if', is there any reason to keep them this way ?
example:

6352     if((source[i]&0x30000)==0) // BC1F
6353     {
6354       emit_mov2imm_compact(ba[i],addr,start+i*4+8,alt);
6355       emit_testimm(s1l,0x800000);
6356       emit_cmovne_reg(alt,addr);
6357     }
6358     if((source[i]&0x30000)==0x10000) // BC1T    <--- why no 'else' ?
6359     {
6360       emit_mov2imm_compact(ba[i],alt,start+i*4+8,addr);
6361       emit_testimm(s1l,0x800000);
6362       emit_cmovne_reg(alt,addr);
6363     }
6364     if((source[i]&0x30000)==0x20000) // BC1FL    <--- why no 'else' ?
6365     {
6366       emit_testimm(s1l,0x800000);
6367       nottaken=(int)out;
6368       emit_jne(0);
6369     }
6370     if((source[i]&0x30000)==0x30000) // BC1TL    <--- why no 'else' ?
6371     {
6372       emit_testimm(s1l,0x800000);
6373       nottaken=(int)out;
6374       emit_jeq(0);
6375     }

Actually a switch statement would be even better. Compilers are really good at optimizing switch statements; twenty years ago I worked for a compiler company that had three strategies for switch statements (jump table, perfect hash, binary if tree). More readable and faster--a double win!

 

[ari64]

In a company, that kind development in software leads to a very bad maintenance or evolution if the program author leaves the company, because he/she rarely documents his/her source or doesn't even create a document to help to understand the layout.

http://pandorawiki.org/Mupen64plus_dynamic_recompiler

The function that you quoted (pagespan_assemble) is described under "Mapping executable pages".

By the way, I can see a lot of sequence of 'if' without 'else if', is there any reason to keep them this way ?

No.

Actually a switch statement would be even better. Compilers are really good at optimizing switch statements; twenty years ago I worked for a compiler company that had three strategies for switch statements (jump table, perfect hash, binary if tree). More readable and faster--a double win!

Well, in storelr_assemble(), there is a four-case if-then in assembly language. The structure is like so:

 tst r0,#2
 bne L2
 tst r0,#1
 bne L1
 ...
 b L4
L1:
 ...
 b L4
L2:
 tst r0,#1
 bne L3
 ...
 b L4
L3:
 ...
L4:

If I was to do this as a jump table, it would be like so:

 and r1, r0, #3
 ldr pc, [pc, r1, lsl #2]
 nop
 dw L0
 dw L1
 dw L2
 dw L3
L0:
 ...
 b L4
L1:
 ...
 b L4
L2:
 ...
 b L4
L3:
 ...
L4:

It's not any better, and actually takes up more space, and another register. The break-even point for this sort of thing is typically around five or six cases.

 

[hlidegp2x]

By the way, I can see a lot of sequence of 'if' without 'else if', is there any reason to keep them this way ?

No.

Actually a switch statement would be even better. Compilers are really good at optimizing switch statements; twenty years ago I worked for a compiler company that had three strategies for switch statements (jump table, perfect hash, binary if tree). More readable and faster--a double win!

Well, in storelr_assemble(), there is a four-case if-then in assembly language. The structure is like so:

 tst r0,#2
    bne L2
    tst r0,#1
    bne L1
    ...
    b L4
   L1:
    ...
    b L4
   L2:
    tst r0,#1
    bne L3
    ...
    b L4
   L3:
    ...
   L4:

If I was to do this as a jump table, it would be like so:

 and r1, r0, #3
    ldr pc, [pc, r1, lsl #2]
    nop
    dw L0
    dw L1
    dw L2
    dw L3
   L0:
    ...
    b L4
   L1:
    ...
    b L4
   L2:
    ...
    b L4
   L3:
    ...
   L4:

It's not any better, and actually takes up more space, and another register. The break-even point for this sort of thing is typically around five or six cases.

huh I would expect for the compiler to choose the best strategy, not necessarily a jump table which is usually selected when the number of cases is pretty high and their values not too sparse : simple if/else or binary tree for instance. Unless you can prove it's not the case with arm-gcc...

 

[notaz]

Time for r5 release.
http://notaz.gp2x.de/releases/pcsxr/pcsx_rearmed_r5.pnd

The compatibility should go up a good deal thanks to fixed cdrom code from newer PCSX-Reloaded and my recompiler fixes. All final fantasies work now, for FF8 you need to enable BIOS and play it from the beginning (don't use old savestates with HLE, they are bad). There is now support for more compressed CD image formats, but before reporting bugs try .cue/.bin/.img first.

I'll probably stop working on compatibility now, it's really time to start my optimization efforts. You can keep doing compatibility lists for future reference though.

Here is complete changelog:

• added support for .bz format, also partial support for .znx and eboot.pbp formats
• merged latest cdrom code from PCSX-Reloaded project
• fixed remaining savestate incompatibilities between PCSX4ALL and P.E.Op.S. GPU plugins
• fixed channel disable preventing irqs in P.E.Op.S. SPU plugin
• fixed some alignment issues
• added handling for branches in delay slots
• fixed some unexpected drops to menu
• fixed lots of recompiler related issues (see GIT)
• added watchdog thread to detect emulator lockups
• minor frontend adjustments

 

[Dead1nside]

That all sounds awesome Notaz, thanks again, makes a guy want to get a premium Pandora just to try this

 

[larry]

It's not any better, and actually takes up more space, and another register. The break-even point for this sort of thing is typically around five or six cases.

I'll take your word for it. My point was, if you express your code as a switch statement the compiler will (hopefully) figure out which implementation is fastest.

p.s. hooray for r5!

 

[jonlad1]

Thanks so much Notaz,

The last donation I made to you got rejected for some reason?

Will try again....

EDIT: Just been trying R5 on SOTN Jap patched version and the sound seems way improved. Unfortunately the graphics seem to have suffered a little. Characters are more dark than on R4, walls flickering blue. Tried R4 straight afterwards, doesnt have this problem.

This is the game I play most of, so I notice issues. I will test with more though.

Anything in particular I can test/try out Notaz?

 

[Michoko]

Wooot, thanks a lot Notaz, I can't wait trying this new release!

 

[EvilDragon]

Wooooooot
Suikoden 2-time, finally
Thank you soooo much

 

[Emnasut]

Thanks so much Notaz,

The last donation I made to you got rejected for some reason?

Will try again....

EDIT: Just been trying R5 on SOTN Jap patched version and the sound seems way improved. Unfortunately the graphics seem to have suffered a little. Characters are more dark than on R4, walls flickering blue. Tried R4 straight afterwards, doesnt have this problem.

This is the game I play most of, so I notice issues. I will test with more though.

Anything in particular I can test/try out Notaz?

Why are you playing the Jap patched version of SOTN, is it better? I only know SOTN from xbox live and was looking forward to playing it on my Pandora.

Also, thanks for all the great work notaz!

 

[tarwin]

Thx for pushing out the updates so fast Notaz! =D

Ive got FF7 to run, the image was faulty.
Next item on my NEED-list is Wipeout 3.

The game loads fine and runs very good, but crashes at exact the moment when you drive over a blue line.
In R4 the game froze, but you where still able to get into the Menu. In R5 the whole emu crashes.
I've toyed around with the compatiblity settings, but only succeeded in slowing everything down. Chrashes pesist.

Has anyone found a working setting?

-T

 

[Abaddon666]

Time for r5 release.
http://notaz.gp2x.de/releases/pcsxr/pcsx_rearmed_r5.pnd

The compatibility should go up a good deal thanks to fixed cdrom code from newer PCSX-Reloaded and my recompiler fixes. All final fantasies work now, for FF8 you need to enable BIOS and play it from the beginning (don't use old savestates with HLE, they are bad). There is now support for more compressed CD image formats, but before reporting bugs try .cue/.bin/.img first.

I'll probably stop working on compatibility now, it's really time to start my optimization efforts. You can keep doing compatibility lists for future reference though.

Here is complete changelog:

• added support for .bz format, also partial support for .znx and eboot.pbp formats
• merged latest cdrom code from PCSX-Reloaded project
• fixed remaining savestate incompatibilities between PCSX4ALL and P.E.Op.S. GPU plugins
• fixed channel disable preventing irqs in P.E.Op.S. SPU plugin
• fixed some alignment issues
• added handling for branches in delay slots
• fixed some unexpected drops to menu
• fixed lots of recompiler related issues (see GIT)
• added watchdog thread to detect emulator lockups
• minor frontend adjustments

Hi and thanks for the new build. I seem to have missed r4 since I didn't know about this thread. I would like to add my compatibility issues here, just to let you know. They didn't change at all between r3 and r5 (yes, I deleted the corresponding appdata-folder).

1) Diablo: Still starts fine but doesn't accept any Input and plays the Intro instead. No chance of starting a game (intro runs fine and smooth though).
2) Resident Evil DC: Doesn't start at all (black screen)
3) Crypt Killer: same as 2
4) Myst: same as 2, did load up to the main screen in r3 though.
5) Tomb Raider: hangs during initialization of main screen
6) Tomb Raider 2: main screen, screen stay black after starting a game

All the above mentioned games except for Crypt Killer are in bin/cue format. Crypt Killer runs fine in PSX4Pandora.
Also I don't get what you mean by saying "you need to enable bios". Where do you do that? I couldn't find any option allowing me to use a specific bios file.

One last thing: Does altering the region option have any effect on compatibility at all? I noticed that leaving it at ntsc works fine for most us and european games and that setting it to pal doesn't improve compatibility for the above mentioned games.

EDIT: Would you mind adding the possibility to map save/load state to the keypad in a future release? Would be great. And thanks again for this great piece of software so far.

 

[meandu229]

Time for r5 release.
http://notaz.gp2x.de/releases/pcsxr/pcsx_rearmed_r5.pnd

The compatibility should go up a good deal thanks to fixed cdrom code from newer PCSX-Reloaded and my recompiler fixes. All final fantasies work now, for FF8 you need to enable BIOS and play it from the beginning (don't use old savestates with HLE, they are bad). There is now support for more compressed CD image formats, but before reporting bugs try .cue/.bin/.img first.

I'll probably stop working on compatibility now, it's really time to start my optimization efforts. You can keep doing compatibility lists for future reference though.

Here is complete changelog:

• added support for .bz format, also partial support for .znx and eboot.pbp formats
• merged latest cdrom code from PCSX-Reloaded project
• fixed remaining savestate incompatibilities between PCSX4ALL and P.E.Op.S. GPU plugins
• fixed channel disable preventing irqs in P.E.Op.S. SPU plugin
• fixed some alignment issues
• added handling for branches in delay slots
• fixed some unexpected drops to menu
• fixed lots of recompiler related issues (see GIT)
• added watchdog thread to detect emulator lockups
• minor frontend adjustments

Hi and thanks for the new build. I seem to have missed r4 since I didn't know about this thread. I would like to add my compatibility issues here, just to let you know. They didn't change at all between r3 and r5 (yes, I deleted the corresponding appdata-folder).

1) Diablo: Still starts fine but doesn't accept any Input and plays the Intro instead. No chance of starting a game (intro runs fine and smooth though).
2) Resident Evil DC: Doesn't start at all (black screen)
3) Crypt Killer: same as 2
4) Myst: same as 2, did load up to the main screen in r3 though.
5) Tomb Raider: hangs during initialization of main screen
6) Tomb Raider 2: main screen, screen stay black after starting a game

All the above mentioned games except for Crypt Killer are in bin/cue format. Crypt Killer runs fine in PSX4Pandora.
Also I don't get what you mean by saying "you need to enable bios". Where do you do that? I couldn't find any option allowing me to use a specific bios file.

One last thing: Does altering the region option have any effect on compatibility at all? I noticed that leaving it at ntsc works fine for most us and european games and that setting it to pal doesn't improve compatibility for the above mentioned games.

EDIT: Would you mind adding the possibility to map save/load state to the keypad in a future release? Would be great. And thanks again for this great piece of software so far.

The BIOS option was added in r4 (which since you missed you wont have seen before,) Its in the options menu you can then choose the bios which needs to be in the appdata folder

 

[Johannes]

I'll probably stop working on compatibility now, it's really time to start my optimization efforts.

To me this means that for games which aren't working for now or have serious faults, like Music 2000, there is nothing to hope in the future. Right? The speech dialogues in Resident Evil 2 are corrupted with r5. In the case of Music 2000 I only see a chance with another sound plugin. Are they any other ones for using with this emu?

 

[Abaddon666]

The BIOS option was added in r4 (which since you missed you wont have seen before,) Its in the options menu you can then choose the bios which needs to be in the appdata folder

Ok, this solves all of my problems remaining problems so far. Thank you and thanks a lot for the great work notaz.

 

[notaz]

Most of the games i tested so far made no problems. The biggest errors i encountered were with "The X-Files (US)" and "Dead or Alive (US)".
The first game runs, but the visuals are messed up. The picture is split in 18 columns and beginning with the second column they are disordered. The whole picture is a great puzzle . This happens with every gpu plugin.
Dead or Alive runs fine, but crashes at or after the first fight.

Looks like I messed up Dead or Alive, and X-Files got broken by PCSX-Reloaded mdec rewrites - opened an issue in their tracker.

Thanks so much Notaz,

The last donation I made to you got rejected for some reason?

I've responded to it here

EDIT: Just been trying R5 on SOTN Jap patched version and the sound seems way improved. Unfortunately the graphics seem to have suffered a little. Characters are more dark than on R4, walls flickering blue. Tried R4 straight afterwards, doesnt have this problem.

This is the game I play most of, so I notice issues. I will test with more though.

Anything in particular I can test/try out Notaz?

That's weird, I haven't changed the GPU plugins. Tell me how to reproduce it, maybe you can post a savestate or something?

I'll probably stop working on compatibility now, it's really time to start my optimization efforts.

To me this means that for games which aren't working for now or have serious faults, like Music 2000, there is nothing to hope in the future. Right? The speech dialogues in Resident Evil 2 are corrupted with r5. In the case of Music 2000 I only see a chance with another sound plugin. Are they any other ones for using with this emu?

No it just means it might happen later, after performance is improved (or at least was tried to be improved). Some games are painfully slow on this emu, so in general it would benefit more from performance increase than compatibility improvements (which take huge amounts of time to do).

I'll still look at regressions (if something worked in previous versions but not later ones) so keep reporting those, I'll just stop fixing stuff that never worked for the time being.

About plugins, there is only one at the moment but standard psemu plugin interface is supported, so any PC plugin can be recompiled and used.

 

[meandu229]

@Notaz
Is it possible to be able to save SPU plugins on a per game basis,
Its annoying having to change them each time I boot certain games (and if possible save controls per game?),,
Great work though will be trying r5 out after work.