How Does Pandora Emulate So Well?

[(naw)mcx]

So even when we get the Dreamcast emulator at least full speed some games just won't work? Why?

Because you touch your self at night.
edit: i thought you were being sarcastic, this response looks out of place now

 

[WizardStan]

So even when we get the Dreamcast emulator at least full speed some games just won't work? Why?

Because WinCE is licensed and the devs working on the emulator didn't want to touch that can of worms with a 10 foot pole.

 

[1magus]

So even when we get the Dreamcast emulator at least full speed some games just won't work? Why?

Because WinCE is licensed and the devs working on the emulator didn't want to touch that can of worms with a 10 foot pole.

Is this why people who write emulators never attempt to reverse engineer older versions of Windows to get to emulate?
Does anyone think that someone will eventually just write a WinCE version as an Anonymous poster?

 

[Exophase]

Because WinCE is licensed and the devs working on the emulator didn't want to touch that can of worms with a 10 foot pole.

That has nothing to do with it. WinCE is on the game discs, so as far as the emulator is concerned it's just another stream of instructions. The reason WinCE games won't work is because they use the SH4's MMU, and emulating the MMU requires a pretty different approach to memory emulation that can have a steep performance penalty, especially given some of the tricks nullDC employs. Other games that don't use WinCE but use the MMU also won't work.

From a performance point of view it makes sense on embedded platforms to not use the MMU in production code unless you have a particularly compelling reason to (like virtual memory), because using it incurs TLB misses adding additional overhead. So most Dreamcast games, like most N64 games, don't use it. But if you're running in a protected operating system like WinCE you have no choice.

 

[WizardStan]

That has nothing to do with it.

Yeah, I'd written up a long thing about technical reasons as well, but decided that *I* didn't want to touch that can of worms with a ten foot pole and erased it, sticking with the straightforward and simple answer.
Sorry, I should have known better

 

[1magus]

Well does Sonic Adventure use the MMU? How do we even know what games do and do not use it?

 

[Exophase]

Yeah, I'd written up a long thing about technical reasons as well, but decided that *I* didn't want to touch that can of worms with a ten foot pole and erased it, sticking with the straightforward and simple answer.
Sorry, I should have known better

I'm sure there are better ways to give short and simple answers than just making one up >_> I mean no offense or nothin'

Well does Sonic Adventure use the MMU? How do we even know what games do and do not use it?

I guess wait and see? No point getting all worked up about what they have no intention of emulating long before there's an emulator out.

Sonic Adventure doesn't use the MMU.

Here's a list of WinCE games:

http://www.dcemu.co.uk/vbulletin/showthread.php?t=43804

(first hit on google for "winCE dreamcast games", next time you try)

 

[1magus]

No I found that list but I don't see how you can even know how a game uses MMU inside the Dreamcast when all those games are closed source?

 

[WizardStan]

I'm sure there are better ways to give short and simple answers than just making one up >_> I mean no offense or nothin'

I didn't think I was making it up. I thought it was a little from column A, a lot from column B, C, D, E, and F. I know a little more about what would go into column A, so that's what I wrote. Turns out there's no A involved at all. I deserve all the offense I get.

 

[jonlad1]

It's Tekken the piss.

I think I pee'd a little after reading that.

 

[ari64]

That has nothing to do with it. WinCE is on the game discs, so as far as the emulator is concerned it's just another stream of instructions. The reason WinCE games won't work is because they use the SH4's MMU, and emulating the MMU requires a pretty different approach to memory emulation that can have a steep performance penalty, especially given some of the tricks nullDC employs. Other games that don't use WinCE but use the MMU also won't work.

It's not just emulating the MMU, but also accurately emulating page faults. Doing so requires keeping track of what's in the register cache.

What tricks does nullDC employ which would make this difficult?

No I found that list but I don't see how you can even know how a game uses MMU inside the Dreamcast when all those games are closed source?

When the game is run it will execute an instruction which enables the MMU. The emulator either handles this properly, or fails. Then you know.

BTW I made the N64 emulator log when games do this, so you can tell which ones do it by looking in /tmp/pndrunmupen64plus...

 

[Exophase]

It's not just emulating the MMU, but also accurately emulating page faults. Doing so requires keeping track of what's in the register cache.

Given that SH4 only has 16 general purpose registers there is good reason to not opt for dynamic register allocation at all. I'm probably sounding like a broken record though, I know this exact thing has come up multiple times before.

What tricks does nullDC employ which would make this difficult?

Because SH only uses 16-bit instructions its immediate space is very limited, and it's a common pattern for larger immediates to be loaded PC-relative, in particular for branches. Since indirect branches are normally much slower than direct branches nullDC assumes that PC-relative data is constant, and only revises this by detecting modifications via write protection. When the address space can be changed underneath you tracking this becomes more problematic.

 

[ari64]

Given that SH4 only has 16 general purpose registers there is good reason to not opt for dynamic register allocation at all. I'm probably sounding like a broken record though, I know this exact thing has come up multiple times before.

Okay, so doing MMU emulation needs another register, and makes this register allocation scheme a little more complicated.

What tricks does nullDC employ which would make this difficult?

Because SH only uses 16-bit instructions its immediate space is very limited, and it's a common pattern for larger immediates to be loaded PC-relative, in particular for branches. Since indirect branches are normally much slower than direct branches nullDC assumes that PC-relative data is constant, and only revises this by detecting modifications via write protection. When the address space can be changed underneath you tracking this becomes more problematic.

This problem is not unique to SH4, nor to PC-relative loads. Basically I have the recompiler stop at the page boundary in this situation.

 

[Exophase]

Okay, so doing MMU emulation needs another register, and makes this register allocation scheme a little more complicated.

And more importantly, potentially makes loads and stores (those operations that tend to take up 30+% of the instruction stream) more expensive, depending on what you have to do in the first place.

This problem is not unique to SH4, nor to PC-relative loads. Basically I have the recompiler stop at the page boundary in this situation.

I think you must be misunderstanding what I'm describing. I'm talking about treating PC-relative memory loads as if it's accessing a pool of constants that is valid at compile time. The loads are then turned into constant propagation assignments, and are folded in wherever they're later accessed, including changing indirect branches to direct ones. The host's memory protection is then used to detect if that region is ever modified, so the recompiled code can be invalidated - usually PC-relative regions that are then referenced this way remain constant. If you change address space you have to change what you're protecting and what you're tracking, which can be a pretty complex arrangement.

Honestly I don't even know what situation it is you're referring to. Blocks crossing pages in MMU translated space?

 

[ari64]

Okay, so doing MMU emulation needs another register, and makes this register allocation scheme a little more complicated.

And more importantly, potentially makes loads and stores (those operations that tend to take up 30+% of the instruction stream) more expensive, depending on what you have to do in the first place.

Significantly different from this? http://pandorawiki.org/Mupen64plus_dynamic_recompiler#Translation_lookaside_buffer_emulation

The overhead from this is only around 5-10%. Usually the page table entries are in L1 cache. The performance problem that you really have to worry about is swapping executable code pages causing the dynarec cache to be flushed.

Honestly I don't even know what situation it is you're referring to. Blocks crossing pages in MMU translated space?

Yes. So if a branch crosses a page boundary then it has to go through indirection or the dynamic linker, and if a load crosses a page boundary then it can't be treated as a constant.

It's either that, or come up with some way of ensuring that none of the page mappings have changed.

 

[Exophase]

Significantly different from this? http://pandorawiki.o...uffer_emulation

The overhead from this is only around 5-10%. Usually the page table entries are in L1 cache. The performance problem that you really have to worry about is swapping executable code pages causing the dynarec cache to be flushed.

The overhead in your emulator is that (I don't know if that's on the recompiler alone or the whole emulation), not necessarily what it'll be in another emulator under pretty different conditions.

Yes. So if a branch crosses a page boundary then it has to go through indirection or the dynamic linker, and if a load crosses a page boundary then it can't be treated as a constant.

It's either that, or come up with some way of ensuring that none of the page mappings have changed.

And this problem has nothing to do with the one I was talking about, I hope I don't have to convince you further of that..

 

[Pleng]

OK Steady on there guys!

I'm trying to follow the basic flow of the conversation. My conclusion is that one of you (not sure which one is debating on which side at this point!) believes that adding MMU emulation to the Dreamcast may theoretically be possible without adding too much additional overhead. Though clearly nobody is working on that at this point.

Am I correct? Is Sega Rally a theoretical possibility on the Pandora?!!

 

[Exophase]

Pleng, the point is more that drk||Raziel said he isn't going to do MMU emulation, not that it's "not possible." We could talk about overhead all day but it's pretty silly when we don't even know how cripplingly slow DC emulation might be without doing MMU emulation in the first place. It's amazing that you guys get so worked up over this stuff long before you have any hope of running even a single game, as if that's all a foregone conclusion.

It'd be a big design change, and contrary to what Ari64 might be implying it'd be a bigger design change than the one needed for his dynarec... maybe Ari himself will be willing to add support one day, but I don't think he's very optimistic about DC emulation in the first place.

 

[ari64]

The overhead in your emulator is that (I don't know if that's on the recompiler alone or the whole emulation), not necessarily what it'll be in another emulator under pretty different conditions.

Recompiler alone. In what way are the conditions different? The only major difference that I know of is the page size (1K vs 4K) which does make SH4 emulation a bit inefficient.

Yes. So if a branch crosses a page boundary then it has to go through indirection or the dynamic linker, and if a load crosses a page boundary then it can't be treated as a constant.

It's either that, or come up with some way of ensuring that none of the page mappings have changed.

And this problem has nothing to do with the one I was talking about, I hope I don't have to convince you further of that..

The problem that I'm thinking of is an instruction like the following:

7FFF4: MOV.L @(64,PC),r1

Suppose the page beginning at 80000 was swapped out and marked invalid. If the address was in the same page, then you could convert it to an immediate load, but in this case it will cause a page fault.

 

[Pleng]

Pleng, the point is more that drk||Raziel said he isn't going to do MMU emulation, not that it's "not possible."

Yes I'm aware of that.

It's amazing that you guys get so worked up over this stuff long before you have any hope of running even a single game, as if that's all a foregone conclusion.

To be fair I did fill my posts full of 'theoretically's and and 'possibly's. I'm just trying to decipher what the hell you two are going on about! Anyway, there's nothing wrong with dreaming. 2 years ago I dreamed of actually *having* a Pandora, and look: now, against all odds, I have one