Wiz Performance Not As Expected?


Exophase said:
Maybe I should see what the costs are when you cross memory rows. On GP2X the initial configuration was very poor, hence why the improved RAM timings helped things. Normally you will be bottlenecked by the column addressing and burst read/write, not the row changing time, so it would have to be pretty significant to affect things this much.
I've just ran some tests on this, and it looks like this might be indeed one of major bottlenecks. When I write words and post-increment address by 1kB, my test runs more than 8 (!) times slower than with post-increment of 32 bytes or less. It seems write buffer won't even work then, there is no difference if it's enabled or not. I thought this could have something to do with TLB cache misses so I set up single 1MB section in MMU, ran the test there and got mostly the same results. I can stuff some nops between those accesses without affecting measurements, so it must be memory latency issue.

Exophase said:
One nice thing about later ARM CPUs is that they have performance monitoring counters that you can use to help better profile where cycles are going. I don't know, maybe this one has them and I just missed it, but I don't think so.
Nah, there is nothing like that in documentation.

EDIT: Well, after increasing buffer size I can see TLB misses hit pretty hard, as expected. Although hitting different page on every access is extremely unlikely, that scenario slows things down at least 5 times compared to exact same code with no TLB misses (which can be achieved thanks to 1MB section setting in MMU).
 
Last edited by a moderator:
notaz said:
I've just ran some tests on this, and it looks like this might be indeed one of major bottlenecks. When I write words and post-increment address by 1kB, my test runs more than 8 (!) times slower than with post-increment of 32 bytes or less. It seems write buffer won't even work then, there is no difference if it's enabled or not. I thought this could have something to do with TLB cache misses so I set up single 1MB section in MMU, ran the test there and got mostly the same results. I can stuff some nops between those accesses without affecting measurements, so it must be memory latency issue.

It's not the MMU/TLB misses, it's the DDR interface. 1KB is a typical size of DDR rows. If you keep accessing within the same rows then you just need to supply a column address, and this can overlap with read/write cycles (pipelined) allowing you to sustain full bandwidth. If you need to move to a different row then you need to do row address strobing AND row precharge and possibly another step that I'm forgetting.

My guess is that Wiz is performing exactly one full precharge/row/column cycle on every non-cached memory load, cacheline load, and cacheline evict, with only the write buffer being able to detect same row access between consecutive accesses. You probably also get a ton more efficiency using ldm on non-cached regions than several ldrs, because of this. I think the GP2X is probably the same way, but the relative damage is much worse here because the memory clock to core clock ratio is 1:4 instead of 1:2, and the DDR aspect of the RAM only applies to data transfers, not all of the clocks spent on address/precharge.

Like the GP2X it seems like the values selected for the timings of these operations are way too high - you should look into lowering them and trying the test again, if you have the databook. You can play around with these registers:

Code:
typedef enum
 {
   MEM_CFG         = 0x14800,
   MEM_TIME0       = 0x14802,
   MEM_TIME1       = 0x14804,
   MEM_REFRESH     = 0x14808,
   MEM_CONTROL     = 0x1480A
 } wiz_memory_regs_enum;

My tests were much more sequential than 1KB so I didn't see much difference, didn't bother to do wider ones (should have).

notaz said:
Nah, there is nothing like that in documentation.

Yeah, I didn't expect it.

notaz said:
EDIT: Well, after increasing buffer size I can see TLB misses hit pretty hard, as expected. Although hitting different page on every access is extremely unlikely, that scenario slows things down at least 5 times compared to exact same code with no TLB misses (which can be achieved thanks to 1MB section setting in MMU).

It's not just hitting a different page on every access, it'd be causing a TLB miss on every access. The TLB should be pretty effective almost all the time for most programs, MUCH more effective than icache/dcache, and a TLB miss isn't that expensive since it's hardware TLB. Modifying Linux to use 1MB page regions is probably out, so you'd have to use the upper memory/mmuhack to get this. Maybe someone can try putting their huge arrays here but I doubt it'll make a big difference.
 
Last edited by a moderator:
Quick question here to save on new thread clutter:
I see the Wiz has a few optimization problems. I kind of expected this would happen with lackluster performance right at launch. While I don't need a pocket media player by tomorrow, I'm still curious as to how long this will take to sort out before the Wiz is as mature as it's predecessors or at least mature enough to consider over the competition when I could get something that does all I currently want the Wiz to do for much cheaper already. So my question is:
How long did it take before the worst of the kinks were ironed out of the GP2X? It's been about 2 months (6-ish weeks I heard?) nearly with the Wiz out, how long before the GP2X was promising to non-company fanboys?

Something about memory bottlenecks? Could this be a fatal design flaw, or when developers finally start using the Wiz's dedicated graphics chip will that be negated?
 
El Idiot said:
Quick question here to save on new thread clutter:
I see the Wiz has a few optimization problems. I kind of expected this would happen with lackluster performance right at launch. While I don't need a pocket media player by tomorrow, I'm still curious as to how long this will take to sort out before the Wiz is as mature as it's predecessors or at least mature enough to consider over the competition when I could get something that does all I currently want the Wiz to do for much cheaper already. So my question is:
How long did it take before the worst of the kinks were ironed out of the GP2X? It's been about 2 months (6-ish weeks I heard?) nearly with the Wiz out, how long before the GP2X was promising to non-company fanboys?

Something about memory bottlenecks? Could this be a fatal design flaw, or when developers finally start using the Wiz's dedicated graphics chip will that be negated?


What you're looking at is the problem being solved as well as it can by the hard core community, who are doing a damn fine job of narrowing down the issues to the real wire. I would imagine the results of this investigation is going to be useable in the next 2 or 3 firmware releases ..
 
Last edited by a moderator:
Exophase said:
Like the GP2X it seems like the values selected for the timings of these operations are way too high - you should look into lowering them and trying the test again, if you have the databook.
Some results here:
http://www.gp32x.de/board/index.php?/topic/48575-wiz-ram-timings-test/

My 'pathological' write test which most likely hits different DRAM row on every access is now 2 times faster (still 4 times slower then sequential access), but non-sequential reads/linefills are not affected much by this, maybe 10%. Sequential reads/writes are mostly unaffected.
 
Last edited by a moderator:
Today I've noticed one thing: ARM926EJ has a 4-way set associative cache, while ARM920T has 64-way! This means Wiz has completely different cache architecture which might be much worse for some programs and better for others. This might explain why some things are slow and others (like Quake2) are fast.
 
notaz said:
Today I've noticed one thing: ARM926EJ has a 4-way set associative cache, while ARM920T has 64-way! This means Wiz has completely different cache architecture which might be much worse for some programs and better for others. This might explain why some things are slow and others (like Quake2) are fast.
That's nuts! You think that's something they would've looked into. Then again probably not. :blink:
 
Last edited by a moderator:
notaz said:
Today I've noticed one thing: ARM926EJ has a 4-way set associative cache, while ARM920T has 64-way! This means Wiz has completely different cache architecture which might be much worse for some programs and better for others. This might explain why some things are slow and others (like Quake2) are fast.

I doubt it really makes a big difference. Most CPU L1 cache is only 4-way. 64-way is quite excessive for only 16KB of cache, especially for icache.
 
Last edited by a moderator:
But it does make a difference. Now that I have single PicoDrive binary running on both F100 and Wiz, I can see that Wiz is very sensitive to code placement, i.e. if I shuffle object file order I can see up to 30% performance difference for some Sega CD games, while on F100 there is none. Granted the code path is not exactly the same, but only a few functions really differ.
 
notaz said:
But it does make a difference. Now that I have single PicoDrive binary running on both F100 and Wiz, I can see that Wiz is very sensitive to code placement, i.e. if I shuffle object file order I can see up to 30% performance difference for some Sega CD games, while on F100 there is none. Granted the code path is not exactly the same, but only a few functions really differ.

AFAIK Franxis reported similar things about MAME4all.
 
Last edited by a moderator:
quartercast said:
sbock said:
AFAIK Franxis reported similar things about MAME4all.

Sorry, what was your name again?

What do you mean? Sorry, I dont understand your question.
 
Last edited by a moderator:
notaz said:
But it does make a difference. Now that I have single PicoDrive binary running on both F100 and Wiz, I can see that Wiz is very sensitive to code placement, i.e. if I shuffle object file order I can see up to 30% performance difference for some Sega CD games, while on F100 there is none. Granted the code path is not exactly the same, but only a few functions really differ.

Wow. I wish we had cache performance profiling functions! I ran into a situation that felt similar to this on GP2X but it makes no sense given all the associativity we had there.

This is going to be extremely hard to optimize for. Especially for something like gpSP that relies on executing generated code, although that actually might be in its favor since it kind of randomizes the way distribution.
 
Last edited by a moderator:
sbock said:
quartercast said:
sbock said:
AFAIK Franxis reported similar things about MAME4all.

Sorry, what was your name again?

What do you mean? Sorry, I dont understand your question.

He was, rather childishly, telling you that you're not a notable dev and therefore should shut up. Put him on ignore and carry on as you were, is my advice.

D.
 
Last edited by a moderator:
Uhhh.. no. I was poking fun at the fact he likes to end almost every post with "Regards, Stephan", and that post he hadn't. Thanks for making slanderous comments about me, I don't give a rats if you're a dev or not (I know you are, Dunny).
 
quartercast said:
Uhhh.. no. I was poking fun at the fact he likes to end almost every post with "Regards, Stephan", and that post he hadn't. Thanks for making slanderous comments about me, I don't give a rats if you're a dev or not (I know you are, Dunny).

Then please feel free to disregard my comments - you did come across as quite confrontational though. Most times, when one person comes into a conversation and someone asks "and who are you then?" they usually mean that they don't have any place (or enough knowledge) to contribute. And I wasn't the only one who came to the same conclusion...

However. If that was not your intention, then I'm also apologising for overstepping the mark.

D.




PS - yes, I am quite arrogant.
 
Last edited by a moderator:
Apology accepted. You must be one of those special humble-arrogant types.
 
Back
Top