Gpmark


Exophase said:
CKeichel said:
Until today I wait for a proof for that.

And what do you want from me? We've explained that it can't be happening, you've implied several times now that it's just a lie...

CKeichel said:
And I really don't see the need for longer tests, cause the Caanoo results are pretty much the same for everybody and I find it very unlikely, that the results are inaccurate as you suggest. At least, this benchmark is public and it's results are public for everyone. Maybe the benchmarks, you were referring to earlier had some flaws?

I said they can be inaccurate - multiple people getting the same thing does NOT suggest that the time base is correct, just that if it's off it's the off the same for everyone. This can easily happen if bad scale factors are used for clocks.

If the tests Lordus did had some flaws it wouldn't be due to screen rotation because I know his emulators weren't doing it, something we already KNOW happens with SDL on Wiz. But who cares, if you're going to deny Optimus's can have problems then why claim Lordus's could?

And yes, I do think emulators are representative of what people run on the platforms, or at least more representative than these sort of demos.

Note, that I don't say it's a lie to say the Caanoo is 10% slower then the Wiz, but if you come up with this, you should be able to say what you benched, otherwise it is completely pointless, cause nobody can verify or falsify these results.

Now you change the terms again, now you speak of "emulators" in general, before you spoke of fast emulators that don't use SDL. Note that there are emulators on the Caanoo, that use SDL.
I don't say, there can't be problems with Optimus benchmark, all I say is, that it is strange to say the Caanoo is 10% slower then the Wiz, and refuse to say, were these benchmark results come from or what exactly was benched and then, when a public benchmark shows something else, claim, that this benchmark may be faulty.
 
Last edited by a moderator:
Darkknight512 said:
Fda said:
My Wiz results. FW 1.2.6, no overclock (533mhz):

Blitting test: 332.2
Plasma: 242.9
Rotozoomer: 169,6
Rotozoomer near: 233.9
Rotozoomer far: 73.9
Radial blur: 20.7
3d bunny 5.4

no idea why they're so different from Optimus's........maybe the SD class (mine is class 6)?

Mine are +/- 0.1 from you, Wiz at 533 Mhz running 1.2.6. Class 10 SD card.

I also tried these tests at 800 Mhz, about a 50% higher clock. It gives about 23-32% increase in performance. I'm guessing that is because of the lack of cache?

So, SD class has no influence whatsoever (as there is no difference from a minimum write speed of 6mb/s and 10mb/s......and, aside from that, when the program is copied into memory it doesn't matter anymore how much is the write speed of the medium, i think....). It could be the fact that Optimus kinda 'messed' with its own test, using a different sdl library, if i understood well...

Anyway, there's another thing. I remember (from the time i was simply lurking, in here) that different people got different performance peaks with emulators like Mame4All or Pcsx4All, some reaching 830 mhz and others crashing at 790. In my case Mame4All reach 830 and Pcsx4All 850, so it could be something that has to do with internal build, meaning that some people have a wiz able to squeeze a little more perfomance than others...........but, then again, i could be totally wrong and it wouldn't be the first time ^^
 
Last edited by a moderator:
I don't want to stir any more controversy but those are my results by just calculating the effects without rendering anything (commented SDL_flip, I hope it's enough).


Caanoo:

Blitting Test: 1266.4
Plasma: 580.3
Rotozoomer: 251.0
Rotozoomer Near: 572.2
Rotozoomer Far: 68.1
Radial Blur: 24.0
3D Bunny: 5.2


Wiz:

Blitting Test: 1013.5
Plasma: 434.1
Rotozoomer: 223.4
Rotozoomer Near: 400.6
Rotozoomer Far: 73.2
Radial Blur: 18.2
3D Bunny: 4.5

Still, if someone with a Wiz got the same results as Caanoo that might say that my Wiz is underperforming for some reasons. I am also surprised about those Caanoo 1.5.0 results that are similar to 1.0.6. I would like to look at those faulty timer issues mentioned somehow.
 
I also tried these tests at 800 Mhz, about a 50% higher clock. It gives about 23-32% increase in performance. I'm guessing that is because of the lack of cache?

I would say it's memory bottleneck and so it seems that X percentage improvement on Mhz speed doesn't necessarily mean X percentage of improvement in performance. However, there is something I still don't understand about performance (since I am not familiar with hardware architecture in general) and I have raised a question here. I would be very curious to know.
 
Last edited by a moderator:
i knew i wasn't crazy, I did notice a performance hit with 1.5.0

jb0yx said:
is it just me or did the speed of the snes emulator take a huge dive after this update?

might have a corrupt download

I'm going to go back to 1.0.6
 
Last edited by a moderator:
CKeichel said:
Note, that I don't say it's a lie to say the Caanoo is 10% slower then the Wiz, but if you come up with this, you should be able to say what you benched, otherwise it is completely pointless, cause nobody can verify or falsify these results.

Now you change the terms again, now you speak of "emulators" in general, before you spoke of fast emulators that don't use SDL. Note that there are emulators on the Caanoo, that use SDL.

You're nitpicking.

Where performance matters those SDL based emulators are going to be replaced by emulators not using it. Until then they're simply at a disadvantage compared to Wiz where the commonly used emulators that have any kind of performance dependency aren't using SDL already. So yeah maybe SNES on Caanoo is using SDL right now, but it isn't on Wiz, and eventually it's PocketSNES that should be on Caanoo. Just for example.

Where performance isn't limited in either case the whole question becomes moot.

CKeichel said:
I don't say, there can't be problems with Optimus benchmark, all I say is, that it is strange to say the Caanoo is 10% slower then the Wiz, and refuse to say, were these benchmark results come from or what exactly was benched and then, when a public benchmark shows something else, claim, that this benchmark may be faulty.

First of all, he did say a benchmark where he's observed 10%, it was in Picodrive - you're more than welcome to test it yourself and come up with your own findings to verify or deny this.

I'm not saying what other emulator he ran because he doesn't want it to be public. You may think it's "strange" but it's not that hard to imagine some reasons why someone isn't at liberty to talk about something like this. Think about it a little harder.

What you're leaving out of your explanations is that I have a validation behind everything I say. Caanoo is often slower, up to 10% so, than Wiz because CAS latency. Empirical support that this would happen - on Wiz, tweaking parameters that are also proportional to memory usage but less frequently can effect by at least 4-5%. Your rebuttal - changing CL on some PC caused a much smaller result. I gave several reasons why this PC would be less sensitive to CL but you just ignored it.

So now we see a test where Caanoo is faster. We already know it's not the same software running (we were comparing hardware, you know), we know the SDL is different and furthermore we know of variables that can effect this a lot.. so there are viable explanations for why this could be happening that have nothing to do with the software. What explanation is there that Caanoo hardware can be faster than Wiz hardware? In the scientific method experimental results are theoretically supreme, but an experiment can always be flawed and contained uncontrolled variables or incorrect methodology. Good science should always question experimentation with theory as much as it questions theory with experimentation.

You're saying now that it's not about doubting my comments, but you've posted saying this benchmark proves that my comment that Caanoo can be 10% slower is false. Well sorry but it doesn't prove that. Yes, it shows that there are software situations where this can happen, that are very sensitive to firmware version, but so what? That's always the case. I don't know why you're fighting me so hard on this if you say you're not against my comment. Lordus has already confirmed himself as my source and confirmed my comment so that really removes me from the equation.. if you have a problem with his comment either you think he's lying or you think he's incompetent. Otherwise what's your problem exactly?

Anyway, Optimus already said he'd do one that doesn't blit the screen and therefore removes SDL screen update performance out of the equation. Maybe we can get him to use the framebuffer directly, like everyone else who cares about performance should be doing.. he can use libcastor or something. If he wants direct 320x240 it'll mean diagonal tearing but I think for the sake of this benchmark that should be the preferred route.

EDIT: Okay, I see Optimus posted new results.

Honestly, I think the best thing to do is to do now is to a) make sure the time base is correct, and probably use hardware timers anyway B) check it out on other people's rigs, and make sure all the pertinent registers are set the same (clocks, RAM timings) as much as the hardware allows to confirm the same thing is tested c) release the source code so it can be thoroughly analyzed to see exactly where the cycles are spent...

Here's another question for you Optimus:

What kind of RAM are you accessing, is it purely stuff allocated with malloc or are you mmapping stuff that's in the "high" memory section? Or, despite not calling SDL_Flip, are you writing to an SDL_HWSURFACE? If you're writing to RAM allocated by SDL it's possible it's in the high region, and it's possible that one version has write buffering enabled and another doesn't. This makes a big difference in performance (which is why we had mmuhack/wARM stuff on GP2X and Wiz)
 
Last edited by a moderator:
I am not fighting for anything Exophase, but I still think, if you say, the Caanoo is slower then the Wiz, because of the Cas Latency and you even give a number (10%), you are saying something, which you can't proof and I am still saying, that it seems, at least at the moment, that you are wrong. Optimus new posted results seem to support that the Caanoo isn't slower then the Wiz, but in most cases (a little bit) faster.
And about the non-public benchmarks, it gets weirder and weirder, Lordus isn't here to say, why his benchmarks of emulators aren't public, he may have his reasons, honestly I really don't care, non-public benchmark results are nothing, that is of any interest to me.
To generally say Picodrive is 10% slower on the Caanoo doesn't help either. Which games he tried, how long did the benchmark run, why he didn't measure a performance hit, when updating from Firmware 1.0.1 to 1.0.6 that had a higher system speed and in general increased performance, etc.? Lots of questions, that can't be answered, if the results aren't public.
I think it doesn't help, if you make mysterious remarks about that, without going into detail.
 
I said it can be 10% slower, I think you have some problem understanding what "can be" means. Maybe these non-public numbers mean nothing to you personally, but maybe other people have less problem believing this. As far as I'm concerned, with this benchmark's source code not being available it has just as many questions, but you trust it completely.

We're going in circles. I've said everything pertinent to your most recent post. I'll ask Lordus to give a thorough explanation of how he tested Picodrive, but with as invested as you are in this I'd expect you'd have formed your own test.
 
What kind of RAM are you accessing, is it purely stuff allocated with malloc or are you mmapping stuff that's in the "high" memory section? Or, despite not calling SDL_Flip, are you writing to an SDL_HWSURFACE? If you're writing to RAM allocated by SDL it's possible it's in the high region, and it's possible that one version has write buffering enabled and another doesn't. This makes a big difference in performance (which is why we had mmuhack/wARM stuff on GP2X and Wiz)

I was curious about this. I still write to the pixel buffer provided by SDL. Maybe I should omit that and try running the tests again. My other arrays are allocated in normal map I think (all are static arrays).
To have a look at writing natively to the videoram instead of using SDL it will take some time but it will be interesting to me. However SDL seems quite fast from GP2X and on, I wouldn't think an emulator would loose significant performance by using SDL.

I will release the source code in the next version probably (sorry, have to clean up mess and unify the different handhelds code in one project).
 
Something strange I just noticed while trying to figure out why some people get better score on Wiz.

Run the benchmark and immediately after that slide the switch to the input lock position.
Everything performs better!

My Wiz 1.2.6 results with this trick are finally similar to Caanoo:

Blitting test: 329.3
Plasma: 240.9
Rotozoomer: 168.2
Rotozoomer near: 232.1
Rotozoomer far: 73.2
Radial blur: 20.7
3d bunny 5.4

Now, isn't that crazy? If you also try sliding the switch in and out of lock you see the speed difference on realtime.
Of course it's silly, because you can't play any emulators without input. But why the hell does this happen? (well, does it happen in your Wiz too?)
It doesn't only happen in my benchmark but also other software that display some framerate, e.g. batterytest.

I think I am getting crazy today :p
 
That's some messed up shit Optimus. I'm at a big loss for this one. Maybe the switch position is somehow flooding the kernel with IRQs?

Can you cat /proc/interrupts and see if those numbers are changing?
 
Optimus said:
Something strange I just noticed while trying to figure out why some people get better score on Wiz.

Run the benchmark and immediately after that slide the switch to the input lock position.
Everything performs better!

My Wiz 1.2.6 results with this trick are finally similar to Caanoo:

Blitting test: 329.3
Plasma: 240.9
Rotozoomer: 168.2
Rotozoomer near: 232.1
Rotozoomer far: 73.2
Radial blur: 20.7
3d bunny 5.4

Now, isn't that crazy? If you also try sliding the switch in and out of lock you see the speed difference on realtime.
Of course it's silly, because you can't play any emulators without input. But why the hell does this happen? (well, does it happen in your Wiz too?)
It doesn't only happen in my benchmark but also other software that display some framerate, e.g. batterytest.

I think I am getting crazy today :p

Mmmmmh, nope. Even doing this trick, mine are exactly the same....
 
Last edited by a moderator:
Nope, no change with the slider on or off of lock. I'm beggining to think you have a funky Wiz Optimus, maybe do a reflash?
 
Ok, it took me a while to get used typing in termula2X and then find out how to set a delay before the command so that I can lock in the input before it executes but I found it. I hope I did it right (sorry, still not familiar with unix/linux environments :p ). Those are the results, I don't know what to make of them.


Normal:

4: 63095 pollux-level POLLUX Timer Tick
14: 64036 pollux-level pollux_sdi0
32: 0 pollux pollux-i2c
44: 562 pollux-uart0 pollux-uart
45: 0 pollux-uart0 pollux-uart
97: 36198 pollux-gpio GPIO
99: 1 pollux-gpio pollux_sdi0
143: 5 pollux-gpio GPIO
159: 1 pollux-gpio GPIO
Err: 0


With locked input:

4: 73886 pollux-level POLLUX Timer Tick
14: 64036 pollux-level pollux_sdi0
32: 0 pollux pollux-i2c
44: 562 pollux-uart0 pollux-uart
45: 0 pollux-uart0 pollux-uart
97: 41773 pollux-gpio GPIO
99: 1 pollux-gpio pollux_sdi0
143: 5 pollux-gpio GPIO
159: 1 pollux-gpio GPIO
Err: 0

Also notice that, I forgot to mention, my Wiz has broken pixels and sometimes something is wrong with the input. It's preety much going to die soon. Though I am not sure if that's the reason. I would expect Fda or Darkknight wouldn't get better result with that lock since they already got the optimum results, but I am curious for those who had initially the same slower results as me in Wiz.
 
Hmmm, i don't think this test is going to be of any use.......values 4, 14, 44 and 97 litterally change every time cat /proc/interrupts is executed, so they probably act like a clock/timer of some sort......with the only difference that my 97 ones go around 60-70, not 41000 like yours. But, then again, must be something that have to do with how much time/how many programs have been executed since then.........

99, 143 and 159 are fixed, and they gave my the values of 1,1, and 0. Maybe after running some program they'll change, who knows? I'll try later ^^

Even doing the lock-out trick (i used "sleep 5;cat /proc/interrupts"), the results won't change anyway.

By the way, my Wiz have 1 or 2 dead pixels. Even a small hole in the touchscreen layer. And let's not talk about the fact i had to change rubber pads and volume buttons (worn out and broken, respectively). But, no, it doesn't look like it's going to die anytime soon.........i hope.......... :)
 
Just running it after each doesn't say a lot, I'd say you need to run it immediately before and immediately after each test. Nonetheless, based on what Fda said it does at first glance appear that pollux-gpio IRQ is getting hammered for you and not for him. And that is of course the GPIO I would expect input to be generating if input lock is off. You might have a bad connection in controls somewhere that's triggering input constantly. I don't know of what else internally would be connected as GPIO, maybe someone has some ideas.
 
jb0yx said:
i knew i wasn't crazy, I did notice a performance hit with 1.5.0

jb0yx said:
is it just me or did the speed of the snes emulator take a huge dive after this update?

might have a corrupt download

I'm going to go back to 1.0.6
I'm back at 1.0.6, noticeable (however slightly) improvement in performance on emulators
 
Last edited by a moderator:
Hi,

Here are the GPMark results on my WIZ (FW 1.1.0 @ 533 Mhz) without any tweaking.

Blitting test: 340.4
Plasma: 250.0
Rotozoomer: 173.6
Rotozoomer near: 240.6
Rotozoomer far: 74.6
Radial blur: 21.5
3D bunny: 5.5

@Optimus: tried with lock-slider on, but it didn't change anything.

Apart from that: really nice application for a first version! Can't wait for the next release ;)
 
Yep, I wouldn't expect someone who made the fast results to get any more faster with this trick.
I would be more curious about the people with results similar to mine. It could be either that only my Wiz is in bad state or this happens in some of the Wiz manufactured.

p.s. My Wiz is in horrible state. I think it happened when I left it in my bag that traveled with the airport. Then dead pixels or over the place and sometimes I have to reset because the controls are working alone (like someone is pressing everything together). But this happens only few times. The test always makes something wrong. Then, the menu moves so slow with the controls but I remember this was always the case with Wiz.
 
Hello

Here the results for my caanoo FW 1.0.7.

It was connected to USB for charge (it makes differences?)


Blitting test: 297.6
Plasma: 216.6
Rotozoomer: 145.8
Rotozoomer near: 215.9
Rotozoomer far: 56.6
Radial blur: 19.5
3D bunny: 4.7

Someone knows what to do for improve so poor perfomances ?

:(

Thank you

Xabarass
 
Back
Top