[announce] c64_tools (DSP loader and IPC)


However, "copy paged to paged" performs at ~990 MBytes/sec so something "fishy" is going on there -- the shared memory (contiguous mem) reads should not be 3 times slower than this.
I have a theory about this: pandora's RAM module actually has 2 256M RAM chips on 2 different chip selects. Paged memory most likely ends up on first chip (PA < 0x90000000), and if you look at dmesg, CMA tends to reserve space at the end of physical address space, so ends op on the second chip. For whatever reason second may be slower than the first.
when I first encountered this issue some weeks ago, I had a similar idea. That's why I ran the benchmarks with statically reserved memory (@96M offset) back then, and the results were the same -- still 3 times slower. We can probably rule this out as the root cause for this issue.
 
And here's the PND with the examples/demos: http://tkscript.de/c64_tools/c64_tools_tests.pnd

It works fine in XFCE4 and minimenu.

The applications should show up under "Other->DSP (c64_tools)" but the subcategory element does not seem to work as described in "PXML Fileformat.pdf" (all applications are placed directly in "Other" instead)

Pick "Open c64_tools testcases shell", then issue "./c64_tc" (or run any other test/example).

There are additional menu entries for the dsprite and fractal demos.

I tried uploading it to repo.openpandora.org but that did not work because of some obscure error message:


WARNING: there seems to be an error with the data from your <package> block.
Because your pnd contains more than one <application> block
We need you to fix your PXML with a <package> element suitable for your package
and then re-upload.
Thanks!

MISSING:
xml elements: descriptions | - package block

No idea what that is supposed to mean, the <package> element has a <description>, so does each <application>.

..and please, no comment about the logo -- the repo demanded one so I was forced to unleash my mad pixel skillz :p

EDIT: here's the PXML.xml . Does anyone have an idea why the repo won't accept it ? http://tkscript.de/c64_tools/tmp/PXML.xml
 
Last edited by a moderator:
I've played with memory tests a bit more, and I think it's just some kind of luck that your paged memcpy test almost hits 1GB/s. It's quite easy to break it, for example disabling USE_S2 already puts it ~500MB/s or below, changing mallocs with memalign(64, ..) also has similar effects. Another test here reports only ~350MB/s, which is consistent to what you get on shared memory.

The paged test probably gets the pages scattered around in physical memory in some more lucky order so that it needs less SDRAM column/row reprogramming than when they are placed in successive order in shared memory.
 
Last edited by a moderator:
This link doesn't look like pxml for me ;)


If the apps don't show up in the right categories, you most likely made a typo or something. Or you choose a non standart categoryname.


Edit:


I get this:


c64_tools (DSP) testcases + examples c64_tools (DSP) testcases + examples c64_tools (DSP) testcases + examples Open c64_tools testcases shell c64_tc shell c64_fractal_dspgpp Julia attractor fractal (DSP+GPP) c64_fractal_dsponly Julia attractor fractal (DSP only) c64_fractal_gpponly Julia attractor fractal (GPP only / FixedPoint) c64_dsprite (256 32x32 ARGB32 sprites, premul. srcover blending) Sprite test c64_dsprite (1500 32x32 ARGB32 sprites, alphatesting) Sprite test c64_dsprite (4000 8x8 ARGB32 sprites, alphatesting) Sprite test c64_dsprite [CC] (2000 8x8 ARGB32 sprites, alphatesting) Sprite test


At this link: http://tkscript.de/c64_tools/tmp/PXML.xml
 
Last edited by a moderator:
the osversion tags looks suspicious. I was expecting something like 1.55, not 3.2.52.919?
I do not know, it is not specified clearly. When I started looked at PND packaging this morning I found an example PXML.xml that used the kernel version. That made sense to me so I did the same.

I've played with memory tests a bit more, and I think it's just some kind of luck that your paged memcpy test almost hits 1GB/s. It's quite easy to break it, for example disabling USE_S2 already puts it ~500MB/s or below, changing mallocs with memalign(64, ..) also has similar effects. Another test here reports only ~350MB/s, which is consistent to what you get on shared memory.


The paged test probably gets the pages scattered around in physical memory in some more lucky order so that it needs less SDRAM column/row reprogramming than when they are placed in successive order in shared memory.
I have a log of the paged->paged memcpy and memset benchmark that shows 1.4 GByte/sec. That was regardless of whether USE_S2 was defined or not. Running the memperf benchmark (the one I sent you via PM some time ago) directly after that showed just ~366 MBytes/sec, though.


Now I constantly see ~1.0 GBytes/sec with e.g. c64_tc 12.


It is really interesting (and utterly confusing!) why the memory performance changes by +-1.1 GBytes/sec. That's quite a lot!

It would probably be worth the trouble to investigate why that is and how the 1.4 GBytes/sec can be achieved consistently! (bold text since this is really quite a remarkable difference! the blocksize was too large to fit into the L2 cache)

By the way: The DSP eDMA3 peaks at ~614.4 MBytes/sec (memcpy), although that was measured with lots of 32x32 copies. IIRC, with 800x480 blocks the performance was ~1GB/sec.

The alphatested (i.e. with dst+src reads) ARGB32 sprites in the current release perform at 32*32*1500*76 => 116 mpixel/sec (466 MBytes/sec). That's not too bad but it should be ~153 mpixel/sec once the process loop is properly optimized.

This link doesn't look like pxml for me ;)
If the apps don't show up in the right categories, you most likely made a typo or something. Or you choose a non standart categoryname.
Hmm the link you posted is the same one I posted before and it works fine here (!?).

In the meantime I have finally managed to persuade the repo to accept the .pnd.


The repo insists on a <descriptions> tag with at least one <description> child node, plus an additional <description> element directly within <package>.


That is strange since according to http://pandorawiki.org/PXML_specification the descriptions node is not required.


The next problem was that after the upload, the repo complained that the pnd was not using a valid freedesktop application category. That was a bit easier to fix although it was not quite obvious that in order for PND apps to show up in the XFCE "Multimedia" menu, the "AudioVideo" category has to be specified in the .xml.


Since subfolders are not supported (I initially thought that <subcategory> could be used for that), I prefixed the PND application titles with "dsp:" -- the poor man's folder, so to speak ;)

( the updated .xml is here: http://tkscript.de/c64_tools/PXML_new.xml )

Here's the PND repo link: http://repo.openpandora.org/?page=detail&app=c64_tools_tests

EDIT: @M-HT: thanks, this http://tkscript.de/c64_tools/PXML_validated.xml version of the PXML is now valid (fixed empty <title> and removed <package>.<description>)
 
Last edited by a moderator:
Wow!  cool demos!  millions of pandora logos flying around, millions little orange dots/cubes flying around, fractal stuff swiveling around!  Don't know what these tests mean but they are nice! :)   Are they supposed to stop in couple seconds? Because all demos I ran automatically exited after few seconds.
 
Last edited by a moderator:
Hi everyone,

Again, to all devs involved in this project, THANK YOU VERY MUCH :wub:

As a potential use case, do you think DSP could be useful for Saturn emulation, to help with its many processors (VDP 1 & 2, SCSP, SCU, etc...) ?

Bye,

Magic Sam
 
Wow!  cool demos!  millions of pandora logos, little orange dots, fractal stuff!  Don't know what these tests mean but they are nice! :)   Are they supposed to stop in couple seconds? Because all demos I ran automatically exited after few seconds.
These are just some simple DSP sprite rendering tests. I was too lazy to add keyboard/joystick support that's why they exit after a certain number of iterations/frames.

As a potential use case, do you think DSP could be useful for Saturn emulation, to help with its many processors (VDP 1 & 2, SCSP, SCU, etc...) ?
I am completely unfamiliar with the Saturn so I really cannot say. The answer is probably yes and the biggest problem probably is to find someone willing to actually work on this.

Ican't seem to get it working: I updated using the online OS upgrade. Download the example PND and tried to run a demo.

It can't find the shared lib. Do I have to install something too?

Hmm no, that should do it. It works here (and apparently on Shenmue's Pandora).


Do you have /usr/lib/libc64.so.1 ?


What does "readelf -d c64_dsprite" say ? (type it in the terminal window opened by "Multimedia" -> "dsp: Open c64_tools testcases shell")
 
Wow!  cool demos!  millions of pandora logos, little orange dots, fractal stuff!  Don't know what these tests mean but they are nice! :)   Are they supposed to stop in couple seconds? Because all demos I ran automatically exited after few seconds.
These are just some simple DSP sprite rendering tests. I was too lazy to add keyboard/joystick support that's why they exit after a certain number of iterations/frames.

As a potential use case, do you think DSP could be useful for Saturn emulation, to help with its many processors (VDP 1 & 2, SCSP, SCU, etc...) ?
I am completely unfamiliar with the Saturn so I really cannot say. The answer is probably yes and the biggest problem probably is to find someone willing to actually work on this.

Ican't seem to get it working: I updated using the online OS upgrade. Download the example PND and tried to run a demo.

It can't find the shared lib. Do I have to install something too?
Hmm no, that should do it. It works here (and apparently on Shenmue's Pandora).


Do you have /usr/lib/libc64.so.1 ?


What does "readelf -d c64_dsprite" say ? (type it in the terminal window opened by "Multimedia" -> "dsp: Open c64_tools testcases shell")
Nope I don't have the shared lib, which suggests that something went wrong with my OS update.

I think I need to reflash then try the update - I think it didn't like that I installed a deb package for my wifi dongle
 
If DSP+GPP can make Pandora 1 emulate Sega Saturn at playable speed, it would be the first handheld device!  Man!  That would be awesome!

...Well one step at the time :) N64 first.  ptitSeb? sebt3? :)
 
Last edited by a moderator:
It works here after normal update, too. Thanks!


There is one unexpected result with vsync on, might be a bug with timing, I cannot say. If I run the bullet hell with 4500 sprites, it reports 60FPS but there is a black horizontal stripe near the top of the screen with one or two rows of active pixels above. If I run it with 5000 sprites I get a perfectly smooth display at 30FPS. With vsync of, 4500 sprites report an average 64FPS and 5000 an average 58FPS. This explains the second result, but the first doesn't look right.
 
Last edited by a moderator:
I've noticed that too, it seems to be some kind of double buffering issue that happens when the 60Hz limit is about to be reached. Might be a bug in my app. code.

Like you already noticed, the actual number of 'bullet hell' sprites should be higher, e.g. 4800 sprites + clearscreen should work at vsync (w/o I measured 60.024 FPS)

EDIT: Anyone here with a CC or Rebirth Pandora ? I really would like to know where the sprite limits are on these devices.

Please open the "testcases shell" from the menu, then type


./c64_dsprite 1 4800 1 100 1.0 0 1
and decrease the number of sprites (4800) until the avg FPS is ~60

Then do the same for the 32x32 size sprite test:


./c64_dsprite 1 1720 1 100 1.0 0 0
and please post your results.
 
Last edited by a moderator:
Here are some results on my Rebirth Pandora:

bash-3.2$ ./c64_dsprite 1 3639 1 100 1.0 0 1
[...]         mode: MODE_ALPHATEST
[...]     #sprites: 3639
[...]   clear_mode: GPP
[...]  #iterations: 100
[...]        slomo: 1.000000
[...]        vsync: NO
[...]       img_nr: 8x8
[...] omapfb_plane_enable: ok, plane 0 is now disabled.
[...] omapfb_plane_setup: ok, plane 1 is now disabled.
[...] omapfb_plane_setup: OK, plane 1 is configured size=(800; 480) geo=(0; 0; 800; 480) virt=(816; 992) bpp=32.
[...] omapfb_plane_setup_dsp_mem: plane 1 virt_addr=0x40767000, phys_addr=0x9c900000 size=0x00317000.
[...] omapfb_init: OK.
[...] coff_load_overlay: text=0x00000ca0  data=0x00000068  bss=0x00001110
[dbg] fshm_img: phys_addr=0x5cf0e000, virt_addr=0x40125000
[dbg] fshm_img: DSP phys_addr=0x10f0e000
[dbg] loc_init: loc_sprite_init() ok.
[...] 100 iterations in 1661 millisecs.
[...]  ==> avg FPS=60.205
[...] omapfb_plane_enable: ok, plane 1 is now disabled.
[dbg] omapfb_exit: [1] unmapped 3239936 bytes @virt=0x40767000 (phys=0x9c900000).
[...] omapfb_plane_enable: ok, plane 0 is now enabled
Bye, Magic Sam
 
Back
Top