[announce] c64_tools (DSP loader and IPC)

[mcobit]

I could think of offloading mpeg2 decoding to it in Daphne. There would need to be a decoder written for it though.


Something I would not be able to do. Heck, even plugging something like this into the existing emu would not be possible for me.


But maybe this is the kind of example app bsp is looking for as a usecase.


Edit: Sources are here *wink,wink*

http://mcobit.openpandora.org/source/daphne-1.0-pandora-src.tar.bz2


Edit2: Oh, there already seems to be some mpeg2 codecs

http://software-dl.ti.com/dsps/dsps_public_sw/codecs/C64XPlus_Video/index_FDS.html

 

[bsp]

@exophase:

you said "DSP is only going to be useful for emulation if you have really big or mostly autonomous tasks, not things that rely on quick round-trip communication".

that's not right and here's the "proof" (src + binaries will follow this weekend):


pandora:/home/bsp# ./go64.sh

go64.sh: ARGS="" DSPIMAGE="./c64p_simple_dm3730.out"
[...] loading DSP image from "./c64p_simple_dm3730.out"
[dbg] coff_open: file size=220698 (0x00035e1a)
[...] reading 52 section headers....................................................
[...] initializing DSP memory and copying sections..
[...] text=0x000063a0 data=0x0000134f bss=0x002103bc
[...] DSP image loaded successfully. starting DSP..
[dbg] waiting for DSP startup..
[...] DSP is up and running!
pandora:/home/bsp# ./c64_tc 6
[...] selected testcase 6 ("TC_FASTCALL_RPC_ADD")
[...] starting fastcall benchmark
[...] fastcall benchmark finished. => 1000000 iterations in 2458 millisecs.
pandora:/home/bsp#


Configuration:

[ ] flush_cache_all()
[x] DSP L1D
[ ] DSP L2D
[x] use L1DSRAM for IRQ-driven IPC
[x] fastcalls


==> 0.002458 millisec messaging overhead per roundtrip/RPC.
==> 406834.825 roundtrips/RPCs per second. (6781 RPCs per 60Hz video frame)

==> 284.784 times faster than DspLink (based on M-HT's measurement, i.e. ~0.7ms/roundtrip)
==> 17.37 times faster than c64_tools-18Sep2013

p.s.: that's still not fully optimized, at least ~1.5 million RPCs per second can be done. My current code is just very defensive and does a lot of error checking..

 

p.p.s.: "but just having to keep the DSP on at all probably comes with a big chunk of static leakage power" (exophase) --> I have not noticed any reduced battery life and during the past days (including today) I had my Pandora on for up to 13 hours (with the DSP running all the time). According to my colleagues, RAM access (and the display, of course) is what consumes power. SRAM does not seem to be much of a problem. (and yea, at the end of the day the battery usually was at least 30% full).

 

[milkshake]

I like this thread

 

[Linux-SWAT]

More than ARM vs x86 ?

 

[milkshake]

Yeah that thread is just like watching kids compare bicycles

 

[bsp]

"look at how fast I can ride my bicycle"

srsly, now we are getting somewhere. As of today (well yesterday here), this is getting fast enough so that the DSP can not just be used for "bulk processing" but for rather small subroutines (>14 calls per scanline is not bad, eh? I think we can at least quadruple that if all the function calls and error checking are removed (that's for expert devs, of course)) (did I mention I really like TI hardware and think some of the software they hand out does not do it justice?). Anyway, beer time for me (have a day off tomorrow).

 

[milkshake]

Well I hope someone attempts to use what you have come up with in a project and if they do I hope it's Awesome!

 

[ptitSeb]

Yes. This is interesting. At one time, I'll probably try to use this. I have in mind a sound plugin for mupen64plus for example. That not to complicated (yet not trivial). And even if doesn't bring a huge speed up, it could be an interesting test case.

But I'l not ready to start working on that yet.

 

[Linux-SWAT]

Where are you from bsp ?

 

[Exophase]

@exophase:

you said "DSP is only going to be useful for emulation if you have really big or mostly autonomous tasks, not things that rely on quick round-trip communication".

Well I guess we have a different idea of what constitutes big or small - rohezal was talking about emulating individual instructions here. For that to even be up for consideration we'd be talking tens of millions per second territory, so yeah it's too small.

p.p.s.: "but just having to keep the DSP on at all probably comes with a big chunk of static leakage power" (exophase) --> I have not noticed any reduced battery life and during the past days (including today) I had my Pandora on for up to 13 hours (with the DSP running all the time). According to my colleagues, RAM access (and the display, of course) is what consumes power. SRAM does not seem to be much of a problem. (and yea, at the end of the day the battery usually was at least 30% full).

Uh, no offense but that doesn't exactly sound like the most scientific testing there. The OMAP3530 and DM3730 datasheets do talk about power consumption on the DSP rail (yes, it gets a rail to itself), it's definitely nothing to take lightly.

 

[notaz]

@Notaz: (almost posted the message then saw your reply): You are more proficient at Linux kernel internals than I am so if you could do the necessary changes to the kernel and the c64 module and post a patch, I will gladly try it out / adapt the rest of the software when it works (I have everything set up to compile new kernels).

I've decided to do a test module from scratch to test the functionality instead, and it works for me.


First you need an updated kernel:

http://notaz.gp2x.de/tmp/uImage


( source: http://git.openpandora.org/cgi-bin/gitweb.cgi?p=pandora-kernel.git;a=shortlog;h=pandora-3.2-c64_tools )


With that installed (copy to /boot/uImage-3), try this module:

http://notaz.gp2x.de/tmp/mod_cma_test.tar.gz

(EDIT: don't forget to remove your usual memhole bootargs)


if that works, integrate it with yours.


I would still like some demo app before we pull this into firmware.. Maybe some graphics effect or something.. For example, spinning cube on the corner of the screen on separate DSS layer would be cool. Imagine the joyful view of 0% CPU usage in top or some emu running as usual while the cube spins in the corner at 60fps unaffected by anything. It would be impossible to resist pulling c64-tools into firmware if a demo like that exists.


I can offer to code the base app to set up the DSS layer and give you fb physical address, and you would draw some effect of your choice to it with the DSP (or you can do everything yourself of course).

 

[PokeParadox]

I agree with Notaz... that would be cool.

Incidentally this is making me wonder if I could perform the AI operations of CromoZome on the dsp... if so then I am itching to do a CromoZome 2!

 

[levi]

Uh, no offense but that doesn't exactly sound like the most scientific testing there. The OMAP3530 and DM3730 datasheets do talk about power consumption on the DSP rail (yes, it gets a rail to itself), it's definitely nothing to take lightly.

Still, if you can get the DSP doing stuff that's suited to it (like 3d vector transforms, I assume) it should be able to do that work for less mAh than doing it on the CPU would cost. Would the cost of setting the DSP up and pulling off the results and just having the DSP on in the first place be that hard to offset?

That said, for some apps I'd just like to say sod the battery life. My Pandora spends 80% of its time (safely) overclocked, just so I can get those extra FPS in my emulator of choice.

 

[_wb_]

This is a very interesting thread.

I wouldn't try to use the DSP in individual games or applications - except maybe some emulators.

I think we should spend most effort in trying to use the DSP in frequently used libraries where it makes sense: e.g. audio/video/image decoding, SDL, ALSA, lunixboch's OpenGL, and so on. That way many existing PNDs could benefit from it instantly.

If we could somehow offload mp3/ogg/flac decoding (mostly) to the DSP, that would probably have the biggest impact.

 

[Linux-SWAT]

Is our DSP the C6000 ?

https://en.wikipedia.org/wiki/TMS320C6000

--edit: seems so.

 

[notaz]

If we could somehow offload mp3/ogg/flac decoding (mostly) to the DSP, that would probably have the biggest impact.

I doubt it, these things use like ~5% CPU on ARM if proper libraries are used, so difference would not be noticeable.You'd be replacing well tested and proved ARM decoders with freshly baked DSP code that would have high risk of instability.

 

[milkshake]

So what is the best use scenario for our dsp?

 

[Linux-SWAT]

A demo ^^.

 

[_wb_]

If we could somehow offload mp3/ogg/flac decoding (mostly) to the DSP, that would probably have the biggest impact.

I doubt it, these things use like ~5% CPU on ARM if proper libraries are used, so difference would not be noticeable.
You'd be replacing well tested and proved ARM decoders with freshly baked DSP code that would have high risk of instability.

It would be only a small performance difference in games, but it could potentially be a big difference in battery life when you just want background music with otherwise no real cpu use (e.g. using the Pandora as an audio player, or reading a PDF while listening to music), no?

Would it be possible to decode and scale JPEG faster using the DSP? I would be interested in an image viewer that can load and zoom in/out on high-res pictures fast. Try viewing a series of 5MB or bigger JPEG files from your camera's SD card with ristretto. It's too slow. Maybe that's because ristretto itself needs some optimization, but still. If libjpeg could be speeded up while reducing CPU usage, it could also help to make web browsers load pages a tiny bit faster.

Maybe truetype font rendering and stuff like that could also benefit from the DSP? Probably not worth the effort though.

I'm just trying to find some potential places where the DSP could maybe be put to use. But maybe we should first try to do some benchmarks to get an idea of its performance and power consumption, to see if it's all worth the trouble.

 

[Crionic]

As it was said before, dspbridge and dsplink have already been available for a while. I think, the reason why the dsp was hardly (if ever) used is not because of high latency or (too) low execution time. I'd say it's because of the limited usefulness for most applications. The dsp performs best, when large chunks of mathematical operations, that can be executed in parallel, are thrown at it. And the vast majority of apps we have just don't need to do that. I even can't see a major improvement for emulators, except maybe some image post- or preprocessing (not sure about sound emulation though).