[announce] c64_tools (DSP loader and IPC)

[bsp]

Shenmue: Unhelpful answer: what ekianjo said. more helpful: a list of documents/manuals that I have gathered so far (there are lots more!):

 sprugn4r.pdf [http://www.ti.com/lit/ug/sprugn4r/sprugn4r.pdf] - "AM/DM37x Multimedia Device Silicon Revision 1.x Version R Technical Reference Manual"
 spruf98x.pdf [http://www.ti.com/lit/ug/spruf98x/spruf98x.pdf] - "OMAP35x Applications Processor Technical Reference Manual" (more complete than the previous document. the differences between DM3730 (1Ghz Pandora) and OMAP3530 seem so small that I'd recommend this manual.)
 spraao8.pdf  [http://www.ti.com/lit/an/spraao8/spraao8.pdf] - "Common Object File Format" (DSP images)
 Chapter10.ppt [http://www.ti.com/ww/cn/uprogram/share/ppt/c6000/Chapter10.ppt] - Introduction to interrupts, Types of interrupts and sources, Interrupt timeline, Handling and processing interrupts using C and assembly code.
 spra829.pdf [http://www.ti.com/dsp/docs/litabsmultiplefilelist.tsp?sectionId=3&tabId=409&literatureNumber=spra829&docCategoryId=1&familyId=326] - "DSP/BIOS Timers and Benchmarking Tips"
 spru656a.pdf [http://www.ti.com/lit/ug/spru656a/spru656a.pdf] - "TMS320C6000 DSP Cache User’s Guide"
 spru425a.pdf [http://www.ti.com/lit/ug/spru425a/spru425a.pdf] - "TMS320C6000 Optimizing C Compiler Tutorial"
 spru190q.pdf [http://www.ti.com/lit/ug/spru190q/spru190q.pdf] - "TMS320C6000 DSP Peripherals Overview / Reference Guide" (contains many links to other documents)
 spru198k.pdf [http://www.ti.com/lit/ug/spru198k/spru198k.pdf] - "TMS320C6000 Programmer’s Guide"
 spru871k.pdf [http://www.ti.com/litv/pdf/spru871k] - "TMS320C64x+ DSP Megamodule Reference Guide"
 spru732j.pdf [http://www.ti.com/litv/pdf/spru732j] - "TMS320C64x/C64x+ DSP CPU and Instruction Set"
 c6xbios.hlp [installed with CCS, on my system its @ C:\ti\bios_5_41_13_42\packages\ti\bios\help\doc] - "TMS320C600 DSP/BIOS Help"

 spru007i.pdf [http://www.ti.com/lit/ug/spru007i/spru007i.pdf] - "DSP/BIOS 5.40 Textual Configuration (Tconf) User’s Guide" (in case you're wondering what those .tci and .tcf files in the example DSP image project are used for)

 

[WizardStan]

So the very high level end-to-end process basically goes: write a DSP component in C/ASM (presumably tailored specific to the way the dsp functions), build with Code Composer into the dsp object (until overlays are a thing that happens), load that object to the dsp with go64.sh. ARM code then loads data into shared memory and sends an RPC message to the dsp telling it to run the new component on that data and then buggers off to do whatever else it needs to do, eventually checking back for an RPC response which contains the shared memory location where the result of the request can be found.

Multiple processes can use the DSP by virtue of the fact that it single-threadedly handles messages as they come in.

Is that right?

 

[bsp]

Yep, that's pretty much it.

If it's pure "C", you don't have to change existing code since the c6x compiler does quite a good job already.

You will probably want to use fixed point math (e.g. IQmath lib) instead of float for performance reasons but this could even be true when programming the ARM (have to benchmark that).

The "load overlay" feature, once implemented, will still require you to build an .out file with CodeComposer.

The difference to regular DSP images / .out files will be that it will have to contain a specially named linker section, which c64_load/go64.sh will look for. I will provide an example project for that, of course.

Unless you place the shared data in either the DSP cache memory ("L1DSRAM") or uncached RAM , you have to take special care of cache coherency between the two cores: After you've modified the memory on core A, you have to writeback its caches and invalidate the caches for that memory in core B (I called it a core, see what I did there ). The details are described in the readme.txt.

EDIT:

Multiple processes can use the DSP by virtue of the fact that it single-threadedly handles messages as they come in.
Is that right?

yes that's right. It would technically be possible to use multi-threading on the DSP-side but I strongly advise against that because  1) DSP-development is already hard enough as it is and  2) the DSP OS timer is not set up correctly ATM so multitasking simply does not work.

p.s.: and before you start pulling out your hair wondering why a DSP image won't load resp. start if a different image was loaded previously: I am currently trying to figure out a solution for that. Just reboot before reload (for now). EDIT 5:16pm: just did some tests. The previous statement is not correct, loading different DSP images (without reboot) works fine. There seems to be something 'magical' (i.e. not understood by me so far) about the main() function. I placed a simple volatile tmp; for(i=0..1000) tmp=i; loop in there, the image would then not load. Moving the loop to the "mlb" task function fixed that. Maybe this is some kind of timing issue. For now, do not do any heavy lifting in main() and you should be fine.

p.p.s: Although it's mentioned in the readme.txt: Do not increase the optimization level of the main project file or the code will crash. Looks like a compiler bug to me. Do like I did with the "demo_fractal" project: Make your code a .lib and link it to the main project.

p.p.p.s.: The component registration is done on the DSP during startup. After you've created a new component, edit "components.h" (add #include and enum entry). Then edit "main.c" and add the appropriate mlb_component_register() call. After that you can use RPCs to invoke the component functions on the ARM side.

p.p.p.p.s.: As you can see in the examples, you do not have to place all args / return values in shared memory. Each function has up to two 32bit arguments, and up to two 32bit return values.

 

[M-HT]

You will probably want to use fixed point math (e.g. IQmath lib) instead of float for performance reasons but this could even be true when programming the ARM (have to benchmark that).

I took the number-crunching part of your fractal benchmark and tested floating point and fixed point versions on ARM.

floating point:
1200 iterations in 12669 millisecs.
fixed point:
1200 iterations in 3851 millisecs.
---Does you package require Code Composer Studio ?

AFAIK that's a commercial product. (The free version is a 30 day evaluation version).

If yes, are you planning a version that can be compiled with the free compiler (by TI) ?

 

[WizardStan]

AFAIK that's a commercial product. (The free version is a 30 day evaluation version).

If yes, are you planning a version that can be compiled with the free compiler (by TI) ?

The bare compiler is freeware. I'm playing around with it now. No IDE, no debugging, no commercial use, but it still seems to spit out binaries (that I haven't been able to test at all, maybe tonight or tomorrow)Nevermind, I may not know what I'm talking about. I've got it setup as an Eclipse plugin that I thought was license free but now I am not sure.

 

[bsp]

M-HT, nice, thanks for trying that out! I am assuming that you overclocked your Pandora by ~15.5% which would explain the slightly better benchmark numbers.

This means that the DSP is 3851/(1393/1.55) = ~3.19 faster at executing the same code (same source code, to be exact).

I scaled the DSP time to account for overclocking (the DSP can be overclocked, too, you probably read my post on the previous page).

So, that's quite cool.

And this is with a render function that is absolutely not DSP-friendly since it contains branches in its inner loop!

Your results also indicate that fixed point optimization on the ARM is not worth it (~the same speed as the FPU loop).

Regarding the CCS license: I did not buy a CCS license, downloaded it straight from the TI website. I had to create an account there but did not have a problem with that (not if I am given worthwhile things). There was no message warning me about a 30 days evaluation limit, nor is there any in CCS itself.

Here's a screenshot of "Help->Code Composer Licensing Information":

Btw, I was really glad to see TI invest into FOSS over the past few years (e.g. the Linux kernel). Their IDE is free "as in beer". Other manufacturers leave the IDE part to 3rd parties who charge so much money for it that hobbyists cannot afford to buy it (I am talking about thousands of dollars per license).

AFAIK, CCS is not available for Linux, though. You can probably run it in wine (EDIT: wrong, there is a Linux version). The compiler itself is available as a native version, though (also for free download).

EDIT:

If yes, are you planning a version that can be compiled with the free compiler (by TI) ?

that could be arranged but is not exactly one of my priorities.

OK: The IDE already auto-generates (GNU) makefiles, so there you go.

I have to admit that I am using CCS mostly to build the project. Source code editing I do in emacs (well, maybe half/half). Having a UI for the compiler is still of great help, especially if you are not so familiar with it.

EDIT#2:

CCS / the IDE is not completely free-as-in-beer (just had another look at the TI site). I think the limitation is that you must not use the free edition for commercial development (which would perfectly make sense).

Can anyone who installs CCS from scratch / creates a license file provide a screenshot or something of the "create license" dialog ? I may be mistaken but that there might have been an option "educational / open source".

p.s.: as an anecdote: some days ago I requested access to the sources of some closed source TI library. Had a quick chat with someone from their software marketing team, told him that I was just interested to see how it works and voila, I was granted access the next day. I am not allowed to redistribute the source, though, but that's fine. Regarding CCS: If you really feel unsure, use the command line compiler to build your releases and publish your source codes under a FOSS license. I cannot imagine that TI will send SWAT teams after you for doing that.

p.p.s.: last year I bought an MSP430 "Launchpad" EVM (~7 euros), for tinkering with electronics/GPIOs. Maybe that one came with the CCS license ? I really cannot remember but on their website it says that CCS licenses are often bundled with such things.

 

[M-HT]

Like I said, I took the number-crunching part from your benchmark - I removed copying the data to framebuffer and logging from the loop so my results are not directly comparable to your results (doubly so, because I'm using my 600MHz CC Pandora versus your 1GHz Pandora). I was only interested in floating point versus fixed point performance on ARM - fixed point is 3.28 times faster than floating point in this benchmark.

Regarding free CCS license - according to this page there's time limited evaluation version, code size limited version and version usable only with onboard emulation on development boards.

EDIT: I took a look at the generated makefiles - they contain absolute paths to the compiler and libraries in various files, so if you install the compiler to different path or your using linux version of the compiler, you need to change the files.

 

[bsp]

ok, thanks for the update. so, fixed point on ARM is still worth the effort. that makes things easier in so far that adding DSP acceleration to one's application does not require float->fix rewrites if the application is already properly optimized.

EDIT: and if I understood the numbers correctly, using the DSP for computations like in the fractal benchmark is performance-wise similar to having an additional ARM core (almost twice as fast, actually EDIT: ~48.6% faster). The DSP version should take ~1943 EDIT: ~2591 (=1393 * (800Mhz / 430Mhz)) milliseconds on your 430Mhz CC Pandora DSP.

and yes, also thanks for confirming that the DSP stuff works on CC Pandoras.

I took a look at the generated makefiles - they contain absolute paths to the compiler

that's not so nice but easily fixed with a shell script. maybe there's a way to setup CCS so it does not do that / uses env. variables instead.

 

[Exophase]

Did you see where I said that floating point on Cortex-A8 is, by default, very slow? Of course fixed point is worth the effort.. or converting it to NEON.

Don't use floating point on Pandora unless you're willing to use NEON or you don't care about performance.

 

[M-HT]

and yes, also thanks for confirming that the DSP stuff works on CC Pandoras.

Sorry, but I didn't test the DSP stuff.

 

[bsp]

heh, I was just coming back to add an "edit:" saying that you did not really confirm that the DSP stuff works.


Was there a technical reason that prevented you from doing so or did you just not have the time to do so (maybe just waiting for this to be included in the official firmware) ?

 

[WizardStan]

So what compiler do I need to run in order to build a DSP image? In the Eclipse plugin, creating a new project, you choose device Family and Variant, and I chose the C6000 family and TMS320C64XX variant. Is that right? I built a binary (basic "hello world") but I'm not in a position to test it.

 

[bsp]

@Exophase: Yes I read that but you did not say how slow. I always use fixed point on embedded devices myself so I did not have an idea what the performance advantage of Fix vs. FPU on the Pandora would be. EDIT: What do you mean by "unless you are willing to use NEON" . Was there anything wrong with the compiler options I used for the float version (see previous posts or the source code / makefile) ?


@WizardStan: If you want to make life easy for you, just install CCS and look at how the c64_tools example projects are set up (or did you already do that and that's the Eclipse plugin you are talking about?). Select "New->CCS Project", enter the project name (and maybe the filesystem location), accept the defaults (Family: "C6000", Variant: "Generic C64+ Device"), click ok and that's it.

 

[M-HT]

I didn't really have time to play with the DSP, I just quickly hacked the benchmark together.

Also, you said that latest kernel from notaz is needed to test your package - is that the kernel from the latest Pandora OS upgrade or something different ?

 

[bsp]

it's the kernel from the latest Pandora OS (SZ 1.55, kernel 3.2.45), with slight modifications. No need to rebuild all modules.

EDIT:

p.s.: I just decided that I will provide C64+ makefiles in the next release of c64_tools. I am a makefile-kind-of-guy after all and IDEs can cause unnecessary confusion. Still, quite handy if dealing with something unknown / new.

p.p.s.: In addition to that, I think it would probably be a good idea if I registered an account at the Pandora Wiki and wrote a step-by-step DSP development tutorial / introduction (WizardStan already suggested that).

@M-HT: would it be possible for you to send me your fixed point ARM variant of the benchmark (PM or email) ? I would like to include that in the next release.

 

[Exophase]

@Exophase: Yes I read that but you did not say how slow. I always use fixed point on embedded devices myself so I did not have an idea what the performance advantage of Fix vs. FPU on the Pandora would be. ?

I did give specifics about performance:

"Figure about 7 cycles per instruction for single precision. Much worse if you're doing a bunch of divisions. Also huge penalties if you're converting from floats to ints, or if you're doing flow control based on floating point comparisons"

You can get an idea of performance vs fixed point. Integer add/sub and shift have one cycle latency and can dual issue with independent instructions - shifts can also be folded along with instructions in ARM, so that reduces some of the overhead of fixed point. Really depends on what your instruction mix is like though. Integer multiplications are worse (IIRC 2 cycles to issue 32x32-bit, 1 cycle for 16x32 or 16x16), but still not nearly as bad as VFP. There's no integer division instruction so that's slow no matter what, although floats probably tend to beat 32-bit integer divisions in software.

EDIT: What do you mean by "unless you are willing to use NEON" . Was there anything wrong with the compiler options I used for the float version (see previous posts or the source code / makefile)

Compiler will only use NEON with what it can auto-vectorize, which it's not that great at. It won't force NEON for scalar code, at least not last I was aware. So I mean using NEON deliberately through intrinsics or assembly code.

 

[notaz]

As we now have more interest of doing something on DSP than ever, it sounds like I should start firmware integration, but we still haven't decided on image loading. I guess merge of go64.sh to current op_dsp.sh that would also take path to DSP image and do c64_load on it would do?

 

[bsp]

@Exophase: Thanks for the insight. I would love to see a skilled dev like you getting his hands on the DSP. With (E)DMA(3) and L1DSRAM trickery, there's lot of potential of implementing e.g. fast graphics blitters on the DSP.

@notaz: Yes, we should. I would recommend to keep go64.sh a separate script that gets called from op_dsp.sh but please let me think about this for a few days (I have a couple of holidays coming up, will use some of the time to work on this project, implement the "load overlay" feature, etc)

EDIT / p.s.: go64.sh currently takes up to two arguments, e.g. "./go64.sh <DSPImagePathName>" or "./go64.sh -vvv <DSPImagePathName>" for very verbose debug output. (-vv and -v for less debug)

 

[WizardStan]

@WizardStan: If you want to make life easy for you, just install CCS and look at how the c64_tools example projects are set up (or did you already do that and that's the Eclipse plugin you are talking about?). Select "New->CCS Project", enter the project name (and maybe the filesystem location), accept the defaults (Family: "C6000", Variant: "Generic C64+ Device"), click ok and that's it.

Doing so seems to work fine for the very simple "hello world" sample, but attempting to build your example leaves me with a lot of errors about missing include files. Clearly it doesn't automatically setup all the include directories, probably also missing various libraries, which means I've got some further investigations to do.

 

[PokeParadox]

I tried out the fractal example on my Pandora, very nice! My Pandora has now lost its DSP virginity!