Plans For Utilizing The Dsp?

[Maltanar]

*steps out of the shadows*

Hi,

I had asked about this before in April regarding DSP-accelerated video decoding, did a forum search but couldn't find anything so here it goes.

What are Team OpenPandora's plans for DSP utilization on the Pandora? I know there are many more higher priority tasks in the shorter run but I'd still like to hear what you have to say.

The reason why I'm asking this is to bitch about why it's just sitting there and we're not using it....nah not really I'm a Google Summer of Code student working on the BeagleBoard to develop a DSP ease-of-development suite (my project blog's at http://maltanar.blogspot.com and the project page is located at https://gforge.ti.com/gf/project/dspeasy/ ) and a Pandora owner.

I'm willing to help with DSP/Link integration on the Pandora firmware and help people get started with DSP-side development (since the project C6Run is all about making it easier to get started).

For those who aren't so familiar with the DSP core on the OMAP3530: it's a processor core running at 430 MHz which is quite good at tasks like video/audio/image encoding and decoding - think of 720p h.264 decoding while using only 10% of the CPU. TI offers a package of APIs and a kernel module called DSP/Link which allows DSP utilization. Angström has good support for DSP/Link, though installing it from the repos probably won't work due to kernel version issues.

edit: my bad - Pandora's DSP clock speed isn't 530 but 430 Mhz, thanks to urjaman for the correction.

 

[urjaman]

I have nothing else to say (or i did, but whatever), but our DSP is "just" 430Mhz (as our OMAP is rated for 600Mhz, it's the model before TI started binning OMAP3530s at 720Mhz (and still available)). My guess would be that after 2.6.35 is working well on the pandora, then we can start to have some DSP fun.

 

[Maltanar]

Corrected, thanks! I'd still say 430 MHz isn't all that bad

 

[Bosbeetle]

Hehe fun to see the pandora already being awesome without using all of its chips. love to see them used though.

 

[torpor]

[quote name=Maltanar]I'm willing to help with DSP/Link integration on the Pandora firmware and help people get started with DSP-side development (since the project C6Run is all about making it easier to get started).[/quote]

What I fantasize about is an open-source compiler for the DSP component. Its a real bore having to get it from TI all the time, set it up, and work out how to put all the pieces together.

And besides: using the host-CPU to build DSP code, and compile and manage it locally, is a pretty nice idea, actually.

So lets throw all the standard arguments out for a minute, and assume that somehow, we work out how to write a compiler specifically for the DSP, and that compiler runs in the hostOS (PandoraOS/Angstrom/etc.) .. DSP modules could be dynamically assembled to form components in a processing stream, on the basis of emitted code from some other larger application ..

An onboard compiler for the DSP would be a huge boost to the OMAP DSP utilization graph, personally .. I mean, sure, we can also remember that it is a commercially applied component, not everyone wants to share DSP code, I suppose ..

 

[Maltanar]

[quote name=Maltanar]I'm willing to help with DSP/Link integration on the Pandora firmware and help people get started with DSP-side development (since the project C6Run is all about making it easier to get started).

What I fantasize about is an open-source compiler for the DSP component. Its a real bore having to get it from TI all the time, set it up, and work out how to put all the pieces together.

[/quote]

I agree that it isn't pleasant having to obtain the DSP-side compiler (the TI CGT6000) by registration from TI, even though it doesn't cost anything.

But

..and assume that somehow, we work out how to write a compiler specifically for the DSP..

that's one big assumption to make I'd heard of some university researches from Germany who worked on porting GCC to the TMS320 architecture, but I'm not sure how much progress has been made.

I'd think that there would be at least some Pandora users interested in having DSP-accelerated audio/video playback on their devices, and would say getting DSP/Link and TI Gstreamer plugins running on the Pandora as soon as possible would be more appropriate shorter term goals.

 

[greendots]

Welcome!

The devs are busy and were having trouble picking the interface to use (bridge, link, etc) Pandora generally needs to move to a newer kernel or have components back ported.

 

[Maltanar]

Well met!

Since it sounds like we'll be moving on to 2.6.35 anyway, back-porting (actually, it should be just compiling DSP/Link for the current Pandora kernel) sounds less logical. Until then...well, I'll make sure I share anything here in case I have time to play around

 

[Exophase]

Unfortunately choosing DSP-Link vs DSP-Gateway is not simple since IMO at least both have pretty serious drawbacks (Gateway is probably not even an option).

From what I can understand - and please correct me if you know otherwise - this is the current scenario:

DSP-Gateway - Requires operating system code running on the DSP that consists of closed source binaries from TI. This means you end up locking in unknown memory and possibly DSP utilization resources, adding potentially unwanted overhead.
DSP-Link - Uses a purely static memory map for what is visible by CPU, DSP, and what's shared. This is especially problematic since it's implemented such that the space is reserved at kernel boot and can never be used for anything else by the CPU. IMO this is too severe a restriction for DSP.. we want every Pandora to be able to use it but without compromising the amount of available memory, especially when there probably won't be a lot of DSP software.

So I think DSP-Gateway is probably the best path forward, but it looks pretty thorny... I also wouldn't hold my breath for TI to release source to the DSP code.

I do hope this gets sorted out soon because I think that the DSP could be used to greatly enhance pretty much anything that can be multithreaded. Contrary to common belief it's not just an audio/video processor but can beat the CPU in all sorts of tasks.

What I fantasize about is an open-source compiler for the DSP component. Its a real bore having to get it from TI all the time, set it up, and work out how to put all the pieces together.

And besides: using the host-CPU to build DSP code, and compile and manage it locally, is a pretty nice idea, actually.

So lets throw all the standard arguments out for a minute, and assume that somehow, we work out how to write a compiler specifically for the DSP, and that compiler runs in the hostOS (PandoraOS/Angstrom/etc.) .. DSP modules could be dynamically assembled to form components in a processing stream, on the basis of emitted code from some other larger application ..

An onboard compiler for the DSP would be a huge boost to the OMAP DSP utilization graph, personally .. I mean, sure, we can also remember that it is a commercially applied component, not everyone wants to share DSP code, I suppose ..

that's one big assumption to make I'd heard of some university researches from Germany who worked on porting GCC to the TMS320 architecture, but I'm not sure how much progress has been made.

Short of TI open sourcing CodeComposer (very unlikely) I really doubt any open compiler would provide good performance on the DSP. Given GCC's awful performance on Blackfin, a much more limited and narrow VLIW, I don't have good expectations for a C6x GCC port without some very major work. TI's compiler would probably always have the advantage, even if things got quite aggressive. Optimizing C6x is not easy; in fact, if you start optimizing at all it becomes difficult to even maintain correct output due to the effects of the deeply non-interlocking execution.

If the performance isn't remotely competitive then there probably won't be much reason to use the DSP at all. I also don't think self-hosting compilation is going to be very popular for highly performance sensitive code because performance critical code tends to be slower to compile.

 

[jonlad1]

Sounds like an opportunity for Craig's bountys?

Maybe MWeston/Craig could speak to TI about releasing the source? They were in touch with them originally at the start of the whole Pandora project IIRC.

 

[torpor]

Well if we can't see a compiler on the horizon, at least an assembler would be nice ..

But anyway, welcome to the scene, Maltanar .. I've been following your blog progress on DSP hacking and its quite interesting, for sure. I'm hoping we make some more progress in the quest to complete a DSP pilgrimage, as I have a filter or two I'd like to be running on that DSP one day! (Would love to port Tony H-B's Korg MS-20 filter simulation, for example ..)

 

[fanoush]

So I think DSP-Gateway is probably the best path forward

Are you sure you meant DSP-Gateway? You did not mention dsp-bridge as an option in your list at all.
some explanation here http://elinux.org/BeagleBoard/DSP_Clarification

DSP-Gateway was something working on OMAP1 and 2 (which have older generation C55x DSP ) in Nokia's kernels 2.6.16-2.6.21 for 770/N8x0 i.e. something which appeared to be dead for quite some time.

But maybe you really mean it since the wiki page linked above does mention some revival attempt for OMAP3.

Not sure about other solutions but dsp-gateway allowed to load additional custom DSP code that could run in parallel to existing DSP code in some sort of multitasking DSP kernel. At least dsp-link looks like the DSP is reset when loading code which looks like big step back. PCMIIAW regarding OMAP3 dsp solutions, I am more familiar with C55x on Nokia 770/N8x0. AFAIK the Maemo dsp issues were
- closed DSP kernel which did not export access to audio hardware (nokia dsp codecs were sinks so mp3/aac/whatever decoded data went directly to audio hardware, you could not do same for tremor codec done by community)
- C55x is a bit archaic regarding memory addressing (similar to 16 bit CPUs like 8086)
- char type has 16 bits and there are other oddities so C code needed lot of changes
The C64x dsp in OMAP3 looks like it could run plain ANSI C code.

 

[notaz]

yeah sounds like Exo mixed bridge with gateway. I agree that static memory map is major drawback of -link, but I wonder what it's memory requirements are?

DSP-bridge has been recently added to mainline kernel staging area, so it's becoming more attractive from maintenance point of view.

 

[urjaman]

I agree that static memory map is major drawback of -link, but I wonder what it's memory requirements are?

DSP-bridge has been recently added to mainline kernel staging area, so it's becoming more attractive from maintenance point of view.

There are slightly more user-space applications using it, including gst-openmax and gst-goo through TI's OpenMAX IL implementation which is also open source. Leonora is a project implementing the Theora codec on the DSP, and is a good example of non-trivial code running on the DSP and communicating with a process running on the ARM core.

I think these are pretty heavy points in favor of bridge, too.

 

[Maltanar]

yeah sounds like Exo mixed bridge with gateway. I agree that static memory map is major drawback of -link, but I wonder what it's memory requirements are?

DSP-bridge has been recently added to mainline kernel staging area, so it's becoming more attractive from maintenance point of view.

The requirements for DSP/Link itself aren't that high, should be about 2MB for the default config. The CMEM module (for contiguous shared memory allocation) usually takes up more space. As an example, the Beagle Angström demo image's memory map (with which the TI Gstreamer plugins work) reserves ~30MB for all these (including 16MB for CMEM). Though more minimalistic configurations might be possible as well.

DSP-bridge also has the node system to its advantage, whereas with DSP/Link one needs Codec Engine for sharing the DSP between multiple algorithms/nodes (which is kind of messy to work with, but is quite powerful) which also solves the need for DSP powerdown/powerup.

DSP/Link versions 1.40 and above support dynamic configuration, and should be able to change the memory mapping dynamically, although I haven't tried it myself (http://processors.wiki.ti.com/index.php/Configuring_DSPLink_Dynamically)

I've heard from my GSoC mentors (who work at TI) that a newer version of DSP/Link was in the works to address Link's shortcomings. Eventually the two architectures (bridge and link) will probably converge.

Even in the case the devs decide to go along with the bridge (in which case we still have gst-openmax which can be readily used for video/audio acceleration, as urjaman mentioned), my offer for help still stands, although I don't have as much experience with dsp-bridg.

 

[Exophase]

Are you sure you meant DSP-Gateway? You did not mention dsp-bridge as an option in your list at all.
some explanation here http://elinux.org/Be...P_Clarification

http://elinux.org/BeagleBoard/DSP_Clarification

Yeah I mixed up the two, that's all. I meant DSP-Bridge.

 

[Maltanar]

For reference, the Syslink project which will be a unification of dsp/link and dsp-bridge, can be found at

https://gforge.ti.com/gf/project/sl_bios_ipc/

(that syslink will be like a union of link and bridge is what my GSoC mentor at TI has told me - I'm not sure if it'll be 100% backwards compatible for both architectures)

Edit - indeed, the project description reads:

SysLink is the next generation IPC driver developed for OMAP4 and beyond. SysLink is an evolution of both the previous-generation IPC drivers - DSPBridge and DSPLink. It provides a symmetric IPC interface on both the host processor and remote/slave processors. It is easily scalable to support more than 2 processors. SysLink provides features to control and communicate with slave processors enabling parallel processing for multimedia acceleration. SysLink integrates with D-OMX enabling MPU to offload some OMX components to slave processors.

Some of the key features of SysLink are:
Messaging: Ability to exchange fixed size control messages with Co-Processors
Dynamic memory management: Ability to dynamically map files to Co-Processor's address space
Dynamic loading: Ability to dynamically load new components on Co-Processors at run time
Power Management: Static and dynamic power management for DSP
Zero-copy shared memory: Ability to "pass" data buffers to other processors by simply providing its location in shared memory.
TILER-based memory allocation: Allocate 2-D buffers with mirroring and rotation options, suitable for video
Remote function calls: One processor can invoke functions on a remote processor.

 

[notaz]

developed for OMAP4 and beyond

Doesn't sound too good for us. I've read OMAP4 has some hardware support for ARM/DSP synchronization so it sounds like this will depend on that.

 

[fusion_power]

Sounds like TI does alot for the new OMAP4. But what did they actualy do for the OMAP3 to make the life of coders easier? Easy Access of the DSP for example 'n stuff?
For me it still sounds like TI does all the nice hardware but forget to make the propper software (drivers...) for it. :unsure:

 

[zevdawg]

Sounds like TI does alot for the new OMAP4. But what did they actualy do for the OMAP3 to make the life of coders easier? Easy Access of the DSP for example 'n stuff?
For me it still sounds like TI does all the nice hardware but forget to make the propper software (drivers...) for it. :unsure:

3 redundant words will answer this:
"Developers, Developers, Developers!"

No, but in all seriousness - it's a 'hobbyist' device, after all.

Good example, I think should also play key here:
An R/C kit isn't ever pre-assembled (a 'real' one, where everything's built from the ground up). Everything from the body (decals, paint), tires (and rims), and even engine (whether that be gas or electric - both have advantages and disadvantages) are up to the person building it.

Same goes for this hardware (more or less)- but it's up to developers to make it work. TI has supplied the "model kit", it's up to OP and the community to make the most out of it.