Flash + DSP


WizardStan

Mega GP Mania
Joined
May 24, 2008
Messages
16,731
Just got an update on the TI Flash demo. DSP acceleration has been enabled. I will now start throwing money for realz at whomever can get DSP to work on the Pandora for this.
 
I keep hearing that the DSP will be pretty near useless, though. Apparently it takes up a large amount of RAM? Our RAM is limited enough as it is. Some say that it will take at least 32MB of RAM, others say it might need as much as 128MB.
 
I keep hearing that the DSP will be pretty near useless, though. Apparently it takes up a large amount of RAM? Our RAM is limited enough as it is. Some say that it will take at least 32MB of RAM, others say it might need as much as 128MB.
I'd be very surprised if it's impossible to (eventually) make the DSP start/stop at will from kernelland. This would then make it possible to only give up the memory for as long as the DSP is actually in use.


I might be talking out of my back end, though. I've never so much as laid eyes on the OMAP documentation.
 
No one has really had time to dig their teeth in and sort out the options.


At _Worst_ case woudl be a 'pick what you want at boot time' (or 'pick what you want next boot') type menu, to pick the memory impaired kernel or not.


At best maybe it can reduced in size, or memory mapped into space, or something.


Its certainly not useless, just no one has had time to dig into it; any volunteers, step up :)


jeff
 
Its certainly not useless, just no one has had time to dig into it; any volunteers, step up :)


jeff

That's good to hear. From what you're saying it's obvious that we're not forced into choosing a single option (closed drivers/firmware and whatnot). I just really hope that it doesn't take up too many resources. I suppose that if the performance improvements can outweigh any sort of resources taken up, it'll be worth it, but... well, I guess I just don't like what I've heard so far. But I guess that's just a bit of FUD, since we don't really have any experience with it so far.


But back on topic, if there's anything we can do to improve Flash's performance that'd be killer. I can't even find one that works decently. Just that one 10.1, red bar sucky one. :(
 
I keep hearing that the DSP will be pretty near useless, though. Apparently it takes up a large amount of RAM? Our RAM is limited enough as it is. Some say that it will take at least 32MB of RAM, others say it might need as much as 128MB.

One of the available kernel modules requires RAM to be statically allocated at boot.


Another one is more flexible than this, but needs closed source blobs running on the DSP and so is unattractive as well.


All I can say is that until it's a no-cost option present on everyone's Pandora it's going to be hard encouraging anyone to use it. It isn't a very good situation.
 
All I can say is that until it's a no-cost option present on everyone's Pandora it's going to be hard encouraging anyone to use it. It isn't a very good situation.

A no-cost option? Is that even theoretically possible?


*sigh*... Honestly it sounds like no matter what happens with this DSP it'll take quite a while to get any practical use out of it. Oh well. It's still an awesome device, even with its currently-useless DSP.
 
Honestly while its sub-optimal, I don't mind the killing-memory option; sucks, but I reboot often enough .. if I'm planning to watch a long video, then I don't mind booting up the ram-halved kernel and watching it; for gaming, boot the normal kernel. Not a big deal :p


Still, when someone digs into it, hopefully a better option presents itself.


jeff
 
Pipe dream: dynamic DSP memory allocation, 32MB RAM usage, and magical overclocking. Woo!
 
A no-cost option? Is that even theoretically possible?

*sigh*... Honestly it sounds like no matter what happens with this DSP it'll take quite a while to get any practical use out of it. Oh well. It's still an awesome device, even with its currently-useless DSP.

Of course. I just mean something that won't compromise everything else just by being there.


I'd like to use the DSP for something eventually, but I certainly won't need 32MB of memory allocated to it, much less 128MB.


I think the difficulty with the DSP memory mapping is due to it having only some memory mapping, that's not quite compatible with what the CPU's MMU can do (but I can't confirm this precisely). If this is the case then the most preferable option is to allocate physically contiguous memory for the DSP so it doesn't have to do any memory mapping at all. Unfortunately, the only way I know of to guarantee this in Linux is to pull it from memory reserved at kernel boot time. You can ask for it at any time, but if it's not there due to fragmentation you won't get it.


The kernel could get around this by running a defrag routine, or at least to the extent that it can move non-wired pages which I think are pretty minimal these days. I couldn't find a reference to this officially being part of the Linux kernel these days but I might be looking in the wrong places.
 
If I may ask the very simple questions that are probably on the minds of many, why would hardware be sold with advertised features that people can't access? Isn't it like buying a computer with a really slick sound/video card and the manufacturer saying "sorry, no drivers"? Who is using this DSP and in what capacity?


It all feels a bit dumb.
 
Are you sure the DSP needs that big amount of RAM? Any links explaining why?


In the past, I have worked with other TI DSPs for audio processing (filtering, FFT, etc.) and never needed more than a few kilobytes of RAM. I have used a TMS320VC5501 DSP without external RAM and only with 32 KB of internal dual access RAM to code some audio weighting and third octave filters. And I have also used a lot more powerful TMS320C6713 (floating point DSP) with 8 MiB SDRAM to code some more complex audio algorithms. The board I used had 8MiB external SDRAM so I used it, but I could have fit the code+data size into the internal 384 KiB RAM of the DSP.


DSP memory usage tends to be small for most applications because usually DSPs run small algorithms with heavy computational costs. Maybe H264 Full HD video decoding needs a lot of RAM to store the ref frames, but audio decoding, mixing, video filtering, 800x480 H264 video decoding, etc. shouldn't take that much RAM.
 
I know almost nothing about video decoding, but I suppose the largest memory requirement for an H264 decoder is due to the DPB size. I made some numbers and I may be wrong, but DPB size for these profiles is:


4.1 & 4.0: 12 MiB


3.1: 6,6 MiB


3.0 (the one used by PSP): 3 MiB


If I'm not wrong, maybe we could "waste" only 8/16 MB for the DSP to get a good H264 decoder.
 
I think the point is that the DSP maps its address space into a range that is used by physical RAM already .. so the kernel has to be told to mask off that range at boot-time so that the DSP module can use it for shmem-style communication between the DSP and main CPU .. the RAM isn't actually used, its just mapped off.


Seems to me that it would be *possible* to have the kernel module map this address range, and then make it available to apps that want to use it (which aren't going to use the DSP) but this is quite sophisticated and probably would be a bit of work to actually implement ..
 
I think the point is that the DSP maps its address space into a range that is used by physical RAM already .. so the kernel has to be told to mask off that range at boot-time so that the DSP module can use it for shmem-style communication between the DSP and main CPU .. the RAM isn't actually used, its just mapped off.


Seems to me that it would be *possible* to have the kernel module map this address range, and then make it available to apps that want to use it (which aren't going to use the DSP) but this is quite sophisticated and probably would be a bit of work to actually implement ..

I was already supposing that, but... why 32~128 MiB? As I have pointed, and if I'm not wrong, 16 MiB should be enough even for High@4.1 AVC profile (the one used in BluRay movies).
 
Last edited by a moderator:
Unfortunately, so long as the only decode implementations out require lots of RAM and that's the only thing you can use it isn't going to be an easy sell to make a fixed allocation of much less. Maybe a really killer app use case that's < 16MB needs to be done first, but as far as Pandora developers themselves are concerned it's very chicken and egg.


There's also the issue of only letting one application at a time use it. All we need is some insidious background app to claim it to really hose everything.
 
As far as I could tell form the old Flash version it hooks into /dev/dsplink, is that device node DSPLink specific or are other solutions compatible?
 
Maybe a really killer app use case that's < 16MB needs to be done first, but as far as Pandora developers themselves are concerned it's very chicken and egg.

For me a good H264 decoder would be a killer app. A decent flash player would also be a killer app IMHO.


I think the most important point is to give developers the tools to use the DSP. There must be an option to choose the amount of RAM reserved for the DSP even if it's only at boot time or with other tricky methods (using a specially formatted SD card, etc.). When that's done, we will see what devs can or cannot do with the DSP.
 
Last edited by a moderator:
I think the most important point is to give developers the tools to use the DSP. There must be an option to choose the amount of RAM reserved for the DSP even if it's only at boot time or with other tricky methods (using a specially formatted SD card, etc.). When that's done, we will see what devs can or cannot do with the DSP.
The tools are around, it's not like we keep the source only to ourselves. If nobody bothers to even compile the DSP modules it's rather unlikely somebody will write code for DSP (the former is a lot easier then the later).
 
Back
Top