Sdio, .pnd Files, And The Dsp

[sixtyfifthbit]

I was wondering; it looks like Pandora's major bottleneck is going to be SD access - not surprising, HD access is the major bottleneck for desktops and most laptops. Most Pandapps are going to be in compressed (or at least solidified) .PND files. Normally decompression of files takes a while; slower than the difference between loading a compressed file and an uncompressed one, making compressed I/O a net sink in processor cycles. However, the Pandora also has a DSP - a processor specifically designed to do things like decompress data. Given that, I think it's possible that loading compressed data, and running it through the DSP, might outpace loading the same data uncompressed.

I'm wondering if anyone on the dev team has done anything regarding implementing decompression on the DSP for .PND files, or if those who know the hardware best can give harder numbers showing that it's feasible or infeasible.

 

[rabidpoobear]

I can answer for you: no one on the team has tried this yet.
video decoding on the DSP is much higher priority, and the source already exists. If anyone were to do any work involving the DSP at this stage, that's what they'd focus on.

 

[NLS]

IIRC the PND system does not compress

 

[torpor]

Its not really relevant in the case of the Pandora, since block reads from the PND file are going to be cached anyway .. and there is no compression to deal with, so it'll be as fast as the system can handle, for the most part.

 

[Creature XL]

the Pandora also has a DSP - a processor specifically designed to do things like decompress data.

Please correct me if I am wrong, this is not true in general.
It can compress and decompress Video and Sound files very good. As thre is some kind of transformation (like discrete cosine transf.). Whichin turn consist of ALOT multiply-accumulate operations and THAT is the main purpose of a DSP.
Further, a DSP is optimised to operate on streamed data, as in you read sequentially from byte 0 to byte N.
In "normal" compression algorithms (like Huffman-based ones) you need to do tree-lookups for every "char" you read from the stream. Its not too bad as I guess that it has quick addressing modes for that as it *could* be used for sin-table lookups.

Last but not least (and for me the most interesting point): It *could* be that the RAM access for the DSP is slower then for the ARM. Anyone knows abou tthat?

I will use the DSP at least for some experiments, however, I will not be the FIRST to do this. I let the other guys figure out how to communicate with the DSP first

 

[EvilDragon]

PNDs can be compressed using SquashFS (which is cool, as the file is as big as it would be when you compress it as ZIP).
It works fine with games like AlephOne (200MB size!).

Haven't tried if it actually is faster compressed or uncompressed, but I can try during the next days.
It doesn't use the DSP to decompress and I doubt it could really be useful for that.

 

[skeezix]

Squashing can slow some things down, but i really depends on the nature of the app and the data (ie: millions of small files, large heavily compressed files, frequency of access; is the app surfing back and forth on the filesystem or just reading up front?)

It tends to work pretty darned well though in my limited testing; I tend to use non-compressed since most things aren't that big anyway (compared to 8GB+ cards

jeff

 

[EvilDragon]

All the PNDs I create are compressed
Exaile is just 1,3MB big

 

[Nupfi]

All the PNDs I create are compressed
Exaile is just 1,3MB big

Nice; now I'm glad I bought a 32GB SD Card

 

[JayFoxRox]

another implementation for compression would be to use a "dropper" which is done in some linux demos. Basicly the first few bytes of the binary are prefixed with a bash script which unpacks the rest of the file to /tmp/ or directly somewhere to memory. I wouldn't know why any homebrew game would be slowed down by the SD cards though as they usually don't have to load much. The RAM is also large enough to hold even larger roms of the more modern systems.

 

[skeezix]

There sno need for 'droppers' (ie: uuencoded apps etc); in that event, you'd just have the sh-script be the executable in the pnd, and then unzip/gzip/bzip the executable in-place (so that it would go to appdata for temp executable):

(note, you could compress the executable and anything you want this way, do selective compression; unpack some files, leave others always unpacked, etc.)

ie:

myrun.sh:

unzip myexecutable.zip
./myexecutable
rm myexecutable

--> that'd do it all in appdata, as Crom intended.

The advantage of squashfs would be it 'just works' -- you in effect use mksquashfs instead of mkisofs (using pnd_make.sh command line option, or an option to ED's GUI tool for Windows, or whatever), and then it just all works nice and compressed... but if the app is seeking back and forth in a heavily compressed file, it could slow things down. But if it works fine for Quake, I figure its pretty good

jeff

 

[Kramy]

I was planning on just using ImgBurn to make pnd files.

To make compressed ones I have to use something like this?

 

[Letalis Sonus]

If it's only the executable that should be compressed, wouldn't UPX be much better for that? arm-linux.elf is in the list of supported formats.