GP2X Feasibility Of Mpeg4 Audio Decoding

[Epicenter]

Pardon my newbishness to the MPEG4 format, but I was exploring today the options of audio formats to decode in realtime running a game. MP3 is much less 'open' and bears licensing issues, and carries a heavy CPU toll- even with a fixed-point decoder lib. Same for OGG Vorbis, although it is at least entirely open. I'm seeing a 40 FPS hit with audio decoding using either of those two. While a remaining 84 FPS is still nice, I absolutely need 90 FPS+ for my purpose, and will be rendering more than this, so this sort of performance is just unacceptable.

I'm considering some new ideas, like 22khz or 44khz ADPCM, but these are fairly large files, even though they seem to carry no major CPU cycle toll to use. This brings into play more substantial load time, RAM consumption and of course SD card space constraints.

An odd idea hit me-- in my list of possible conversions in the application I was using to test my source material in different formats, I found some MPEG-4 audio solutions. This brings me to wonder, does the MPEG4 format contain some sort of support for audio tracks, or does it merely expect to contain video frames and be laid side-by-side with AC3 or AAC et cetera in a container?

The reason I ask is, if MPEG4 contains support for audio already, perhaps there is the possibility of utilizing the MMSP2's built-in MPEG4 decoding hardware to offload the hefty task of audio decoding from the ARM920T. Unlikely I suppose since it is probably specially geared to work only on the low-level common math tasks for MPEG4 codecs in the aid of decoding video frames. However, it never hurts to ask-- has this been considered and/or explored? Is documentation and understanding of the MPEG4 functionality of the MMSP2 still too poor?

Thanks for any information.

 

[Shikaku]

I don't think the video processor processes audio...

 

[greenmikey]

I don't doubt with full documentation we could do it..but magic eyes are a bit too secretive for us. We probably could use tricks to offload lots of little things on to it...some day someone might do something cool with it.

 

[Epicenter]

I don't think the video processor processes audio...

Right now it doesn't. The question is, could it be made to.

 

[BradN]

I'm not sure there is such a thing as mpeg4 audio, but I wouldn't be surprised if magiceyes slipped some mp2/3/aac decoding assist hardware in there, especially considering the target market. This could give a big boost to video and audio playback time if it exists (and is utilized) as well.

We know (well, at least I think I remember) they have video encoding hardware, why would they stop there and not do audio encoding? If they do audio encoding, why not decoding?

Also, I can't fathom why magiceyes wouldn't want to give out tech docs to gp2x developers - it makes a great demo platform for their chip, and the better it performs, the better their chip looks.

 

[Epicenter]

Because they can charge companies like GPH who are buying their chips because they are on a tight budget, to buy custom-made shitty firmware from them that uses the hardware's true abilities, while keeping how it actually works shielded to secure a stream of constant revenue, from what I hear.

 

[Squidge]

I don't think the video processor processes audio...

Right now it doesn't. The question is, could it be made to.

Yes

 

[TheFr00n]

Hmm. I see where you're going with this, and it is an interesting idea. However, I gotta point out that the *.mp4 format is simply a file container, similar to other file containers such as *.avi and *ogm. It is a box that contains a video and an audio stream.

Therefore, even if an *.mp4 file contains new, proprietary audio streams, they wouldn't be decoded by the MMSP2. Ultimately, the term MP4 refers (AFAIK) to video only.

That said, if you could find a way to use the MMSP2 to decode the audio, might well result in the performance boost you want.

But I have a question: As DivX and XviD are implementations of the MP4 video standard, wouldn't asking the MMSP2 to decode the audio as well actually result in a performance hit, since it's already processing video?

 

[doc5avage]

Isn't Apple's AAC files an Mpeg4 variant?

 

[BradN]

But I have a question: As DivX and XviD are implementations of the MP4 video standard, wouldn't asking the MMSP2 to decode the audio as well actually result in a performance hit, since it's already processing video?

If there is audio decoding circuitry, it's almost certainly separate than the video stuff. That's the nice thing about chips with special purpose hardware - it can usually be used simultaneously with other things, at least if it's well designed. In the case where it'd be doing both (video and audio decoding) at the same time, it might not make a huge difference performance wise, but it'll take some load off the processor and might allow running at a lower clock rate -> better battery life.

Where the performance side comes into play is in games where you want background music... it's possible the decoding hardware is more or less efficient with memory access (so it might be better or worse performance wise than decoding audio on the 940, as far as impacting the 920's access to memory). If you had another job for the 940 anyway (rendering, etc), then it would almost certainly be a win.

 

[PokeParadox]

Isn't Apple's AAC files an Mpeg4 variant?

AAC is the proposed standard audio codec for MP4 streams.
Xvid and DivX are mpeg4 complient video implementations.

MP4, the container can contain mp3 or AAC audio (I think also AC3) + mpeg4 complient video(divX,xvid,3ivx etc...) + subtitles + chapter info.

 

[greenmikey]

I don't think the video processor processes audio...

Right now it doesn't. The question is, could it be made to.

Yes

survey says squidge *might* have a card up his sleeve or is workin on something...*cough*snes*cough*

 

[doc5avage]

Isn't Apple's AAC files an Mpeg4 variant?

AAC is the proposed standard audio codec for MP4 streams.
Xvid and DivX are mpeg4 complient video implementations.

MP4, the container can contain mp3 or AAC audio (I think also AC3) + mpeg4 complient video(divX,xvid,3ivx etc...) + subtitles + chapter info.

my bad

 

[Epicenter]

Where the performance side comes into play is in games where you want background music... it's possible the decoding hardware is more or less efficient with memory access (so it might be better or worse performance wise than decoding audio on the 940, as far as impacting the 920's access to memory). If you had another job for the 940 anyway (rendering, etc), then it would almost certainly be a win.

Well, I tried OGG decoding on the 940T with Dzz's modified libtremor for it. But it dropped my game's framerate to HALF and did the same to someone else's. So I think they are competing for memory too much...

 

[BradN]

Personally, I bet hardware audio decoding would be a lot easier on memory access than through software - all the lookup tables are probably built into the chip, etc...

 

[Epicenter]

For the time being, I suppose I'll just use PCM. I wonder if there is something smaller than ADPCM that doesn't sound awful though ...