Audio Driver (with bounty!)


13.1 ABE Overview
The audio back end (ABE) module is a subsystem that manages various audio and voice uplink and
downlink streams between the initiator
Click to expand...
13.1.1 AE Subsystem
The AE is the core of the ABE subsystem. It performs the real-time audio processing within the ABE:
• Mixing
• Muxing
• Volume control
• Smooth muting
• Sampling rate conversion
• Side-tone equalization
Click to expand...
It sounds like AESS is a small part of ABE. If the ABE sets up how to connect to ram and other parts of the system for AESS to do the work with the sound and output it to speakers, then ABE would need to be set up before AESS?
I don't know how much work is involved in putting those components into ready states. The code might have been consolidated. It might have been missed that you need both parts.
It's amusing that naming is the hardest part of software/hardware development.
-edit-
Good work on the **mobile** development
 
Risca said:
I finally got some time to work on this and I got some news!
View attachment 39456

Before you get too excited: no, I haven't been able to play sound yet :( but I've made some realizations.

The old Linux-3.8 driver actually have 2 audio "platform drivers":
  • omap-pcm.c - introduced in 2008 with support for Nokia N810
  • omap-abe-mmap.c - added in 2012 as a "ABE low power" driver.
ABE (Audio BackEnd) is somehow related to AESS. Maybe one of them is a component of the other? I haven't looked it up

The Letux port of the audio driver only has 1 "component driver" (it has changed name from "platform" to "component"):
  • omap-aess-pcm.c
It looks to be a port of omap-abe-mmap.c. I can't blame anyone for going with the driver that clearly states that it's meant for the ABE/AESS when porting the code and disregard the older driver.

However, it looks like there's a symbiosis between the drivers happening in the old code.

One of the problems I've had on a newer kernel is that I couldn't even begin to play any streams unless I chose the "OMAP ABE Media LP" device. Choosing this device triggers some special branches in the omap-aess-pcm.c code to let the kernel know about the hw constraints of the device. If I chose any other device, that branch was not hit and the kernel refused to play.

I started investigating what code did the same thing on the older kernel, and I found that it was done in 2 places. In omap-abe-mmap.c when using the "OMAP ABE Media LP" device and in omap-pcm.c otherwise.

It's clear to me that parts of the old omap-pcm.c driver has to be ported over to omap-aess-pcm.c, but I don't know what yet.

I'm based off of https://git.goldelico.com/?p=letux-kernel.git;a=shortlog;h=refs/heads/work-aess-noprint-v9 if anyone else want to help out!
I tried using the letux/aess-v13 head, but things have changed in the kernel memory management and I wasted about a day debugging stack corruption. I wouldn't recommend it.

That's it for now. Sorry for the delay
Click to expand...
Just a quick note: I have a 3.15 kernel working on the PandaES. It still has omap-pcm.c but no omap-abe-mmap.c
So this might be a key to get things solved.
 
hns said:
Just a quick note: I have a 3.15 kernel working on the PandaES. It still has omap-pcm.c but no omap-abe-mmap.c
So this might be a key to get things solved.
Click to expand...
Looks like that Linux-3.15 repo is simply missing the two commits that adds omap-abe-mmap.c: https://github.com/Risca/linux/commits/omap5_uevm-3.8/sound/soc/omap/omap-abe-mmap.c
Notably, the second commit is some kind of big bang "Just fix it" commit before release (my interpretation).

However, if you've successfully been able to use the AESS on that branch, it means that omap-abe-mmap.c is not really needed. Mind you, I was not able to load the AESS firmware when I tried running a kernel from there. The firmware format might have changed between Linux-3.8 and Linux-3.15. Unfortunately, I've been unable to build a new firmware based on Linux-3.15 branch :/

The story continues...
 
Risca said:
Looks like that Linux-3.15 repo is simply missing the two commits that adds omap-abe-mmap.c: https://github.com/Risca/linux/commits/omap5_uevm-3.8/sound/soc/omap/omap-abe-mmap.c
Notably, the second commit is some kind of big bang "Just fix it" commit before release (my interpretation).

However, if you've successfully been able to use the AESS on that branch, it means that omap-abe-mmap.c is not really needed. Mind you, I was not able to load the AESS firmware when I tried running a kernel from there. The firmware format might have changed between Linux-3.8 and Linux-3.15. Unfortunately, I've been unable to build a new firmware based on Linux-3.15 branch :/

The story continues...
Click to expand...
Yes, the firmware format must always match the kernel expectations. It appears that you have to run /root/build-aess-fw to build the firmware binary from source. It is included in the omap-audio-3.15 branch. What I don't know is if it works on the OMAP5. Just tested on the PandaES (where I have access to the handsfree speaker through expansion connector J6).
 
After quite a lot of fiddling I at last got the AESS firmware to build for Linux-3.15 today. Wasn't able to play any audio with my Letux-3.15 kernel, but will have to debug another time. Probably messed up my config trying to slim it down too much :oops:

Getting audio working with the Letux-3.15 kernel would bring me a bit closer in time (and code) to latest mainline.

Attaching the firmware here for later reference
 

Attachments

  • omap_aess-adfw-3.15.bin.zip
    15.4 KB · Views: 22
Alright, got AESS running on Letux-3.15 kernel now - it was as I expected: too many drivers disabled to cut down on build/boot time.

According to htop, playing a 96 kHz audio sample consumes around 1-2% cpu :) same with a 44.1 kHz audio sample.

Next step will be to compare this branch to latest Letux kernel
 
Risca said:
Alright, got AESS running on Letux-3.15 kernel now - it was as I expected: too many drivers disabled to cut down on build/boot time.

According to htop, playing a 96 kHz audio sample consumes around 1-2% cpu :) same with a 44.1 kHz audio sample.

Next step will be to compare this branch to latest Letux kernel
Click to expand...
That sounds (pun intended) great :)
I haven't had time to test/look into it recently. May become better in future.

BTW: I have AESS running on 3.15 on the PandaES for quite a long time but didn't find out what really differs in our latest kernels and setups.
So I wish you have more luck (or better eyes). It certainly is just a needle in a haystack.

If you have progress, please let me know for testing.
 
Today I found an old audio driver related project I had been working on during the pandemic...
It's purpose is an attempt to disassemble the AESS firmware (https://git.ti.com/cgit/glsdk/abefw...c?id=eb79849a8c4c50b5f87f5923482162c3dea56147).

I think I had derived from some documentation that it is likely some TMS320C6xx code and tried capstone tms320c64x but I am not sure since the disassembly makes no sense. It could also be tms320c66x: https://www.ti.com/lit/ug/sprugh7/sprugh7.pdf?ts=1678518947564 Then it needs to contribute our own disassembler to capstone.

Here is the code if anyone wants to try.

Bash: 
#!/bin/bash
# use https://www.capstone-engine.org
[ "$(which cstool)" ] || sudo port install capstone || sudo apt-get install capstone-tool # for MacPorts || Debian

[ -r abe_firmware.c ] || wget -O abe_firmware.c https://git.ti.com/cgit/glsdk/abefw-omap4plus/plain/fw/009590/abe_firmware.c?id=eb79849a8c4c50b5f87f5923482162c3dea56147

MODE="$1"
[ "$MODE" ] || MODE=2

function hexify { # process abe firmware file
ADDR=-20    # skip 5 words header (version, pmem+cmem+dmem+smem size)

while read LINE
do
    case "$LINE" in
        */'*' | '*'* )    continue;;    # comment
        '' )    continue;;    # empty
    esac
    HEX="$(echo "$LINE" | cut -d ',' -f 1)"
    if false
    then    # attempt byte swap - has 2144 ERRORs instead of 1634 ERRORs without swap...
        H=$[$HEX]
        HEX=$(printf "%02x%02x%02x%02x" $[HEX&0xff] $[(HEX&0xff00)>>8] $[(HEX&0xff0000)>>16] $[(HEX&0xff000000)>>24])
    fi
    [ $ADDR -ge 0 ] && echo $HEX
#    printf "%08x\n" $[$(echo "$LINE" | cut -d ',' -f 1)]
    ADDR=$[ADDR+4]
done <abe_firmware.c
}

A=0

case $MODE in
    1) # disassemble single lines (for debugging)
        hexify | while read HEX
            do
            cstool tms320c64x "$HEX" 0x$A 2>/dev/null || echo $A: invalid assembly code: $HEX
            B=0x$A A=$(printf "%04x" $[B+4])
            done
        ;;
    2 ) # disassemble full file
        HEX=$(hexify)
        #echo $HEX

        cstool -s tms320c64x "$HEX" $A
        ;;
esac
It is self-contained and downloads the capstone tool and the abe_firmware.c source file (which just contains the binary for #include into an C array).
I hope sharing this helps in any way.
 
Last edited:
Okay, got a bit further debugging this today.

Looks like the older drivers used a "platform driver" called "omap-pcm-audio" but this driver was removed and another, generic driver was introduced called "sdma-pcm". This new driver is not really a standalone driver, but more of a way for other, existing drivers to register as platform drivers... I think. See these commits for a bit more detail:
* https://git.goldelico.com/?p=letux-...ff;h=dde637f2daf19daf7e0d551ef47bec6819504910
* https://git.goldelico.com/?p=letux-...ff;h=31d2f5db92a7a64737e329370b45489ffc686f54
* https://git.goldelico.com/?p=letux-...ff;h=cfb53a56d1024bfa547a55ba599a19f882253003

My main issue right now is that I'm not getting past snd_pcm_hw_constraints_complete() because some driver is not filling in the data we need to get past this step. I tracked it down to the "omap-pcm-audio" platform driver on Linux-3.15. It calls snd_soc_set_runtime_hwparams() with the necessary data. This data now lives in the aforementioned "sdma-pcm" platform driver, or maybe you could say that this data now lives in every driver that is registered using sdma_pcm_platform_register(). I just need to setup the proper DAI links so the data from the "sdma-pcm" driver gets loaded... I think.

It's way too late and I need some sleep now |-)
 
Great that you find a little time for deeper analyses.

I am not sure that this is AESS related at all. IMHO this a driver the McPDM interface between the OMAP4/5 and the external twl6040 audio chip.(TI calls this ABE = Audio Back End).

And that one is working if we do not use AESS. At least my v6.13 based setup has multiple channels for this sound card: a channel 11 (legacy) which goes around AESS and works and a channel 5 which should go through the AESS DSP. This "consumes" the audio stream but there is no sound output.

But what could be is that setting up the McPDM for sending audio streams coming from the AESS DSP must be done differently from sending it in legacy mode.

And then, the rules of snd_soc_set_runtime_hwparams() https://elixir.bootlin.com/linux/v6.13-rc2/source/sound/core/pcm_native.c#L2620 may be too strict.
BTW: the 3.15 version https://elixir.bootlin.com/linux/v3.15/source/sound/core/pcm_native.c#L1941 and it doesn't look to be significantly different.
 
You must log in or register to reply here.

Similar threads

EvilDragon
Information about OS / Kernel / Prototypes
2 3 4
Replies
79
Views
46K
docbroke
docbroke
TrashyMG
Other odd bugs that could use a bounty.
Replies
17
Views
3K
TrashyMG
TrashyMG
Farox
Beta FastTracker 2 Clone
Replies
4
Views
3K
levi
levi
Bosbeetle
Help needed by pyra team
Replies
1
Views
3K
levi
levi
Wally
Pyra Audio Driver - PulseAudio
2
Replies
23
Views
9K
Letalis Sonus
L
Back
Top