Audio Driver (with bounty!)


Risca said:
I have a really hard time to grasp how the whole audio topology works, and where all the pieces come from :/
Click to expand...
Same for me... This is the reason why this is not a stroll but climbing Mt. Fuji :) And why it takes that long.
I just have started my PandaES to check how the working setup was built. And it turned out that v6.5-rc2 changed the device tree subdirectory structure (to the better) but that is not handled properly by my build scripts.
Why do I work on PandaES? Because it has one of the handsfree speaker drivers wired to a jumper. And it is easier to have a remote controllable setup than the Pyra. And, I have (had?) a working 3.15 AESS setup - but did not yet find the SD card.
 
hns said:
Same for me... This is the reason why this is not a stroll but climbing Mt. Fuji :) And why it takes that long.
Click to expand...
I see what you did there :D
hns said:
I just have started my PandaES to check how the working setup was built. And it turned out that v6.5-rc2 changed the device tree subdirectory structure (to the better) but that is not handled properly by my build scripts.

Why do I work on PandaES? Because it has one of the handsfree speaker drivers wired to a jumper. And it is easier to have a remote controllable setup than the Pyra. And, I have (had?) a working 3.15 AESS setup - but did not yet find the SD card.
Click to expand...
I got sound from the legacy interface with a Letux kernel and device tree I received from @hns. I still get "Invalid argument" when trying the AESS sound path, so not much progress. I'll try and replicate the kernel build in my own build environment.

On a related topic, I found this talk today:

They present a tool for producing firmware files for the Linux audio SoC subsystem. The tool is called "alsatplg" (ALSA topology) and is part of the official ALSA project. It comes bundled with alsa-utils on any modern Linux distro. The tool takes plaintext configuration files (and blobs) and produces a firmware file to be loaded by the Linux kernel. Support for the firmware format was merged in Linux-4.2, and considered stable in Linux-4.8. The current firmware ABI version is 5, but ABI version 4 is also compatible.

What's especially interesting in the talk is that they mention that the tool was initially developed by Liam Girdwood when he was working for TI and the OMAP processors. He later joined Intel and continued working on it. Liam Girdwood is also the name you find in most commits for the AESS firmware repositories. The official AESS firmware, released by TI for Linux-3.8, has firmware ABI version 1.

@hns has spent a considerable amount of time to update the AESS firmware repositories to produce an AESS firmware that can be loaded by mainline Linux. The only way to produce it is by compiling the tools and binary on a compatible board itself, and this compilation breaks from time to time when kernel headers change. Apart from that, the code for generating the firmware is quite bad and hard to understand IMHO. This is not really a good situation.

So, an idea took form: since the firmware @hns have converted can successfully be loaded by Linux, it should be compatible with firmware ABI version 4 or 5. What if we can use the alsatplg tool to decode it back into plaintext configuration files (and blobs)? Then we would have the firmware information in a much more readable format and a well-maintained tool for compiling it! Unfortunately, it wasn't that easy :(

Code: 
$ alsatplg -d omap_aess-adfw.bin -o omap_aess-adfw.bin.decoded
ALSA lib decoder.c:98:(snd_tplg_decode) first block must be manifest (value 3)
failed to decode context omap_aess-adfw.bin: Invalid argument

The manifest block is actually described in the video above and shouldn't be that hard to inject into the binary, but I haven't tried yet.
Anyone feeling adventurous could have a try! :D

Since we already have a stable firmware binary, I will not focus that much on it for now. I'll focus on getting AESS support working (unless @hns beats me to it)
 
Great idea and good find for the alsatplg utility...really hope we find a way to finally enable and use the AESS.
Unfortunately on Pyra Alsautils (and maybe Alsa too) is too old and don't have the -d option to decompile a firmware.

On a side note...today i tested again the firmware you compiled on the bookworm image ( i know that kernel was not able to use...but still i tried)...an i noted a strange thing....when loaded the USB ports were disabled...and i noted this because the same thing happened on standard pyra os.
Maybe this are unrelated...but who knows.
 
Risca said:
What's especially interesting in the talk is that they mention that the tool was initially developed by Liam Girdwood when he was working for TI and the OMAP processors. He later joined Intel and continued working on it. Liam Girdwood is also the name you find in most commits for the AESS firmware repositories. The official AESS firmware, released by TI for Linux-3.8, has firmware ABI version 1.
Click to expand...
Maybe you can get some help or pointings to the right direction from Liam Girdwood himself. Who knows, it costs nothing to ask... Maybe he's even pleased by your efforts...
 
Farox said:
Great idea and good find for the alsatplg utility...really hope we find a way to finally enable and use the AESS.
Unfortunately on Pyra Alsautils (and maybe Alsa too) is too old and don't have the -d option to decompile a firmware.
Click to expand...
You don't have to decode the firmware on the Pyra. Any OS which can run alsatplg may do it. I tried it on Arch and got the error posted above.
Farox said:
On a side note...today i tested again the firmware you compiled on the bookworm image ( i know that kernel was not able to use...but still i tried)...an i noted a strange thing....when loaded the USB ports were disabled...and i noted this because the same thing happened on standard pyra os.
Maybe this are unrelated...but who knows.
Click to expand...
Strange indeed :confused:
 
Risca said:
Farox said:
Great idea and good find for the alsatplg utility...really hope we find a way to finally enable and use the AESS.
Unfortunately on Pyra Alsautils (and maybe Alsa too) is too old and don't have the -d option to decompile a firmware.
Click to expand...
You don't have to decode the firmware on the Pyra. Any OS which can run alsatplg may do it. I tried it on Arch and got the error posted above.
Click to expand...
Decoding is only needed once in life of mankind either... As soon as it has been converted into a .conf file only the compile mode is needed to build the firmware from scratch. And there is nothing wrong using the binary format of the firmware.
Anyways this topology file conversion is not the road block. It is "cosmetics" to make future life easier.
The real roadblock is that the AES DSP isn't running properly yet. Hence we can use only the legacy mode (with the existing firmware files).
 
Finally got some work done on this today, and I might have found the reason the AESS firmware is not running as it should!

You see, early during firmware loading we do this:
C: 
static int aess_load_fw(struct snd_soc_component *component,
                        struct snd_soc_tplg_hdr *hdr)
{
        struct omap_aess *aess = snd_soc_component_get_drvdata(component);
        const void *fw_data = snd_soc_tplg_get_data(hdr);

        /* get firmware and coefficients header info */
        memcpy(&aess->fw_hdr, fw_data, sizeof(struct fw_header));
...
        /* store AESS firmware for later context restore */
        aess->fw_data = fw_data;

        return 0;
}
Notice how aess->fw_data is pointing to the start of a struct fw_header.

Later, we do this:
C: 
/**
 * omap_aess_load_fw_param - Load the AESS FW inside AESS memories
 * @aess: Pointer on aess handle
 * @data: Pointer on the AESS firmware (after the header)
 *
 * Load the different AESS memories PMEM/DMEM/SMEM/DMEM
 */
static void omap_aess_load_fw_param(struct omap_aess *aess)
{
...
        pmem_ptr = (u32 *) aess->fw_data;
...
        omap_aess_write(aess, OMAP_AESS_BANK_PMEM, 0, pmem_ptr, pmem_size);
...
}
We're writing the fw header straight into PMEM memory bank! :mad: There's even an outdated comment about a missing data parameter to this function where they specifically mention "after the header".

The fix is easy:
Diff: 
diff --git a/sound/soc/ti/aess/omap-aess-fw.c b/sound/soc/ti/aess/omap-aess-fw.c
index 8daff9fb1488..48cd6d7e3d33 100644
--- a/sound/soc/ti/aess/omap-aess-fw.c
+++ b/sound/soc/ti/aess/omap-aess-fw.c
@@ -658,7 +658,7 @@ static int aess_load_fw(struct snd_soc_component *component,
        dev_info(aess->dev, "AESS Firmware version %x\n", aess->fw_hdr.version);
 
        /* store AESS firmware for later context restore */
-       aess->fw_data = fw_data;
+       aess->fw_data = fw_data + sizeof(struct fw_header);
 
        return 0;
 }

With this patch, I can dump pmem from /sys/kernel/debug/omap-aess/pmem however much I like and I don't get any segfaults any more!
I'll send the patch to the Letux and Pyra kernel mailing list, once I get my mail client setup again. :oops:

Still can't play any sound though

EDIT:
I can confirm that PMEM dumped from Linux-3.8 and my latest kernel are identical now with this patch. Reading PMEM from /sys/kernel/debug/omap-aess/pmem reads it directly from the AESS memory banks.
 
Last edited:
I've sent a patch to the Letux mailing list and notified @hns about the findings. He sounded very excited in his reply :D He was gonna do some testing on his PandaES asap. The PandaES is a similar architecture to the Pyra.

I've also pushed the code I used for debugging this issue: https://github.com/Risca/dump_pmem
It's a kernel module with all the boiler plate needed to load the AESS firmware, parse it, write the DSP bits to the DSP, then dump the PMEM memory area.

It was when I was putting together all the pieces in one, single place when I could grasp the whole picture and realize what was going on.
 
Yes, it works for me as well. Congratulations to Risca to have found the reason for this issue.

The fix I will add is a little different. It will be in omap_aess_load_fw_param() and not aess_load_fw().
The reason is that the code in omap_aess_load_fw_param() is exactly the same as in the 3.15 kernel but aess_load_fw() had been reworked and obviously a bug was introduced (by me).
So it will be fixed where it was really wrong: https://git.goldelico.com/?p=letux-...ff;h=52383c444c44db67b991dbe9c35aac8774fba6f7

With the DSP obviously running now, we can use the ABE tools for further tests: https://github.com/omap-audio/audio-test/tree/master/tools/abe
Especially abe_check now reports that buffers are exchanged between the Linux system on omap4/5 core and the AE DSP.

Great progress and kudos to Risca!
 
I finally got some time to work on this and I got some news!
I do some of my best coding while on a train


Before you get too excited: no, I haven't been able to play sound yet :( but I've made some realizations.

The old Linux-3.8 driver actually have 2 audio "platform drivers":
  • omap-pcm.c - introduced in 2008 with support for Nokia N810
  • omap-abe-mmap.c - added in 2012 as a "ABE low power" driver.
ABE (Audio BackEnd) is somehow related to AESS. Maybe one of them is a component of the other? I haven't looked it up

The Letux port of the audio driver only has 1 "component driver" (it has changed name from "platform" to "component"):
  • omap-aess-pcm.c
It looks to be a port of omap-abe-mmap.c. I can't blame anyone for going with the driver that clearly states that it's meant for the ABE/AESS when porting the code and disregard the older driver.

However, it looks like there's a symbiosis between the drivers happening in the old code.

One of the problems I've had on a newer kernel is that I couldn't even begin to play any streams unless I chose the "OMAP ABE Media LP" device. Choosing this device triggers some special branches in the omap-aess-pcm.c code to let the kernel know about the hw constraints of the device. If I chose any other device, that branch was not hit and the kernel refused to play.

I started investigating what code did the same thing on the older kernel, and I found that it was done in 2 places. In omap-abe-mmap.c when using the "OMAP ABE Media LP" device and in omap-pcm.c otherwise.

It's clear to me that parts of the old omap-pcm.c driver has to be ported over to omap-aess-pcm.c, but I don't know what yet.

I'm based off of https://git.goldelico.com/?p=letux-kernel.git;a=shortlog;h=refs/heads/work-aess-noprint-v9 if anyone else want to help out!
I tried using the letux/aess-v13 head, but things have changed in the kernel memory management and I wasted about a day debugging stack corruption. I wouldn't recommend it.

That's it for now. Sorry for the delay
 
You must log in or register to reply here.

Similar threads

EvilDragon
Information about OS / Kernel / Prototypes
2 3 4
Replies
79
Views
45K
docbroke
docbroke
TrashyMG
Other odd bugs that could use a bounty.
Replies
17
Views
3K
TrashyMG
TrashyMG
Farox
Beta FastTracker 2 Clone
Replies
4
Views
2K
levi
levi
Bosbeetle
Help needed by pyra team
Replies
1
Views
3K
levi
levi
Wally
Pyra Audio Driver - PulseAudio
2
Replies
23
Views
9K
Letalis Sonus
L
Back
Top