must be some code that can grok the FAT/ext filesystem and the autoboot.txt/boot.scr file before it can boot from SD
Right, baked into the SoC must be some microcode that can access various boot devices, understand MS-DOS partitioning, and FAT just well enough to load the MLO. The MLO probably implements the same functionality to load uBoot, which does all the rest - the files autoboot.txt/boot.scr are configuration for uBoot. So, whatever's inside the SoC is probably quite similar to the MLO, and I think that is only a tiny amount of code (I don't have a Pandora to check).
That code in the SoC is probably not replaceable, or else you'd be able to permanently brick it, and it would be a security issue too. The chip designer might allow signed microcode updates, but that adds a lot of complexity, and I hope they decided it wasn't worth it and didn't bother. It might be one-time programmable in the factory.
The situation with modern laptops, with UEFI and "Secure Boot" - trying to implement a 2000+-page specification - has been a long history of security flaws. Unless you physically open the laptop, attach a clamp to SPI flash and rewrite it, you are otherwise forced to trust UEFI, its security, and that it is not tampered with (since malware could hide itself in there permanently).
In the boot sequence, malware in the early stages can be harder to detect and eliminate, but might be limited it what harm it can do (especially if it is limited in size); malware that loads later on can be larger, more intelligent, and better make sense of data in memory, and do much more harm. So I don't worry much about microcode in the SoC that handles the very start of the boot process.
The brilliant thing with the Pandora is, and with the Pyra will be, being able to completely bypass at boot the internal eMMC, the MLO, uBoot image, kernel, initramfs - and be able to boot trusted ones from an external SD card. It minimises the amount of stuff you *have* to trust, when you first receive the device, or any time you suspect it was tampered with and want to reset it to factory state. This gives me a lot of confidence in its security because I no longer *have* to trust any of those things (a smaller Trusted Computing Base).
But now I am wondering exactly what Notaz reflashed in the 1.73 update that let slower/larger SD cards be used.
I guess that just updated uBoot, on the internal eMMC.
I'd like to produce an SD card image for verifying and/or reflashing the early boot bits on the eMMC, once I eventually get hold of a Pyra. Would be equally useful as either a repair or security tool.
Or a very secure way to use the Pyra might be: to only ever boot from removable SD (by holding START+POWER buttons), as then you could turn the whole internal eMMC into encrypted (and tamper-proof) storage for the OS and your data. I love security, as you can probably tell.