The worst part of Raspberry is not only uese a lot o blobs, even a blob to boot, even worst is that closed OS for GPU is always on, it has real control (and its backdoors plus bugs) and highest priority, and the user OS you load after it is on lower priority. So you have a OS controlling your machine and you don't control that OS, even you know nothing about it (this also occurs on modern PC x86 CPUs).
It's almost the same. When a SOC needs a blob to boot, the code in that blob is in a position to stay resident, keep control on the system, and in some way set up a virtual machine for the rest of the system to work in.
I don't mean that they always do, just that they always could.
The details vary depending on the architecture and for simple microcontrollers what I'm saying could be false, but as a general rule whoever boots first can keep the control, and can decide what payload is allowed to boot next.
Then the SOC may have more or less infrastructure to keep a "secure world" separate and invisible from the OS and applications, or have it rougher and simply brick the device if it doesn't like what you installed,
but at the very least it has always the decision on what can boot next.
If that boot code is in a ROM it can still be bad, but at least it cannot be changed to worse once you have the system installed. If it is loaded from storage then you have to keep trusting the provider than any update really fixes a
vulnerability instead of introducing one or adding antifeatures. And trust than the blob won't update itself.
The boot chain is a control chain, which can be used (plus memory protection) to set up a trust chain. Whoever comes first rules over the next code.
I don't need to know that the OS in the GPU is controlling the system. The simple fact that I can't understand or change what it does when it boots is enough to worry that it could be so.
Now in the case of the Raspberry, I'm not sure whether the code for the GPU that boots first could be changed and the problem is "only" lack of manpower and documentation, or it is signed and the GPU won't boot anything you would give it if you could write it.
The later is worse, but both are bad enough.