Dreamcast does NOT run just fine on Pandora, it's slow as hell, and Gamecube is a lot more powerful than Dreamcast. Frankly, this question is like asking an astrophysicist why going to Alpha Centauri in a year is so unachievable. You should try doing a little more research on your own, at least before casting doubt on this.
Look at how Dolphin performs on computers that are much more powerful than Pandora. It's pretty decently optimized there, so I can tell you right off the bat that you'll have a hard time getting much better performance, if you can at all.
You can divide Dolphin's emulation into three big performance critical sections: CPU, GPU, and DSP (audio). Any decently written Gamecube emulator will probably follow the same pattern. All three of them need some level of CPU resources on the host machine.
To emulate Gecko (Gamecube's CPU) you need a CPU which is much more powerful, because of the overhead involved. In the case of emulating it on ARM that means you have to spill registers because there are fewer on ARM, you have to do something to map the different sort of flags and handle competition between the emulated flags, registers, etc and those needed additionally by the emulator. You have to perform extra work for translating indirect branches, counting cycles, and so on. But maybe the biggest cost is that you have to check all memory accesses to map them to emulated memory, which typically includes the game's attempts to directly access hardware peripherals. You can use the OS/MMU to help you to some extent but usually not 100%, because then you end up with too many expensive traps (which cost hundreds of times more than direct emulation). And somewhere in there you have to check for code modification.
Clock for clock Pandora's CPU is probably not as fast as Gecko, at least not when running the same kind of code. Gecko is mildly out of order with register renaming, has a much smaller pipeline and therefore much smaller branch misprediction penalty, has faster branches in general (because of a BTIC that can hide branch fetch bubbles), has lower latency caches and fewer interlock conditions. And Gecko has full speed double precision floating point while on Pandora it's super slow. Pandora has a little more clock speed to compensate, or a lot more (twice as much) on the 1GHz models. But the real killer is the memory latency. Gamecube is optimized to use a fast pool of 1T SRAM that's really low latency. Pandora's main RAM, on the other hand, is really high latency. There's no way you can make up that difference.
The GPU is a massive hurdle too. Gamecube's GPU Flipper is probably already more powerful than Pandora's SGX530. Add to that both CPU and GPU overhead for texture caching and mapping primitives that aren't directly supported. Even worse, on Pandora the GPU drivers need a ton of CPU time handling scenarios that are very inexpensive on consoles (lots of state changes). They tend too much CPU time regardless. This takes a big hit on N64 emulation and it'll probably take a bigger hit here. Worse than on Dreamcast emulation, because Dreamcast games have to prepare an entire scene in advance and are a closer fit to Pandora's GPU since they have the same lineage.
DSP needs more CPU time yet, especially if using the LLE emulation which is necessary to get a lot of games sounding right.
You might ask why someone doesn't port Dolphin anyway. The reason is because there's a ton of platform specific work that hasn't been done, in order to get the best performance - and the difference between a direct port (assuming it even has an interpreter and soft renderer) and an optimized one would probably be at least an order of magnitude, so you'd most likely be looking at below 1FPS. That's not worth anyone's time even if that port just takes a few days. But the work needed to bring it to optimization parity with the PC versions would at least require an ARM recompiler (probably targeting NEON for paired singles) and GPU emulation targeting OpenGL ES 2 and all its limitations vs whatever the current plugins can use. This is a huge amount of work. It could require hundreds or even thousands of man hours to develop and test.