Already got a possible system in place. UnionFS seems to be perfect, except that it is rather confusing. It allows you to load a series of .mo objects into the file system as though they were always there. That way you can have the following:
base.mo (base system: mounted under / )
gcc_x86.mo (GCC X86 files: mounted under /usr/)
gcc_arm.mo (GCC for 2X: mounted under /opt/loca/gp2x/)
gp2x_libs.mo (Various Libraries for 2X: mounted under /opt/local/gp2x/)
Then ontop of that you can have a USB memory stick loaded in to handle any changes you make to the filesystem. That way when you load the system up again after making changes they are not lost
Its rather ingenious, but might be difficult to impliment. I am talking with the developers of a few different LiveCD's based on UnionFS for pointers on getting it working. Will let you know
Of course for those that want, you could always just install GP2niX to a HDD and not worry about all this
but where's the fun in that?