I have the same problem, the gp2x works fine without a memory card at all but doesn't get beyond the loading screen when one is in. Seeing as you cannot transfer files to the internal memory without having a memory card inserted as far as I know, my gp2x is as good as useless right now.
I also have a 1gb sandisk card and have tried formatting it correctly multiple times and doing this does not sort anything.