Ah okay... so pretty much what I was assuming. Well, I really don't know if the basis of the stick can be replaced so easily. If you are good at soldering it shouldn't be too hard though.
I don't know if there is any spare sticks for sale. I would ask craigix if he has got some lying around or maybe EvilDragon. Both of them are resellers for the gp2x.
If they don't have any, try the marketplace section of the forum and ask for a broken unit, with the stick intact. Maybe someone can sell you their broken gp2x for a couple of quid and you can use it for spare parts. I have seen people give away their broken gp2x for free so someone else could repair theirs.