The thing is, the majority of games for it are on the net. If you have net access say at work, but don't have a CD Burner there, you'll be pretty screwed for getting downloaded software onto it. The exception to this is for old system Roms. For example, the C64 disk images are only 180K per side of disk, so you can get a few onto a floppy. But the ADIC games (GP games written for the GP) usually weigh in at around a handful of megs.
If you know how spanning floppy disks works in your favourite compression package (read Winzip for most people) you could get the software home that way.
To get the GP32 set up so it can accept home-brew applications in the first place it does need to be hooked up to the net once.
HTH