When GPH said they wouldn't be offering a web browser, my guess is that they were talking about the OS they'll be releasing the GP2X with. It doesn't mean that one couldn't be added as a third party program or maybe as part of a replacement/alternative OS.
There's LOADS of software available for linux, but that has nothing to do with the feature list GPH need from the GP2X OS. It seems that they've decided not to support web browsing - that cuts out a whole load of stuff they'll have to get working in their OS, making the development a lot easier. The hardware will still be capable of it later on (That's based on the assumption that the hardware for SDIO will all be wired up from release - I don't know enough about the internals of SDIO, but am hoping).
But yeah - let's wait and see!