The final release (at least by me) of the gp2x-gpe, a port of the GPE palmtop environment to the GP2X is out! This release includes pretty much everything except the kitchen sink. AbiWord, Evince, Gaim, Dillo, Gnumeric, XChat, The GIMP, GpsDrive, GCC are included along with all the standard GPE applications.
The internet is not a program, so no, it doesn't have it. If you are networked through USB then XChat (irc), Dillo (browser), and GAIM (instant messenger client) should work.
Its slow as hell unless you install it on an ext2 formated SD card and make the nessaccary changes to the .gpu file along with wirteing a script to mount that partition (that info can be found on the board).
In my opinion, its still a little slow when using it on a ext2 SD card. I think it is a direct port from another handheld, just recomplied for the GP2X. I suspect this other handheld had more RAM and a faster processor.
But once I get my wireless internet working, I may change my overall view of GPE because I'll have internet connectivity then. We shall see.
As for new releases, I dont know of anything. It would be nice though if someone went through and optimized the code for the GP2X. Maybe post in the "Cool ideas..." forum is in order.