I have two issues with my GP2X. The first is that it doesn't have built-in lithium batteries that will charge when plugged in. I've been using two sets of rechargeables (one set charges while the other is in the device), but constantly swapping them is annoying. The rest of my devices (smartphone, PSP, PocketPC, N-gage :blink: ) just charge when I plug 'em in, which fits my lifestyle better.
My only other grievance is with the shoulder buttons. They require noticeably more pressure and leverage than others I've used (Xbox controller, PSP), so I don't use them as much. It's too bad, because it was a great design feature to include them.
Otherwise, I love everything else about the controls, display, weight, size, etc.! And unlike some folks, the joystick controller has never bothered me.
The biggest positive for the device is the strong community support. If you like emulators, you really can't go wrong with the GP2X. And there's a pretty good homebrew selection, too! So you end up paying for the hardware, and getting insane amounts of software for free.