That does sound really slow. Unless you're transferring 10+GB every time onto a really bad Class 2 SD card: that's the only way I can see it taking hours.
You may want to try a different USB cable, and check to make sure the port you've plugged it into on your computer is USB2.0, not USB1.1. My laptop has that problem, one USB2.0 port and one USB1.1 port so if you plug it into the wrong one it goes slow.