I used to work at Software Etc. (a mall software / video game / limited hardware store).
I worked the launch for the Xbox, and while some people were annoyed at the size of the controler, most liked it AFTER they gave it a chance.
I remember a huge guy that came in and complained about how bad the controler was compared to the ps2 controler. I asked him if he had an xbox, and he said he didn't want one because of how stupid and large it was (I tried very hard not to point out the irony in that statement to him). I handed an original xbox controler him to hold, and his fingers almost overlapped on the backside. After which he grudgingly agreed that it may be ok for large guys.
In my honest opinion, the original xbox controler was correctly made for its target audience (16+ males) and perhaps was not as useful to 12 year old girls with midget fingers as the playstation controler was.