Except in cases where the documentation tells you where your files should go (check the Documentation menu in Xfce, or highlight the app in question and press the "Y" game button in Minimenu), it's entirely up to you where to put them.
Personally, in the root of my SD Card I have one directory per system, with the exception of my arcade one, which for MAME 0.106 has subdirectories in it for the various manufacturers whose stuff I own.