Yep, depending on the program, you need to quite work around a few bits.
Chromium, for example, doesn't like the the SocketLock file on a FAT32 (permission problem). So I couldn't simply move the Chromium settings directory to the SD Card - Chromium wouldn't run.
Instead, I made a script. Now the settings directory is moved to /tmp/chromium. Chromium will create the SocketLock file in there and it works (as it has Linux permissions).
Inside that directory, I made a symbolic link to the profile-directory on the SD-Card within appdata...
When you close Chromium, the dirs will be removed from /tmp again.
So yeah - depending on what the programs want, it needs to be some smart scripts to make them work