I think the most pertinent question is; how did you install xfce? Did you start from the pyra image with mate and uninstall mate and install xfce? Or did you start from some more basic debian command line and installed xfce from there? If the former you should have uninstalled a package that handles headphone plugging and unplugging, and we may be able to identify it. If the latter it's a little more complicated, and you'd need to look at the dependencies of mate all the way down, and compare that against the packages you did have installed when using xfce. I assume mate isn't coming from the pyra repos, indicating it may well have been patched itself, rather than coming from debian and being the same codebase that's running on rpi and others. Of course an rpi doesn't have to cope with headphone removal events, since it doesn't have onboard speakers, so if you want any sound out it has to go over headphones or maybe the HDMI channel.