The problem is that alsa/pulseaudio sees the channel mapping as :
Channel Map: front-left,front-center,front-right,rear-center
Instead of the 4.0 setup it should have by default (which would be front-left,front-right,rear-left,rear-right)
I tried forcing this in several places in the pulseaudio...