Until we know even how to get the m4s to start running it's probably a little premature to start allocating work for them. Although the code written for a driver to periodically run can probably run as well on an M4 if we can figure them out, so there's unlikely to be work lost even if we do switch to using the M4s somewhere down the line.
Personally I'm dubious that the M4s even have access to the GPIOs directly though, so it won't be able to read the VSync signal from the rotation chip. They're probably sat behind a very simple MMU.