Gala is the name of the WM, its compositing features are based on libmutter, but I believe it can run without them, although I haven't tested it. But if not - it's Linux, you can just swap Gala for whatever you wish.
Still, compositing in Pantheon/Gala is done very conservatively, nothing...