So I played a bit with Speed-Dream tonight.
It is quite slow for now. I played a bit with the graphics settings, lowering everything I can, and it seems most time is lost in the textures. I reduce ingame texture from 512 (only settings that doesn't produce white/missing texture) to 64 and even 32, and I have smoother gameplay (at the expence of the graphics of course).
I will probably try to recompile speeddreams with the experimental OSG driver instead of SSG, just to see if it changes something (OSG if OpenSceneGraph, and it's the graphic engine used in FlighGear for example).