From what I understand the main limiting factor for SD card speeds is the controller in the SD card. This controller must do wear leveling for the flash chips inside. All the while, the controller needs provide the block device abstraction to whatever the SD card is inserted into.
The controller is a processor, with a program that executes the above functions. So you can imagine that some controller chips might be faster or the software they run might be better optimized. This is why SD card speeds varies.
If I'm not mistaken, within the same SD spec, the electrical interface (including clock speed) is the same. But in a new SD spec that clock speed can change. Devices not designed to handle those speeds will then be incompatible.
I'm guessing that so far the clock speed has remained the same and SD card controllers have been playing catch up. But now (with SD 4.0) they are increasing the clock speed, so that the controllers can continue improving.