D.C. said:
So why can't the x86 architecture compete with ARM in the mobile sector on perf/watt basis? Is it mainly due to architectural differences (e.g. x86's legacy)?
They're actually not extremely far apart right now. It looks worse than it is because most Atoms are on more clock optimized processes and most high-end ARMs are on power optimized ones. Both actually have numbers for both sorts of paths.
The best number we have for power-optimized Atom is Z500 @ 800MHz having a TDP of 0.65W. There are numbers for Lincroft (Z600) but they're higher because that chip includes a GPU. 0.65W for 800MHz is 0.8125 mW/MHz at 45nm. ARM claims something likw 0.31mW/MHz/core for Cortex-A9 at 800MHz on 40nm process. But afaik that's without L2 cache. You also get 0.5/mW/MHz for both cores, so there are definite savings with going dual core even at full power.
It's difficult to say how much of the difference is due to x86, how much is due to more experiencing with low power for ARM, and how much is just inaccurate or misleading.
The biggest disadvantage the instruction set itself imposes is that the instruction encoding space is really complex and even finding the length of an instruction is a decent amount of work. So a lot of burden is placed on fetch and decode which translates into more power consumed to keep up. I've seen some people insist that this doesn't actually add up to much once you hit superscalar and especially OoOE designs. I don't know if I completely believe this. As issue width increases this cost scales up, where others don't necessarily as much (a lot of multi-issue is leveraging execution units you already had). And even if you don't use that much power you still pay in pipeline stages, which makes branch mispredicts more costly and more energy wasting. There are mitigating technologies that can diminish this, like if future low-power Intel CPUs start employing uop caches like in the upcoming Sandy Bridge.
The instruction set itself has a few performance disadvantages (compared to ARM) as well. Low-power Atoms are held back by not running x86-64 code and therefore being more limited in register count. Two address operations increase register pressure and make it harder to schedule operations in an in-order processor. Same with inability to prevent flag setting for ALU operations. Predicated operations on ARM reduce expense of branches over small sections, especially difficult to predict ones. Cortex-A9 being out of order and having a lower pipeline length helps a lot too, of course.
Atom has some advantages too. It can perform read/modify/write operations to memory in a single cycle. It can do 2x 128-bit simple integer SIMD instructions per cycle (Cortex-A8 and A9 can do one). Hyper-threading can help a lot, but is supplanted by the fact that we'll actually see dual core Cortex-A9s in mobile devices where we probably won't see dual core Atoms any time soon.
In theory Cortex-A8 is supposed to have much better L2 cache latency, but in practice it looks like Atom has better (at least vs OMAP3530) and Cortex-A9's is only worse.