In my opinion, the lack of integrated storage makes this a non-starter for going up against other microcontrollers for most applications. The small amount of integrated SRAM and lack of external parallel memory interface makes things even worse. For many applicatoins you'll have a hard time fitting your program and data on the SRAM without having to regularly (and slowly) pull data over an external serial flash. The 320MHz one is especially hard to understand, I don't know what applications can benefit from that performance but are okay with only 16KB of code + data space. It's good that they're very upfront about this limitation and are talking a lot about their forward plans to resolve this. But given this situation they should have included much more SRAM.
I also have to say that I really hate their feature comparison table which fails to list common peripherals that they don't provide. And I've always found RISC-V's uA/MHz comparisons to be very misleading because they don't include static power consumption and are comparing against microcontrollers tuned for low static and standby power consumption at the expense of poorer dynamic power. The chip selection they used to compare the HiFive with is especially mind boggling, they should have went with Cortex-A5s or ARM9s or PIC32 or something.
