Raid Across The Sd Cards, Potential For Speedup?

[azmodean]

I was wondering if the hardware gurus that we have on the forum could weigh in on this. Is there potential for a RAID 0 or 1 array built on the SD cards to increase read performance?

There are lots of variables to this that I don't know about:
1. Overhead, will the cpu (or DSP?) cycles needed to do this outweigh the potential speedup?
2. Payoff, is the potential speedup worthwhile?
2a. Bottleneck, is the read from the SD cards a bottleneck in the first place?
2b. Other Bottleneck, is there a bus or controller bottleneck such that RAID won't give a (or enough) speedup?
3. Implementation, how hard would it be to do this?
4. hybrids
4a. 0 vs 1, 0 gets you read speedup and redundancy, but 1 (might?) get you more speedup as well as write speedup.
4b. some kind of system where part of the card is RAIDed and part isn't, so you might have a performance area of the SD cards set aside to put games, the kernel, libraries, etc that get the speedup, but another part of the drive that is JBOD style so you get to use all the storage space available.

And I'm sure there are other issues I'm missing, just wondering if it would even be worth looking into more.

 

[Lineus]

I'm far from an expert, but I believe that the speed of the SD card itself (class I, II, etc.) would determine if there was any speedup when using a RAID array - but I could be wrong.

I like idea 4b, though
~Lineus

 

[Svartalf]

I was wondering if the hardware gurus that we have on the forum could weigh in on this. Is there potential for a RAID 0 or 1 array built on the SD cards to increase read performance?

Raid 0's a bad idea, but will provide a speedup of less than a factor of two because of striping- but at the expense of robustness (blow one of the SD's and the whole thing's toast, obviously...).

Raid 1's "okay" (for relatively small values of it...) but you'll degrade performance in mirroring where you wind up making the CPU and SD controller write the data not once, but twice.

RAID's only useful for integrity and speed when you consider something like Raid 5, where you've got a parity bit and striping- and to be truthful, it's only of limited value in something like this handheld.

 

[Kramy]

But doesn't the Pandora have two SD controllers/slots? RAID-0 should be possible with a fairly minimal overhead increase? The CPU overhead would scale similarly to the increased data transfer speed, I would think, assuming the controllers are the chips that handle all communication with the card.

I would think the battery life implications(~5% lower?) would be more severe than the CPU overhead increase. But maybe I'm wrong.

For RAID-1, the CPU would definitely take a hit, and don't even bother with RAID-5

 

[Megamixman]

RAID-0 overhead could be a lot more then you think. You're effectively forcing addressing of flash to the software level; where SoC's usually have flash interfaces and hardware level addressing, which the Pandora is using.
I can't see a huge need for RAID on Pandora. Good RAM caching is likely to be more effective where needed, such as games. On a mobile platform, that level of data throughput is generally not needed, and when rarely needed, it can be replaced with RAM caching.

 

[Pleng]

I was wondering if the hardware gurus that we have on the forum could weigh in on this. Is there potential for a RAID 0 or 1 array built on the SD cards to increase read performance?

There are lots of variables to this that I don't know about:
1. Overhead, will the cpu (or DSP?) cycles needed to do this outweigh the potential speedup?
2. Payoff, is the potential speedup worthwhile?
2a. Bottleneck, is the read from the SD cards a bottleneck in the first place?
2b. Other Bottleneck, is there a bus or controller bottleneck such that RAID won't give a (or enough) speedup?
3. Implementation, how hard would it be to do this?
4. hybrids
4a. 0 vs 1, 0 gets you read speedup and redundancy, but 1 (might?) get you more speedup as well as write speedup.
4b. some kind of system where part of the card is RAIDed and part isn't, so you might have a performance area of the SD cards set aside to put games, the kernel, libraries, etc that get the speedup, but another part of the drive that is JBOD style so you get to use all the storage space available.

And I'm sure there are other issues I'm missing, just wondering if it would even be worth looking into more.

What do you hope to achieve with such a setup?

Are you going to be spending *that* long loading apps as opposed to using them? Or do you have an app in mind which will make heavy use of the disks?

 

[Sphinxter]

That would be even slower.

 

[azmodean]

My primary thought with this was load times, not continuous drive accesses.

 

[Exophase]

Raid 0's a bad idea, but will provide a speedup of less than a factor of two because of striping- but at the expense of robustness (blow one of the SD's and the whole thing's toast, obviously...).

I don't think that the integrity is an issue in the first place. People aren't going to be using their SD cards to back things up (or at least they shouldn't be). The topic was made asking if speed could be improved. If given an option, I don't really see a problem with requiring consistency between the two cards, and if it goes bad you lose the sector/file. At least with SD cards you're probably much less likely to see a failure of the entire thing like with a harddrive.

 

[azmodean]

My thoughts overall:
write speed: don't care, I'm not doing anything that writes significant amounts of data
goal: decreased startup time (both apps and the system as a whole)
raid 0 vs raid 1:
raid 0 benefits: theoretically better read speed
raid 1 benefits: SD cards I could actually use outside the raid configuration

What I don't know is the technical feasibility, which has only been addressed slightly with the mention that the SOC probably has most of the SD accesses handled in hardware, and doing any kind of RAID would require all of that to be done in software in addition to the raid stuff itself, which is of course a major problem with it.

Overall looks like most think it wouldn't help all that much, but I'm not seeing much certainty.
We'll see, seems pretty unlikely to be feasible at this point, but I'll keep looking.

 

[Blah]

I have my doubts as to whether raid 0 style SD access would be any faster, but its an exciting proposition. As for robustness, I'd give that up for speed in some cases. Just keep a mirror on your PC (should be easy enough to do with rsync or similar).

 

[Svartalf]

Sigh... Let's try going through this again, shall we?

RAID 0 is about shedding seek times and spooling data off the disk.

With SD's you don't have seek times compared to HD's.
The data rates off the modern generation of SDHC's is right there with HD's of a hardware generation ago.
The data rates off of premium non HC SD's is in the same ballpark.
This is something on the order of 9-25MB/s.
At the top-end, with ONE SD, this is more than the Pandora is likely to deal with.

Even if it weren't, the best you're going to see, because you're making the Pandora's CPU do the read/write work twice, is 2/3rds more speed than the best your SD's will do. Just because you have two of them, does not go into making it twice as fast. No hardware to make it nearly twice as fast, the CPU not being largely able to utilize the boost if it were there, the only reason you'd do this is because you're trying to be "cool".

As for doing it, as long as you configure the kernel to support Linux Multidisk (Software RAID) and configure the thing to use the two identically sized partitions you've reserved for the RAID 0/1 configuration, it'll just work.
It just won't work the way you guys think it will. It doesn't work the way people typically think it does- mostly because they read up on what the hardware RAID offers and then think they'll get it with the software versions.

 

[azmodean]

Ah, now that's pretty key, I didn't know that the data rates were up that high, and I hadn't looked into RAID enough to realize that striping was mostly to avoid seek time delays. Thanks for the info.

 

[jhoff80]

What about the possibility of spanning the storage across the two SD slots so they're seen as one drive?

It'd be very helpful for multimedia purposes, instead of keeping track which card has more space, just copy songs and video across to both cards, in addition to being able to plays songs from both cards in the same queue.

 

[Vladsharp]

I'm not sure about the performance penalty (if significant) of LVM2, but what you suggest is certainly doable.

What filesystem is everyone considering putting on the SDHCs? I assume FAT12 because that's what they come formatted with, but perhaps someone has some benchmarks for which journaling FSs use the smallest % of cycles?

 

[sindbad]

I'm not sure about the performance penalty (if significant) of LVM2, but what you suggest is certainly doable.

What filesystem is everyone considering putting on the SDHCs? I assume FAT12 because that's what they come formatted with, but perhaps someone has some benchmarks for which journaling FSs use the smallest % of cycles?

jffs2 or yaffs. But I doubt there are any windows drivers for those.

 

[Exophase]

Sigh... Let's try going through this again, shall we?

RAID 0 is about shedding seek times and spooling data off the disk.

I don't know what you mean by "spooling data", but it improves bandwidth as effectively as it improves latency (if the latency is a function of access distance as it is for hard drives)

Even if it weren't, the best you're going to see, because you're making the Pandora's CPU do the read/write work twice, is 2/3rds more speed than the best your SD's will do. Just because you have two of them, does not go into making it twice as fast. No hardware to make it nearly twice as fast, the CPU not being largely able to utilize the boost if it were there, the only reason you'd do this is because you're trying to be "cool".

Please explain why RAID 0 would require reading/writing anything twice. It should be alternating which drive the stripes access, not mirroring anything (and even that would only apply to writes).

As for doing it, as long as you configure the kernel to support Linux Multidisk (Software RAID) and configure the thing to use the two identically sized partitions you've reserved for the RAID 0/1 configuration, it'll just work.
It just won't work the way you guys think it will. It doesn't work the way people typically think it does- mostly because they read up on what the hardware RAID offers and then think they'll get it with the software versions.

What you're saying is that people shouldn't care if it can only provide a 66% improvement instead of a 100% one, right?

I don't really see why you're opposed to the notion. If the SD controllers are not a limiting factor then I say go for it.

 

[Holokauston]

I'm not sure about the performance penalty (if significant) of LVM2, but what you suggest is certainly doable.

What filesystem is everyone considering putting on the SDHCs? I assume FAT12 because that's what they come formatted with, but perhaps someone has some benchmarks for which journaling FSs use the smallest % of cycles?

If my SDHCs comes formatted with FAT12 I'll change that asap, a 32MB maximum limit on data on a card that should hold a few GB feels wrong

Done some searching for benchmarks on the matter but doesn't seem to be any, at least not reliable, since "all cards are different" and "all readers" and so on I guess the fact is that benchmarking file systems isn't as interesting as testing readers on the same card or different cards in the same reader.

 

[azmodean]

As for filesystem, I'll probably be using ext2. All my systems are linux and all my drives are ext2 except the thumb drives I use for small data transfer, so being able to read the SD card on windows isn't an issue, which I think is the only reason to use fatXX. If there are better alternatives for use on SD cards I'd be glad to hear about it though.

It shouldn't matter though. One thing I'm excited about is that I should be able to load data onto the SD cards wirelessly. I hate having to reboot my gp2x whenever I load new apps or data onto it, especially when I'm developing something.

 

[Pickle]

Ive seen benchmarks for SSD's in raid and there is a performance increase, but I pretty sure the MB supported Raid with HW, which in the case of Pandora it would need to be done through SW.
I dont know what hit we would see because of that. The other things that would be bad is the cards would always have to be in the pandora and could never be taken out.
Another thing that might be worth looking into is if the SD controller would have any negaitve/positive effects on this kind of setup.

Overall, I dont think we need the speed improvement. For the things we are running SD access speeds are more than enough. Its not like we are going to be running XP