Filesystem shootout

[chaotix]

I tortured my SD-card for more than 34 hours straight until it bled and screamed for merci to get a definate answer to the question: Which filesystem is best suited for the Pandora?


The contenders were ext2, ext3 and ext4 and the clear winner is... ext4.


You can read about the testing procedure and more thorough results on http://lxtek.de/pand...stem-benchmark/


Lars

 

[relliker]

This is very interesting. Well done!


When looking at the results though, I find that while the difference between the varioius ext formats is clear however does the difference between mounting with relatime, noatime and dirsync matter that much? I can see that with relatime and then with dirsync can make a difference from let's say, between 15MB/s to for ex. 16MB/s, i.e. 1MB/s difference. Is this enough to get rid of dirsync just to gain the 1MB/s? In real life (using percieved time versus actual time), would you know or care if your 1GB file finished writing 10 seconds later or faster? If the difference was between a gain from say 12MB/s to 16MB/s then percieved time would matter but from your benchmarks I'd prefer the safer options for the minimal difference they produce in the same ext format.


EDIT: FYI, I just threw values at random. I didn't perform any mathematics to get the real differences so in reality I will probably be off by a good amount.

 

[relliker]

May I ask HOW to get rid of the block boundaries issue during partitioning?


EDIT: Thanks for the link

 

[Prometheus]

Due to the differences between the cards of different manufacturers (for example, some use controllers that are simply not suited to the ext* filesystems at all), I'd be curious to know how many different types and brands SD Cards were used in your testing. Your article only references using "a 64 GB SDXC Class 10 card", but not which brands were tried.

 

[notaz]

http://lxtek.de/pand...stem-benchmark/

Unfortunately on the Pandora ext4 is compiled as a module. So it is not available at system boot time and presently cannot be used for your root filesystem.

Could compile it in for next firmware update I guess.

 

[kaprikawn]

Interesting. I have a 64gb Transcend card that I recently replaced because it was throwing up errors when I tried to delete some stuff (something about a bad nfs mount or something to that effect). I don't like the idea of having my files on a card that is giving those types of errors. That was running on ext2, I don't know whether that was a factor, I didn't treat it that badly.


I've replaced it with a 64gb Sandisk (luckily the prices aren't crazy anymore). I've formatted that to ext2, but I may reconsider, and do another format over the weekend in light of your recommendations.


And Notaz, I for one would appreciate ext4 comiled natively.

 

[chaotix]

does the difference between mounting with relatime, noatime and dirsync matter that much? I can see that with relatime and then with dirsync can make a difference from let's say, between 15MB/s to for ex. 16MB/s, i.e. 1MB/s difference. Is this enough to get rid of dirsync just to gain the 1MB/s?

You are right. There is no real difference between using "dirsync" or not for I/O in large blocks. That's because it really does not do anything differently for this kind of access. So it even is not really safer to use.


"dirsync" just makes a difference when you write metadata. That is reflected in the file creation and deletion tests. So e.g. using ext3 1740.0 files were created per second. Using "dirsync" this dropped to 91.0 files / second. And this you will notice when you are creating a lot of files (like extracting a large ZIP-archive) and "dirsync" actually does anything.


And on another note: Pulling out the SD-card without unmounting it first is reckless even when using "dirsync" as only directory data is written to disk at once. You file contents will probably still be in the system cache. If you want to pull out the card without unmounting you should actually use the "sync" option. But I guess this will probably kill I/O performance for larger blocks as well.


So "dirsync" is actually more a poor man's journal. A journaling filesystem (by default) only journals metadata and will make sure that it is always in a correct state even if the power fails (or you eject the card). So in my opinion just use a journal instead of using "dirsync". It should be faster and safer.


Regarding "noatime" and "relatime". Yes, I did not see any performance differences there. That's why I prefer "relatime" as it might prevent some problems.

May I ask HOW to get rid of the block boundaries issue during partitioning?

You may There's a link at the top of the page to http://lxtek.de/pandora/flashbench/ It explains how you can guess your erase block size and partition your card accordingly.

Due to the differences between the cards of different manufacturers (for example, some use controllers that are simply not suited to the ext* filesystems at all), I'd be curious to know how many different types and brands SD Cards were used in your testing. Your article only references using "a 64 GB SDXC Class 10 card", but not which brands were tried.

I made the extensive benchmarks on only one card. The manufacturer is Platinum. But as I understand it that doesn't really mean anything. Most SD-card manufacturers use whichever controllers are available and cheap at the time. (Maybe with the exception of SanDisk which seem to use own controllers if I remember correctly).


If there is interest I could do a smaller testrun (without the different mount options) on another card (extremememory Performance 16 GB Class 6).

http://lxtek.de/pand...stem-benchmark/

Unfortunately on the Pandora ext4 is compiled as a module. So it is not available at system boot time and presently cannot be used for your root filesystem.

Could compile it in for next firmware update I guess.

That would be fantastic.

 

[Linux-SWAT]

http://lxtek.de/pand...stem-benchmark/

Unfortunately on the Pandora ext4 is compiled as a module. So it is not available at system boot time and presently cannot be used for your root filesystem.

Could compile it in for next firmware update I guess.

Oh uboot can manage it ?

 

[chaotix]

Interesting. I have a 64gb Transcend card that I recently replaced because it was throwing up errors when I tried to delete some stuff (something about a bad nfs mount or something to that effect). I don't like the idea of having my files on a card that is giving those types of errors. That was running on ext2, I don't know whether that was a factor, I didn't treat it that badly.

Of course I don't know if your card has some failing erase blocks, but my card also produced some write errors when I first started using it on the Pandora with an unaligned partition. After aligning the partitions I did not have a single message in my kernel log reganding I/O errors. And I did benchmarks tests for over 50 hours on the same card....


So I wouldn't throw away that Transcend card yet.

 

[chaotix]

Oh uboot can manage it ?

There are patches available that make it work with ext4 but I don't believe the Pandora version is patched.


Although it would be nice to have ext4 support from the bootloader it really is not a showstopper. The bootloader just needs to access the kernel image somewhere. So just put it either on NAND or in a seperate ext2 /boot partition on your SD-card.

 

[Prometheus]

If there is interest I could do a smaller testrun (without the different mount options) on another card (extremememory Performance 16 GB Class 6).

I don't know about anyone else, but I'd certainly be interested in that.

 

[Christoph.Krn]

You might be interested in SDcard formatting HOWTO on olpcnews.com's forum.


-----

If there is interest I could do a smaller testrun (without the different mount options) on another card (extremememory Performance 16 GB Class 6).

I don't know about anyone else, but I'd certainly be interested in that.

Me too. The results probably won't be transferable to other SD cards, but the differences (or lack thereof) might be interesting nonetheless.


Also, in case you (chaotix) will be doing more tests, it might be interesting to have some extended comparisons between a card in a 'trashed' state, and the same card in a non-trashed state (confused controller vs. 'fresh' controller memory); as well as the same card with some unformatted space and no unformatted space (possibly on a FAT filesystem, or any other filesystem the controller may understand) ( http://boards.openpandora.org/index.php?/topic/1062-sd-cards/page__st__20#entry20300 ).

 

[chaotix]

So. I updated the page. It now includes benchmarks for the second card and also for vfat.


Results are "interesting". ext2 and ext3 got faster and ext4 and vfat slower on the other card although ext4 is still ahead.

 

[Linux-SWAT]

Great job.


Do you plan to also check cpu usage during file managements ?

 

[notaz]

Was this tested an 256M or 512/rebirth pandora?

 

[chaotix]

Also, in case you (chaotix) will be doing more tests, it might be interesting to have some extended comparisons between a card in a 'trashed' state, and the same card in a non-trashed state (confused controller vs. 'fresh' controller memory); as well as the same card with some unformatted space and no unformatted space (possibly on a FAT filesystem, or any other filesystem the controller may understand) ( http://boards.openpa...__20#entry20300 ).

I'm not sure the controllers of SD-cards are so sophisticated. As the SD-card specs don't really include a TRIM command, I don't know if they really include such SSD-features as background garbage collection.


But I'm pretty certain the controller knows which blocks have been written to before. When I ported the flashbench util and used it on my new card, results were pretty confused and random (which could be explained by reads from blocks that had not been written to before so the controller could just returned an empty block without actually reading it). After benchmarking (and writing a lot of GB of data to the cards in the process) I ran it again and now the results are consistent.


I also really don't think that any controller really "understands" FAT (But I'm sure some are optimised for the access patterns that FAT occur when using FAT). Making decisions on this "understanding" of which blocks are in use and which aren't would be a recipe for desaster. So the only way to be sure a block is unused (lacking a TRIM command) is either because it was never written to before or the controller used a free block on a write. This way it can mark the old block as unused.


This directly implies that if the card actually features a background garbage collection not using an area of the card could be beneficial. Although maybe leaving space explicitly unused might not be necessary. According to https://en.wikipedia...al#DRM_features roughly 10% of the card's capacity are exclusively reserved for the "secure" area, so cannot be used anyway. But as the rest this is only speculation.


Maybe someone could use the flashbench utility (http://lxtek.de/pandora/flashbench/) on the Pandora on a used card, then format it with this "magic" Windows SD card formatter (https://www.sdcard.o...ds/formatter_3/) and then use flashbench again. If the results are different it would at least show that the Windows formatter is able convince the SD-card that some blocks are indeed unused which would be necessary for a garbage collection to work at all.

 

[chaotix]

Do you plan to also check cpu usage during file managements ?

I thought this was already enough data But if you are interessted, cpu usage data is available in the raw bonnie output that is available on the page.

Was this tested an 256M or 512/rebirth pandora?

512 MB Pandora.

 

[Grench]

http://lxtek.de/pand...stem-benchmark/

Unfortunately on the Pandora ext4 is compiled as a module. So it is not available at system boot time and presently cannot be used for your root filesystem.

Could compile it in for next firmware update I guess.

Would it be possible to include EXFAT while you're there? I'd like to be able to read/write my camera's cards in my Pandora. Camera only likes EXFAT on the big SDXC cards.


Thank you!

 

[Grench]

I tortured my SD-card for more than 34 hours straight until it bled and screamed for merci to get a definate answer to the question: Which filesystem is best suited for the Pandora?


The contenders were ext2, ext3 and ext4 and the clear winner is... ext4.


You can read about the testing procedure and more thorough results on http://lxtek.de/pand...stem-benchmark/


Lars

Excellent work Sir! It's amazing how much better EXT4 is. It looks like you have a sound and repeatable methodology set up. I'd like to see this expanded out.


How about a PND that can execute the test battery (with card-ID information) and save the output to a text file that the user can upload to an online database linked to the FAQ? It would enable us to have this level of detail by file system by card for everyone who is willing to participate and put their card's info up.


I do not see this as a duplication of the existing SD card information on the FAQ. Half the cards there seem to be listed as the manufacturer's claimed speed - the other half may be on a Pandora, but with inconsistent methodology.


VERY cool stuff - thank you!

 

[chaotix]

Would it be possible to include EXFAT while you're there? I'd like to be able to read/write my camera's cards in my Pandora. Camera only likes EXFAT on the big SDXC cards.

Although there are implentations for Linux, I don't think it is possible to include exFat in the stock firmware for legal reasons.


From https://en.wikipedia.org/wiki/ExFAT

Microsoft has not released the official exFAT file system specification, and a restrictive license from Microsoft is required in order to make and distribute exFAT implementations. Microsoft also asserts patents on exFAT which make it impossible to re-implement its functionality in a compatible way without violating a large percentage of them.[11] This renders the implementation, distribution, and use of exFAT as a part of free or open-source operating systems or of commercial software, for which the vendors could not obtain a license from Microsoft, not only technically difficult, but legally impossible in countries that recognize United States software patents.

Maybe in can be included in the codec pack. But this is more a legal than a technical issue.