Filesystem? Please Don't Default To Fat32

[Zarneth]

What file system is Pandora going to use for it's internal storage? Will it support other file systems on the SD cards on top of FAT32 out of the box?

FAT32 is ancient and unreliable. But it's all the GP2X supported. My SD card often becomes corrupted, usually killing GMENU2X or other things and only formatting the card will fix it. I've gotten the card replaced once but it still happens. It could be an incompatibility but I suspect FAT32 is likely contributing to the problem if not the cause itself.

EXT3 or NTFS on the other hand are much more reliable and handle unexpected power loss much better.

EXT3 is probably the most likely choice for the built in storage being the most common Linux format. (Though it can now read/write NTFS quite reliably) Using EXT3 for the internal storage makes it slightly tricker for windows users to access it, but given how small it is and the many alternate methods of accessing it (wifi for samba and ftp, SD cards) and how small it is this is unlikely to be accessed often directly by the user so shouldn't be an issue.

It's a shame it's only 256mb really. 512mb would allow for a much more functional linux install.

 

[simonb]

FAT32 is ancient and unreliable.

Thats a pretty sweeping statement. The main problem with FAT fs' is the directory structures getting fragmented.
QUOTE

But it's all the GP2X supported.


Nope, it supports ext2 too, and ext3 by definition.
QUOTE

EXT3 or NTFS on the other hand are much more reliable and handle unexpected power loss much better.


ext3 is a journalling fs, it's not "more reliable". indeed, ext3 doesn't do checksumming when writing to the journal, which can lead to very serious problems if a crash happens while the kernel is doing this. Also, because of the way it works, you'd half the life of your SD card as all data has to be written twice....

There's always ReiserFS, the FS with killer performance...

 

[Magnulus]

It's a shame it's only 256mb really. 512mb would allow for a much more functional linux install.

I'd say Damn Small Linux is pretty functional, and that's what? 50 megs?

 

[juanvvc]

FAT32 is ancient and unreliable. But it's all the GP2X supported.

Not true. You can format your SD cards in EXT2 and the Gp2x will support them. Just modify /etc/fstab in your Gp2x to mount EXT2 as the default filesystem for SDs. You can even write a simple script to mount the SD as EXT2 and if it fails, try FAT32.

You'll lose compatibility with Microsoft systems but if it does not bother you, go ahead.

I'm with simonb, using EXT3 you have to manage the journaling in a system that does not shutdown the normal way in *nix, but switching off the power. You will corrupt the journal quite often. But as I recall ReiserFS uses journals as well. In this sense, EXT2 seems the best alternative to prevent fragmentation.

 

[Zarneth]

Thats a pretty sweeping statement. The main problem with FAT fs' is the directory structures getting fragmented.

And fragmentation isn't really a problem on flash drives. But FAT32 hasn't been used since the days of dos (and win98 which ran on top of dos) and has problems like needing scandisk to run whenever it isn't shut down properly. (not that i didn't always skip scandisk anyway) It's been a while since I've looked into the various file systems but I was always under the impression FAT32 was the older and less reliable system.

Nope, it supports ext2 too, and ext3 by definition.

Wow. that's the best kept secret ever. Certain I looked into it thoroughly. I shall have to try it.

ext3 is a journalling fs, it's not "more reliable". indeed, ext3 doesn't do checksumming when writing to the journal, which can lead to very serious problems if a crash happens while the kernel is doing this. Also, because of the way it works, you'd half the life of your SD card as all data has to be written twice....

There's always ReiserFS, the FS with killer performance...

I'm actually a ReiserFS user myself. I assumed Ext3 would be better choice because it's usually the default in linux. Definitely needs more research into the ideal filesystem regardless.

QUOTE
I'd say Damn Small Linux is pretty functional, and that's what? 50 megs?
Very good point, but the amount of work to get things to that size, over something more flexible and easy to implement.... It's functional but it isn't pretty.
The pandora was also going to support debian ARM packages to some degree right? I've no experience with Debian ARM but I'd wager it's no 50mb lightweight. Don't forget swap space too. The Pandora has limited ram after all. If they plan to run Firefox 3 and FF3 is anything like it's predecessor then it'll need all the swap/cache space it can get.

We'll just have to have faith in our good dev's. If they manage to pull off the Pandora (with an option to ship to Australia) then they'll be gods in my eyes. B)

 

[davidgutierrez]

The pandora was also going to support debian ARM packages to some degree right? I've no experience with Debian ARM but I'd wager it's no 50mb lightweight. Don't forget swap space too. The Pandora has limited ram after all. If they plan to run Firefox 3 and FF3 is anything like it's predecessor then it'll need all the swap/cache space it can get.

Debian ARM isn't a 50 MB distro, but I think Pandora (should) be configured in a way you could install programs / packages in the SD. It's very easy to mount the SD as /usr/bin or something like that.

Talking about Firefox3, Mozilla says it would require LESS RAM that Firefox 2. On the other hand, Mozilla is developing "Firefox Mini", a Firefox based browser for mobile phones. In the worst case, we would use konqueror or another web browser.

 

[Zarneth]

I'm with simonb, using EXT3 you have to manage the journaling in a system that does not shutdown the normal way in *nix, but switching off the power. You will corrupt the journal quite often. But as I recall ReiserFS uses journals as well. In this sense, EXT2 seems the best alternative to prevent fragmentation.

Yes RiserFS uses Journaling. I could be wrong but I think NTFS did too.
But I thought the purpose of journaling was to STOP the problems with corruption when the power is shut off? :blink:
Wouldn't MS and Linux still use fat32 if it was superior. I was under the impression that FAT32 was used on portable drives because of it's universal compatibility.

Wikipedia sais:
QUOTE
journaling file system is a file system that logs changes to a journal (usually a circular log in a specially-allocated area) before actually writing them to the main file system. Such file systems are less likely to become corrupted in the event of power failure or system crash.

Debian ARM isn't a 50 MB distro, but I think Pandora (should) be configured in a way you could install programs / packages in the SD. It's very easy to mount the SD as /usr/bin or something like that.
Probably best to stick to windows style (one folder per program) for games and such on the SD that aren't part of the core system and common libraries. Less headache, and you don't need to teach all the windows users how the linux file structure works.

 

[davidgutierrez]

Probably best to stick to windows style (one folder per program) for games and such on the SD that aren't part of the core system and common libraries. Less headache, and you don't need to teach all the windows users how the linux file structure works.

The idea was that Pandora could do that without the user has to worry about that.

1 - I insert SD1 in Pandora.
2 - The user doesn't know it, but Pandora will mount the SD as /usr/bin or something like this.
3 - I open the package manager and install a game or application.
4 - The user can use that game or application only when SD1 is inserted on Pandora.

As you can see, you don't have to teach the Windows user the internals of Linux file structure.

 

[simonb]

I'm actually a ReiserFS user myself.

Sorry, I've not been on this forum long enough for people to get my twisted sense of humour...

http://en.wikipedia.org/wiki/Hans_Reiser#N...s_disappearance

 

[javaJake]

2 - The user doesn't know it, but Pandora will mount the SD as /usr/bin or something like this.
3 - I open the package manager and install a game or application.
4 - The user can use that game or application only when SD1 is inserted on Pandora.

Indeed, I had forgotten about linking. You can do all sorts of magical things if you want to have your filesystem on an SD card, and still be able to apt-get install stuff

 

[Pickle]

Theres also some interesting things by using ext2, for example with the current X11 server runs much faster on ext2.
Theres also supposed to be a new fs by MS for flash type drive, cant remember the name...

Also arnt filesystems inherit by using linux kernel in general?
I could forsee using one card with a linux fs and the other with a windws one.

(A little off topic) The other cool thing is going to the wireless, whos says everything needs to be on the cards. I am for one looking forward to playing back movies/ps1 over the wireless.

 

[klikklak]

actually mounting the sd card on /usr/local would be the logical choice as that wouldn't cause problems with the internal memory (mounting two directories on /usr/bin, no way). As for the filesystem, I think using ext3 by default would cause problems for the majority of users so because of that it would be undesirable. It's meant to stay on for ages, right? so no hard reboots all the time, instead suspend and recharge when needed would take away a lot fo the wear and tear on fat32.

 

[juanvvc]

But I thought the purpose of journaling was to STOP the problems with corruption when the power is shut off? :blink:
Wouldn't MS and Linux still use fat32 if it was superior. I was under the impression that FAT32 was used on portable drives because of it's universal compatibility.

Sure, you're right, journal helps when power switch off abruptly But journal means accessing several times to the SD, and it is not acceptable on battery powered devices. Besides, the journal system needs to be restored in case of a power crash. Try in your RaiserFS: it waste many seconds during boot.

Nobody says that fat32 is superior to any modern filesystem. That will be absolutely wrong. Fat32 is not superior but simpler: little devices need little and fast drivers to access to a fat32 filesystem, and that's the main reason of its surveillance.

Fragmentation really is a problem in flash drivers where files with different sizes are often added and removed, like gamers do. My SD card really needs a defragmentation since fsck does not help any more

 

[Pickle]

Sure, you're right, journal helps when power switch off abruptly But journal means accessing several times to the SD, and it is not acceptable on battery powered devices. Besides, the journal system needs to be restored in case of a power crash. Try in your RaiserFS: it waste many seconds during boot.

I think there are ways to turn the journaling off. If your really interested had over to the eee comunity. This topic has been fully discussed. http://www.eeeuser.com/

Fragmentation really is a problem in flash drivers where files with different sizes are often added and removed, like gamers do. My SD card really needs a defragmentation since fsck does not help any more

I dont see your point, there really is no latency between the sectors of a flash drive like a moving hard drive, fragmentation is really irrelevant. Unless your refering to wasted space with frag's.

 

[Orkie]

actually mounting the sd card on /usr/local would be the logical choice as that wouldn't cause problems with the internal memory (mounting two directories on /usr/bin, no way). As for the filesystem, I think using ext3 by default would cause problems for the majority of users so because of that it would be undesirable. It's meant to stay on for ages, right? so no hard reboots all the time, instead suspend and recharge when needed would take away a lot fo the wear and tear on fat32.

Sorry, I'm confused as to why you'd want to do this?. It's be much simpler to just have /mnt/sd like the GP2X does... Oh, and you can mount two filesystems on the same directory using unionfs.

 

[Squidge]

But I thought the purpose of journaling was to STOP the problems with corruption when the power is shut off? :blink:

Correct. However, since on the gp2x the power is normally always shut off before the device is unmounted, that means every time you boot the machine with that card in it's going to trawl through the journal looking to see whats faulty due to the 'dirty' bit being set.

Fragmentation really is a problem in flash drivers where files with different sizes are often added and removed, like gamers do. My SD card really needs a defragmentation since fsck does not help any more

An SD card is always fragmented, even if you do a full format and then copy a single file to it. Defragging an SD card does nothing but reduce the life cycle. This is because when you write a block to an SD card, the first thing it does is choose a random location to write the block to. This is called wear levelling, and prevents certain parts of the card failing before others (eg. when the FAT is constantly updated).

 

[juanvvc]

I think there are ways to turn the journaling off.

I do not want to turn journaling off in my PC. I'm really happy with Ext2/3 and ReiserFS since I am using disks formated with any of them for years without problems. I cannot say the same with any FAT or NTFS. I just think than journaling is not a good idea for little, battery powered devices. But, hey, it is just an intuition

QUOTE
I dont see your point, there really is no latency between the sectors of a flash drive like a moving hard drive


I can see your point: flash drives access instantly to any sector so fragmentation is not a problem at all. But it takes a long time to save big files in my SD, up to ten minutes for a 300MB file. Last time I formated the SD the same file moved in less than 30 seconds. Fsck says than my SD is clean and I cannot explain that time difference without fragmentation... unless I cannot explain with fragmentation either

Did I defrag my SD to check writing speed with a clean SD? No, I do not own any Windows, neither at home nor at work, and I do not know about any defrag in Linux

Edit: the Squidge's post really make me think that I'm absolutely wrong in my assumptions. But the SD in the Gp2x is used mainly to read, not to write, and the journaling will slow down the booting process to solve problems as little as rare. Anyway, thanks for your explanations, Squidge!

 

[Pickle]

I think there are ways to turn the journaling off.

I do not want to turn journaling off in my PC. I'm really happy with Ext2/3 and ReiserFS since I am using disks formated with any of them for years without problems. I cannot say the same with any FAT or NTFS. I just think than journaling is not a good idea for little, battery powered devices. But, hey, it is just an intuition


No i also meant for the Pandora, I wasnt refering to turn journaling off for a regular pc.

I can see your point: flash drives access instantly to any sector so fragmentation is not a problem at all. But it takes a long time to save big files in my SD, up to ten minutes for a 300MB file. Last time I formated the SD the same file moved in less than 30 seconds. Fsck says than my SD is clean and I cannot explain that time difference without fragmentation... unless I cannot explain with fragmentation either

Did I defrag my SD to check writing speed with a clean SD? No, I do not own any Windows, neither at home nor at work, and I do not know about any defrag in Linux

Edit: the Squidge's post really make me think that I'm absolutely wrong in my assumptions. Thanks for your explanations, Squidge!

I would think its your os taking it's time to write to the flash card. Defraging doesnt make it go any faster.
Looks like your picking up on that from your edit.

 

[Sphinxter]

Wouldn't touch FAT with your ten foot pole;
http://www.news.com/Microsofts-file-system..._3-6025447.html

 

[Squidge]

it takes a long time to save big files in my SD, up to ten minutes for a 300MB file.

I've had exactly the same problem - taking ages (anything upto 5 minutes) to save a ~300MB file to an SD card. I changed the sd card reader, and now it takes 40-50 seconds. Both are USB2.0 high speed devices. The slow one is SD card only, the fast one is a '7 in 1' device. Maybe the processor is faster, or the way of talking to the SD card is different in the '7 in 1' device.

Wouldn't touch FAT with your ten foot pole;
http://www.news.com/Microsofts-file-system..._3-6025447.html



The FAT file system isn't patented, that was rejected with "prior art". These days, only some features of Microsoft's implementation of the FAT system is patented. So, you can still read a fat file system without getting sued.