File system question.

Kelvin

Pixel Mercenary
Joined
Jul 22, 2013
Messages
128
Website
www.kelvinshadewing.net
So, I've come across a problem that's reared its ugly head a few times using the Pandora, and I just want to know if this is still going to be around with the Pyra.

Apparently, to boost performance or something, the Pandora doesn't actually modify SD card contents when you hit save, but instead saves the file to some sort of cache or something. Problem is, this has caused me to lose a lot of data, as I've had the Pandora die or crash while editing sometimes, and even though I've saved, it doesn't actually save until I close the program using the file.

It's been a serious problem for me in the past, and has even caused me to abandon a few projects because I was so frustrated at having to do all that work over again. I just want to make sure this isn't going to be the case, since even something like bumping the unit and making the battery jump will cause this. I tried using the paper wad trick, but that's only a temporary fix, one which wouldn't be needed if saving a file actually meant saving a file.
 

TrashyMG

Sarcasm Dispenser
Staff member
Joined
Jan 18, 2010
Messages
10,607
What you're most likely describing isn't cache or anything like that it's caused by the slow write times that SD cards have in general (especially older ones). I have encountered the same thing on my Linux desktop/laptops as well, the Pyra won't be any different. If it was a just a matter of ejecting the SD before it was finished writing I would suggest that you hit the eject button or run the sync command at the command prompt before ejecting, But you have a flakey battery connection and that can complicate things.
 

lukey

Rare Species
Joined
Jun 17, 2015
Messages
481
Location
Germany
Code:
for n in /media/*; do sudo mount $n -o sync,remount; done
Oh and issue "sync" Command from Console if you really want to be sure.
 

FBnil

Ready to Champion the Pyra to the World...
Joined
Dec 14, 2012
Messages
2,805
Location
Yurp
I used ext3 instead of ext2 for a while. This protected me from file corruption, but lowered the life of the SD card to a mere year (daily usage).
 

Kelvin

Pixel Mercenary
Joined
Jul 22, 2013
Messages
128
Website
www.kelvinshadewing.net
I guess I could make a script and have it on the desktop for such an occasion, but will the Pyra at least have faster write speed?

The thing that's weird though is I've never had a file get corrupted or disappear as a result of this, unless it was a brand new file; it always acts as though I simply hadn't saved at all, even if I saved multiple times while editing, and with time between them, so it makes me feel like it really is just not trying to update the SD card until the app closes.
 

TrashyMG

Sarcasm Dispenser
Staff member
Joined
Jan 18, 2010
Messages
10,607
but will the Pyra at least have faster write speed?
Yeah it should in theory, but the SD itself determines a lot of that too.

Honestly, what you're describing sounds more like an SD card issue.
 

Letalis Sonus

Advanced Member
Joined
Mar 5, 2009
Messages
1,286
The thing that's weird though is I've never had a file get corrupted or disappear as a result of this, unless it was a brand new file; it always acts as though I simply hadn't saved at all, even if I saved multiple times while editing, and with time between them, so it makes me feel like it really is just not trying to update the SD card until the app closes.
There are several reasons why such actions are being buffered by default, it's not just for performance reasons. For example, the mount manpage warns about sync increasing the wear of write-limited media (that is: flash memory like SD cards and SSDs). It is just reasonable to watch out for all the structural data changes while organizing the write access batches to keep the filesystem itself healthy - additionally, the journaling of modern filesystems (ext3/4, NTFS...) should allow for a full filesystem recovery even when things go wrong. Keep in mind that block-based devices (HDDs) and those emulating such behaviour (SSDs, SD cards) take rather large chunks instead of single bytes at a time, it shouldn't be that easy to actually corrupt such a block transfer with everything involved in actually getting it written.
 

Kelvin

Pixel Mercenary
Joined
Jul 22, 2013
Messages
128
Website
www.kelvinshadewing.net
OK, I don't think it's write speed, because if I save a markdown file I'm writing, which is larger than any of the story chapters I've lost, I can instantly load it in Firefox, so if it was taking its time being written to, wouldn't Firefox have trouble loading it?
 

kaprikawn

Very Active Member
Joined
Sep 28, 2008
Messages
372
Location
UK
Website
kaprikawn.wordpress.com
What filesystem will be used by default on the Pyra? I would've thought something like F2FS would be best given it's specifically geared towards towards flash memory?
 

lukey

Rare Species
Joined
Jun 17, 2015
Messages
481
Location
Germany
What filesystem will be used by default on the Pyra? I would've thought something like F2FS would be best given it's specifically geared towards towards flash memory?
Hmm... F2FS looks very promising, benchmarks show it's faster than ext4 (At least on Flash Storage), but takes up more Disk space. So that or ext4 with Journaling Disabled.

Edit: Oh and there's still UBIFS (used on the Pandora).
 
Last edited:

Letalis Sonus

Advanced Member
Joined
Mar 5, 2009
Messages
1,286
F2FS was developed especially for SD cards and eMMC, it only supports block based storage devices. In this case "optimizing for NAND based storage devices" did not include doing wear-leveling.
 

Rubis Drake

Member
Joined
Apr 26, 2012
Messages
37
f2fs-tools lack some significant features like FS shrinking. Error checking and fsck is still iffy I've heard. Might not be true. I don't have direct experience with it.
I prefer Btrfs at the moment, even if it has its problems, too. if bcachefs gets more development, I'd rather use that. Maybe disabling COW would save writes and improve performance, at the cost of having snapshots.

I hate dealing with the retarded lost+found directory. Anything that gets rid of that is okay in my book. Yeesh. So yeah, even ext2 would be better.
 

sebt3

homebrew player (P. & C.)
Joined
Sep 9, 2008
Messages
4,747
Age
39
Location
France
Website
sebt3.openpandora.org
I hate dealing with the retarded lost+found directory.
then you hate how Unix filesystems works. All have that lost+found directory (xfs, ext*, btrfs, zfs...). And if you find it "retarded" you surely never had to recover critical data from a failling disk.
 

PCXT

Very Active Member
Joined
Sep 14, 2016
Messages
232
Age
30
I think a solution may be:
1. To minimize wear put the noatime to fstab, it's always good in SSDs and minimizes writes by the cost of a rarely used metadata.
2. cron the script which checks for some activity (this needs to be worked on) and if no significant activity is present, syncs the card.
Linux is the system in which caches play important role, in fact I found that in most cases more RAM = more cache = significantly faster usage experience.
 
Top