File system question.

[Kelvin]

So, I've come across a problem that's reared its ugly head a few times using the Pandora, and I just want to know if this is still going to be around with the Pyra.

Apparently, to boost performance or something, the Pandora doesn't actually modify SD card contents when you hit save, but instead saves the file to some sort of cache or something. Problem is, this has caused me to lose a lot of data, as I've had the Pandora die or crash while editing sometimes, and even though I've saved, it doesn't actually save until I close the program using the file.

It's been a serious problem for me in the past, and has even caused me to abandon a few projects because I was so frustrated at having to do all that work over again. I just want to make sure this isn't going to be the case, since even something like bumping the unit and making the battery jump will cause this. I tried using the paper wad trick, but that's only a temporary fix, one which wouldn't be needed if saving a file actually meant saving a file.

 

[TrashyMG]

What you're most likely describing isn't cache or anything like that it's caused by the slow write times that SD cards have in general (especially older ones). I have encountered the same thing on my Linux desktop/laptops as well, the Pyra won't be any different. If it was a just a matter of ejecting the SD before it was finished writing I would suggest that you hit the eject button or run the sync command at the command prompt before ejecting, But you have a flakey battery connection and that can complicate things.

 

[lukey]

for n in /media/*; do sudo mount $n -o sync,remount; done

Oh and issue "sync" Command from Console if you really want to be sure.

 

[FBnil]

I used ext3 instead of ext2 for a while. This protected me from file corruption, but lowered the life of the SD card to a mere year (daily usage).

 

[Kelvin]

I guess I could make a script and have it on the desktop for such an occasion, but will the Pyra at least have faster write speed?

The thing that's weird though is I've never had a file get corrupted or disappear as a result of this, unless it was a brand new file; it always acts as though I simply hadn't saved at all, even if I saved multiple times while editing, and with time between them, so it makes me feel like it really is just not trying to update the SD card until the app closes.

 

[FBnil]

You might not need remount, if you also flush the buffers

However, are you sure it is not the SD card? Try another brand...

 

[TrashyMG]

but will the Pyra at least have faster write speed?

Yeah it should in theory, but the SD itself determines a lot of that too.

Honestly, what you're describing sounds more like an SD card issue.

 

[Letalis Sonus]

The thing that's weird though is I've never had a file get corrupted or disappear as a result of this, unless it was a brand new file; it always acts as though I simply hadn't saved at all, even if I saved multiple times while editing, and with time between them, so it makes me feel like it really is just not trying to update the SD card until the app closes.

There are several reasons why such actions are being buffered by default, it's not just for performance reasons. For example, the mount manpage warns about sync increasing the wear of write-limited media (that is: flash memory like SD cards and SSDs). It is just reasonable to watch out for all the structural data changes while organizing the write access batches to keep the filesystem itself healthy - additionally, the journaling of modern filesystems (ext3/4, NTFS...) should allow for a full filesystem recovery even when things go wrong. Keep in mind that block-based devices (HDDs) and those emulating such behaviour (SSDs, SD cards) take rather large chunks instead of single bytes at a time, it shouldn't be that easy to actually corrupt such a block transfer with everything involved in actually getting it written.

 

[Kelvin]

OK, I don't think it's write speed, because if I save a markdown file I'm writing, which is larger than any of the story chapters I've lost, I can instantly load it in Firefox, so if it was taking its time being written to, wouldn't Firefox have trouble loading it?

 

[slaeshjag]

before anything is written to disk, it is cached. Any program on the system uses the same cache if it's the same file.

 

[kaprikawn]

What filesystem will be used by default on the Pyra? I would've thought something like F2FS would be best given it's specifically geared towards towards flash memory?

 

[lukey]

What filesystem will be used by default on the Pyra? I would've thought something like F2FS would be best given it's specifically geared towards towards flash memory?

Hmm... F2FS looks very promising, benchmarks show it's faster than ext4 (At least on Flash Storage), but takes up more Disk space. So that or ext4 with Journaling Disabled.

Edit: Oh and there's still UBIFS (used on the Pandora).

 

[Silent-Hunter]

I think UBIFS is fine, it's what we're using now and it works quite well.

 

[Letalis Sonus]

I think UBIFS is fine, it's what we're using now and it works quite well.

UbiFS only works on directly accessed NAND memory, eMMC has an FTL and therefore requires a filesystem for block based storage devices.

 

[Silent-Hunter]

UbiFS only works on directly accessed NAND memory, eMMC has an FTL and therefore requires a filesystem for block based storage devices.

What about F2FS? Does that work on block based ones?

 

[Letalis Sonus]

F2FS was developed especially for SD cards and eMMC, it only supports block based storage devices. In this case "optimizing for NAND based storage devices" did not include doing wear-leveling.

 

[RubisDrake]

f2fs-tools lack some significant features like FS shrinking. Error checking and fsck is still iffy I've heard. Might not be true. I don't have direct experience with it.
I prefer Btrfs at the moment, even if it has its problems, too. if bcachefs gets more development, I'd rather use that. Maybe disabling COW would save writes and improve performance, at the cost of having snapshots.

I hate dealing with the retarded lost+found directory. Anything that gets rid of that is okay in my book. Yeesh. So yeah, even ext2 would be better.

 

[sebt3]

I hate dealing with the retarded lost+found directory.

then you hate how Unix filesystems works. All have that lost+found directory (xfs, ext*, btrfs, zfs...). And if you find it "retarded" you surely never had to recover critical data from a failling disk.

 

[RubisDrake]

Btrfs doesn't have a lost+found directory.

 

[PCXT]

I think a solution may be:
1. To minimize wear put the noatime to fstab, it's always good in SSDs and minimizes writes by the cost of a rarely used metadata.
2. cron the script which checks for some activity (this needs to be worked on) and if no significant activity is present, syncs the card.
Linux is the system in which caches play important role, in fact I found that in most cases more RAM = more cache = significantly faster usage experience.