1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Dismiss Notice

File system question.

Discussion in 'Pyra OS (Debian GNU/Linux)' started by Kelvin, Jul 8, 2016.

  1. Kelvin

    Kelvin Pixel Mercenary

    Joined:
    Jul 22, 2013
    Messages:
    119
    So, I've come across a problem that's reared its ugly head a few times using the Pandora, and I just want to know if this is still going to be around with the Pyra.

    Apparently, to boost performance or something, the Pandora doesn't actually modify SD card contents when you hit save, but instead saves the file to some sort of cache or something. Problem is, this has caused me to lose a lot of data, as I've had the Pandora die or crash while editing sometimes, and even though I've saved, it doesn't actually save until I close the program using the file.

    It's been a serious problem for me in the past, and has even caused me to abandon a few projects because I was so frustrated at having to do all that work over again. I just want to make sure this isn't going to be the case, since even something like bumping the unit and making the battery jump will cause this. I tried using the paper wad trick, but that's only a temporary fix, one which wouldn't be needed if saving a file actually meant saving a file.
     
    Tags:
  2. TrashyMG

    TrashyMG Sarcasm Dispenser Staff Member

    Joined:
    Jan 18, 2010
    Messages:
    10,154
    What you're most likely describing isn't cache or anything like that it's caused by the slow write times that SD cards have in general (especially older ones). I have encountered the same thing on my Linux desktop/laptops as well, the Pyra won't be any different. If it was a just a matter of ejecting the SD before it was finished writing I would suggest that you hit the eject button or run the sync command at the command prompt before ejecting, But you have a flakey battery connection and that can complicate things.
     
    FBnil likes this.
  3. lukey

    lukey Rare Species

    Joined:
    Jun 17, 2015
    Messages:
    459
    Location:
    Germany
    Code:
    for n in /media/*; do sudo mount $n -o sync,remount; done
    Oh and issue "sync" Command from Console if you really want to be sure.
     
    FBnil likes this.
  4. FBnil

    FBnil What is laugh? Baby don't hurt me....

    Joined:
    Dec 14, 2012
    Messages:
    2,304
    Location:
    Yurp
    I used ext3 instead of ext2 for a while. This protected me from file corruption, but lowered the life of the SD card to a mere year (daily usage).
     
  5. Kelvin

    Kelvin Pixel Mercenary

    Joined:
    Jul 22, 2013
    Messages:
    119
    I guess I could make a script and have it on the desktop for such an occasion, but will the Pyra at least have faster write speed?

    The thing that's weird though is I've never had a file get corrupted or disappear as a result of this, unless it was a brand new file; it always acts as though I simply hadn't saved at all, even if I saved multiple times while editing, and with time between them, so it makes me feel like it really is just not trying to update the SD card until the app closes.
     
  6. FBnil

    FBnil What is laugh? Baby don't hurt me....

    Joined:
    Dec 14, 2012
    Messages:
    2,304
    Location:
    Yurp
    You might not need remount, if you also flush the buffers

    However, are you sure it is not the SD card? Try another brand...
     
  7. TrashyMG

    TrashyMG Sarcasm Dispenser Staff Member

    Joined:
    Jan 18, 2010
    Messages:
    10,154
    Yeah it should in theory, but the SD itself determines a lot of that too.

    Honestly, what you're describing sounds more like an SD card issue.
     
  8. Letalis Sonus

    Letalis Sonus Very Active Member

    Joined:
    Mar 5, 2009
    Messages:
    976
    There are several reasons why such actions are being buffered by default, it's not just for performance reasons. For example, the mount manpage warns about sync increasing the wear of write-limited media (that is: flash memory like SD cards and SSDs). It is just reasonable to watch out for all the structural data changes while organizing the write access batches to keep the filesystem itself healthy - additionally, the journaling of modern filesystems (ext3/4, NTFS...) should allow for a full filesystem recovery even when things go wrong. Keep in mind that block-based devices (HDDs) and those emulating such behaviour (SSDs, SD cards) take rather large chunks instead of single bytes at a time, it shouldn't be that easy to actually corrupt such a block transfer with everything involved in actually getting it written.
     
  9. Kelvin

    Kelvin Pixel Mercenary

    Joined:
    Jul 22, 2013
    Messages:
    119
    OK, I don't think it's write speed, because if I save a markdown file I'm writing, which is larger than any of the story chapters I've lost, I can instantly load it in Firefox, so if it was taking its time being written to, wouldn't Firefox have trouble loading it?
     
  10. slaeshjag

    slaeshjag ¯\_(ツ)_/¯

    Joined:
    Apr 8, 2010
    Messages:
    2,687
    Location:
    ~Stockholm, Sweden
    before anything is written to disk, it is cached. Any program on the system uses the same cache if it's the same file.
     
    _jr_ likes this.
  11. kaprikawn

    kaprikawn Member

    Joined:
    Sep 28, 2008
    Messages:
    346
    Location:
    UK
    What filesystem will be used by default on the Pyra? I would've thought something like F2FS would be best given it's specifically geared towards towards flash memory?
     
  12. lukey

    lukey Rare Species

    Joined:
    Jun 17, 2015
    Messages:
    459
    Location:
    Germany
    Hmm... F2FS looks very promising, benchmarks show it's faster than ext4 (At least on Flash Storage), but takes up more Disk space. So that or ext4 with Journaling Disabled.

    Edit: Oh and there's still UBIFS (used on the Pandora).
     
    Last edited: Jul 27, 2016
  13. Silent-Hunter

    Silent-Hunter Advanced Member

    Joined:
    May 29, 2010
    Messages:
    2,686
    I think UBIFS is fine, it's what we're using now and it works quite well.
     
  14. Letalis Sonus

    Letalis Sonus Very Active Member

    Joined:
    Mar 5, 2009
    Messages:
    976
    UbiFS only works on directly accessed NAND memory, eMMC has an FTL and therefore requires a filesystem for block based storage devices.
     
  15. Silent-Hunter

    Silent-Hunter Advanced Member

    Joined:
    May 29, 2010
    Messages:
    2,686
    What about F2FS? Does that work on block based ones?
     
  16. Letalis Sonus

    Letalis Sonus Very Active Member

    Joined:
    Mar 5, 2009
    Messages:
    976
    F2FS was developed especially for SD cards and eMMC, it only supports block based storage devices. In this case "optimizing for NAND based storage devices" did not include doing wear-leveling.
     
    _jr_ likes this.
  17. Rubis Drake

    Rubis Drake Member

    Joined:
    Apr 26, 2012
    Messages:
    37
    f2fs-tools lack some significant features like FS shrinking. Error checking and fsck is still iffy I've heard. Might not be true. I don't have direct experience with it.
    I prefer Btrfs at the moment, even if it has its problems, too. if bcachefs gets more development, I'd rather use that. Maybe disabling COW would save writes and improve performance, at the cost of having snapshots.

    I hate dealing with the retarded lost+found directory. Anything that gets rid of that is okay in my book. Yeesh. So yeah, even ext2 would be better.
     
  18. sebt3

    sebt3 homebrew player (P. & C.)

    Joined:
    Sep 9, 2008
    Messages:
    4,738
    Location:
    France
    then you hate how Unix filesystems works. All have that lost+found directory (xfs, ext*, btrfs, zfs...). And if you find it "retarded" you surely never had to recover critical data from a failling disk.
     
  19. Rubis Drake

    Rubis Drake Member

    Joined:
    Apr 26, 2012
    Messages:
    37
    Btrfs doesn't have a lost+found directory.
     
  20. PCXT

    PCXT Active Member

    Joined:
    Sep 14, 2016
    Messages:
    190
    I think a solution may be:
    1. To minimize wear put the noatime to fstab, it's always good in SSDs and minimizes writes by the cost of a rarely used metadata.
    2. cron the script which checks for some activity (this needs to be worked on) and if no significant activity is present, syncs the card.
    Linux is the system in which caches play important role, in fact I found that in most cases more RAM = more cache = significantly faster usage experience.
     

Share This Page

Loading...