guarding against sd corruption

[sepulep]

so a few days i noticed some popping when playing an ogg on my pandora - turns out the sd card was getting bad.. the thing is I only noticed this by comparing file contents against a backup and checking with ogginfo: the filesystem itself checked out ok; no dmesg errors or anything. Now what got me really worried was that when I compared hashes something like 1 in 3 files in the music library (thousands of files) where actually affected! maybe strangest of all, this was also the sd containing the root fs...

my question: what would be a good way (some tool or script to run) to verify/maintain the integrity of the data on an SD card??

 

[Linux-SWAT]

First, get a good card.

Second, use some EXT filesystem as the tools to repair it are shipped in any Linux.

 

[elw3]

A _good_ card may remap broken bits till the reserved error space is full.

I wonder if it is possible that  the remap function breaks a file with that. 

Guess if you dd the card with /dev/zero and then count one by one if all those bits are really zeros you should see any broken parts. And of course repeat it wit "one".

 

[levi]

You're better off writing a pattern to the drive and checking that than blanket 0s or 1s - if the controller is duff and feeding you the wrong data, you won't know what's right if it's all the same. Also, assuming most cards these days are using MLC flash, flat 1's and 0's are likely to be the extreme edges of the voltage range, so not the best test for flash errors.

 

[alxm]

A journaling file system is a must, I use ext4 on a 32GB SanDisk without any issues. I used ext2 before and files occasionally got corrupted when I exited applications with the Pandora button instead of the proper way.

 

[ekianjo]

A journaling file system is a must, I use ext4 on a 32GB SanDisk without any issues. I used ext2 before and files occasionally got corrupted when I exited applications with the Pandora button instead of the proper way.

Yeah, but isnt there something of an issue, such as you can't boot a system from a ext4 card on Pandora?  

 

[levi]

I may be misremembering, but IIRC you can boot an OS on an EXT4 partition just fine, the only problem is the bootloader can't read the autoboot.txt file from such a partition, so you need another small partition for that.


Personally I've got my boot.txt and my kernel on a simple ext2 partition and the rest of my OS on an ext4 partition, which works fine except every time the kernel gets updated when I update the OS I need to remember to copy it over.


The system can boot from cold to an ext3 partiton just fine though, and that has journalling.

 

[Linux-SWAT]

Mmm I'm not sure OP can boot on ext3.

 

[TrashyMG]

Mmm I'm not sure OP can boot on ext3.

I'm fairly sure it can.. need to use ext3load vs ext2load in the boot arguments.

 

[dimag0g]

What about making a RAID level 1 array with two SD slots? Did someone investigate that subject?

 

[fusion_power]

I only use FAT32 onto my Pandora SD's, would like to use NTFS but that does not work sadly.

If using EXT, how to connect it to Windows then? Can Windows access EXT formatted SD's? Maybe through the USB mass storage mode?

 

[levi]

I believe there are third party tools for Windows to allow it to read EXT-formatted partitions, but by default, WIndows can't touch EXT partitions, true.  Mass storage mode should work though, and there are scp network copying tools I've used in the past on Windows, if your Pandora's wifi is up to it.

 

[Null]

Can Windows access EXT formatted SD's?

http://www.ext2fsd.com/

 

[Linux-SWAT]

What about making a RAID level 1 array with two SD slots? Did someone investigate that subject?

I did it a while ago. It works.

Wasn't the full SD, only non-system partitions on two SD cards.

I didn't try with a usb drive .

 

[TrashyMG]

I only use FAT32 onto my Pandora SD's, would like to use NTFS but that does not work sadly. If using EXT, how to connect it to Windows then? Can Windows access EXT formatted SD's? Maybe through the USB mass storage mode?

NTFS should work if you install the codec pack. I have it on one of my SD cards so I can deal with files over 4GB that can also talk to filthy Windows machines.

 

[levi]

You won't be able to boot off an NTFS partition though, or load an OS. If you're leaving the OS on the NAND though, you can run the codec pack installer on a temporary FAT or EXT partition, then swap to using both cards as NTFS.

 

[fusion_power]

Thanks for the tips. But I guess I stay with FAT32 for now, don't use my Pandora that often and so far files seems to be fine, no noticable errors. ^^

 

[Yoyobuae]

I only use FAT32 onto my Pandora SD's, would like to use NTFS but that does not work sadly.

If using EXT, how to connect it to Windows then? Can Windows access EXT formatted SD's? Maybe through the USB mass storage mode?

Mass storage mode is nearly the same as using an SD card reader. So Windows won't be able to read EXT formated cards thru USB mass storage either.
What you can do is create an NTFS image file on the EXT formatted SD then expose that to Windows using Mass storage mode.

First download ISO Optical Drive PND:

http://repo.openpandora.org/?page=detail&app=iso_optical_drive

Run it and leave the window which ask to choose an ISO file open. Now copy the file /mnt/utmp/iso_optical_drive/mountisodrive.sh (this file won't appear unless the PND is running) into the respective appdata dir on your SD card (ie: /media/PANCARDA/pandora/appdata/iso_optical_drive my SD card is labeled PANCARDA).

Open that file (the one in appdata dir) in a text editor and change this line:

modprobe g_file_storage file="${selection}" stall=0 cdrom=1Into this:

      modprobe g_file_storage file="${selection}" stall=0

(ie. delete the cdrom=1 parameter)
Now the ISO Optical Drive PND will expose the ISO image as a USB HDD instead of CDROM drive.

Now create the ISO image file (on a terminal window, also remember my SD card is named PANCARDA):

truncate /media/PANCARDA/disk.iso --size 4GActually the above command only creates an empty sparse file. Sparse means that the file doesn't consume space despite being 4GB in size. It will only consume space once you start writing stuff into it (pretty cool, huh?).
Now you can load the (empty) ISO file using the ISO Optical Drive PND, then plug in Pandora to Windows machine, format the USB HDD that appears to NTFS (make sure you are formatting the right one, check the disk size matches the size of the iso image) using the Windows PC and copy stuff to it or something.

If you want to access the contents of the ISO image from Pandora itself you can do on a terminal window (assuming you installed the Community Codec Pack):

mkdir /media/PANCARDA/disk/
sudo mount -t ntfs -o loop /media/PANCARDA/disk.iso /media/PANCARDA/disk/

(/media/PANCARDA/disk/ is a directory I created on my SD card to mount the image)

 

[Linux-SWAT]

Neat, is it on the wiki ?

 

[sepulep]

First, get a good card.

Second, use some EXT filesystem as the tools to repair it are shipped in any Linux.

A journaling file system is a must, I use ext4 on a 32GB SanDisk without any issues. I used ext2 before and files occasionally got corrupted when I exited applications with the Pandora button instead of the proper way.

well, it was a Patriot EP card, so not the cheapest

Also, I am aware of fsck etc, and in fact I would run the card through fsck every time the system crashes (and some more, as it does not often crash  ) ..in this case the file system seemed clean and repaired, but a lot of files would differ from a backup - and reading a broken file multiple times would also often give different results. It was an ext2 filesystem, but I don't think a journal would help here, cause that only helps tracking unfinished writes in case of a power off, it does not check or maintain file hashes..