32GB SDHC card problems...

[cosurgi]

I bought a class 10 32GB kingston (the red one), and I had nothing but problems with it.


I made two partitions: first a very big one in ext2, and at the end a small 500MB for swap. Then I tried to install Pandebian.pnd on it. And I got lots of filesystem errors. I tried several times, and everytime I got ext2 filesystem errors. So I decided to run

badblocks -c 4096 -s -w -t random -v /dev/mmcblk0

on it. It showed no errors. So I formatted again in ext2 and I had problems again. I returned this card to shop on warranty.



The shop owner is my friend, and he borrowed me another 32GB card, so that I will have anything when warranty is processed. I can give it back without any problems, so at least no money lost. Because the second card isn't working either! That is a class 10 pretec 32GB 233x. So I went into more investigations: the card is not fake - it definitely has 32GB capacity. I have the impression that it just hates ext2 filesystem. Here's what I did:



1. expecing troubles I made four partitions: 10GB, 4GB, 10GB, and the rest. I formatted each one with ext2. Right after formatting (and making sure that it is unmounted) I ran

e2fsck -C 0 -fy /dev/mmcblk0p1

on each partition. I was thinking that if card is fake, then at least the first partition would go without any errors. But all fsck's were showing thousands of errors! And even after the errors were fixed (fsck finished), a second, and third consecutive run of fsck was showing exactly similar looking thousands of errors (they scroll in terminal for 5 minutes at minimum). Then I thought that maybe card is a fake 4GB so I made first partition to be very small 3GB. And it still couldn't pass a simple test of

mkfs.ext2 ; fsck

- it had tons of errors.



2. I decided to make a different test: I formatted whole card into a single fat32 partition, and I created a random file with this commmand:

dd if=/dev/urandom of=./test1 bs=1MiB count=500

. And then I copied this file under different names until whole 32GB on the card was filled. Then I ran

md5sum *

on all files. And the result was correct: not a single file got corrupted. That is very similar to "badblocks" test, except that it is done by hand. All works.


Fortunately I can give back this pretec card back. But what is the problem? Can't I just format SDHC with ext2 ?


Do you have any idea?


Those cards are not the most expensive ones - roughly $40 - $50. I can find sandisk 32GB cards for $100. Does it mean that to have a working 32GB with ext2/ext3 I must pay extra and buy for $100 ?


BTW - during all that testing Pandora got unusually warm.

 

[Sugarfix]

Not that I would believe everything on Wikipedia, but...


the wiki article on ext2 suggests that it's the prefered filesystem for SD & USB due to it's lack of a journal which reduces the number of writes. Apparently.


So it would appear from my cursory glance at the interwebs that you should be OK with ext2, but I'm afraid I can't offer you any advice as to why you're having such problems.


Enter Prometheus, stage left.

 

[Sugarfix]

Here's another option that someone else suggested: Try ext3 or ext4 and see if you get the same errors.

 

[cosurgi]

Here's another option that someone else suggested: Try ext3 or ext4 and see if you get the same errors.

ext3 and ext4 don't work either. I just tried.

mkfs.ext4 /dev/mmcblk0p1 ; fsck -C0 -fy /dev/mmcblk0p1 ; fsck -C0 -fy /dev/mmcblk0p1

Even though the first fsck claims that all errors are fixed, the second (and 3rd and 4th) one still finds the same errors, and fixes them again.


EDIT: but remember that badblocks test is passing, and also copying back and forth 32GB of files in fat32 test is passing too. It's just ext2/3/4 and its bitmap or inodes that just can't work.


EDIT2: and ext2/3/4 can't work on two different SDHC 32GB cards (kingston & pretec). This is so strange.


- http://images.google.com/images?q=class+10+kingston+32GB


- http://images.google.com/images?q=class+10+pretec+32GB+233x

 

[mindlord]

I hate to say it, but you're just having some bad luck there. I have a 16GB card that simply will not work with an EXT filesystem as well. But I have a 16GB Kingston (blue) and Transcend 32G class 10 card that works like a charm. The Pandora just seems to not like EXT and some cards... for some reason.

 

[Prometheus]

Enter Prometheus, stage left.

:lol:


I doubt this is a fake card, in this instance. I know that several people, myself included, are encountering problems using fast cards (Class 6 and Class 10) with the Pandora at the moment, and in some cases this extends to using particular file-systems on those cards. I presume it's a driver issue, but because it's very difficult for me to replicate the issues I'm encountering, I can't submit it on the bug-tracker.

 

[pisiiki]

I have problems with a transcend 64GB SDXC card too. No way of making it work, I have been trying it for days now without success. Btw it is not that the card doesn't "like" ext2 or ext3 as you said because my card works flawlessly in a pc with ubuntu or gentoo, so I guess it is a mmc driver problem or something related. Also confirmed that it is not a sdhc slot problem in my pandora because it fails in both slots and other sd cards that I have tested up to 8GB work also in both slots.


I will try to do a loopback ext2 within a fat32 to see what happens, maybe this does the trick until it gests fixed. If someone tests this before me please post your results.

 

[cosurgi]

I will try to do a loopback ext2 within a fat32 to see what happens, maybe this does the trick until it gests fixed. If someone tests this before me please post your results.

IIRC there's one problem with loopback on fat32 - max file size limit in fat32 is 4GB. So to get 32GB fs on fat32 we would need to try lvm or mdadm (to join the 4GB loopback devices together)


EDIT: maybe I'll try that later.

 

[Letalis Sonus]

The UBIFS documentation (the filesystem used on the Pandora's NAND) might reveal the problem:

FTL stands for "Flash Translation Layer" and it is software which emulates a block device on top of flash hardware. At early days FTL ran on the host computer. For example, old PCMCIA flash devices were essentially raw flash devices, and the PCMCIA standard defined the media storage format for them. So the host computer had to run the FTL software driver which implemented PCMCIA FTL. However, nowadays FTL is usually firmware, and it is run by the controller which is built into the storage device. For example, if you look inside an USB flash drive, you'll find there a NAND chip (or several of them), and a micro-controller, which runs FTL firmware. Some USB flash drives are known to have quite powerful ARM processors inside. Similarly, MMC, eMMC, SD, SSD, and other FTL devices have a built-in controller which runs FTL firmware.

[...]


Good FTL must be a rather complex piece of software, especially if it deals with MLC NAND (which is used in most modern mass storage devices). Implementing it in firmware might be a difficult task, and running it might require a powerful controller. Instead, we suspect that vendors may user various tricks or compromises to keep their devices "good enough" and cheap. For example, it is known that some vendors optimize their FTL devices for FAT, and if you start using ext3 on top of one, you might face some unexpected problems or the device may become worse than you would imagine. Of course, it is often difficult to verify this with closed FTL.

Maybe someone should set up a Wiki page with a list of SD cards known to be working good with ext filesystems.

 

[Prometheus]

^ The only problem is, these sorts of problems are also occurring on fast non-ext cards...

 

[Letalis Sonus]

IIRC there's one problem with loopback on fat32 - max file size limit in fat32 is 4GB.

NTFS is a FAT based filesystem as well and might work. It would at least remove the 4GB limit.

 

[Poem58]

I am not a technical genius like others here but I had a thought. Could it be possible the the memory controllers on certain cards could be the problem? Since I have no Linux experience and thus none with these filesystems, I cannot do anything but guess. If the controllers themselves are not playing nice with those filesystems could they be causing the errors while they wear level etc. Might be something to look into, but I doubt it's easy to find out what's controlling different cards to make a database or even enough info to verify that's the problem.


My apologies if I am way off.

 

[pisiiki]

I will try to do a loopback ext2 within a fat32 to see what happens, maybe this does the trick until it gests fixed. If someone tests this before me please post your results.

IIRC there's one problem with loopback on fat32 - max file size limit in fat32 is 4GB. So to get 32GB fs on fat32 we would need to try lvm or mdadm (to join the 4GB loopback devices together)


EDIT: maybe I'll try that later.

You are right, the max file size will kill the effort. Maybe a kernel rebuild with ntfs support is a faster test?

 

[BaDToaD]

Just for info. I've got a class 4 32GB Dane-elec card that I've been using for a couple of months for PSX and N64. I have also recently partinioned 512MB off for swap via Gparted and it performs perfectly. It is however formatted as FAT32

 

[Prometheus]

You are right, the max file size will kill the effort. Maybe a kernel rebuild with ntfs support is a faster test?

You can add ntfs support by installing the Community Codec Pack. Or is there some reason that rebuilding the kernel is a better option for this?

 

[cosurgi]

Maybe someone should set up a Wiki page with a list of SD cards known to be working good with ext filesystems.

OK, I created a new page on our wiki. Can you guys add your cards?


pandorawiki: SDHC compatibility list

 

[cosurgi]

pandorawiki: SDHC compatibility list

Gruso, maybe you could encourage people to add their cards to this list, by posting on PandoraPress?

 

[Christoph.Krn]

Typically, at least with ext3 the journaling isn't a problem if you're using the "noatime" mount option. See "SSD’s, Journaling, and noatime/relatime" by Theodore Ts'o.

^ The only problem is, these sorts of problems are also occurring on fast non-ext cards...

I've heard of that too (I don't have a Pandora yet) and I think it's quite weird. Did I understand you correctly in that the assumption that these problems arise for the same reasons is not yet verified?


Here's another idea: maybe trying NILFS on an SD card that's being used with the Pandora will yield new insights. With NILFS, all writes will automatically be sequential, which even most (if not all) of the cheaper controllers should "get" and stop getting confused. Anyone attempting to use NILFS on an SD card that wasn't working correctly with another file system before should use the official Panasonic SD card formatter if possible (i.e. if you have Windows) before formatting the card with NILFS, so the controller's mind will be reset as much as possible.

I am not a technical genius like others here but I had a thought. Could it be possible the the memory controllers on certain cards could be the problem? Since I have no Linux experience and thus none with these filesystems, I cannot do anything but guess. If the controllers themselves are not playing nice with those filesystems could they be causing the errors while they wear level etc. Might be something to look into, but I doubt it's easy to find out what's controlling different cards to make a database or even enough info to verify that's the problem.


My apologies if I am way off.

You are actually very right! [1] However, the controllers /might/ be the problem, but then again, maybe it's something else entirely that causes these problems.

Maybe someone should set up a Wiki page with a list of SD cards known to be working good with ext filesystems.

OK, I created a new page on our wiki. Can you guys add your cards?


pandorawiki: SDHC compatibility list

The problem with such kind of list is that the characteristics of some brands of SD cards may be different depending on, amongst other things, which brand it is, where you bought it, and when you bought it. [1]


The list could at least need an additional data field that tells the reader whether or not it is verified that a brand's innards have rather constant characteristics across cards. However, this information is usually hard to get / find out [1], so we might end up with very little reliable information. But at least in my humble opinion that's better than having loads of information of questionable usefulness.


------


[1] http://boards.openpandora.org/index.php?/topic/1046-whos-is-going-run-the-os-from-the-sd-card/page__view__findpost__p__19461

 

[Prometheus]

^ The only problem is, these sorts of problems are also occurring on fast non-ext cards...

I've heard of that too (I don't have a Pandora yet) and I think it's quite weird. Did I understand you correctly in that the assumption that these problems arise for the same reasons is not yet verified?

You understand me correctly, yes. I have no way of knowing if the reasons are the same, but they seem quite similar on the surface, at least - that is, "fast cards = problems with corrupted files and other issues".


My own experience has been that the very same cards behave perfectly fine with any other Linux or Unix system that I use them with...

 

[cosurgi]

...

OK, I created a new page on our wiki. Can you guys add your cards?


pandorawiki: SDHC compatibility list

The problem with such kind of list is that the characteristics of some brands of SD cards may be different depending on, amongst other things, which brand it is, where you bought it, and when you bought it. [1]


The list could at least need an additional data field that tells the reader whether or not it is verified that a brand's innards have rather constant characteristics across cards. However, this information is usually hard to get / find out [1], so we might end up with very little reliable information. But at least in my humble opinion that's better than having loads of information of questionable usefulness.


------


[1] http://boards.openpandora.org/index.php?/topic/1046-whos-is-going-run-the-os-from-the-sd-card/page__view__findpost__p__19461

OK, so we need to add an appropriate disclaimer there. For now it looks like nobody has a working 32GB or 64GB card.


For example, based on your lenghty explanations in [1] I started to look for 32GB Sandisk Eextreme class 6, but apparently it has never been produced:


* class 6 sandisk extreme (max 16GB): http://www.sandisk.com/products/camcorder/sandisk-extreme-hd-video-sdhc-cards


* class 10 sandisk extreme (max 32GB): http://www.sandisk.com/products/imaging/sandisk-extreme-sdhc-cards-


From what Prometheus says class 10 are more likely to fail, so I would rather prefer to stay off. There is however some chance that I could test this card before buying it - I'll need to talk with my friend in the computer store.


Also, from what you say "Sandisk Ultra" are not going to work? I ask, becuase the 64GB card exists only in this one flavour:


* class 4 sandisk ultra, 64GB: http://www.sandisk.com/products/dslr/sandisk-ultra-sdxc-cards


Also I looked for Panasonic, but they are not sold in Europe at all - no chance to get one.


I would have never suspected that buying a ext2/3/4 working 32GB card would be such a PITA. Could you help me a bit with this? (I already RMA-ed kingston 32GB (red), but in fact it's just lost money for me - now it turns out that it was broken "by design")


What should I do?


* try a 32GB class 10 sandisk extreme?


* settle for 16GB class 6 (safest bet)?


* risk 64GB ultra?


There are also "Extreme III" variants, which nobody tried yet, apparently. But they seem to be faster ones, therefore more eligible to not working with ext2?


other options?