hdonk
Active Member
- Joined
- Sep 12, 2008
- Messages
- 572
That's interesting as I have a 32Gb Transcend that doesn't work... I'll try it again when my Pandy returns today.
32GB SDHC card problems...
- Thread starter cosurgi
- Start date
Christoph.Krn
Advanced Member
Neither of both. It looks like this misunderstanding was caused by my non-perfect English, for which I'm sorry. I edited my previous post:I'm not sure if I've been misread or if I'm misreading you here, so either way please pardon me, but I have no idea about combinations of these cards. I don't have access to a Class 10 one - I just said that I've heard separate reports of a few instances of problems with Class 6 as well as Class 10.
"That being said, as Prometheus said there ARE indications that the combination of /some/ higher class (class 6 as well as class 10) SD cards with a (particular kind of?) Pandora does lead to problems."
Yes, I was aware that you specialize in warning about fakes. It's good that you say you're not an expert. If someone calls him-/herself an "expert", more often than not that's a very bad sign (unless there's a very, VERY good reason for him/her to do so). By various logical conclusions, an expert would have to have comprehensive knowledge, which is something that's hard (usually impossible) to reach, no matter about what subject.
The amount of knowledge a person has doesn't really matter - no one is born with broad knowledge about SD cards, or Linux, or quantum mechanics. What matters a lot though is that, unlike many other people, you are very aware of the limitations of your knowledge and abilities (example), and that is a VERY GOOD THING™. I'm not an SD card "expert" either. But I'm usually aware of what I don't know, and I usually don't decide anything before I haven't evaluated all possible series of events that I can conceive. Another factor is being in-depth with everything that I'm interested in, so I can utilize knowledge that others would call trivial and unimportant in order to better construct or gather information about what I still don't know - ever heard about Megaman? What I'm trying to say is: you probably have more knowledge (and, more importantly, "not-knowledge") than you believe you do, but then again what do I know...
Your posts often contain useful information that is "sanified" (i.e. people can usually trust the useful information that can be extracted from your posts). Information that you provide can typically be relied upon, which is what matters most.
Well done. (I don't say that easily.) (You will now probably realize how little this "Well done." means.
)</uninteresting-sidenote>
No doubt this is a good idea. We could also need as many reports of working cards as possible.
Hey, that would be a good idea! Of course, this would only tell us if a card had corrupted files on it and give no clue about the actual cause, but that would still be better than nothing.Do we have a good reproducable test?
It woudl be cool to make a pnd that just does a few file-creates-and-md5sums tests and reports if SD seems working or not. For 1) test we can all use, repeatably and easy, and 2) Peace of mind
Maybe the following could be an acceptable approach for testing SD cards on a symptomatic level:
- Format the card using the official Panasonic SD card formatter if possible (unfortunately, Panasonic's SD card formatter is not open source and only available for Windows).
- The test fills the card half with files the size of the SD card's erase block size, checking the integrity of each file before creating the next one.
- It then deletes all files and repeats step 1, only this time until the card is filled with files, then check integrity of all files.
- delete every alternate file
- Fill with files, then check integrity of all files.
The result of the test should contain the information if the card was formated with the official Panasonic SD card formatter (1.).
I dont know enough about various filesystems, and there are probbly be other things that I have not taken into account here, so that test might not be a good one at all - it's only good on a symptomatic level (it can only tell us if a card is probably not corrupt, but does not maximize the chance that we can use it to find the cause). But for various reasons I still think it's better than a more random file-create-and-md5summing.
Maybe this is helpful for you: http://boards.openpandora.org/index.php?/topic/1062-sd-cards/page__view__findpost__p__20300
Last edited by a moderator:
Prometheus
Well-Known Member
- Joined
- Mar 8, 2008
- Messages
- 9,472
^ This is thoroughly not on-topic, but I just wanted to say thankyou for what you said just there, Christoph.
I wish I could contribute a bit more with testing these issues, but I have no machines running Microsoft Windows (and actually know nobody who has one), and the Panasonic SD Formatter doesn't work under WINE... (Unless there is some trick I'm missing, in which case I'll be grateful if someone can point me to it so I can implement it on my system.)
Curse you, Panasonic!
I wish I could contribute a bit more with testing these issues, but I have no machines running Microsoft Windows (and actually know nobody who has one), and the Panasonic SD Formatter doesn't work under WINE... (Unless there is some trick I'm missing, in which case I'll be grateful if someone can point me to it so I can implement it on my system.)
Curse you, Panasonic!
Last edited by a moderator:
cosurgi
http://janek.kozicki.pl/
Yes, this test is better than md5summing. However I suspect that this test will still pass, while ext2/3/4 fsck will fail.
That is because:
1) What I've done is inferior, but I have have done this test a lot by hand: creating and md5summing files. And copying them, it was always OK.
2) also I had run a badblocks test (see OP), it was always OK.
3) but still mkfs + fsck failed monumentally (think: terminal was scrolling the errors for 5 minutes at least, that's like a few tens of thousands of errors)
I know little about ext2 architecture. The only thing that means anything to me, was when I was running an e2defrag tool (screenshot), which was displaying disc contents like old DOS diskscan. I saw that fs-specific data (inodes, directories, attributes) are scattered all along the disc (those cyan rectangles). In FAT all that data is stored at the beginning of the disc. When you write anything to ext2 bits of data are updated in several of those rectangles (at least), maybe even all of them. They have backups of each other scattered all along the disc, and so on.
From what I know about flash memory, and I know just a little too, and maybe wrong, too - is that writing a puny small piece of data to flash memory (even just a single byte), requires overwriting a whole physical block in the flash memory chipset. This block may be 512 bytes, or sth like that. Even if you write only only byte, whole 512 bytes are first read, single byte modified, then all 512 bytes are wrote back.
It is obvious that optimizing for FAT writes is really simple: FAT is organized in IIRC 512 blocks, it writes data sequentially, from start to end. And it doesn't do any weird lookups at many places at a time, unless there's heavy fragmentation. And even if data is fragmented it still is sliced into 512 byte chunks.
IIRC ext2 is not fighting the "traditional" organization of physical media into 512 byte blocks, or 4096 bytes, etc... It just uses it a bit more extensively than plain FAT (since FAT just writes from start to end).
You wrote a nice piece about not being an expert - so of course I am not. That is what I roughly remember, and how I am connecting the dots here to explain the mysterious flash problems.
EDIT: and worse thing: ext2/3/4 have bitmaps. It means, that it is often updating just a single bit of data. And the overhead is to read & re-write whole 512 bytes. The worst is that I can't refrain from using posix filesystem, I would rather have a properly working SD card.
Last edited by a moderator:
Elanzer
Member
- Joined
- Aug 29, 2008
- Messages
- 194
- Age
- 38
Just got around to trying it with one of my Patriot LX Class 10 32GB SDHC cards, using the testing method cosurgi outlined on page 2 with ext2 formatting and e2fsck right after.
It spammed pages and pages of numbers, which I assume are errors, then e2fsck said it had to fix them and modify the filesystem.
It looks like these cards also failed the test. I haven't experienced any file corruption under FAT32 yet however.
edit: on the upside, these seem to have the best performance in read/write so far of all the 32GB cards listed on the wiki.
It spammed pages and pages of numbers, which I assume are errors, then e2fsck said it had to fix them and modify the filesystem.
It looks like these cards also failed the test. I haven't experienced any file corruption under FAT32 yet however.
edit: on the upside, these seem to have the best performance in read/write so far of all the 32GB cards listed on the wiki.
Last edited by a moderator:
cosurgi
http://janek.kozicki.pl/
exactly, that's how it looks like. Also - it claims to have fixed the errors, but didn't. If you run fsck again - they will be still there. Even if you run consecutively ten fsck checks, the errors will still be there.
Last edited by a moderator:
Neelix
Insecticidal Maniac
I just picked up two 32gb SD cards and tested them with a variation of cosurgi's test:
Sandisk 32GB SDHD Class 4
loaded in gparted
had 4 Mib unallocated area + 29.72 GiB fat32 partition
deleted partition
created & formatted new ext2 partition with minimum unused space (1 Mib)
e2fsck 1.41.9 (22-Aug-2009)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mmcblk0p1: 11/1949696 files (0.0% non-contiguous), 133613/7791488 blocks
Patriot LX series 32GB SDHC Class 10
loaded in gparted
had 4 Mib unallocated area + 29.90 GiB fat32 partition
deleted partition
created & formatted new ext2 partition with minimum unused space (1 Mib)
e2fsck 1.41.9 (22-Aug-2009)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mmcblk0p1: 11/1962240 files (0.0% non-contiguous), 134401/7838464 blocks
No errors on either card. After I completed both tests I realised that my Pandora was over-clocked at the time, so I rolled it back from 900 @ OPP4 to 600 @ OPP3 and ran the second fsck again with exactly the same results.
I'm now intending to use the patriot as my boot card, so I'll see how I go with it once I have a rootfs on there and a debian extend, but I have to say, so far so good.
Sandisk 32GB SDHD Class 4
loaded in gparted
had 4 Mib unallocated area + 29.72 GiB fat32 partition
deleted partition
created & formatted new ext2 partition with minimum unused space (1 Mib)
e2fsck 1.41.9 (22-Aug-2009)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mmcblk0p1: 11/1949696 files (0.0% non-contiguous), 133613/7791488 blocks
Patriot LX series 32GB SDHC Class 10
loaded in gparted
had 4 Mib unallocated area + 29.90 GiB fat32 partition
deleted partition
created & formatted new ext2 partition with minimum unused space (1 Mib)
e2fsck 1.41.9 (22-Aug-2009)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mmcblk0p1: 11/1962240 files (0.0% non-contiguous), 134401/7838464 blocks
No errors on either card. After I completed both tests I realised that my Pandora was over-clocked at the time, so I rolled it back from 900 @ OPP4 to 600 @ OPP3 and ran the second fsck again with exactly the same results.
I'm now intending to use the patriot as my boot card, so I'll see how I go with it once I have a rootfs on there and a debian extend, but I have to say, so far so good.
Provost Dhorkan
Still Fresh
- Joined
- Feb 9, 2011
- Messages
- 69
Any one tried one of these? I have 2 waiting for my Pandora, hope they work...
http://www.shopping.com/PQI-PQI-16GB-SDHC-Class10-150X-flash-memory-card/info
http://www.shopping.com/PQI-PQI-16GB-SDHC-Class10-150X-flash-memory-card/info
Granitehead
Advanced Member
- Joined
- Oct 16, 2009
- Messages
- 3,011
As far as I know (best you don't believe me) Transcend uses various different chips which they buy from different companies depending on what is cheapest at the moment. So it would be normal for some to work and for others not to work.
Several cards are now marked yellow in the wiki, as "partially working". I don't think flash controllers that attempt to guess which blocks are free by looking at the file allocation table should really be considered "working", as these are basically defective by design.
I have a Kingston card that I suspect may be doing this, and FAT32 seems to work at first, but I got it to fail by writing a large amount of data at once. Perhaps due to the write-back cache, the FAT doesn't get updated fast enough, so it erases the wrong blocks. The erase timing is probably just enough to work in Windows and most cameras, but can't cope with >100MB writes without the FAT being rewritten.
I have a Kingston card that I suspect may be doing this, and FAT32 seems to work at first, but I got it to fail by writing a large amount of data at once. Perhaps due to the write-back cache, the FAT doesn't get updated fast enough, so it erases the wrong blocks. The erase timing is probably just enough to work in Windows and most cameras, but can't cope with >100MB writes without the FAT being rewritten.
Last edited by a moderator:
hdonk
Active Member
- Joined
- Sep 12, 2008
- Messages
- 572
I did some investigating with the Kingston card to determine why it fails. With the card formatted FAT32, I wrote several large files, verified them using md5sum, then deleted the files. Looking at the raw block device, most of the data remained intact. It seems to erase only a few megabytes at a time.
The way flash controllers usually do wear levelling is before it erases a group of blocks, it copies any valid data (blocks that haven't been overwritten) to another area. Then it erases the group of blocks. The Kingston card seems to cheat a bit on the copying step, by not copying blocks that it thinks are unused. So a handful of 'unused' blocks will get erased from time to time as other blocks are written.
Basically this is a scam to increase write-speed benchmarks at the expense of data integrity. This is yet another false-spec flash ripoff, and should be regarded as such. Unsurprisingly, the write speed of this card varies a lot. It starts out fast, but gets slower depending on how many blocks it can erase but not copy when writing new blocks.
The thing is, when it works, it works really well. I got 9.5 MB/sec writes on this class 4 card, and the md5sums verified. The problem is, the card is unreliable and writes sometimes fail.
The way flash controllers usually do wear levelling is before it erases a group of blocks, it copies any valid data (blocks that haven't been overwritten) to another area. Then it erases the group of blocks. The Kingston card seems to cheat a bit on the copying step, by not copying blocks that it thinks are unused. So a handful of 'unused' blocks will get erased from time to time as other blocks are written.
Basically this is a scam to increase write-speed benchmarks at the expense of data integrity. This is yet another false-spec flash ripoff, and should be regarded as such. Unsurprisingly, the write speed of this card varies a lot. It starts out fast, but gets slower depending on how many blocks it can erase but not copy when writing new blocks.
The thing is, when it works, it works really well. I got 9.5 MB/sec writes on this class 4 card, and the md5sums verified. The problem is, the card is unreliable and writes sometimes fail.
Last edited by a moderator:
I don't know what's going on with this Kingston card. I can successfully format it ext2, and it works and blocks don't get erased. But if I write a lot of data, the controller simply crashes and becomes unresponsive.
What I do know is that I'm not buying any more cards from Kingston.
What I do know is that I'm not buying any more cards from Kingston.
cosurgi
http://janek.kozicki.pl/
yeah, I hate kingston too. From now on I'm buying only sandisk.
I received that 32GB class 4 sandisk ultra, and it was the first 32GB SDHC card that worked with ext2. I updated the SDHC list accordingly - it works really good.
Good that I was able to return pretec 32GB without losing any money, but I can't do this with my kingston 32GB. I totally wasted money on this one. I am thinking that we could do some lawsuit against kingston and demand that they label their cards as "FAT Only !!!". Too bad I have no time for that. But maybe collectively we all do?
Last edited by a moderator:
Well it's not really "FAT Only" because this card does work with ext2, but it's got other issues. The problem is that Kingston is buying chips from different manufacturers and you never know what you're getting.yeah, I hate kingston too. From now on I'm buying only sandisk.
I received that 32GB class 4 sandisk ultra, and it was the first 32GB SDHC card that worked with ext2. I updated the SDHC list accordingly - it works really good.
Good that I was able to return pretec 32GB without losing any money, but I can't do this with my kingston 32GB. I totally wasted money on this one. I am thinking that we could do some lawsuit against kingston and demand that they label their cards as "FAT Only !!!". Too bad I have no time for that. But maybe collectively we all do?
Read this for an investigation of what was actually inside some Kingston cards.
I would also recommend Sandisk.
Christoph.Krn
Advanced Member
Cards that get "confused" by a lot of stress (whatever "a lot of stress" may mean) are one of the reasons for why suggested to test cards the way I did in this previous post.
Maybe a "combined" test that combines consurgi's formatting test with a write test similar to what I recommended would be most useful, as that would seem to cover most of the experienced problems found so far.
------
Maybe a "combined" test that combines consurgi's formatting test with a write test similar to what I recommended would be most useful, as that would seem to cover most of the experienced problems found so far.
------
This is true, many other manufacturers also do this and it's probably the biggest source of sd card related problems in general. Two exceptions are Sandisk and Panasonic, which I already recommended in http://boards.openpandora.org/index.php?/topic/1046-whos-is-going-run-the-os-from-the-sd-card/page__view__findpost__p__19461
Last edited by a moderator:
I sold my Kingston card and bought a Lexar 64GB card, which works.
This card is seriously fast, with 19.6MB/sec reads and 15.1MB/sec writes. However the pandora is limited to about 16.2/11.3 respectively. Seems fine with ext2/ext3, no fsck errors. I wrote several GBs of files and md5ed them.
My only concern with this card is that the power draw may be a little high. It consistently fails with one of my older USB readers, either failing to power up or shutting down within a few seconds of starting any read/write operation. It does seem to work okay in the pandora, but I'm not sure what might happen in a low battery situation. I tested it with the CPU overclocked, copying 5GB of files from a flash drive plugged into the USB port, and the md5sums matched.
Although I couldn't get the Lexar card to fail in the Pandora under high load, I'm a little nervous about it since it does fail in one of my other devices. If you don't need the speed, it might be worth it to get the Sandisk card even though it's only $5 less.
BTW, both the Sandisk and Lexar 64G cards came with the partition table configured with the first sector of the partition being 32768. I left it as is, although it wastes a few MB of space. I'm guessing this is for block alignment purposes.
This card is seriously fast, with 19.6MB/sec reads and 15.1MB/sec writes. However the pandora is limited to about 16.2/11.3 respectively. Seems fine with ext2/ext3, no fsck errors. I wrote several GBs of files and md5ed them.
My only concern with this card is that the power draw may be a little high. It consistently fails with one of my older USB readers, either failing to power up or shutting down within a few seconds of starting any read/write operation. It does seem to work okay in the pandora, but I'm not sure what might happen in a low battery situation. I tested it with the CPU overclocked, copying 5GB of files from a flash drive plugged into the USB port, and the md5sums matched.
Although I couldn't get the Lexar card to fail in the Pandora under high load, I'm a little nervous about it since it does fail in one of my other devices. If you don't need the speed, it might be worth it to get the Sandisk card even though it's only $5 less.
BTW, both the Sandisk and Lexar 64G cards came with the partition table configured with the first sector of the partition being 32768. I left it as is, although it wastes a few MB of space. I'm guessing this is for block alignment purposes.
Here's a little educational note about the ext filesystems and journaling on flash drives. It's basically not true, except perhaps if write amplification is a concern:
http://thunk.org/tytso/blog/2009/03/01/ssds-journaling-and-noatimerelatime/
Here's the Google cache version, since the above link seems to be down at the time of this posting:
https://webcache.googleusercontent.com/search?q=cache:thunk.org/tytso/blog/2009/03/01/ssds-journaling-and-noatimerelatime/
There's also a mangled repost of it here in case the Google cache link becomes useless too:
http://www.linuxfoundation.org/news-media/blogs/browse/2009/03/ssd%E2%80%99s-journaling-and-noatimerelatime
http://thunk.org/tytso/blog/2009/03/01/ssds-journaling-and-noatimerelatime/
Here's the Google cache version, since the above link seems to be down at the time of this posting:
https://webcache.googleusercontent.com/search?q=cache:thunk.org/tytso/blog/2009/03/01/ssds-journaling-and-noatimerelatime/
There's also a mangled repost of it here in case the Google cache link becomes useless too:
http://www.linuxfoundation.org/news-media/blogs/browse/2009/03/ssd%E2%80%99s-journaling-and-noatimerelatime
