Formatting a SD card : demystifying !

[Christoph.Krn]

Do you want the color too ?

It is your decision whether or not you want to take this seriously.

According to your link, using hardware tools (or software bench) must be wayyyyyy faster than manually making tries.

The information contained therein is not limited to the Arduino platform. An automated software-based benchmark would of course be faster than manual software-based inspection, but I do not know of any reliable SD card benchmarking software. There probably is none, for lack of a proven and well-known systematic way of testing that would be generally applicable.

 

[Linux-SWAT]

Can't take SD cards seriously the way they are. "Secure Digital" rofl.


Maybe, like some hardware pieces, only defense departments can use it 100% safely.


I think black cards are better than blue haha.

 

[Grench]

I think that this 4MB thing may be how internet myths are born.


Gparted works just fine thank you.


The wiki isn't always right. For example, I use 32MB Patriot LX Class 10 cards. I use Fat/32 on the right slot, the left slot is Ext2 (boot), Fat/32 (applications) and Linux SWAP.


Both cards have the first partition at the start of the card.


I've been using them heavily since April and never had an issue.


I also have Patriot 16GB cards (2), a Kingston Class 4 32GB card, and several Transcend, A-data, Kingston, SanDisk and Patriot cards of 128 to 4GB in capacity.


I have NEVER used that formatting tool or wasted the first 4MB of a card under formatting.


The ONLY time I have EVER had an issue with a card was on a "8GB" card that I bought on Ebay - it was a fake card and I got Ebay to refund it (pain as it was). Won't do that again.


Linux-SWAT - there is no way that you have had the issues you described across brands and card types as you described it.


If you are having "issues with the first and last portion of a card" then state the exact manufacturer, card model, card age and where you purchased it from. Otherwise you're myth/scare mongering.

 

[Linux-SWAT]

Troll.


K thx bye.

 

[Prometheus]

The wiki isn't always right. For example, I use 32MB Patriot LX Class 10 cards. I use Fat/32 on the right slot, the left slot is Ext2 (boot), Fat/32 (applications) and Linux SWAP.


Both cards have the first partition at the start of the card.


I've been using them heavily since April and never had an issue.

That doesn't necessarily mean that the wiki isn't right, though.


The likely cause of the disparity here is the fact that some manufacturers are not consistent with what parts they use and are restickerers - for example, some users have said the same of the Kingston cards listed there, whilst others had issues. And there was also the case of the cards sold by 7DayShop that were popular here for quite some time, until they started selling a batch that had different parts in the cards which consequently didn't play well with the Pandora even though the previous batch of the exact same cards did, at which point recommendations for them seemed to stop.

 

[Linux-SWAT]

Agree.


And take a look at this :


http://hardware.slashdot.org/story/10/02/16/1454246/quality-concerns-for-kingston-microsd-cards

 

[Letalis Sonus]

There could be several reasons to align a partition properly, leaving those almost 4MB unused, and it may as well be important for some dirty tricks of the FTL. If the FTL is confused, corruption of data is the consequence. It's the same reason why many cards don't play nice with ext file systems: The FTL's dirty tricks may expect a file allocation table at a certain position.

 

[Grench]

Obviously though - I'm not going to take the original poster's word for it as I have a pile of real world examples of SD and SDHC cards that directly conflict with those statements and NO specific examples have been stated in support of the claim. Challenges to this have been met with non-answers.


From what I can tell, the topic of this thread has not been proven true on any branded card acquired from a reliable and 100% legit retail source. No examples of X card from Y brand acquired from Z source on D date was malfunctioning after a successful long format and doing -this- for partitioning then doing a long format resulted in a card that was reliable of N0,000 random read/write cycles over the card.


The fact is - no example of any specific card has been given that this thread's assertion applies to.

 

[Asmo]

EDIT - wrong thread

 

[Stephane Hockenhull]

+1 to Grench


for the part at the beginning,


it may align the filesystem structures in a more SD-card friendly way so that data ends up better aligned to flash erase blocks.


4MB is still overkill nowadays IMHO, even 512KB would be enough for a lot of SD cards.


it depend on the erase block size.


as cards get larger and erase block sizes gets larger as well you might try to align to larger values but I doubt it'll make much of a difference past 4MB.


as far as I know the part left out at the end is only useful on a fresh SD card (or SSD) if you've never used it or can somehow tell the flash controller to discard all data.


and only if the wear-leveling controllers will use never-been-used blocks to do extra wear leveling and move data around, you're basically reserving this area of the flash as "spare".


but without knowing exactly how the FTL and wear-leveling controller works on a particular SD card it may end up being just wasted space if the controller does nothing with it.


IF and it really is *IF* the controller uses never-been-used space as spare, you can reserve as much space as you want at the end, the card may last longer with frequent writes, it may write faster by keeping pre-erased blocks for quicker writes and relocations, etc.


but that's a big >>> IF <<<


NOTE: personally, I don't bother doing this and don't intend to but its an interesting theory, and aside from wasting space it can't hurt the card.

 

[Linux-SWAT]

And it doesn't hurt the troll so the fun is safe ^^.


Anyway, there is not IF when formatting with the official tool, and then watching the geometry with GParted or whatever.


But i never checked with a white card, only blue and black, so this is not a rock-solid statement .

 

[Christoph.Krn]

Anyway, there is not IF when formatting with the official tool, and then watching the geometry with GParted or whatever.

Please, unless you have verified that this is everything Panasonic's official SD card formatter is doing, do not state it as a fact, because otherwise some people may think did verify it. SD cards can have different erase block sizes, which means that the ideal formatting layout can vary between SD cards. Aside from any other shenanigans the official formatter might be doing, it appears unlikely that it would not respect this.

 

[Grench]

Christoph.Km & Stephane Hockenhull


Be careful - asking for actual data to back up blanket statements or calling BS on them may result in your being labeled as a troll. Being able to back up your own statements will be deemed irrelevant. Quoting back the repeated blanket statements will be stated as your misinterpretation.


Still - thank you for voices of reason.


Being a troll isn't so bad in the end. The tolls from the bridge are pretty good. With the world population boom there is plenty to eat. Challenging people to back up and re-consider what they 'know' - although the responses can be entertaining, that isn't the main purpose. Statements become myths become legend become lore - what we all think we know is often based on assumptions or outliers.


In this case, we have blanket statements supposedly covering all cards. Attempts to find out which cards the OP has experience in applying his methodology on and whether or not they fail without it have been met with sarcastic non-answers. Ya, I'll probably get 4 page replies chastising my existence again.

 

[Christoph.Krn]

Be careful - asking for actual data to back up blanket statements or calling BS on them may result in your being labeled as a troll.

Whenever I do that, I have no intention to denounce someone whatsoever, and also I do not find situations in which someone does not have a random piece of knowledge that I do have to be entertaining in the slightest. I'm just trying to be as unbiased as possible because I want to understand the truth.


It's a fallacy to assume it made sense to be a troll -- it is never a logical solution to a dispute. Anyone who chastises your existence surely won't make the world a better place by doing it. In a similar vein, sometimes finding being a troll to be not so bad in the end won't stop alleged misinformation from being spread, or fallacies from being made, either.


Rien ne va plus, back to topic everyone please -- agreed?

 

[gruso]

This SD card information war is getting ridiculous. There is a lot of good information being shared, which is unfortunately being tainted by character attacks. Different points of view are expected (this usually stems from different personal experiences, after all) but it would be nice if people could present these views on their own merits, without taking jabs at others.

 

[Grench]

This SD card information war is getting ridiculous.

Agreed.

There is a lot of good information being shared, which is unfortunately being tainted by character attacks. Different points of view are expected (this usually stems from different personal experiences, after all) but it would be nice if people could present these views on their own merits, without taking jabs at others.

There is also a lot of incomplete, poorly supported and outright misinformation being shared too.


Requests for the actual circumstances and or proof of the claims being made are being answered with clearly inaccurate statements:


"FAT16, FAT32, EXT2, EXT3, ReiserFS.


Many well-known brands, kingston, transcend, sandisk... SD, Micro-SD and Mini-SD.


Do you want the color too ?"


and dismissal:


"Troll. K thx bye."


With lack of substantiation, it SHOULD be assumed that the original claim was limited to the circumstances around the claimant's experience - which is left unstated. In this case the main questions are:


1. What card(s) was this scenario found on?


2. What were the symptoms when this method was not followed?


3. Are there any cards in his (or other's) experiences where this situation did not exist.


Any requests for this type of information are meet with derisive and inflammatory comments.


So - what actual 'good information' is really coming out of this thread? The original poster isn't being overly cooperative with documenting the claim. Those of us who are interested in actually understanding his stated experience are requesting more information. He's dodging the questions.


I have not made any personal attacks. I have merely requested additional information. When said information was not provided, I stated that I did not believe the blanket statements were applicable to all scenarios.


Clearly I'm a troll for having done so.

 

[OmegaPhil]

I have recently got hold of my first SDHC card (SanDisk Extreme 16GB) and the Transcend USB 3.0 Card Reader - based on all the fradulent stuff I've been reading about flash cards I did the following test (I use Debian Testing):

1. Get the size of the device in bytes: fdisk -l <device>
   
2. head -c <device size> /dev/urandom > test_pattern
   
3. sudo dd if=test_pattern of=<device path> bs=10k
   
4. md5sum <device>
   
5. md5sum test_pattern
   

This showed that the card or reader had corrupted the data. I then looked a little closer:


cmp --verbose test_pattern <device>


This showed that 4075 bytes differed between the two, and these differences all occurred before 20KB. I then did another comparison, and the same number of bytes continued to differ, however within those differing bytes, 2 of them changed value. The third (last) comparison agreed with the results of the second.


So much for a high quality make of card and supposedly good card reader I can see why you are recommended to leave the first 4MB untouched though - I will now do this in my paritioning scheme.

 

[Linux-SWAT]

Well, I don't recommend it, it's just a conclusion i came across seeing how the official SD partitioner works ^^.


Note that for 64 G and higher cards, 16384 K is needed.


Tell us if your problem is fixed.

 

[relliker]

I wonder if some space is allocated not just for alignment but for allocating alternate blocks when there are unusable blocks. Does anyone know if sd writers are able to allocate alternate blocks when bad blocks are encountered?

 

[OmegaPhil]

Tell us if your problem is fixed.

It'll only get fixed when the manufacturer cares


For the next test I created an ext3 partition in the remaining space, then filled it up with another test file and hashed - I was able to demonstrate the data was read back correctly, however as with an earlier test, I had difficulty creating a file that was as large as the space available. If a UNIX guru is around, I would appreciate looking this over - after mounting the partition, I used 'df' to get the number of free 1K blocks then multiplied it by 1024 for the size. However when the cp operation failed, the final file was ~14.4MB smaller. This doesnt match up with a 1000B df block size either.