Making an image of an SD Card - Is this possible?

[Mr_Loon]

Is it possible to make an image of an SD card, in a similar way you can make an ISO image of optical media?


How does this relate to Pandora? :


If an image of a Pandora OS, ready and waiting to run the first boot wizard, complete with swap file & perhaps some preinstalled PNDs could be created, it would then hopefully be a simple process for users to extract this image to an SD card and enjoy the benefits of running the OS from an SD card.


It's appreciated that there are pre existing methods of installing OS's to SD cards but if this could be turned into a 'one click' or a wizard based process then I'm pretty sure more users would give it a try.


Images could be created for different 'flavours' of Linux. If an image could be created complete with a selection of 'essential' software then this might well prove popular with reviewers.


I'm not too sure about the implications of this in terms of differing sizes of SD card's but my guess is that if images were created for 2GB cards then it would be a relatively straightforward process to use something like GParted to alter the partition size to suit the size of card you are using.

 

[Linux-SWAT]

It's possible, and it's a piece of cake ^^:


http://pandorawiki.org/How_to_back_up_a_Root_File_System_%28on_an_SD_Card%29_to_an_SD_Card

 

[WizardStan]

That's not quite the same thing. You'd still need to format the destination card appropriately as you un-tar the image to it in order to make it work.


On the other hand, you could use some form of dd to take a proper image of one card into an ISO, and then dump that image back to a different card. Actually nevermind, I'm 99% certain this would be more trouble than it's worth though: the card would necessarily have to be exactly the same size as the original, plus a bunch of other minor problems. Stick with the tar/format/untar process, it's one extra step that actually makes things much easier. Isn't that what the existing SD installer already does? Problem solved!

 

[Mr Rob]

Though I completely agree with @WizardStan, for the sake of completeness, this is one way (a Unix way) to make a disk image (in case kids want to do it anyway for whatever reason).

dd if=/dev/"block_device" | gzip -c  > filename.img.gz

Where "block_device" is the drive you're backing up. On the Pandora, it usually looks something like this: /dev/mmcblk0p1. On a Linux laptop, it would be something like this: /dev/sdb. The above command will pipe it through gzip to compress it and output it to filename.img.gz. To restore:

gzip -c filename.img.gz | dd of=/dev/"block_device"

Which takes the compressed file, uncompressed it and writes it directly to the card.


As I said, WizardStan is right, and this would be rife with problems especially with different card sizes (and possibly problematic for different cards with identical card sizes). But for other reasons to make drive images (backups, archival, fun) the above works well.

 

[Linux-SWAT]

The problem with a low-level dd, is that it will read and compress even deleted files.


So it's not secure if you want to share it, and it can produce huge images full of useless stuff.

 

[Mr Rob]

The problem with a low-level dd, is that it will read and compress even deleted files.


So it's not secure if you want to share it, and it can produce huge images full of useless stuff.

A clever hack around this is to make a big file of zeros to take up all the extra space on the filesystem.

dd if=/dev/zero of=big-ass-file-of-zeros.img

Which will get compressed very well.


Though still, kind of hackish.

 

[Linux-SWAT]

The problem with a low-level dd, is that it will read and compress even deleted files.


So it's not secure if you want to share it, and it can produce huge images full of useless stuff.

A clever hack around this is to make a big file of zeros to take up all the extra space on the filesystem.

dd if=/dev/zero of=big-ass-file-of-zeros.img

Which will get compressed very well.


Though still, kind of hackish.

Then you'll have to do this before each clean installation followed by a saving...


Not worth the time/pain.

 

[Mr_Loon]

As I said, WizardStan is right, and this would be rife with problems especially with different card sizes (and possibly problematic for different cards with identical card sizes).

Why would this be a problem if the destination card was bigger than the source card? I'm not disputing the fact, just ignorant as to why this would be a problem.

 

[Linux-SWAT]

It won't be a problem in this case.


Anyway, you will have to partition and format the following empty space.


Do like you want, but the procedure described on the wiki is the cleanest and easiest one.

 

[McLovin]

well, if someone sets up a 2gig sd card, blasts all unused space with an /dev/zero image, deletes said image and creates an dd image, it would be a good start


then setting up a new sd would be just dd said image back and extend partition...

 

[WizardStan]

Why would this be a problem if the destination card was bigger than the source card? I'm not disputing the fact, just ignorant as to why this would be a problem.

The problem is that it would recreate the disk exactly as the original, down to the partition. At the very least you'd need to create a second partition from the free space, but for optimal performance you'd need to repartition the entire disk.


Also make note that no SD card is perfect: that 16GB card you *just* bought has at least a few damaged sectors on it; it's pretty unavoidable and entirely invisible to you, but it means that your 16GB disk image has equal chance of not fitting on another 16GB card as it does of having a little bit of space left over.


The simplest solution is to make card images of 2 and 4GB sizes and then repartition after dumping the image, but that is another step in the process, and so long as you have need of an extra step it may as well go first with the less error prone and more featureful formatting of the card.

 

[Mr Rob]

Why would this be a problem if the destination card was bigger than the source card? I'm not disputing the fact, just ignorant as to why this would be a problem.

I could be wrong, but could you end up with a wacky partition table if you do this? Or am I thinking if you dd the whole device, not just the partition.

well, if someone sets up a 2gig sd card, blasts all unused space with an /dev/zero image, deletes said image and creates an dd image, it would be a good start


then setting up a new sd would be just dd said image back and extend partition...

To be fair, I'm not sure how this is easier than the current method. Going back to the original post, isn't the point to find an easy way to make bootable SD cards? I'd think downloading the image and extracting it onto a card is just as easy.

 

[Linux-SWAT]

well, if someone sets up a 2gig sd card, blasts all unused space with an /dev/zero image, deletes said image and creates an dd image, it would be a good start


then setting up a new sd would be just dd said image back and extend partition...

Will simply take more time, especially when doing this natively on the OP.


Also, every cards of the same size doesn't have the same block counts


http://pandorawiki.org/SD_compatibility_list


So you're just running into troubles.


:rolleyes:

 

[Prometheus]

Going back to the original post, isn't the point to find an easy way to make bootable SD cards?

I'm unsure if this is of any help, but the Raspberry Pi Fedora Remix appears to have a way of handling this, and I seem to recall that they were planning on making sure that it would be doable from at least Linux, Mac OS X, and Microsoft Windows, in time for the educational release later this year. See "raspberrypi-resize", here. Perhaps this would be a good starting point?

 

[bismuthdrummer]

PING would perhaps do the trick, but I could not unfortunately get an image made from the SD card. PING wouldn't recognize it, even though it's supposed to handle FAT and ext3 filesystems... somebody more savvy might be able to help here, or at least explain the disconnect with PING and the Pandora-formatted card.


If PING could create a backup image, you could have a bootable dvd iso to work with. However, PING is very dangerous and could blow away the system drive if the user makes the wrong move. So even if we got an iso out of the deal, I am not certain it would be 100% friendly to new or inexperienced users.

 

[Letalis Sonus]

Don't forget you're talking about flash memory here. FLT chips don't like it when you fill the whole medium at once, SSDs are well known to freak out becasue of this when using backup tools that work that way. Any solution should actually take care not to write sectors that don't contain any data.

 

[bismuthdrummer]

Sure, but I have used PING for years and have always found its write time to be proportional to the size of the image being written. Hence, it doesn't write to the whole drive, at least as far as I know it doesn't.


For example, I would buy a new Windows box, take a snapshot of the drive, test restoring it, then proceed to strip the system of all the bloatware. Another snapshot, and the second restore from this smaller image would take minutes less to complete.


Also used it to restore several times on my eee 701 netbook which had a 4 gb ssd, no problems there at all.

 

[Stephane Hockenhull]

Don't forget you're talking about flash memory here. FLT chips don't like it when you fill the whole medium at once, SSDs are well known to freak out becasue of this when using backup tools that work that way. Any solution should actually take care not to write sectors that don't contain any data.

[ citation needed ]


because its the first time I hear of this, and I find this very hard to believe. if that was true SSDs would also freak out if you wrote a 4GB+ file as this would also be a long continuous write spanning multiple erase blocks.


and don't count the very early completely broken SSDs that would destroy themselves over time by ever-increasing internal fragmentation of the FTL.


new controllers defragment themselves.


most-likely if they programmed the controller correctly rewriting a whole erase-block-sized area at once should cause the controller to write the whole thing unfragmented.


also, you can tell the SSD drive and SD/MMC card to discard everything or parts and start over: http://en.wikipedia.org/wiki/TRIM


which help with both performance and longevity (potentially less moving stuff around)

 

[WizardStan]

because its the first time I hear of this, and I find this very hard to believe. if that was true SSDs would also freak out if you wrote a 4GB+ file as this would also be a long continuous write spanning multiple erase blocks.

Also remember class 10 cards are designed for 10MB/s continuous writing for several minutes or even hours; they're meant for HD video recorders.

 

[Letalis Sonus]

because its the first time I hear of this, and I find this very hard to believe. if that was true SSDs would also freak out if you wrote a 4GB+ file as this would also be a long continuous write spanning multiple erase blocks.

No, you didn't get my point. The problem isn't how it is being written, the problem is that you are writing the whole media, i.e. almost all blocks exposed by the FTL. It simply breaks the wear leveling resulting in data loss because the FTL has not enough free room to react correctly.