Files - Best Fit to Disc(s)

[Ian J]

I have a number of files that I want to archive on discs (DVD-r)

Some of the files must be kept together so are in folders.

Is there any software where I can type in the file/folder sizes and the size of my media and the software will calculate the optimum layout to use the minimum discs and the maximum space per disc?

I used to have something similar on the Amiga years ago for floppy discs but can't find anything comparable for PC.

 

[Fzero]

Disc-Spanning looks to be the term.
Just found this, not used it myself but looks like what you need

http://www.howtogeek.com/76264/how-to-burn-data-across-multiple-dvd-or-cd-discs/

Use the free s/w CD Burner XP

At the dialog where you would click the 'Split' button, I reckon you'd want to check the option for 'Do not split folder contents unnecessarily' ... that should keep them in order you need

 

[lukey]

The Unix approach would be: "tar -cf - <your Files> | lrzip | split -b <4700M or 8500M depending on your DVDs> - disk.iso_". This will generate the files "disk.iso_aa", "disk.iso_ab" and so forth. But DVDs are very unreliable so i suggest a more robust compression method like Info-Zip, which can decompress broken zip files (http://www.info-zip.org/) together with dd_rescuse.

 

[levi]

I don't think he's asking for disc spanning tools. He's looking for a way to layout files so that even if one disc is destroyed, he'll still have complete files on other discs, I think.

It shouldn't be impossible to knock up a quick python script or something to calculate this, based on a csv file input. I'm not sure without too much thought whether first filling up the first disc using the files that best fit the space, then repeating with remaining files and remaining discs would always produce an optimal result. And presumably it wouldn't usually be necessary to try every permutation of the files, as that's not a scalable algorithm and probably of O(n!) complexity I think.

Hmm, it's an interesting classical computing question. If I were doing it by hand I'd fill the disc to say, 80% full then look for files that fit the remaining 20%, which is O(n) I think. But that 80% figure clearly isn't suitable for all situations. Perhaps sort the files by size, biggest first, then simply keep trying to put the biggest file on that will fit. That's also O(n) by my reckoning, and eliminates any hard coded values, and I can't immediately think of any corner cases to break it. That may be enough for most cases therefore.

 

[FBnil]

This should work for you https://sourceforge.net/projects/discfit

Arranges a big set of files or directories in order to use the minimum number of phisycal media (CD, DVD, BD...) pieces. You can drag the resulting sets directly over your burning software

(found thanks to my awesome duck-duck-go-fu skills!)

Alternatives:

Look at the image in this link: https://cdburnerxp.se/help/Data/disc-spanning
and
http://hcidesign.com/dvdspan/manual.html

Or look at these 3:
https://en.wikipedia.org/wiki/Disc_spanning

honorable mention, the now defunct archivecreator

or, if you want, I can whip out a perl script that does the spanning for you...

 

[Ian J]

Disc-Spanning looks to be the term.
Just found this, not used it myself but looks like what you need

http://www.howtogeek.com/76264/how-to-burn-data-across-multiple-dvd-or-cd-discs/

Use the free s/w CD Burner XP

At the dialog where you would click the 'Split' button, I reckon you'd want to check the option for 'Do not split folder contents unnecessarily' ... that should keep them in order you need

I have used this in the past and was less than impressed, I thought I'd give it another go though and my findings are still the same.

Given the total size of the files and the number of discs CDBXP said it would use, the sorting for filling discs isn't done very efficiently, I did better with an Excel spreadsheet and a couple of hours copying and pasting.

I don't think he's asking for disc spanning tools. He's looking for a way to layout files so that even if one disc is destroyed, he'll still have complete files on other discs, I think.

Exactly right.

This should work for you https://sourceforge.net/projects/discfit

Arranges a big set of files or directories in order to use the minimum number of phisycal media (CD, DVD, BD...) pieces. You can drag the resulting sets directly over your burning software

(found thanks to my awesome duck-duck-go-fu skills!)

Alternatives:

Look at the image in this link: https://cdburnerxp.se/help/Data/disc-spanning
and
http://hcidesign.com/dvdspan/manual.html

Or look at these 3:
https://en.wikipedia.org/wiki/Disc_spanning

honorable mention, the now defunct archivecreator

or, if you want, I can whip out a perl script that does the spanning for you...

I'll give those links a try.

Thank You
[doublepost=1456081741,1456076628][/doublepost]Discfit works well but I wish it showed the % of disc used, a feature that has already been requested but it looks like development has ceased.

It's the best solution I've been shown so I'll use it unless something more efficient is recommended.

Weird with the amount of media we all store on our PC's now that something like this isn't actively being developed to archive them.

 

[FBnil]

something like this isn't actively being developed to archive them

Well, today you can do raid5 cheaply (where, if a disk dies, data can still be recovered with the other disks... ) and backup with rdiff/full backup to recover data... so I think backup has gone that route.

You know what is weird too? The fact that when I am copying a directory to USBstick, then add a second directory, both jobs run simultanously... there is no option to "queue copy" to start it only after the current copy has finished. And I have tried a lot of FileManagers in Linux...

 

[Ian J]

I just like having a folder full of discs that I can grab in case of emergency.

I'd struggle to pick up my server and run out of the house if there was a fire

 

[lukey]

You know what is weird too? The fact that when I am copying a directory to USBstick, then add a second directory, both jobs run simultanously... there is no option to "queue copy" to start it only after the current copy has finished. And I have tried a lot of FileManagers in Linux...

If you set the disk cache high enough (with "echo 90 > /proc/sys/vm/dirty_background_ratio; echo 90 > /proc/sys/vm/dirty_ratio; echo 100 > /proc/sys/vm/swappiness; echo $((10*60*60)) > /proc/sys/vm/dirty_expire_centisecs; echo $((10*60*60)) > /proc/sys/vm/dirty_writeback_centisecs" ), the first copy will be already synced with the VFS before you start the second one.

 

[Wally]

I wouldn't rely on Discs for anything these days given their reliability, possibility of warping unless kept in dark places and a good cd wallet.

Hardware RAID would be a better option.

 

[Ian J]

Touch wood I've never had an issue.

I have some CD's burnt on a HP 6020i drive must be before 1999 and they still read fine.

I've always burnt onto branded discs and burn 2 copies, 1 stays here in a folder and the other goes to my in-laws and is saved in a folder.

Not that I'm OCD

 

[levi]

I still use DVDs for containing the bulk of my ripped CDs for example, as I don't own any machine with discs big enough to actually contain it all at present. I could just pick up a cheap 2TB USB drive, but so far I also touch wood, and also haven't had any failures in my discs. I generally try to burn a fresh copy every five years or so, and they're all stored in a black CD wallet thing.

 

[TrashyMG]

Redundant Servers each with over a dozen Terabytes of storage in a raid 5. I Just run an rsync script on a cronjob every 24 hours in the middle of the night. So nothing too extreme.

 

[levi]

That many discs must make a bit of a racket, at least on reboot. Only my Pandora actually runs 24 hours a day, and that's mainly because I broke the power switch on it.

 

[TrashyMG]

That many discs must make a bit of a racket, at least on reboot. Only my Pandora actually runs 24 hours a day, and that's mainly because I broke the power switch on it.

my new system it’s just three drives and fairly silent. My old system is comprised of 8 drives and even then I don't really hear the drives over the fans.

 

[levi]

Is that running on stock cooling? I was factoring the fan noise into the equation, since modern drives do tend to run and seek a lot quieter despite usually spinning faster, somehow.

 

[TrashyMG]

Is that running on stock cooling? I was factoring the fan noise into the equation, since modern drives do tend to run and seek a lot quieter despite usually spinning faster, somehow.

yeah I'm using the default CPU fans bundled with the AMD processors. I do however have a few extra case fans in both systems. To be fair I also have a ton of other things running in my finished basement that can make noise, DLP projector, the furnace, home theater system, my gaming system... etc

Definitely quieter than the 5 drive raid of 15K RPM Ultra320 HDDs in the PPC server in my office at work.

 

[Pocak]

Hmm, it's an interesting classical computing question.

It's the bin packing problem in one dimension.

You know what is weird too? The fact that when I am copying a directory to USBstick, then add a second directory, both jobs run simultanously... there is no option to "queue copy" to start it only after the current copy has finished. And I have tried a lot of FileManagers in Linux...

Krusader can do that.

 

[levi]

It's the bin packing problem in one dimension.

Ah, I thought it might be NP-hard, but I'm still learning those.