Smc Space Optimization

[ratx]

Apologies if someone previously had discovered this but searching didn't find anything...

I have been playing around with a win32 port of "mkdosfs" ( a linux tool for making FAT/12/16 filesystems ) and have discovered 2 things;

1 - You can use this tool to make SMC filesystems in XP via a cardreader, meaning my SMCrecover thing is no longer needed.. yay?

2 - With mkfdosfs you can reduce the number of bytes in each "allocation sector" to 8192,4096 or even 2048 as opposed to the default 16384.

With the default as 16384 it means _any_ file on the SMC is allocated 16k even if its only 1k - not good if you have a SMC full of 8bit roms/images which are often only a few k.

Eg: a chkdsk of my "emus" SMC using a filesystem created by the gpmanager app:

131,006,464 bytes total disk space.
868,352 bytes in 49 folders.
126,287,872 bytes in 1,097 files.
3,178,496 bytes available on disk.

16,384 bytes in each allocation unit.
7,996 total allocation units on disk.
194 allocation units available on dis

chkdsk on a SMC with the same data formatted via mkdosfs w/8k block size:

130,957,312 bytes total disk space.
450,560 bytes in 49 folders.
121,880,576 bytes in 1,097 files.
8,626,176 bytes available on disk.

8,192 bytes in each allocation unit.
15,986 total allocation units on disk.
1,053 allocation units available on disk.

So as you should be able to see I have gained 5mb simply by reformatting with a block size more suited to my data... but remember changing the block size won't make any difference if your SMC is full of files greater than 16k in size.

Anyway I have tested my SMC for a while and it seems to work fine.. but I'm hoping a few other brave souls might want to try and see what results they have...

I have attached mkdosfs.exe; it doesn't seem to "guess" the correct number of blocks for my 128M SMC's so the command line I was using was:

mkdosfs.exe -s 16 -F 16 -I d: 255950

where -s is the block size * 2, d: is the drive of my cardreader, and 255950 is the forced number of blocks. You'll have to work out that figure if you have a 64 or 32 SMC.

cheers

 

[Blah]

Now if only I had a card reader. (GPDRIVE doesn't work for low-level operations like this)

 

[cozmic]

would it work with Phodrive?

 

[Blah]

Phodrive has the same issue (probably). It doesn't access the SMC directly (I think), instead it can just copy files, delete files, and create folders.

 

[ratx]

I just tested it out and it works - I created a SMC with just gpdrive, started gpdrive and formatted my new H: drive with 8k block size, reset and its all working fine....

 

[cozmic]

Wow, then it's really good news for me. It's kinda like having a 5% bigger smc...

 

[don]

So as you should be able to see I have gained 5mb simply by reformatting with a block size more suited to my data... but remember changing the block size won't make any difference if your SMC is full of files greater than 16k in size.

​

You're wrong here (or probably it's just a bit misleading) - it does make a difference if the files are larger than 16k too. As we're talking about _block_ size and the data is arranged in blocks, you also benefit if you got larger files. Imagine files with 16,1kb size, they take up 2 blocks 16kb each, summing up to 32kb. so changing the block size is always a good idea. Okay, doesn't matter that much when you're packing a 120mb movie on your smc.

 

[tattooman]

That rocks, I will have to try it out because I am always needing a few extra space for either those few game syou just want to try out or game saves....

Thanks so much Ratx for sharing this infomation with us. I love these poststhat give you knowledge and advice to get the most out of our gp32_console

 

[ratx]

@don "You're wrong here (or probably it's just a bit misleading)"

Yes you're right, I wasn't clear As Don points out you will also see a benefit in the situation he describes. I was just trying to make a point heh..

 

[jegHegy]

good thinking!

 

[A600]

I tried this:

mkdosfs.exe -s 4 -F 16 -I d: 255950

130.772.992 bytes total disk space.
130.772.992 bytes available on disk.

2.048 bytes in each allocation unit.
63.854 total allocation units on disk.
63.854 allocation units available on disk.

and it's working perfectly

 

[Blah]

Does it erase your files like a regular format or no? I'm wondering if I have to backup the files first.

 

[A600]

Yes, you must backup your files first.

 

[Squidge]

GpDrive (and Phodrive, which is an updated and improved GpDrive) both no nothing about files and folders on your SMC, they only know about sectors, and windows only ever asks about writing and reading particular sectors.

 

[aapje89]

Does this makes a lot of difference for the read speed (because the blocks are smaller)?

 

[Squidge]

Only if you are reading lots of small files - the SMC file system uses 528-byte blocks, so there's 262,144 of these on a 128mb card, and before you say that's a strange number, or that it equals 132mb, 4mb is reserved for system use (error checking, etc), and some other blocks are reserved to be used as replacements should bad blocks develope.

So a 16kb sector would use up 32 of these blocks (16 bytes are reserved per block for system stuff). An 8kb block size would obviously only use up 16 blocks, so would be quicker to read

 

[ratx]

I've done some timing and further tests on my original set of data

http://l33t.spod.org/ratx/gp32/filesystem/

Seems to show only writing is really affected ( slower )

 

[aapje89]

what files did u use to test it?, I dont think I have a lot of files smaller than 2k on my card.

 

[aapje89]

So a 16kb sector would use up 32 of these blocks (16 bytes are reserved per block for system stuff). An 8kb block size would obviously only use up 16 blocks, so would be quicker to read

​

I dont really understand this sentence, do you mean that it would be faster to read 1 16k block than a 8k block?

I meant, if you have larger files on a 16 block size they use less blocks than on a 8k block size, so I thougt it would be faster to read those files (or is that only with hd's?)

 

[tattooman]

I tried this last night and it basically gave me like an extra 5% space as was mentioned earier!!!! Thanks again for sharing!!!!!!!!! Next I will have to try this on my 128MB USB memory stick...