Smc Space Optimization

[Blah]

I think the people who set the original standards knew what they were doing, so I'll leave it as such.

 

[Squidge]

So a 16kb sector would use up 32 of these blocks (16 bytes are reserved per block for system stuff). An 8kb block size would obviously only use up 16 blocks, so would be quicker to read

​

I dont really understand this sentence, do you mean that it would be faster to read 1 16k block than a 8k block?

No, I say reading 8kb would be faster than reading 16kb. Or at least that's what it says above in your quote of my wibbling

I meant, if you have larger files on a 16 block size they use less blocks than on a 8k block size, so I thougt it would be faster to read those files (or is that only with hd's?)

​

SMC's use a fixed 512-byte block size, and this can not be changed as it's built into the hardware of the SMC. The only thing you can change is the dos filesystem that sits on top of this. Therefore, the less you have to read, the better. There is however, at least two disadvantages to small block sizes : More work for the gp32's processor (although this is insignificant at 132Mhz), and that you could end up with a badly fragmented (=slow) file system if you do a lot of deleting and replacing of files.

 

[ratx]

what files did u use to test it?, I dont think I have a lot of files smaller than 2k on my card.

​

It was a pretty varied mixture, mainly files in the range of 8-10k but with a few 100-200k

cheers

 

[aapje89]

No, I say reading 8kb would be faster than reading 16kb. Or at least that's what it says above in your quote of my wibbling

LOL, yeah i swapped 8 and 16k

"you could end up with a badly fragmented (=slow) file system if you do a lot of deleting and replacing of files."

aha, I already thought so, so what do you think 16 or 8k?

 

[reaper79]

I think the people who set the original standards knew what they were doing, so I'll leave it as such.

​

Hmmm, oddly enough, I checked my 128mb SMC and the block size for it is 4k. This disk was formatted using the newest version of the GPmanager downloaded from GamePark's home site. Your right, maybe they do know what they were doing AND decided to change it. I checked my other two 64mb disks formated using Slubman's FW and they are "normal".

I was looking through my files and recently took a look at the homebrew GPWar game, and there are a tons of tiny files in this game. Size of game is around 8.5 meg, size on disc is around 11 meg or so. And that is with 4k blocks.

Cheers!

 

[ratx]

Just found this page with some interesting FAT info and a tool that will test how "space optimised" your SMC is...

http://www.project9.com/fat32/#l

 

[aapje89]

Just found this page with some interesting FAT info and a tool that will test how "space optimised" your SMC is...

http://www.project9.com/fat32/#l

​

This program helps to calculate how much space could be gained with the use of FAT32.

 

[bast525]

ugh what a pain in the ass...


well I was trying to do this on a 32mb card that I bought specifically for Mame... since soooo many of the romsets do not even take up one full kb (yet each was being allocated 16kb... what a waste!) I figured this would definately make a difference.

unfortunately it took forever to figure out what forced block size to use but I finally found it.

So for anyone who's actually using a 32mb card, set the forced block size to 63965.

well I didn't think to test before and after *doh* so I can't say exactly how much space it freed up but I can definately fit quite a few more roms on there now, almost the full set of 700. Thanks for the tip!

 

[ratx]

Just found this page with some interesting FAT info and a tool that will test how "space optimised" your SMC is...

http://www.project9.com/fat32/#l

​

This program helps to calculate how much space could be gained with the use of FAT32.

​

Yeah the point is it shows how much your SMC is under utilised with the current block size for FAT 16/12 etc the entire web page is about fat32 so its no wonder he/she states that.

 

[starpause]

thank you ratx & bast525!!! :lol:

i was having real trouble w/ a 32meg SMC. Just trying to connect, GPManager was giving me "an error occured in GP32 durring process"

so i threw it in my cardreader, typed

C:\GP32\FIRMWARE>mkdosfs.exe -s 4 -F 16 -I h: 63965
mkdosfs.exe 2.8 (28 Feb 2001)
Win32 port by Jens-Uwe Mager <jum******an.de>
Warning: block count mismatch: found 48195 but assuming 63965.

C:\GP32\FIRMWARE>

it worked really fast!

and now i can try some of those crazy-big mods for BOR :rolleyes:

 

[starpause]

WOW, i almost tore my hair out in the last 30 min h34r:

trying to write a 20 meg file to my SMC via a cardreader in windows, i had an error.

i figured: no big deal, removed the card reader from USB and reattached it.

but: i couldn't delete the file i was writing, or over write it! formating the card via windows doesn't work! mkdosfs returned "failed whilst writing reserved sector" !!!

so it apeared i'd royaly fuked my SMC :huh:

the last thing i tired was updating my firmware to slubman's (with it's SMC formating). that did the trick! my SMC was saved ... some bad sectors were found but the card worked fine after that. B)

maybe the block mismatch was a serious error i should have halted at, i need to read more about SMC and the mkdosfs utility before i try to milk an extra few megs out of any more cards :rolleyes:

 

[Blah]

The settings you used for formatting it were for a 128mb card, that caused the problem.

 

[trooper]

Silly billy.

Trooper

 

[starpause]

ok, trying to figure out what i did wrong then ...

mkdosfs.exe -s 4 -F 16 -I h: 63965

-s 4 (4 sectors-per-cluster / "block size" *2, ie 2k sectors here?)
-F 16 (fat-size 16)
-I (volume-id)
... then forced-block-size/block-count on the end

it looks like the only things i could have missed on are the sectors-per-cluster and the forced block size. bast525 suggested 63965 forced block size for a 32meg card. i must have my sectors-per-cluster off ... i was running with A600's 2kb sector size, bast525 must have been running with ratx's 8kb sector size? honestly, i'm not sure how these numbers relate to each other--especially the block count/size (which is it?) :blink:

where -s is the block size * 2, d: is the drive of my cardreader, and 255950 is the forced number of blocks. You'll have to work out that figure if you have a 64 or 32 SMC.

is that the proper use of blocks? i thought all SMC had 528-byte blocks. Squidge really seems to know what's going on, what data sheets has he been reading ... heh, wikipedia's article isn't even that helpful!

entirely aside, slubman's firmware 1.00b will format a card so it has 16k bytes per sector.

one last question: the format in slubman's firmware found some bad blocks on my SMC. if i use mkdosfs will it know about those bad blocks?

 

[ratx]

Squidge explains it earlier on in the thread, yes the SMC has 528 byte blocks at the hardware layer, the FAT filesystem is another "layer" above that; at which you can choose, within reason, your block size. So when making the file system the mkdosfs.exe utility it has to calculate the number of blocks it can allocated based on your cluster/allocation size. Looking at the source it seems to "guess" this based on what it can physically see on the device. So therefore I'm guessing it might actually differ from SMC to SMC... I guess people will just have to experiement and see what works best for them.... Anyway in answer to your question mkdosfs.exe should detect and mark bad the same blocks slubmans does.

 

[starpause]

Anyway in answer to your question mkdosfs.exe should detect and mark bad the same blocks slubmans does.

​

thanks ratx! i really should remember this shit from my machine architecture classes <_<

it's strange that mkdosfs/windows was constantly failing at formating the SMC, but slubman's firmware tackled it without a glitch. that's the only reason i asked if mkdosfs should see bad blocks. if they are supposed to be the same kind of format maybe the difference in speed could be chalked up to the smc reader in the gp32 being slow, but it was a BIG difference in time ... mkdosfs worked almost instantly and i had to set the gp32 down and walk away when doing slubman's format.

anyway ... are there any firmwares that support formating with different block sizes? or is mkdosfs/latest GPManager the only tools for the trick?

 

[bast525]

Starpause - sorry I haven't been watching this thread. I have kinda forgotten what all I did when I formatted my SMC like this. For one, I forget if I set the block size to 2kb or 4kb but I'm gonna guess 2kb because that makes more sense to me now, so it must have then too.


I did figure out that this part:
"Warning: block count mismatch: found 48195 but assuming 63965"
means you are setting your forced block size too big. Now, that is the number that worked for me on my 32mb, that much I'm positive of, but maybe for whatever reason your 32 has less blocks available?

So, when it says found 48195 but assuming 63965, and then it formats you're card, you're only going to end up with about 2/3 of the space on the card available... about 20mb.

What I had to do was just trial and error. I knew that the 32mb had around 64000 blocks, so I started at 63000, and started working my way up in increments of 100 until it gave me that error message... then back and forth, lower and higher... increments of 50, then 10, then 1 at a time... until I found the absolute highest number I could use that would NOT give me that message.

Now, you don't HAVE to be so exact about it... you could just try 63000 which will probably work fine for you, but then you are still 'wasting' some of the space on your card. If even 63000 gives that message I'm guessing you have some bad sectors on your SMC, and you'll have to figure out for yourself just what to set the forced block size at.

Hope this helps.

 

[christo930]

Apologies if someone previously had discovered this but searching didn't find anything...

I have been playing around with a win32 port of "mkdosfs" ( a linux tool for making FAT/12/16 filesystems ) and have discovered 2 things;

1 - You can use this tool to make SMC filesystems in XP via a cardreader, meaning my SMCrecover thing is no longer needed.. yay?

2 - With mkfdosfs you can reduce the number of bytes in each "allocation sector" to 8192,4096 or even 2048 as opposed to the default 16384.

With the default as 16384 it means _any_ file on the SMC is allocated 16k even if its only 1k - not good if you have a SMC full of 8bit roms/images which are often only a few k.

Eg: a chkdsk of my "emus" SMC using a filesystem created by the gpmanager app:

131,006,464 bytes total disk space.
868,352 bytes in 49 folders.
126,287,872 bytes in 1,097 files.
3,178,496 bytes available on disk.

16,384 bytes in each allocation unit.
7,996 total allocation units on disk.
194 allocation units available on dis

chkdsk on a SMC with the same data formatted via mkdosfs w/8k block size:

130,957,312 bytes total disk space.
450,560 bytes in 49 folders.
121,880,576 bytes in 1,097 files.
8,626,176 bytes available on disk.

8,192 bytes in each allocation unit.
15,986 total allocation units on disk.
1,053 allocation units available on disk.

So as you should be able to see I have gained 5mb simply by reformatting with a block size more suited to my data... but remember changing the block size won't make any difference if your SMC is full of files greater than 16k in size.

Anyway I have tested my SMC for a while and it seems to work fine.. but I'm hoping a few other brave souls might want to try and see what results they have...

I have attached mkdosfs.exe; it doesn't seem to "guess" the correct number of blocks for my 128M SMC's so the command line I was using was:

mkdosfs.exe -s 16 -F 16 -I d: 255950

where -s is the block size * 2, d: is the drive of my cardreader, and 255950 is the forced number of blocks. You'll have to work out that figure if you have a 64 or 32 SMC.

cheers

You can accomplish the same thing at the dos command line (even from Winnt or XP), just run cmd (or command for win9x) from start-run and when at the dos prompt type
format x: /fs:fat /a:2048 Where x is the drive letter, the A option sets the allocation unit size (the GP32 won't recognize 1k allocation units which is possible on a 64meg or less drive). A 2k allocation unit should be valid all the way to 128MB cards. No messing around with blocks and no compatability problems. I got 8mb more space from the standard 16KB allocation unit.

Chris