No More Hundreds Of Small Files

Executer

Still Fresh
Joined
Dec 30, 2005
Messages
13
A little wish to the homebrew developers.
I like what u´ve done for the community, but some games come with hundreds of little files which needs just too much time to transfer to the SD-card.
So how about to use an uncompressed archive to store the hundreds of little pics (1-2kb).
It takes me sometimes around 15min to transfer a 5mb game with card reader, which just isn´t really necessary :)
 

Vimacs

Don't be evil!
Joined
Oct 23, 2003
Messages
5,762
Age
33
Location
Germany
Website
Visit site
sounds like your cardreader is damn slow :)
also you should activate cacheing wich will have the affeckt that it haves to write way less frequently to the cart and should improve speed on small files.
 

Squidge

Certified Guru
Joined
Nov 16, 2003
Messages
8,495
Location
UK
Website
Visit site
Windows by default disables write caching of USB connected media that can be disconnected without warning to try and keep the media from being corrupted. However, if your happy to use the "safely remove hardware" feature, then Windows will cache the files instead, which means a big speedup in the transfer of small files as they will be transferred in batches rather than singly.
 

Executer

Still Fresh
Joined
Dec 30, 2005
Messages
13
Squidge posted on Apr 25 2006 at 09:24 AM said:
Windows by default disables write caching of USB connected media that can be disconnected without warning to try and keep the media from being corrupted. However, if your happy to use the "safely remove hardware" feature, then Windows will cache the files instead, which means a big speedup in the transfer of small files as they will be transferred in batches rather than singly.
I enabled now the cacheing on my card reader, transfering supertux (700files, 16mb) still takes around 7-8 minutes, a single 35mb file is transfered in max 30 secs...
I think the speed of my card reader is okay, it is used in usb 1.1 mode (12Mbit), i think the filesystem entries slows the whole thing alot.

--- edit
and much better, deleting supertux folder takes as long as copying :D
 
Last edited by a moderator:

Dzz

stmia r0!, {r2-r9}
Joined
Jan 30, 2006
Messages
1,098
Website
Visit site
I agree with your point and will endeavor to make sure that I keep the file count to a reasonable number on anything I write. A lot of the stuff we get for the gp2x are more-or-less quickly done ports and writing a thing to collect data files into fewer files is a lot more work than the porters are planning to spend.
 

Executer

Still Fresh
Joined
Dec 30, 2005
Messages
13
Does the gp2x FW support any image formats, that can bea easy mounted? Like a ISO file or something else..
That could be easily done and needs no programming. Just mount the image at app load, and unmount when finishes.. and for sure this is only a good idea for static things, like needed files and not for rom folders


The tar method could be used, but untar on app launch is "slow" and copy the tar and let it untar by the gp2x to the sd-card doesnt sounds practicable, i´m sure the card reader is faster ;)
 

Parkydr

Developing for the GP2X since 2006
Joined
Dec 3, 2005
Messages
1,360
Website
www.parkydr.ezeedsl.co.uk
Executer posted on Apr 27 2006 at 07:06 PM said:
Does the gp2x FW support any image formats, that can bea easy mounted? Like a ISO file or something else..
That could be easily done and needs no programming. Just mount the image at app load, and unmount when finishes.. and for sure this is only a good idea for static things, like needed files and not for rom folders
GP2X doesn't seem to support iso9660 (not surprising really)

The tar method could be used, but untar on app launch is "slow" and copy the tar and let it untar by the gp2x to the sd-card doesnt sounds practicable, i´m sure the card reader is faster ;)
You wouldn't need to untar every time, just once using an install script. I tried it with supertux and it took 2.5 minutes - still not good
 
Last edited by a moderator:

reallynotnick

The smartest newb
Joined
Aug 20, 2005
Messages
871
Website
Visit site
While I am no coder, I think some people can do better then having 250 different files for just one game. I find it really slow to transfer and I am sure it frags up your card even more too. Theres really no excuse to have over 50 files for one game, unless you have loads of music and videos like Final Fantasy.

While I am no coder, I think some people can do better then having 250 different files for just one game. I find it really slow to transfer and I am sure it frags up your card even more too. Theres really no excuse to have over 50 files for one game, unless you have loads of music and videos like Final Fantasy.
 

Executer

Still Fresh
Joined
Dec 30, 2005
Messages
13
NoidZ posted on Apr 30 2006 at 01:25 PM said:
Just buy a new card reader :p
A good reason to get a new card reader is when i can't adress SD over 1gb and such things. My card reader IS fast (and only 3M old), only when putting those houndreds small files on it it slows down, which just dont has to be.
I think my critic is constructive, your card reader comment not...
 
Last edited by a moderator:
Top