paulguy
Member
Good god just shut up, nobody cares.
Sdhc Compressed File System?
- Thread starter TitanUranus
- Start date
TitanUranus
Active Member
Do you guys have a long standing fued or something? Paulguy, I just noticed you joined the day after Drak. I'm begining to think you might be a schitzophrenic person who's personalities clash! Drak is definitely the anally retentive self proclaimed intelect, while Paulguy is the average computer Joe, who is happy to add his piece to the debate. Well, I'm happy enough to hear what you both have to say. Paulguy don't let Drak put you off. For a guy with his encyclopedic (or more likely wiki-pedic) knowledge he's only contributed 119 posts in 2 years. I'm sure Drack if you put your mind to it you could be a lot more useful to the Pandora comunity if you concentrated less on the intricate details of a post you disagree with. Add your ideas and arguments to the contrary certainly, but please try and be a little less condecending.
By the way, if you do happen to be schitzophrenic, can you answer me a question please (and no I'm not taking the piss out of people with mental ilness). If you have 2 personalities, each with contrasting political views, how do you decide which way to vote?
By the way, if you do happen to be schitzophrenic, can you answer me a question please (and no I'm not taking the piss out of people with mental ilness). If you have 2 personalities, each with contrasting political views, how do you decide which way to vote?
WizardStan
Mega GP Mania
- Joined
- May 24, 2008
- Messages
- 16,731
Schizophrenia has nothing to do with multiple personality disorder. Schizophrenics hear voices, but never lose control of their thoughts.TitanUranus said:
Last edited by a moderator:
TitanUranus
Active Member
Actually, the question its self in its original form is still perfectly valid, the misconception was confined to the prior sentence where I confused schitzophrenia with multiple personality disorder. Thanks for clearing that up, but I'm still left wondering - if you have multiple personality disorder, each with contrasting political views, how do you decide which way to vote?
Also, I applaud this current tangent for making ISO ripping seem incredibly close to the original topic.
Also, I applaud this current tangent for making ISO ripping seem incredibly close to the original topic.
Sorry, I didn't read some posts whose first lines indicated a flame, so this might have been brought up already: BTRFS supports zlib compression which speeds up writing a lot as some phoronix benchmarks have shown (Google Phoronix BTRFS) but I'm not sure if BTRFS is suited for flash memory (non-SSD-stuff like SD Cards or USB Sticks) or if compression would speed anything up (as it's done by the CPU which is not as strong as an Intel Core i7)
TitanUranus
Active Member
Thank you fahrstuhl, I'll check it out tomorrow. Right now I'm njot feeling too well so I wont be checking it out tonight. Also, thanks for getting us back on topic. Incidentally, I have an I7, and I dual boot linux, so might give it a shot on the PC if it appeals to me.
Drack
Member
- Joined
- Sep 29, 2008
- Messages
- 210
fahrstuhl said:
You're right. BTRFS version 0.17 and above supports transparent gzip compression, but if you want to use it, the kernel needs support for it. This looks like it would not be difficult, though. Checking the Angstrom repository you can see kernel modules present for omap3evm and beagleboard in the unstable distribution.
To answer those above, no, I don't have multiple personality disorder, just a seething hatred of misinformation.
Last edited by a moderator:
TitanUranus
Active Member
This BTRFS sounds better and better. Does anyone here have the knowledge and motivation to try recompiling the kernel with the necessary additions? Last time I compiled a linux kernel my CRT monitor blew up - literally. I think I made a slight error and it popped in a bright flash and started gushing acrid smoke out the back.
I despise misinformation too - especially intentional misinformation. I have nothing against people making mistakes, they need polite correction and sometimes a new monitor.
I despise misinformation too - especially intentional misinformation. I have nothing against people making mistakes, they need polite correction and sometimes a new monitor.
Drack
Member
- Joined
- Sep 29, 2008
- Messages
- 210
In most circumstances you won't need to recompile the kernel, you'd just need the module, which is a separate file that grants the kernel additional capabilities when it's loaded.TitanUranus said:
The only thing is that you won't be able to do is boot off of a Btrfs-formatted SDCard. There's a workaround hack, though. I don't know about the Pandora in particular (don't have mine yet) but in a standard Linux system you would need two partitions on the SDCard. One as /boot (that would be something like ext2) and one for the rest (Btrfs of course). The kernel and a minimal system called an "initramfs" would sit on the /boot partition. When the kernel first boots, it would execute the scripts in the initramfs which would serve to load the Btrfs module, mount the root partition, and continue booting from there.
According to this post the Pandora runs kernel version 2.6.27.46, which is an older version than the version Btrfs modules I found in the repository are designed for, so you can't just drop in those modules and have them work. A dev will have to compile the Btrfs module and make it available if the Pandora doesn't come with it (It probably doesn't).
Last edited by a moderator:
BTRFS is awesome, I've been using it for a month now on my desktop machine.
My only concern right now is that it might not be suitable for flash-memory whose cells wear out after a number of writes. The reason why you don't use ext3 on flash memory is its journal which gets written on every access or something like that and that wears out the memory faster than non-journaling filesystems would do. As BTRFS is bleeding edge modern it for sure has something similar.
@ Kernel: I know just about nothing about embedded devices, but on desktop machines its not that hard to compile kernels and modules yourself. I use Gentoo on my desktop machine and the neuvoo project aims to bring Gentoo to ARM-Cortex based devices. Neither do I have my Pandora yet nor did I try compiling anything for a non-x86 architecture but if Gentoo on ARM works, stunts like compiling new kernels or experimental drivers don't seem too hard
My only concern right now is that it might not be suitable for flash-memory whose cells wear out after a number of writes. The reason why you don't use ext3 on flash memory is its journal which gets written on every access or something like that and that wears out the memory faster than non-journaling filesystems would do. As BTRFS is bleeding edge modern it for sure has something similar.
@ Kernel: I know just about nothing about embedded devices, but on desktop machines its not that hard to compile kernels and modules yourself. I use Gentoo on my desktop machine and the neuvoo project aims to bring Gentoo to ARM-Cortex based devices. Neither do I have my Pandora yet nor did I try compiling anything for a non-x86 architecture but if Gentoo on ARM works, stunts like compiling new kernels or experimental drivers don't seem too hard
Okay, a google search on the topic brings some interesting results, especially this thread on the Meego devel mailing list which also has some tests involving different filesystems and memorycards.
viridior
Member
fahrstuhl said:BTRFS is awesome, I've been using it for a month now on my desktop machine.
My only concern right now is that it might not be suitable for flash-memory whose cells wear out after a number of writes. The reason why you don't use ext3 on flash memory is its journal which gets written on every access or something like that and that wears out the memory faster than non-journaling filesystems would do. As BTRFS is bleeding edge modern it for sure has something similar.
@ Kernel: I know just about nothing about embedded devices, but on desktop machines its not that hard to compile kernels and modules yourself. I use Gentoo on my desktop machine and the neuvoo project aims to bring Gentoo to ARM-Cortex based devices. Neither do I have my Pandora yet nor did I try compiling anything for a non-x86 architecture but if Gentoo on ARM works, stunts like compiling new kernels or experimental drivers don't seem too hard
Right now we are working on finishing up our neuvoo-0.3.0-official release which will be focused as a light-weight Gentoo-based desktop environment for new/average linux users with armv7a devices. We are spending a lot of time on our build process and automation, and its almost done. This is great because with fairly minimal reconfiguration I can go back and make images using busybox+uclibc, different filesystem support and utilities, along with different window managers + applications. We have also documented this, so if you are interested in really reconfiguring your system and testing out what is best, let me know and we can start a side project and see where it goes.
Last edited by a moderator:
paulguy
Member
This talk of BTRFS is pretty neat, I've wanted to try it out on a more isolated system than my main desktop computer, but I'm still curious for someone who has a pandora to try the ntfs-3g compressed files, for people who want to use their cards in Windows as well. I probably will just stick with ext2 or something, but this could benefit other people.
And as for a feud or something, there's nothing between me and him, I don't know him and have never talked to him before this thread, he's just a douchebag. I'm technically not wrong, just my thought was more or less not expressed well enough and he just chose to nitpick mostly at wording or point out things where he was wrong anyway just to be a troll, then I did some research and found out that someone already did something like I said (Which would also be neat if implemented in to PSX4All, if the format is open.) so IT DOESN'T MATTER ANYMORE.
Also I take offense to being called a "computer Joe". I know I'm not a genius or anything, but I knew what I was talking about, at least in my thoughts... it just didn't come out quite as well as I'd wanted.
So let's just drop it and get some numbers on those I/O benchmarks through ntfs-3g compression!
And as for a feud or something, there's nothing between me and him, I don't know him and have never talked to him before this thread, he's just a douchebag. I'm technically not wrong, just my thought was more or less not expressed well enough and he just chose to nitpick mostly at wording or point out things where he was wrong anyway just to be a troll, then I did some research and found out that someone already did something like I said (Which would also be neat if implemented in to PSX4All, if the format is open.) so IT DOESN'T MATTER ANYMORE.
Also I take offense to being called a "computer Joe". I know I'm not a genius or anything, but I knew what I was talking about, at least in my thoughts... it just didn't come out quite as well as I'd wanted.
So let's just drop it and get some numbers on those I/O benchmarks through ntfs-3g compression!
Drack
Member
- Joined
- Sep 29, 2008
- Messages
- 210
Guys, the drama and name-calling needs to stop.
Let me be the first to apologize. "misinformation" is too strong a word. Paulguy doesn't have any malicious intent as that word implies. I'll try to be more polite in the future.
It's not accurate to call paulguy a "computer joe." I don't think you can call anyone that who posts in a topic about compression knowing what the DEFLATE algorithm is.
Nor is it accurate to call me a "troll." A troll seeks to disrupt the conversation through the off-topic reactions to what he says. While that was the result of our posts, it was not the intent. I want to talk about compressed filesystems in this thread, and am willing to put the past few posts behind us in that effort.
Let me be the first to apologize. "misinformation" is too strong a word. Paulguy doesn't have any malicious intent as that word implies. I'll try to be more polite in the future.
It's not accurate to call paulguy a "computer joe." I don't think you can call anyone that who posts in a topic about compression knowing what the DEFLATE algorithm is.
Nor is it accurate to call me a "troll." A troll seeks to disrupt the conversation through the off-topic reactions to what he says. While that was the result of our posts, it was not the intent. I want to talk about compressed filesystems in this thread, and am willing to put the past few posts behind us in that effort.
Agreed.paulguy said:
Last edited by a moderator:
trix
Very Active Member
- Joined
- Jan 11, 2010
- Messages
- 438
If you think about it, SquashFS can actually be quite useful for the Pandora. Since there are two SD slots, a person could put everything that would not need editing or write access on the SD with SquashFS, and everything else on the one without it. Given the current known max working SD size of 64GB, and the focus of the Pandora on Gaming, which typically responds well to compression, one could store a ROM collection plus things like wikipedia docs and source code that you don't plan to edit to change (just study or compile) on the SD with SquashFS, while keeping emulators (due to config files that change) and programs that edit their own files frequently, on a separate uncompressed SD card.
Also, I'd just like to mention I'm impressed at the level of maturity shown by Drack. It's not often someone on the internet can step outside of their pride and drop the drama. Also paulguy shows maturity in attempting to drop it, but unfortunately kills it by throwing in "douchebag" and "troll".
One question for paulguy, you said for people who want to use their SD cards in Windows, they could use ext2. I admit I don't know much about Windows, but doesn't windows have problems reading anything other then NTFS or FAT? I just have memories of reformatting my USB stick as FAT before loaning it to windows users, as they tended to have problems if I gave it to them formatted ext2. Perhaps Windows Vista or Windows 7 added ext2 support? Sorry for my inexperience, I haven't used a Windows OS in many years.
EDIT: Sorry for the gravedigging. I just found this and I am very interested in using compression to increase the space available on the SD Cards, and a new topic would be wasteful and the information in this one then would be unavailable.
- trix
Also, I'd just like to mention I'm impressed at the level of maturity shown by Drack. It's not often someone on the internet can step outside of their pride and drop the drama. Also paulguy shows maturity in attempting to drop it, but unfortunately kills it by throwing in "douchebag" and "troll".
One question for paulguy, you said for people who want to use their SD cards in Windows, they could use ext2. I admit I don't know much about Windows, but doesn't windows have problems reading anything other then NTFS or FAT? I just have memories of reformatting my USB stick as FAT before loaning it to windows users, as they tended to have problems if I gave it to them formatted ext2. Perhaps Windows Vista or Windows 7 added ext2 support? Sorry for my inexperience, I haven't used a Windows OS in many years.
EDIT: Sorry for the gravedigging. I just found this and I am very interested in using compression to increase the space available on the SD Cards, and a new topic would be wasteful and the information in this one then would be unavailable.
- trix
TitanUranus
Active Member
Welcome trix, I thought this thread was deceased. I don't think resurrecting a pandora thread is really gravedigging though. Since so many people have yet to recieve their pandora it's quite possible there is nothing they can add till they have one, and there's no point opening a new topic when one exists that does the job brilliantly - as this most wonderful topic surely does. Definitely it was started by a genius, and therefore the topic and it's title could never be bettered.
I too was thinking of using squashfs in a similar way to what you suggest, though 2 SD's wouldnt be necessary since you can mount a large block using the loopback technique to effectively have a section of your SD with squashfs. I suspect that this might be the way forward since there should only be a few instances wghere an emulator does not support some kind of zip/rar compression. I haven't checked but maybe squashfs can actually have non compressed files written to it (this would make sense since each file has to be pre compressed, so just by NOT compressing maybe it would allow easy read/write access to some files that need it???). I still can't believe no one has answered my question about using NTFS compressed though - someone must have tried it! And yes, last time I tried some other OS format with windows the results were not pretty, so EXT2 would most likely cause a BSOD or the modern equivelant (where's the damn BSOD in win7? For years it was the most reliable part of Windows OS's!)
Finally, maturity is over rated. After all, if we all behaved like sensible and respectful people this topic would be 3 posts long and burried deeper than memories of catholic sunday school. So in the interest of keeping this topic alive can I just say every one is a dick... discuss.
I too was thinking of using squashfs in a similar way to what you suggest, though 2 SD's wouldnt be necessary since you can mount a large block using the loopback technique to effectively have a section of your SD with squashfs. I suspect that this might be the way forward since there should only be a few instances wghere an emulator does not support some kind of zip/rar compression. I haven't checked but maybe squashfs can actually have non compressed files written to it (this would make sense since each file has to be pre compressed, so just by NOT compressing maybe it would allow easy read/write access to some files that need it???). I still can't believe no one has answered my question about using NTFS compressed though - someone must have tried it! And yes, last time I tried some other OS format with windows the results were not pretty, so EXT2 would most likely cause a BSOD or the modern equivelant (where's the damn BSOD in win7? For years it was the most reliable part of Windows OS's!)
Finally, maturity is over rated. After all, if we all behaved like sensible and respectful people this topic would be 3 posts long and burried deeper than memories of catholic sunday school. So in the interest of keeping this topic alive can I just say every one is a dick... discuss.
trix
Very Active Member
- Joined
- Jan 11, 2010
- Messages
- 438
the only dick in here is the one captain kirk is holding in your avatar. Which is both hilarious and disturbing. Much like your forum name.
I haven't tried NTFS compressed, and to be honest kind of dismissed the idea simply due to my distaste with Microsoft made stuff. To this day I still get headache's trying to copy a DVD image to my USB to give to a friend, because they use windows and my only windows readable USB is FAT32, and a DVD iso is generally 200MB larger then FAT32 allows files to be.
Anyway, some time today (or possibly Sunday, as I have other things to do today) I will muck about with squashfs and compressed NTFS and see where that gets me. Note, however, that I will be testing on a USB stick rather then SD as that is all I currently have available, and will be doing so from linux (or BSD). So hopefully, time permitting, I will have some real results for you from those two methods soon.
- trix
I haven't tried NTFS compressed, and to be honest kind of dismissed the idea simply due to my distaste with Microsoft made stuff. To this day I still get headache's trying to copy a DVD image to my USB to give to a friend, because they use windows and my only windows readable USB is FAT32, and a DVD iso is generally 200MB larger then FAT32 allows files to be.
Anyway, some time today (or possibly Sunday, as I have other things to do today) I will muck about with squashfs and compressed NTFS and see where that gets me. Note, however, that I will be testing on a USB stick rather then SD as that is all I currently have available, and will be doing so from linux (or BSD). So hopefully, time permitting, I will have some real results for you from those two methods soon.
- trix
Drack
Member
- Joined
- Sep 29, 2008
- Messages
- 210
JFFS2 might be worth looking into. It's read-write and also log-structured (helps minimize writes that can wear out flash .. though I believe SDCards have built-in wear leveling these days). Compression is available using zlib, rubin and rtime. Support is built into the kernel via a compile option (Don't know if the Pandora has that option enabled - my Pandora is shipped but not yet received).
TitanUranus
Active Member
Excuse me for grave digging my own topic here, but I've just made the transfer from using windows as my main OS to using Linux (Mint 64x) on my PC. I've tried various versions over the years, but transferring just is not as easy as it should be. Linux is paranoid, seems to want me to give it a password just to browse a website (exaggeration, but not much). I finally gave up on windows as it was just 'causing me grief. I swear, it gets more viruses than a crack whore with a rubber allergy. Literally - I swear (f#@king useless piece of s@*t).
Anyway, I've been using VMware to run windows on the Linux setup to ease transition (I'm now weaned off Windows completely though) but one thing was driving me absolutely mental. That's why I'm here - I figured out something VERY relevant to my original question. I can read and write to the NTFS formatted HDD's I have, and I want to keep them as NTFS 'cause I use them on other peoples PC's. Funny thing is, transmission would read an already downloaded torrent and not connect to any peers, it also wouldn't save to my NTFS HDD. Same with VMware - yet the OS it's self was reading and writing files to it ok. Turns out it's 'cause you MUST uncheck the "compress files and folders" option on NTFS options for the drive/partition or it messes with things. This is odd, it can read files fine, and with compression now turned off transmission does seed the previously downloaded torrents even though the files are still compressed (it's just new files being written to the HDD that aren't being compressed).
To cut a long and tedious post short - linux has problems with some software when accessing a compressed NTFS partition, though if the files are already compressed and are not being written to it works fine. This is irritating, but all in all I like linux - I feel I have proper control over my PC again, like I used to with MSDOS (or my beloved amiga, workbench was so simple and so powerful). If you can explain why NTFS compression is so flakey with linux I'd love to know.
Anyway, I've been using VMware to run windows on the Linux setup to ease transition (I'm now weaned off Windows completely though) but one thing was driving me absolutely mental. That's why I'm here - I figured out something VERY relevant to my original question. I can read and write to the NTFS formatted HDD's I have, and I want to keep them as NTFS 'cause I use them on other peoples PC's. Funny thing is, transmission would read an already downloaded torrent and not connect to any peers, it also wouldn't save to my NTFS HDD. Same with VMware - yet the OS it's self was reading and writing files to it ok. Turns out it's 'cause you MUST uncheck the "compress files and folders" option on NTFS options for the drive/partition or it messes with things. This is odd, it can read files fine, and with compression now turned off transmission does seed the previously downloaded torrents even though the files are still compressed (it's just new files being written to the HDD that aren't being compressed).
To cut a long and tedious post short - linux has problems with some software when accessing a compressed NTFS partition, though if the files are already compressed and are not being written to it works fine. This is irritating, but all in all I like linux - I feel I have proper control over my PC again, like I used to with MSDOS (or my beloved amiga, workbench was so simple and so powerful). If you can explain why NTFS compression is so flakey with linux I'd love to know.
