Advanced Filesystem Hackery


Stuckie

Member
Joined
Apr 7, 2004
Messages
492
Age
38
Location
Tired, and sleeping in the basement...
Website
www.stuckieworld.net
--edit--
This has essentially been superseded by the Extend Utils system.
That said, this still has it's own little benefits - if your SD card is FAT32 formatted and you require "partitions" greater than 4GB, then other than daisy-chaining your Overlays, the Extend Utils can't really help.
This was the original idea too, and probably worth a read at least once to see where the Extend Utils stuff came from.

Extend Utils Wiki Documentation is available here: http://pandorawiki.org/Extend_Utils
--edit--

Another wall of text coming up, in the same style as the other one, where I type it up as I do it.

This time, I'm sacrificing a 2GB SD Card to augment the file system so that I can fiddle about as much as I like, and all files get saved to my SD rather than the NAND.
Disclaimer: we're doing some very scary things here.. although I didn't break my Pandora while doing this, it is possible if you mess up the commands.. so you have been warned!

You may be wondering why is that a good idea? Well, if you've gone through the other guide I posted, you'll notice that when we converted GIMP to a PND, it originally tried spitting files everywhere. My somewhat insane goal is to take as much load off of the NAND as I can possibly get away with, especially seeing as decent sized SD cards are fairly cheap these days, so if one of them dies, I just replace it! That, and if we re-direct all writes to our SD card, we'll have a better idea of just what it IS installing, and where it's sticking temporary files and so on.. so we can catch it and tie it up!

This will also be done entirely on the Pandora, so no need to worry if you don't have a Linux PC handy. ( In fact, after performing this, you'll be able to run through the other guide, substituting the Pandora for the Linux PC :) )

First thing's first, we need an SD card. As said, I'm sacrificing a bog standard SanDisk 2GB SD Card, but you can use any size and type you like. Essentially it depends greatly on what you'll be using your Pandora for, as this will become your "hard drive" for want of a better description. Feel free to use as much as you need, though I'd recommend at least 512MB
Ensure anything you need is backed up off this card, as we're about to format it! It'd also be wise to wipe it anyway, so we can find it in the mount list, especially if you're going to keep your other card in as well, however, if you're paranoid at this point, it may be wise to ensure this is the only card in the Pandora.
For guide purposes, I shall assume you've put it in the right slot ( next to the volume wheel. )

When we put the card in, the system will helpfully auto mount it for us, as we see it's icon appear on the desktop. We don't appear to have direct permissions to unmount the volume ( IE: you can't right click it, and select unmount volume ) so we have to get dirty with the Terminal, so go and open it.
If you type "df" and hit return, you'll get a list of all your mounts, where they are, and how much space is used.
In my case, I can see that root "/" is 72% full, and I've avoided writing to the NAND like the plague!
Either way, what we're looking for is our SD card.. in my case, it's at the bottom - /dev/mmcblk1p1 with 0% being used, and mounted on /media/mmcblk1p1. If you're going to be using the left slot ( next to the headphone socket ) it would be mmcblk0p1 instead.

Let's unmount the bugger: "sudo umount /dev/mmcblk1p1" you'll be asked for your password as usual, and that should be that. Run "df" again just to be sure, and it should be off the list.

Now the fun bit!
We're going to repartition the card to contain an EXT2 partition, rather than a FAT32 partition ( or FAT16 in my case as I'm using an old SD Card. )
This can be very dangerous if you get the device wrong... so pay attention to what you type!
Looking back at our mount point, we had "mmcblk1p1" .. what does this mean? Well, the mmcblk1 part means it's the second MMC ( multimedia card ) block device ( IE: it's our right slot.. most programmatic things start counting from 0, rather than 1.. you get used to it! It then becomes quite jarring when some API or language starts counting at 1 instead of 0 *twitch* ) and the p1 means it's partition 1. We want to repartition the whole card, so we drop off the p1 part, and as we're fiddling with the device itself, our path is /dev/mmcblk1.

Let's sacrifice our SD to the Pandora gods: "sudo cfdisk /dev/mmcblk1"
As this has been formatted on a Windows machine, it's a bit mental in that it's got 0.07MB free space, and a single partition that covers the rest of the 2GB. Why Windows is so determined to keep space for it's own nefarious purposes, when all other systems lets you use the entire lot, who knows. Anyway, press down to select the partition, then press right to move the selection box at the bottom of the screen to highlight [ Delete ] and press return.

We can now do two things here... we can create a single Linux partition to save our NAND, or... we can create the Linux partition and some swap space, to give us a little bit of extra memory when we need it. This is probably only useful for "power users" that will be developing on the machine, writing up insanely large documents, or porting bigger software over to tighten up it's memory - I've noticed that FreeCIV has died on me a few times as it's blown the memory with huge amounts of units running about, so it could do with some fiddling.
Either way, the instructions are pretty much the same.
Back to the task at hand, highlight [ New ] and press return. Select [ Primary ]. Give it a size ( by default it selects your entire card, so you can just hit return here unless you're wanting some swap space, or are up to other things. ) And you'll see a new partition in the list, with the FS Type of Linux.
If you want swap, and you've left some space for it, press down to select the free space, then select [ New ]. Again we want it to be Primary. And we probably want it to fill up the rest of the card, but if you're being fancy, set your size here. We then need to change it's type, so ensure that the second partition is highlighted above, and press right until you select [ Type ]. Now, don't panic! You'll get a massive list of codes for partition types which cfdisk supports, and a press a key to continue message. Press a key, and you'll get some more! Do it again, and even more! However, it doesn't matter as it's default setting at the end, is file system 82. Which, from the list you've probably read and noted down in panic, is Linux Swap. So we can just let it have it's default and hit return.
Back at the partition screen, we should have our two partitions - Linux and Linux Swap. We're done fiddling, so press left and watch as the highlight box wraps around and select [ Write ]. You'll be asked to confirm this by literally typing in "yes" though I only see the "y" for some reason on my terminal. Either way, it'll write to the card, and then complain that DOS cannot boot this.. poor DOS. Select [ Quit ] as we're done here.

That wasn't that scary, was it?
Now, although we've repartitioned the card, we haven't formatted it to the new file system. Again, pay attention to what you type here!
Whether you've created swap or not, your first partition should be formatted as EXT2, so we type: "sudo mke2fs /dev/mmcblk1p1" to format the first partition on the card in the RIGHT SLOT ( I'm stressing this to ensure you don't accidentally format the one in the left slot if you've still got it in. )

If you've made some swap space, we need to format that as well: "sudo mkswap /dev/mmcblk1p2" and we may as well activate it just now to test: "sudo swapon /dev/mmcblk1p2". You can check whether this has worked or not by using the handy dandy program known as "top". It'll tell you, amongst other things, how much memory you're using, and how much swap space you have. It'll also give some stats like up time, how many processes are running, the load average, if there are any zombified tasks, and what processes are currently actively doing stuff. Anyway, if our swap space is active and fine, all is good, so we just need to press Q and quit. Turn the swap off just now, as we'll be needing to eject the card soon: "sudo swapoff /dev/mmcblk1p2" and again, you can check this via "top" if you like.

We now have an EXT2 formatted SD card ( with possible swap. ) Now, we need to have a closer look at the file system, and create our links!
We want to offload stuff direct from root over to the SD card, so let's get to root first "cd /" and see what we have "ls" - bin dev home linuxrc mnt proc sys usr boot etc lib media pandora sbin tmp var - so a few things. Some of these we can safely ignore. What we're interested in, are the following - bin home usr etc lib sbin - Why? What do these actually do?

The bin folder essentially contains user-runnable system software. Stuff like df, mknod, tar, touch, more, mv, cp, echo, etc.. system commands which are quite important, but can be ran by your bog-standard user account.
The boot folder contains our kernel. Fiddling with this is a topic for another day!
The dev folder contains all our devices. We've already been modifying our SD card through here! Essentially, Linux stores devices as special files, so you can effectively treat a device as a file, and perform most file related stuff on them... if you're really insane, you could, for example, cat /dev/random > /dev/dsp ... to get some random noise out the speakers.. why? why not?! I wouldn't recommend actually doing this, however ;)
The etc folder generally contains system-wide configuration stuff. If you've gone through my other guide, you'll know that GIMP tries to store it's initial configuration files here, so that it can install itself afresh for every user.
The home folder is where all your user accounts live. You get a folder here to do as you please in. All your specific configuration and owned files end up here. It is, as per namesake, your home on the Pandora. It's also something we definitely want to move over to SD.
The lib folder contains all system libraries that are needed for your Pandora to operate, including kernel modules.
linuxrc isn't a folder, it's a symbolic link to busybox and gives you most of your standard Linux commands ( ls, for example.. in fact, type "ls -l" and you'll see what linuxrc really is! )
The media folder is where your pluggable devices get automatically mounted for you :) Generally, these are user mounts.
The mnt folder is effectively used for system mounts - like the PND you've just ran.
The pandora folder acts exactly the same as the pandora folder on your SD card - except this stores configuration for any PNDs that you've ran from NAND and any other system specific appdata. We actually want to leave this on the NAND, as it stores some critical things that should we have a hotfix, we want to keep regardless of whether we use our augmented SD or not.
The proc folder is crazy. If you've done "ls -l" you'll see that it was apparently created the 1st January 1970. This is generally because the proc file system is generated on boot ( before the time has actually come into effect ), and like the dev folder, contains special files generated from devices and so on. "less /proc/cpuinfo" will give you some information on the CPU for instance. ( If you haven't used less before, press Q to quit )
The sbin folder holds system specific binaries meant to be run by the system or root. We've already used some from here when creating our file systems, but it also includes other nifty tools like fsck, for checking file systems, and the cfdisk utility we used to repartition.
The sys folder is also crazy and was created on system boot. Essentially, this is like dev, in that it's special files to devices, but organised in a slightly more meaningful manner than being randomly thrown into /dev somewhere.
tmp isn't a folder either! it's another symbolic link, this time curiously enough, to /var/tmp
The usr folder contains all system-wide user specific bits and pieces - including binaries, libraries, documentation, shared files amongst programs, source code, etc.. another prime candidate for moving over to SD!
Finally, there's var, which is a bit of an oddball as it contains various bits and pieces itself. Log files are stored here, your real actual temp folder is stored here, along with various caches and lock files ( for example: opkg will lock itself so that only one process can be adding/removing stuff at one time.. obviously a good idea so things don't conflict! ) This we should leave on the NAND, as there are a few things that get stored in tmp as AUFS files to be remounted ( the PNDs for instance ) and we'd also probably want to keep logs here just incase something hideously breaks.

So, as said, we want "bin home usr etc lib sbin" to be augmented by our SD card. We can do this by the wonders of AUFS, which is a funky file system driver which allows merging of folders as one, and allows to set one as read-only ( our NAND version ) and another as read-write ( our SD version ) and redirect calls as appropriate. Let's mount the card again, and create some stuff!
Quickest and easiest way to remount the card, is to just eject it and put it back in again. It'll show up on your desktop mounted, and you can browse it like normal. It is just a normal SD card, it's just this one will be a bit more specific to your Pandora.

In our terminal, we need to change to our card: "cd /media/mmcblk1p1" and create a bunch of folders: "sudo mkdir bin home usr etc lib sbin".
You may wonder why we did a sudo to do that.. the file system has been mounted in a way that you don't actually have rights yet.. we actually want it this way, save for the folders we augment!
Speaking of which, let's do exactly that just now.
For a test, let's go to our home folder by just typing "cd" and hitting return. Let's create a folder: "mkdir test" and do the same on our card: "sudo mkdir /media/mmcblk1p1/test" Of course, we need to give ourselves permissions here as well: "sudo chown yourusername:yourusername /media/mmcblk1p1/test" obviously replacing yourusername with your actual username! Now, let's touch a file in test on our NAND: "touch test/iamatestfile" this'll create an empty file. You can check it with "less test/iamatestfile" to make sure.
Time to perform some magic: "sudo mount -t aufs -o dirs=/media/mmcblk1p1/test=rw:~/test=ro aufs ~/test"
What's this done? We've mounted our test folder in home as read-only, with the test folder on our card as read-write, and mounted this directly to our test folder in home. How can we test this? Let's fill our test file with some stuff: "nano test/iamatestfile" type some stuff, whatever you like, then press CTRL+X to quit. You'll be prompted to save (y) and the file name ( just hit return. ) Notice the SD activity light? "less test/iamatestfile" and you should see what you put in. In fact, "less /media/mmcblk1p1/test/iamatestfile" should show the same!
Now the final proof.. create another file "touch test/anothertestfile" and you should see the SD activity light flicker. "less test/anothertestfile" and it's empty as we expect. Finally: "sudo umount test" will undone our binding... let's see what's left: "ls test"

Surprised?
You should only see iamatestfile ( which we originally stuck on NAND to begin with ) our other file anothertestfile isn't there! But we modified iamatestfile didn't we? "less test/iamatestfile" not on the NAND we didn't! "less /media/mmcblk1p1/test/iamatestfile" there it is! "ls /media/mmcblk1p1/test" there's the other file as well! Magic :)

With our test completed successfully, let's go and push the entire home folder over! But first, "cd /" so we don't cause any issues by being inside the home folder while we do this.
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/home=rw:/home=ro aufs /home"
All writes to our home folder will now redirect to our SD :) Only the changes will be saved to SD, everything we had on NAND originally will stay as it currently is.
While we're here, might as well do the other folders!
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/bin=rw:/bin=ro aufs /bin"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/etc=rw:/etc=ro aufs /etc"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/lib=rw:/lib=ro aufs /lib"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/sbin=rw:/sbin=ro aufs /sbin"

You may have noticed I've missed off usr. There's a reason for that! Inside usr there's a pandora folder, which contains system specific pandora PNDs and scripts. As fancy as AUFS is, it's not too keen on mounting within mounts - which as PND files essentially get mounted via the very same system we're abusing here; it causes issues. ( NOTE: I didn't realise this till I blindly mounted all of usr, so I've gone back to edit this before posting the topic, to save everyone the hassle! Those using my Simple Dev PND from Torpor's thread will have this issue.. try using the AUFS enabled version, and running one of the Pandora system apps - the overclocker or the nub manager for instance. It won't run, and if you call "dmesg" in a terminal, you'll see it's having issues mounting it. Oops. I'll fix this shortly. )
So, we need to head over to our SD card's usr folder: "cd /media/mmcblk1p1/usr" and create some more folders: "sudo mkdir arm-angstrom-linux-gnueabi bin etc games include lib libexec sbin share src" ( everything in /usr apart from pandora )
We now need to mount these separately.
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/arm-angstrom-linux-gnueabi=rw:/usr/arm-angstrom-linux-gnueabi=ro aufs /usr/arm-angstrom-linux-gnueabi"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/bin=rw:/usr/bin=ro aufs /usr/bin"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/etc=rw:/usr/etc=ro aufs /usr/etc"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/games=rw:/usr/games=ro aufs /usr/games"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/include=rw:/usr/include=ro aufs /usr/include"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/lib=rw:/usr/lib=ro aufs /usr/lib"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/libexec=rw:/usr/libexec=ro aufs /usr/libexec"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/sbin=rw:/usr/sbin=ro aufs /usr/sbin"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/share=rw:/usr/share=ro aufs /usr/share"
"sudo mount -t aufs -o dirs=/media/mmcblk1p1/usr/src=rw:/usr/src=ro aufs /usr/src"

Hideous, isn't it? Oh well, you can now perform installs from the Angstrom repositories without any issue, and it'll get dumped onto your SD card rather than your NAND :)
However, we have some caveats here.
You absolutely cannot remove that SD card while you've linked everything up, else bad things will happen. You may have noticed your SD activity light flash now and then already, this is because background operations are happening all the time - your command history is being logged, for example.
However, you cannot unmount these binds... well, you can unmount some, but not all - bin is one of the key ones which it won't allow you to unmount.
As such, the only way in which to remove the binds, is to reboot, much like the AUFS hack that my Simple Dev PND does ( see Torpor's thread for it )
Also, it's not exactly friendly to do, is it? and every time you reboot, you're going to have to go through the binding again! This can be fixed by creating a little PND or a script that does it for you, though.
Another possible issue, is that you're installing stuff to your SD, so it won't be available on boot until you bind everything up.
Finally, speed.. it does slow down the system ever so slightly depending on how fast your SD card is ( mine isn't terribly fast, it's just a standard SD, not SDHC, so doesn't have all the fancy speed ups of modern cards. It's probably made worse in that it's a microSD through an SD adapter! ) as all writes are being redirected.. however, depending on what you're doing, you probably wont mind, and as it remounts your PND locally anyway, it's not like your games or emulators will be affected unless they touch the root file system :)

We possibly could've taken bits of var over as well, like we did with usr.. however it's somewhat tedious to mount, test, something didn't like that so reboot, rebind, test, etc... so I shall leave that as an exercise for someone else to see what else we can push over.

And that's that! you now have the ability to essentially install whatever you like, and it'll get dumped to your SD card. If you screw up, just reboot, and as your binds don't happen automatically, you're back at your default NAND setup :) So you can just wipe the SD card and start again!
Essentially, this is a sneaky way of having another firmware image on SD, but with the added bonus of all your "default settings" on NAND being pulled over automatically, rather than being a completely separate firmware.

I'll post this over to the PandoraWiki once there's been some proof reads, hopefully it's useful to someone!
 
Interesting read; that's how I simulated overlays on my main PC for some tests a while ago so I know for sure that it works (lacking a Pandora to test it on). Now for a question: how about adding the mounts to /etc/fstab? And how about overlaying the whole root ("/" instead of "/etc", "/bin", "/usr", etc) with an overlay? I don't actually know whether it works, since I don't have device with SD cards to test it on (and I won't sacrifice my main PC since it writes 20 MB to the disk (and thus to the potential overlay) each second on average)

EDIT: see https://help.ubuntu.com/community/aufsRootFileSystemOnUsbFlash
 
You could add the mounts to /etc/fstab if you like, the reason I didn't is you still have the option of a "vanilla" Pandora, so are free to muck about with everything being directed to SD card, and your NAND being clear of any debris should something mess up.
I haven't actually tried this though.

As for mounting root entirely, that may require some further hackery ( much like that guide on Ubuntu describes, which I forgot existed! ) as AUFS really doesn't seem to like mounting within itself ( which is essentially what the PND system would seem to do ) and is why there's so many single mounts everywhere ( and in /usr especially. )

That said, I didn't spend a whole lot of time mucking about with it, and it's a starting point for further hackery at least :)
 
Very cool. I did something similar on Linux where I used just a USB key to boot from. Everything was encrypted and basically read-only, but it didn't act that way because I used unionfs (a similar filesystrem to aufs) and tmpfs. It worked like a livecd in that no data that was changed while it's running was actually saved, but if I needed to do a persistent update I could just boot without the union mounts. It was an experiment on how little information I could store and still have the computer convenient to use, and have what little I did have to store encrypted.

I never thought of using unions to protect and extend the NAND. If I read your post correctly, this is a very cool way to:
-install apps from the angstrom repository in a persistent way without touching the NAND.
-Increase available capacity, transparently, to however big your SD card is.
-If you boot without the union mounts, it's a fresh install, as you haven't actually written anything to the NAND (aside from logs and such that happen before your unions get mounted)

I can see this being very useful with
-a permanent SD card
-A script to activate the union mounts on boot if there's an SD card in that slot. This is the only file you'd need to put on the real NAND to make this automatic.

EDIT: I don't have my pandora yet, but if the kernel uses an initrd with something like busybox, you can probably modify that to mount the entire filesystem from the SD (if present) instead of the NAND. No unions needed and a lot cleaner, though more invasive to install.
 
Yep, that's exactly right.

Also, illusionx pointed out in a thread in Software Hacking ( a forum I didn't notice, as that's where this topic probably should be ) that you could possibly get away with a loopback file sitting on a FAT32 card, meaning you wouldn't have to repartition it and it'd still be readable under Windows - just with a large file sitting there taking up space.
Problem with that is FAT32 has a max single file size limit of 4GB ( probably more than enough, however ) ... and I still don't trust Linux with NTFS ;)

I've basically been using it to compile some stuff today.. the added bonus that I can compile, package it into a PND, and test it on the machine both in my "dev environment" and simply reboot to get a clean Pandora that everyone else has, to ensure I won't be missing any libraries :)

[edit]
Actually.. that gives me an idea... hmmm.. I'll do some fiddling and I may have something in a couple hours!
[/edit]
 
If you're going to tie up an sd that you can't eject/etc .. It's easier to just install a firmware onto SD and boot from there :) ie: use a 16gb card and unpack the official firmware on it (I can write up how tomorrow if you like) - then you've got 15gb free to work in, no problem :) (or whatever your partiont or sd is of course) more stable and fast. Living on sd means things work just like a normal Linux install

there's a lot of awesome aufs tricks but they can make things unstable so be careful :)

for stuff like gcc.. Just build it but wih it using ./ as relative dir (or whatever) so that it happily lives in a pnd without aufs. Ie: going through contortions for aufs is fun, but often it's a lot moreuseful just to build the app, especially if it's just a ./configure away .. But dependancy hell would make the aufs approach a lot easier of course, still I'm more a fan of booting from sd anyway

on the other hand ipks with post install scripts would work great with your evil tricks :) awesome Fu :)

jeffphone
 
Hi Stuckie,

awesome writeup! I'm looking to do something very similar: http://pandorawiki.org/Package_Management

I just spend the weekend downloading the pandora SDK (yep, my Internet is that crappy), so I can build a modified initramfs and have an aufs root.

And skeezix, you do not tie up an sd card, you can eject it and be back to the vanilla OS on the nand.
 
Actually, ejecting an SD card after doing some AUFS mounts to it is probably the worst thing you can do.. you'll confuse the system no end as it can't clean up the mounts properly.
I've tested this, and it just flat out won't let you unbind /bin and a number of other mount points.

However, one mount point that you're relatively safe to abuse is /home and with that in mind, I present a little PND of extend utils: http://www.stuckiegamez.co.uk/apps/pandora/extendutils.pnd
[edit]
Actually, I've removed these as they've been upgraded a bit, please see this thread for more details: http://www.gp32x.de/board/index.php?/topic/54508-simple-dev-pnd/
[/edit]

There's four scripts in there, which if you stick in the menu folder, will lurk under System:-
Create Extend - will create an ext2 formatted loop back file where ever you tell it ( hint: SD card is a good place! ;) ) .. this is HORRIFICALLY slow, so you're much advised to do this on a PC and copy it over.
Home Extend - this will extend your /home to one of the loop back files :)
Home UnExtend - this unbinds the /home extend, so you can remove the card again. ( it's a bit twitchy though, so you may have to do "sudo umount /home" from a terminal yourself.. and clean up the loop back mount point )
Root Extend - the daddy.. this'll bind near enough everything possible to your loop back file. It does tie up the SD card, however, so you can only leave this state by a reboot. I've also set it up so it'll bind everything possible in /usr and /var as well.

It's advisable to use one or the other in regards to Home or Root extend.. using both could cause issues.
I might extend it further so you could have multiple extend loop backs for any folder you wish... would only take a little bit of zenity scripting.

Of course, skeezix has gone and posted on his blog about booting from SD, making most of this a bit superfluous now.. but I dare say not as fun as abusing AUFS for our nefarious deeds ;) that and I'm trying to perfect a mechanism for mounting AUFS mounts properly so we can package up IPKs and DEBs to PNDs in super quick time!
 
Last edited by a moderator:
Stuckie said:
Of course, skeezix has gone and posted on his blog about booting from SD, making most of this a bit superfluous now..

What's the address of his blog?
 
Last edited by a moderator:
Hi,

I get the point, but it could break the system once the SD goes corrupted by accidentally ejecting it.

opkg (and ipkg) are package managers that are made to install apps on external locations (like SD).
opkg has the '-d' option to point to where you want to install, you need ipkg-link to create the symlinks to the NAND.
This has quite a few benefits :
- Angstrom packages have a control file which has information whether they can be installed on SD (system libs and core apps can't)
- the system will run with or without the SD (it just won't find some apps without the SD)
- the apps installed on SD stay there when you flash your NAND, a simple run of ipkg-link brings them all back
- ...
When Angstrom has these options in its package manager, why not just use that instead ?

Have fun,
Chero.
 
Chero said:
Hi,

I get the point, but it could break the system once the SD goes corrupted by accidentally ejecting it.

opkg (and ipkg) are package managers that are made to install apps on external locations (like SD).
opkg has the '-d' option to point to where you want to install, you need ipkg-link to create the symlinks to the NAND.
This has quite a few benefits :
- Angstrom packages have a control file which has information whether they can be installed on SD (system libs and core apps can't)
- the system will run with or without the SD (it just won't find some apps without the SD)
- the apps installed on SD stay there when you flash your NAND, a simple run of ipkg-link brings them all back
- ...
When Angstrom has these options in its package manager, why not just use that instead ?

Have fun,
Chero.
I said that back when the guys around here wanted to create the PND spec, but there was a reason for not using the system, I don't recall what, that seemed very un-obvious to me at the time.

I'm also for pushing the native package format, but nobody else seems to be as well...
 
Last edited by a moderator:
dflemstr said:
I said that back when the guys around here wanted to create the PND spec, but there was a reason for not using the system, I don't recall what, that seemed very un-obvious to me at the time.

I'm also for pushing the native package format, but nobody else seems to be as well...
I'm using it already, with succes. check here : http://www.gp32x.de/board/index.php?/topic/53825-pandora-angstrom-repository/page__view__findpost__p__869445
If the intention is just to work with pnd-s, then I don't see why one should try this setup to enlarge / with a full SD ...
If you want that, you could as well (or better) use the native package manager for that (a lot easier).

Having fun,
Chero.
 
Last edited by a moderator:
To be honest, I wasn't fully aware of what opkg and ipkg could do.
I'm just used to messing with AUFS from my own experiments on my desktop machine, so I wanted to see how far I could abuse it on the Pandora.

At any rate, I currently have a 512mb loopback file on my SD card which is full of dev libraries and stuff. I quite like the fact that I can just bind that when I feel like, and reboot when I'm finished for testing on a "clean" Pandora ( in case I've pulled in libraries that aren't on a stock install. )
I get all the hotfixes and so on of my standard install, and can extend it with my dev environment at will :) should I accidentally remove the SD card, it just confuses the system as it doesn't know where to write anymore.. a simple reboot fixes it.

Every method has advantages and disadvantages, just depends on what your preferences are and what you need, really.
I agree that we should have an all encompassing standard, however. Be it PNDs with a slightly tighter spec and validation, or using what already exists in the form of opkg.
This, what I've been doing with AUFS, shouldn't really become mainstream unless it can be tuned enough that we can safely unmount the binds. It's only mainstream use that I can see, would be the fast porting of Angstrom packages into PNDs for testing. A proper port would still beat it hands down, obviously, but it'd be better than nothing, in some cases.
 
skeezix said:
If you're going to tie up an sd that you can't eject/etc .. It's easier to just install a firmware onto SD and boot from there :) ie: use a 16gb card and unpack the official firmware on it (I can write up how tomorrow if you like) - then you've got 15gb free to work in, no problem :) (or whatever your partiont or sd is of course) more stable and fast. Living on sd means things work just like a normal Linux install

would love to see this, spent half a day trying to mount rootfs.img to no avail :p

plenty of tricks with aufs/mount/chroot though for those with the courage ;)
 
Last edited by a moderator:
There were numerous oddities we didn't like when we fiddled with it; some chunk of it is probably due to me not giving it a lot of timke and lacking experience in some crazy shit they were doing, but a lot of it turned out to be what I like to call "user voodoo"; ie: there was a lot of symlinking magic going on, and it required non-FAT filesystems in order to work, for instance.

We wanted a very basic and easy to use system, and pnd is one such system. It may be displaced in the future with something else, or maybe it'll succeed (so far so good!) .. at the time when we were teswting things, we tried various options that didn't seem to work well for us (and some had driver issues that may be solved now, and we had a system in flux, and so on).

We didn't want dependancy bad stuff (which was a lot of problems I saw for random packages -- installing somethign on SD1, and something else on SD2, and then app on SD2 mysteriuously not working when SD1 was ejected, etc.) A lot of stuff like that. Mind you, those problems could be wiped away with a 'good port' and a private repository with enforced standards; with pnd we do that, but the question remains -- should we have used something like opkg, but with a very well defined custom repository with non-naive ports instead? *shrug*

Anyway, we went for a system that we knew would work well, for everyone, including newbies, espeically those familiar with previous machines like gp2x, etc and so on; the pnd system is a success, in its own right, but it might be silly ;)

Anyway, I'm on my phone replying, so I can't half write well at all I'm afraid :)

jeffphone
 
skeezix said:
If you're going to tie up an sd that you can't eject/etc .. It's easier to just install a firmware onto SD and boot from there :) ie: use a 16gb card and unpack the official firmware on it (I can write up how tomorrow if you like) - then you've got 15gb free to work in, no problem :) (or whatever your partiont or sd is of course) more stable and fast. Living on sd means things work just like a normal Linux install

there's a lot of awesome aufs tricks but they can make things unstable so be careful :)

for stuff like gcc.. Just build it but wih it using ./ as relative dir (or whatever) so that it happily lives in a pnd without aufs. Ie: going through contortions for aufs is fun, but often it's a lot moreuseful just to build the app, especially if it's just a ./configure away .. But dependancy hell would make the aufs approach a lot easier of course, still I'm more a fan of booting from sd anyway

on the other hand ipks with post install scripts would work great with your evil tricks :) awesome Fu :)

jeffphone

Please do this, Thatd be awesome =)
 
Last edited by a moderator:
Great,but I have a problem during formating.

The swap partition cannot be recognized.

sudo mkswap /....... said no such ........

What is the problem?
 
Second questions,
Is the command'
sudo opkg install software' enough to download all the dependencies?

And where can I find the installed software other than using in terminal?


Thx again
 
Back
Top