Discussion About Additions To The Pnd Format

[dflemstr]

Also, the problem of the user writing to the NAND manually (or by accident - most Linux applications need to write to an user's home dir sometimes); those writes will end up on the overlay too if we don't find a good way to avoid that problem (heck, if done incorrectly, even IPKG will write to the overlay).

I suppose that home dir will be mounted from SD card (and tmp and other working directories too) :-O I don't see any reason why write to NAND but firmware upgrade.

Other thought. Standard card use FAT and as dflemstr say, FAT doesn't have symlinks. But can't it be replaced some file stub (as *.lnk file in old Windows98) which can contains path to the original file?

1. It's already decided that /home will be on the NAND. Talk to the devs if you have an idea on how to change that (because the Pandora needs to be able to run even when no SD is inserted so it won't work without some brilliant idea)
2. It's possible to create a small image file (~2 MiB would suffice) that you'd create an ext2 filesystem on. That file could then be the overlay.

 

[dflemstr]

No PND should ever modify NAND, chisel that in fucking stone.

OK, so what solution do you propose to solve the problems?
(and we're not modifying anything, just adding to it With UnionFS, we won't even be doing that)

 

[God Ginrai]

No PND should ever modify NAND, chisel that in fucking stone.

And...... why?

-God Ginrai

 

[Vorporeal]

Also, the problem of the user writing to the NAND manually (or by accident - most Linux applications need to write to an user's home dir sometimes); those writes will end up on the overlay too if we don't find a good way to avoid that problem (heck, if done incorrectly, even IPKG will write to the overlay).

I suppose that home dir will be mounted from SD card (and tmp and other working directories too) :-O I don't see any reason why write to NAND but firmware upgrade.

Other thought. Standard card use FAT and as dflemstr say, FAT doesn't have symlinks. But can't it be replaced some file stub (as *.lnk file in old Windows98) which can contains path to the original file?

1. It's already decided that /home will be on the NAND. Talk to the devs if you have an idea on how to change that (because the Pandora needs to be able to run even when no SD is inserted so it won't work without some brilliant idea)
2. It's possible to create a small image file (~2 MiB would suffice) that you'd create an ext2 filesystem on. That file could then be the overlay.

Totally agree with point 2. I feel the best solution would be making a small ext2 image file (overlay.fs?) in the root of the SD, which would be mounted by the PND daemon when it's loading the PNDs on that SD card. All overlay files would be added to that image file, then it would be UnionFSed on top of the NAND. It would be a simple solution, would totally avoid writing to NAND, and would (from the user's perspective) just be one, small, extra 2MiB file.

 

[peca]

It's already decided that /home will be on the NAND. Talk to the devs if you have an idea on how to change that (because the Pandora needs to be able to run even when no SD is inserted so it won't work without some brilliant idea)

I guess that if user don't use any card that he/she can't store anything. There no reason running AbiWord if no store room. Browse Internet just in RAM, play game in RAM, but not to store anything. Possible suspend to RAM. If user want to store something they must insert a card.

OK, what is done is done. I just hope that this behavior can be changed and home/ mounted from directory on card.

 

[Caine]

Can't we just stack all the /overlays from mounted PNDs into a single union (without the symlinks)? Of course when removing an SD card one has to remove all the mounted overlays from the union as well, but this should not be different from unmounting the PNDs themselves.

As long as the NAND has the highest priority, none of the files on the SD cards should shadow the system files.

So the mounted volumes in the union look something like this (highest priority on top):
- NAND (read-only)
- (optional) Writable image on SD card
- PND0/overlay (read-only)
- PND1/overlay (read-only)

If I understand UnionFS correctly, any lookup operation reads from the NAND first and goes down until the file is found. Same for writes (which skip the NAND because it's mounted read-only).

We could for instance create an Image on the SD card to shadow the home directory on the NAND. This keeps the application data together with the application on a single SD card and protects the NAND from writes.

- Mounting SD cards adds the writable image to the overlay.
- Unmounting SD cards removes the writeable image and all overlays (from PNDs on that SD card) from the union.
- Mounting/unmounting of PNDs simply adds/removes overlays from the union.

-Caine.

<edit>Having a top-level read-only volume is not allowed. Too bad. Thought this could be used to prevent overriding system files.</edit>

 

[multiplex]

I'll try to keep it constructive. I haven't waded through too much of the previous thread, if the answers are there just say so;

The whoel discussion is about how the PND format could be extended to not only package music players or games, but also things such as icon themes or (for the really geeky people among us) web servers or desktop environments.

What features does the PND format as it stands provide for this usage that would make it beneficial to extend it rather than creating something more suited from scratch or from other established package formats? I don't see any technical relation to the original purpose.

the NAND writes are not so limited that we can't afford to write 20-byte files

Slight point, but I don't think the exact size of the file matters much up until you hit the block size.

...and a PND for some reason requires a newer version of libxml2.so (let's say, 2.7.5). It would then, when mounted, create a symlink like this:

/usr/lib/
|-- libxml2.so -> libxml2.so.2.7.3                          (symlink)
|-- libxml2.so.2 -> libxml2.so.2.7.3                        (symlink)
|-- libxml2.so.2.7.5 -> /mnt/pnd7273232/libxml2.so          (symlink) <--
`-- libxml2.so.2.7.3

What happens when two different PND files both include the same library version? (The files could still be different, depending on compiler options, etc.) It just randomly chooses one? Or I can't run GIMP and Firefox at once? (memory jokes aside.)
Is each application actually getting its own /lib/ in some way I'm not quite grasping? Because otherwise you could also do the injection simply by including a newer version of a library and hoping some other application jumps on it, right?

2. It's possible to create a small image file (~2 MiB would suffice) that you'd create an ext2 filesystem on. That file could then be the overlay.

At that point, aren't you using more resident memory than you'd do by just keeping everything in RAM, especially if that's your only ext2 mount? Or was the main purpose the caching? The symlinks are useless before the relevant PND has been mounted, as I understand them.

 

[dflemstr]

What features does the PND format as it stands provide for this usage that would make it beneficial to extend it rather than creating something more suited from scratch or from other established package formats? I don't see any technical relation to the original purpose.

The reason to do this is that the PND format always has to stay "portable"; if I remove a PND at any time or remove any disk at any time, the PND system should not fail (by making a package database corrupt, as would happen with APT or RPM, etc). Sure, programs that are using the PND's files may fail, but that's impossible to prevent. The goal is to keep the "drop PND in folder to install, delete PND to uninstall" principle while still having the same featues as conventional packaging systems.

Slight point, but I don't think the exact size of the file matters much up until you hit the block size.

True, but the write is still small compared to the multi-megabytes that would be required if you'd want to distribute whatever you want to distribute using IPKG instead (because IPKG can only install stuff to the NAND for obvious reasons). Also, NAND won't be a holy place of core files; writes will happen to it whether you like it or not. Your /home and /var/log resides there, for instance.

What happens when two different PND files both include the same library version? (The files could still be different, depending on compiler options, etc.) It just randomly chooses one? Or I can't run GIMP and Firefox at once? (memory jokes aside.)
Is each application actually getting its own /lib/ in some way I'm not quite grasping? Because otherwise you could also do the injection simply by including a newer version of a library and hoping some other application jumps on it, right?

Remember that the Pandora distribution will have a set of libraries that already are present in the core distribution, so the need to include your own libraries should be minimial. Also, the preferred modus operandi is to statically link your application with whatever library you're using, further reducing the need for dynamic library inclusion. Those two facts should leave only application-specific libraries (for your game or similar) or libraries that are obviously provided by the package you hav. For example, if you distribute KDE as a PND, it's kinda obvious that it'll include libkdeui.so and no other package should provide that library unless there are multiple PNDs with KDE inside of them, in which case you won't use them simultaneously anyways. So, the rule will be that when a file already exists for a PND (either as a sane symlink or actual file), the PND may not be mounted at all.

At that point, aren't you using more resident memory than you'd do by just keeping everything in RAM, especially if that's your only ext2 mount? Or was the main purpose the caching? The symlinks are useless before the relevant PND has been mounted, as I understand them.

I'm talking about the situation where the overlay is being stored on an SD-card, and that card uses a FAT32 system. I really don't know why you would do that in the first place; I was just answering the question
Yes, keeping it all in RAM using a RAMdisk with some lightweight FS on it would be preferred, because symlink creation is cheap and only needs to be done when a PND is moutned, as you say.

@Cairne Oh, I never thought about using the PND overlays as the "bottom" of the UnionFS! Things would indeed be very easy if we did it that way. However, I cannot agree with you on your FS stack. This is my alternative (non-final, objectionable) solution to the problem:

1. Have a folder either in a RAMdisk or on an SD card in which you dump all the symlinks from /overlay in all PNDs (resolving the symlinks to point to the correct paths, of course)
2. Make an UnionFS with that folder as the *base* and the root file system as the *overlay*. When you write to the root FS, writes will go to NAND, but when you try to read a file that doesn't exist on NAND but does exist in the symlink folder, the symlinked file will be read.
3. SD cards are mounted in /media without modification; they are unaffected by the whole system.

 

[Sphinxter]

No PND should ever modify NAND, chisel that in fucking stone.

And...... why?

-God Ginrai

It's not an application function. Why one would insist on violating the model and pissing in the pool everybody is swimming in is beyond me. Is reliability and ease of support reason enough not to? No application should be able to fuck up another application's environment or cripple the OS period ever. I plug an SD card in, PND magic happens, unplug that SD card and the device should be in the exact same state as if that PND magic never happened. The reverse should be also true, flashing or updating the OS should not affect the space of a PND. The latter will of course be a bit more difficult but the former will make life much easier for everyone involved.

PND = application on SD
ipkg = OS on NAND

Never cross the streams. Want to alter the OS, put it in the Angstom tree. I disagree with the location of home, should be in memory and check for an .xsession on the sd cards, don't think anything should ever write to NAND short of an OS update. Writing to NAND and the battery dies pfft! corruption. NAND should be sacred. The only log this should keep should be dmesg, it's embedded, there should be nothing to log. What do you want to see, iptables dropped packets, maillog? It's endangering the device, the world's stoutest server can mail itself to death in about fifteen minutes. Logs are not good things in the embedded space.

 

[dflemstr]

I plug an SD card in, PND magic happens, unplug that SD card and the device should be in the exact same state as if that PND magic never happened. The reverse should be also true, flashing or updating the OS should not affect the space of a PND.

Have we implied that this won't still hold true?
Also, sometimes you want to have something that is larger than the NAND has space for (let's say, icon themes or quake level files or something) so it'll be needed to have the ability to add "hooks"

Never cross the streams. Want to alter the OS, put it in the Angstom tree. I disagree with the location of home, should be in memory and check for an .xsession on the sd cards, don't think anything should ever write to NAND short of an OS update. Writing to NAND and the battery dies pfft! corruption. NAND should be sacred. The only log this should keep should be dmesg, it's embedded, there should be nothing to log. What do you want to see, iptables dropped packets, maillog? It's endangering the device, the world's stoutest server can mail itself to death in about fifteen minutes. Logs are not good things in the embedded space.

You'll have to speak with the kernel/init devs then so that they can quell all event sources. Devs in general won't be happy, though.

@ Others also:
Would it be acceptable to have all symlinks in memory, and put them at the bottom of the UnionFS? That way, nothing will write to the NAND, ever, but we'll still have the hooks ability.

 

[multiplex]

if I remove a PND at any time or remove any disk at any time, the PND system should not fail (by making a package database corrupt, as would happen with APT or RPM, etc). Sure, programs that are using the PND's files may fail, but that's impossible to prevent. The goal is to keep the "drop PND in folder to install, delete PND to uninstall" principle while still having the same featues as conventional packaging systems.

PND currently accomplishes this by not having those features. It doesn't seem like a helpful starting point for such a lofty goal.

True, but the write is still small compared to the multi-megabytes that would be required if you'd want to distribute whatever you want to distribute using IPKG instead (because IPKG can only install stuff to the NAND for obvious reasons). Also, NAND won't be a holy place of core files; writes will happen to it whether you like it or not. Your /home and /var/log resides there, for instance.

Also, sometimes you want to have something that is larger than the NAND has space for (let's say, icon themes or quake level files or something) so it'll be needed to have the ability to add "hooks"

I'm pretty sure a not insignificant amount of people will want to boot their system from SD card, with root, home, etc. all on card flash, effectively never writing to the NAND. They'll still want to use PNDs though. As a specification, it would be good if it didn't make too many assumptions about the system.

So, the rule will be that when a file already exists for a PND (either as a sane symlink or actual file), the PND may not be mounted at all.

The example you used, though, included an application needing a newer version of a library than what was included in the base OS. That seems to me like a case where if one application requires it, so will a dozen others. Wouldn't a per-application library directory and a custom LD_LIBRARY_PATH work just as well?
It seems weird to allow such basic gratuitous conflicts in a specification that otherwise takes such great pains to accommodate use cases I'm not quite getting.
The one listed in OP look satisfiable with the most rudimentary of union mounts, no PND needed, except for the library examples, where the only improvement over static linking was the ability to use newer versions from the OS, at cost of the aforementioned conflict problem.

BTW, I mentioned this in the other thread, but have you seen Slax? It's notable for it's heavy reliance on aufs for mounting of application modules.

 

[WizardStan]

1. It's already decided that /home will be on the NAND. Talk to the devs if you have an idea on how to change that (because the Pandora needs to be able to run even when no SD is inserted so it won't work without some brilliant idea)

If no SD card is inserted, /home is mounted as tmpfs. Any card in the primary SD slot gets mounted to /home. Eject the card and it reverts to tmpfs again.

 

[Enverex]

1. It's already decided that /home will be on the NAND. Talk to the devs if you have an idea on how to change that (because the Pandora needs to be able to run even when no SD is inserted so it won't work without some brilliant idea)

If no SD card is inserted, /home is mounted as tmpfs. Any card in the primary SD slot gets mounted to /home. Eject the card and it reverts to tmpfs again.

So every time I insert an SD card all my settings suddenly disappear and I end up having different settings for every program on my machine depending on which SD card I have in at the time? :huh:

 

[dflemstr]

So, the rule will be that when a file already exists for a PND (either as a sane symlink or actual file), the PND may not be mounted at all.

The example you used, though, included an application needing a newer version of a library than what was included in the base OS. That seems to me like a case where if one application requires it, so will a dozen others. Wouldn't a per-application library directory and a custom LD_LIBRARY_PATH work just as well?
It seems weird to allow such basic gratuitous conflicts in a specification that otherwise takes such great pains to accommodate use cases I'm not quite getting.
The one listed in OP look satisfiable with the most rudimentary of union mounts, no PND needed, except for the library examples, where the only improvement over static linking was the ability to use newer versions from the OS, at cost of the aforementioned conflict problem.

BTW, I mentioned this in the other thread, but have you seen Slax? It's notable for it's heavy reliance on aufs for mounting of application modules.

OK, I'll have to correct the rule: A PND can be mounted even if conflicted files exist, but if they already exist, no symlnk will be created that overwrites the existing file.

So, as you saw in my example, the newer library actually didn't overwrite anything in the system. The existing libxml2 library remained unchanged and all applications already on the Pandora would still link to it. Only our PND application would actually "see" the new library file, because it explicitly requests the 2.7.5 version of the library from ld. What I failed to show in my example were the symlinks libxml2.so.2 and libxml2.so that also would be in the PND, that would "add" the libxml2.so library to the system if it wasn't present at all. Please tell me if there are any issues with this system.

chroot setups with LD_LIBRARY_PATH would not be what we want, because it then becomes impossible for applications to find our files (in the case of icon themes, for example). Since no files will ever be overwritten, no conflicts will happen.

And Slax? It probably has a more stable system for something like this, but I think that it now is far too late to switch system for the Pandora. It could be an alternative package manager, however. The PND system as it is defined now, won't conflict with any other packaging system since it uses the UnionFS system to to effectively become the "fallback" file provider.

 

[WizardStan]

1. It's already decided that /home will be on the NAND. Talk to the devs if you have an idea on how to change that (because the Pandora needs to be able to run even when no SD is inserted so it won't work without some brilliant idea)

If no SD card is inserted, /home is mounted as tmpfs. Any card in the primary SD slot gets mounted to /home. Eject the card and it reverts to tmpfs again.

So every time I insert an SD card all my settings suddenly disappear and I end up having different settings for every program on my machine depending on which SD card I have in at the time? :huh:

If necessary. Or settings are stored in a different directory on the NAND, one that looks and acts like /home, but only settings files get stored there.
The problem is the difference between config files being stored in ~ which should persist on the NAND and data files people normally save to ~ which should in fact be stored on the SD card. A quick and dirty way of doing it is with a unionfs which creates all dot files (or files within dot directories) on the NAND and any non-dot files on the SD card (or tmpfs if no SD card inserted).

edit: I mean dot files and dot directories at ~ only, of course.

 

[dflemstr]

1. It's already decided that /home will be on the NAND. Talk to the devs if you have an idea on how to change that (because the Pandora needs to be able to run even when no SD is inserted so it won't work without some brilliant idea)

If no SD card is inserted, /home is mounted as tmpfs. Any card in the primary SD slot gets mounted to /home. Eject the card and it reverts to tmpfs again.

So every time I insert an SD card all my settings suddenly disappear and I end up having different settings for every program on my machine depending on which SD card I have in at the time? :huh:

If necessary. Or settings are stored in a different directory on the NAND, one that looks and acts like /home, but only settings files get stored there.
The problem is the difference between config files being stored in ~ which should persist on the NAND and data files people normally save to ~ which should in fact be stored on the SD card. A quick and dirty way of doing it is with a unionfs which creates all dot files (or files within dot directories) on the NAND and any non-dot files on the SD card (or tmpfs if no SD card inserted).

...or we can simply make the home dir inaccessible in the default GUI, thus forcing users to use SD cards, and warning users that are savvy enough to circumvent the protection and/or install other GUIs not to write to the home dir?

This has, however, nothing to do with our original discussion

 

[Caine]

OK, I'll have to correct the rule: A PND can be mounted even if conflicted files exist, but if they already exist, no symlnk will be created that overwrites the existing file.

Suppose that we mount PND A which creates a symlink to some shared object FOO in its overlay directory. Next PND B is mounted which also wants to create the same symlink to the FOO in its own overlay directory. The second symlink will be skipped and PND B will use PND A's FOO. Now unmounting PND A would require recreation of a new symlink which links to FOO of PND B.

This could get rather complicated.

 

[Enverex]

What's wrong with the /home on the NAND? I'm not keen on having no settings stored on the device itself, it's also where Linux needs to store your logon details and other basics, things that shouldn't be on an SD card...

 

[WizardStan]

Only problem with /home being on NAND arises when they start saving it as their document repository because they don't know any better. /home is often the default directory a lot of programs go to to save their files. And even if you do know better, there still may come a day when you're in a hurry and just forget to change it, and then you wonder why you can't find that cute kitten picture you just downloaded on your SD card.

 

[verm71]

That's why I suggested the symlinks shouldn't be deleted, to get rid of the excessive wear.

The issue I have with that is that if you try-then-delete a lot of software, you're going to end up with lots of symlinks over the years, and who knows what virtual spaghetti that could turn into. I would err on the side of keeping NAND read-only, unless a special kind of PND comes around whose sole purpose is to update the Pandora base firmware (by "base firmware", I mean OS, environment, GUI, sanctioned standard libs, and not media players, emulators, roms, or recently updated libs only required by certain games)

Just my $0.02 worth.