Discussion on repository, packages, etc. (split from Some thoughts thread)

[groovechicken]

I have wanted "fat binaries" with statically linked libraries for a while now. I know it is not trivial. Even PCBSD gave up on the idea eventually, but it is the only sane way to do this. So what if I have 12 different copies of the same lib in the 12 different bundles that need it. Another 64GB SD card solves that.

At the same time, I do look forward to being able to apt-get the stuff I want that maybe a lot of other people don't, like KMyMoney, Kate, Dolphin, even the full KDE, and having that stuff as part of the installed in the real way. But, a custom compiled SNES emu with all the needed libraries bundled together with binary so it will always work even if I apt-get upgrade, and that I can download wherever and double-click... sweet!

This has always been one of the strong points of OSX that I wish more distros would borrow. Maybe we can all band together to show the rest of the Linux world the right way to do it.

While we are at it, let's solve the "crappy cross-platform filesystem for removable media" problem. I have settled on XFS at last, for fault tolerance and good performance, but it means I won't be mounting my SD card that I use for working files on anything but Linux for the foreseeable future.

 

[_wb_]

Isn't a disadvantage of static linking that not just on disk, but also in RAM there are no shared libraries? For single-tasking that does not matter much, but for multi-tasking it does, no?

(The advantage on the other hand is that if you use only part of a library, static linking is more memory-efficient)

 

[bzar]

Another issue of static linking is not getting library security updates and bugfixes to all applications at once, but all applications need to be separately updated to use the new stuff.

 

[Neelix]

Isn't a disadvantage of static linking that not just on disk, but also in RAM there are no shared libraries? For single-tasking that does not matter much, but for multi-tasking it does, no?

(The advantage on the other hand is that if you use only part of a library, static linking is more memory-efficient)

Perhaps so,  but I've noticed that sometimes a PND fails to unmount because some of its libraries are still active after the application has been closed,  and since there's usually no obvious sign of that unless you know to look (and what to look for) that can lead to unexpected results, (like loading the old version of a program after it's PND has been updated) so for PND (or DBP) encapsulated applications perhaps that's less of a disadvantage than it seems.

- Neelix

 

[levi]

Other apps won't be using libs included in the pnd currently mounted, so that's no reason for it not to unmount.  I'm not sure what stops them often unmounting cleanly (although a recent reboot seems to improve the situation), but I'm pretty sure shared libs are not the culprit by virtue of being sharable at least.

I've had a couple of thoughts about dependencies.  Of course, continuing with the Pandora way of doing things where if you need a new lib you have to include it in the package will still work, but pnds did increase in size over time because of this.  At least for some libs I've seen that you can have two incompatible versions installed at the same time, but still get security updates and patches for both.  If the pyra packages can encode dependencies on major versions then we might not have a problem, provided minor version updates don't break compatibility..

It might be a nice goal for all pyra packages to keep updated with the latest and greatest, but I don't think that's ultimately feasible, and maintainers will fall by the wayside from time to time.  Provided we keep compatible libs in /usr/lib or in exceptional cases in the package we should be dandy.

 

[bzar]

PNDManager PND for one would be quite a bit smaller if it didn't contain Qt

 

[_wb_]

PNDManager PND for one would be quite a bit smaller if it didn't contain Qt

Yes. Also I want to apt-get install dbpmanager, not have it as a .dbp

 

[WizardStan]

PNDManager PND for one would be quite a bit smaller if it didn't contain Qt

That's a problem with the size of the Pandora's NAND, that it's too small to contain something as standard as Qt libraries, and is actually a prime example of why building in the mechanics to be able to include all libraries (or at least not coming up with a design that prevents or even discourages it) is a good thing.Besides what, it's 8MB, so even if you weren't including necessary libraries, even if what you need is duplicated across a dozen other PNDs and that's actually 100MB that could be saved, it's not actually that big of a deal, and trying to fix the problem may introduce more headaches than it solves.

 

[_wb_]

If you include standard stuff like gtk or Qt instead of installing it globally, then what bothers me most is not disk space, memory, or bandwidth issues, but not being able to have a global configuration (e.g. a gtk theme).

 

[levi]

Hmm, shouldn't configurations be in /etc or ~/.config (on the nand) so be globally settable wherever the actual code resides? I guess ~/.config will get mapped to go inside the appdata, but /etc should still work.

 

[_jr_]

Other apps won't be using libs included in the pnd currently mounted, so that's no reason for it not to unmount. I'm not sure what stops them often unmounting cleanly (although a recent reboot seems to improve the situation), but I'm pretty sure shared libs are not the culprit by virtue of being sharable at least.

It happens when a PND uses another PND (e.g. wxpython) by prepending that PNDs lib to LD_LIBRARY_PATH and that lib containing a shared object (e.g. libgcc) that is used by pndrun. Then the lib inside the second PND is in use while the first PND is trying to unmount the second. There are several PNDs on the repo that have this problem (e.g. Anki). Probably it should be made sure that different libgcc doesn't leak to exes not compiled with that version of gcc, but in practice it is enough to reset LD_LIBRARY_PATH before unmounting.

I've had a couple of thoughts about dependencies. Of course, continuing with the Pandora way of doing things where if you need a new lib you have to include it in the package will still work, but pnds did increase in size over time because of this. At least for some libs I've seen that you can have two incompatible versions installed at the same time, but still get security updates and patches for both. If the pyra packages can encode dependencies on major versions then we might not have a problem, provided minor version updates don't break compatibility..

While using various binary linux ports of games from humble or gog a typical problem is that dri interfaces have changed incompatibly sometime and it doesn't seem to be easily possible to have different versions (but I may just be too stupid). In the end I think linux's 'backwards compatability is secondary' stance makes it a less ideal environment for binary only distribution models than windows, but that isn't neccessarily a bad thing.

 

[levi]

Yes, that is a problem with raw binary distribution, a la humble bundle games and others.  Binaries+version dependencies ensuring the right backwards-compatible versions of stuff get installed at the same time would fix that problem if done right though, which AIUI is what is being proposed for the Pyra by various people.

 

[_wb_]

Hmm, shouldn't configurations be in /etc or ~/.config (on the nand) so be globally settable wherever the actual code resides? I guess ~/.config will get mapped to go inside the appdata, but /etc should still work.

Yes, but I wasn't talking about configuration files in the narrow sense of the word. I also mean things that would end up somewhere in /usr/share or /usr/lib, things like plug-ins or data files or some kind of extensions or optional stuff, things you may want to customize or tweak or hack in some way or another.