Confliting requirements

[destraht]

The PND system needs to be kept, in my opinion, but enhancing it if need be.


Yes, at home my desktop doesn't need that much space - because I don't play games on it.


Games need A LOT of space (for music, videos, etc.), so they should really be kept on an SD Card.


And a normal SD Card is formatted in FAT32 or exFAT these days.


I doubt Debian could use FAT32 as normal storage space (permission issue) and it will certainly wreak havoc in case the card is being removed (which will happen - it's removable storage!)


We've got PNDManager, we've got the daemon, we've got the repo, we know that system works.


Where's the issue installing normal stuff like FireFox and Libreoffice from Debian repos and most games using PNDs?


SD Cards are cheap, where is the issue including a library with a couple of KB size inside a PND to make sure it works with future firmware versions?


Most standard libraries stay downwards-compatible with new versions, so it shouldn't even be a lot of them.


Until no one can provide a way to use standard .deb files that offers the same advantages as PNDs do (config and data on SD, removing SD Carrd doesn't break things, simply copy the PND onto the card using a normal windows PC, etc.), I really don't see a reason to give up PNDs.

I took the deep plunge and formatted my 64GB SD card as ext4.  Perhaps a simple GUI script could be made for this.

 

[destraht]

No. PNDs without any dependencies would suck. It means you have to include A LOT of libraries (or statically link them), and A LOT of tools like LaTeX and ImageMagick and bash and whatever you need.

Having to include all of that is problematic for many reasons. Size is the least of them. The biggest problem is that having no dependencies forces devs to become packagers/maintainers of not only their own software, but also all of its dependencies. If I want to use a recent version libqt and pdflatex in my program, I need to include those in my PND. If someone else also uses those tools, he also needs to include them. Whenever there's a bugfix or security update or improved implementation of any library or tool, it takes A LOT of effort involving A LOT of people to make sure that it percolates to all software that relies on it. It means EVERYTHING has to be re-packaged all the time and re-downloaded by everyone.

So no, this is not a good option.

The Pandora approach of choosing a fixed base system and allowing PNDs to rely only on that base system is a slightly better option, and it makes sense if you have a very limited internal storage anyway. But it is not a very future-proof option: it essentially means that the base system has to be frozen or at least has very restricted options for updates.

I think it's much better to have a flexible base system and PNDs (or rather .pyra files) that have meta-data for their dependencies. From the user point of view there is barely any difference with the current PND system, especially because .pyra files will mostly be used for games, which tend to have few dependencies and those dependencies will probably be installed already in the default install anyway. The main difference is that instead of getting weird crashes, you get a pop-up box saying "You have to update your base system to run this PND, or else it will not work properly. Click OK to update".

It could be interesting to have different versions of the Pyra base system which specifically lists core libraries.  This would work the same as how you describe and it would give PND developers something to target while still leaving room for downloading DEBs.  People could then appeal to have certain libraries from the Debian repository included in the next Pyra Base.  Hell even if this is ultimately abandoned then it could really help to improve consensus on what should be included by default.

 

[destraht]

I don't propose to merge .deb and .pnd or whatever it is you think I propose.

I merely propose to replace the "OS version" field in the PXML specification (a field that is now simply not used and usually ignored) with a more useful "Depends on:" field, which lists the base system libraries and tools the PND (or .pyra, because for other reasons we may want to define a new file format too) uses. The best way to represent those dependencies is to do it in the same way as .deb packages do it (assuming our base system is Debian or Debian-based).

I think that you should have both such as Uses Pyra Base 1.3 and Depends on: (anything beyond Pyra Base 1.3).  I think if the base system is updated only very sparingly then it could workout pretty well.  Every two years there would be a six month to one year scramble while everyone gets their PNGs setup to work on the next version of Debian.  The point versions of Pyra Base would simply be to add some common dependencies that were left out.

 

[destraht]

Well, that's why suggested to have as many standard libraries as possible into the OS.

Have a meta package for the base installation for the Pyra.

And whenever there's a library that's needed for a new PND that could also be useful for other devs, update the meta-package and the next OS update will include it.

Sure, the dependency lists in the PNDs are a good idea as well and would have a similar result.

Give it a name and a version such as Pyra Base or Pyra Core.

 

[destraht]

Ok apologies for spamming the board.

So PNDs are mostly for gaming right?  If this assumption is valid then force a requirement of a somewhat large Pyra Base 1.2 to allow usage of PNDs.  Then let major versions completely redefine the dependency list.  So let someone use their Pyra as a hard core business machine with only a stripped down Debian install and then if they want to game then they must install Pyra Base.  I think that it would be best to let users f*** off PND support to have a lean system.

If Evil Dragon rocks it the Pyra could get three years on Debian 8 with a full year at the end for migrating PNDs to the next Debian.

Are PND files meant to be forever valid?  Or are they meant to solve storage and portability problems?  Do I really need a PND for a 5MB niche utility that can simply be installed as a .deb?

So I'm seeing it that the majority of current PND programs are better suited to be installed the normal way.  Ur-Quan Masters as a PND then needs to get remade every 2-3 years.  That is not really that big of a deal.

Then if someone can do the research make a Pandora Legacy Base that tries to fill the gap between Debian 8 libraries and older libraries used on the Open Pandora.  Put a marker in the new Pyra PNDs that the newer package manager can read so that it knows not to require Pandora Legacy Base and the absence of it will imply that it is an older format for an older system (specifically openpandora).

So:

* Default system is Debian 8 with of basic stuff like Firefox, SDL

* Pyra [Gaming] Base requires a pretty large set of library packages for a good attempt at minimizing static linking.  This will be redefined every 2-3 years.

* Pandora Legacy Base is an external repo hosted by Pyra Inc. that includes many libraries that are installed in a non-standard place and then the PND loader will setup the environment to load from all of these older library locations.

So maybe I don't personally care about old Pandora PND programs and I only want to look forward and as a new user/owner I will already have too much on my hands.

 

[destraht]

Then offer two Debian images; one with a stripped down Debian 8 and only Pyra LEDs, drivers, etc and the other with both pandora-legacy and pyra-gaming-base installed.  Then make pyra-gaming-base be a meta package that points to the newest version.  Then the stripped down Debian version can install pyra gaming with a simple 'apt-get install pyra-gaming-base'.

 

[vinipsmaker]

So PNDs are mostly for gaming right?

You're late on the discussion and your proposal is difficult to understand.

People seem to agree that only special software would be installed as PNDs. This is mostly games.

The Pyra base you talk about would serve a purpose similar to STEAM runtime and we're defining one set of guaranted libs. In my opinion, most common of hardware-abstracting libs should be part of the set to be more future-proof. It's bad to define a large set of dependencies, because they might break the system in the future.

It'd be possible to deliver two types of PND-next. A light version, which would depend on .deb packages and share its dependencies with the rest of the system and a flat version, more self-contained, heavy and future-proof (?).

This is kind of a summary.

 

[comradekingu]

How about a script that pulls down packages with dependencies, then a GUI tool to batch install with debs, or move them to /var/cache/apt/archives and you can install from package manager without downloading. There are so many ways to do offline installs, i dont see why .pnds or .pyras are the best option.

I may be suggesting dumb options, but at least i know how they work. There are good options to installation restraints too, it just seems to me that people who argue for pnds either dont take into consideration the underlying complexity, or start their reasoning with pnds, forgetting how it was born out of limitations we no longer have.

If it was maintainable and the old pnds could work in a similarly maintainable way on a new OS base, sure, but that doesnt seem to be the case from what i can understand. Still would be very thankful if someone could explain the bigger picture.

 

[vinipsmaker]

There are so many ways to do offline installs, i dont see why .pnds or .pyras are the best option.

They're self-contained, less complicated and more idiot-proof. I'm an idiot and I make mistakes and I don't want to make choices that the software can do better.

And why is sebt3 the only guy who actually showed us a right way to do offline packages installations involving dependencies? http://boards.openpandora.org/topic/15530-the-software-side/page-9#entry311602/URL]

Not a problem. But why are people still repeating a fragile solution? "Ship the .deb dependencies and everything will be fine". How is this fragile solution better than ship a more self-contained package that will not interfere with the rest of the system and will not break things?

 

[bzar]

comradekingu: PNDs also contain other assumed functionality, like appdata overlays stored on the same device as the PND and autodiscovery. Most offline installs don't do either. (I may have misunderstood what you meant here)

Personally, I'm starting to lean towards a versioned common collection of libraries. PNDs can be bound to specific platform version range. Pack everything not in the platform into the PND. When stuff becomes common, add it to the platform, and next versions of PNDs can set new platform requirement and strip the libs. With a system like PUR the packages that need this after a platform update can easily be identified and fixed. I think this will work out much better on Pyra than it did on pandora because we have more space to work with and a limited amount of expected use-cases for PNDs. Unlike in, say android, we won't need to keep backwards-compatible with old versions as long as we know when the current platform version is too old. So this shouldn't lead to people trying to support every platform version and packing everything inside just in case someone is using an old platform.

One thing I don't know how to handle is setting the upper version limit in cases where the latest version works at recipe creation time. Setting it to unbounded causes unhandled breakage when a version breaks compatibility, setting it to current version requires actions for all PNDs on every platform release to test and bump the version. Maybe in those cases setting per-package version requirements would be more sensible? What if these two conflict? How to set the upper version bound for per-package requirements (since not everyone does semver)? We also don't want everyone to upgrade every PND they have on every platform update just to get the version bumps.

This is hard

(used the abbreviation PND for any incarnation of it)

 

[comradekingu]

vinipsmaker:

B but... debs are the system taking care of what it does. If you make a new system on top of that you get to keep the pieces when it breaks (ref pnd),  and it doesnt explain why that is better.

Nor did you explain why my approach is worse, unless you want to also do system dist-upgrades offline too...  That 2014 though. Apt-offline, or any of the other offline repo approaches is not the _right_ way to do that, unless you want it to be. There are daily images of debian, those should work. (!)  Contrary to popular belief, sending machine specific patches upstream, is both legal and encouraged.

Deb dependencies arent fragile, because we arent talking about homemade .rpm packages from the 90s. saxxon-ängstrom was firmware in the bad sense. Debian is an OS.

I would totally agree shipping self contained was the better option, and touching as little as possible if this was windows, but it isnt.

"ship a more self-contained package that will not interfere with the rest of the system and will not break things? " I like how you put _more_ in there, as if you knew what you were saying wasnt quite self contained.

"that will not interfere with the rest of the system and will not break things"  this sure isnt pnds you are talking about, more like how they should have worked. They are inherrently broken and they break too. Try running something on saxxon b2 and even earlier.

The user isnt dumb, and not moreso than the script written to do offline installer, KISS KISS KISS.

B-Zar:   I dont know what an app overlay is, other than that its the same and you are saving seconds on the install and removal. I dont understand what is going (proposed) to be  a pyra package and what is going to be a deb package, but i _do_ understand i want pnd-manager as a frontend for apt-get and everything available to the control of the latter.

You are doing the same as debian upstream are doing, you are implying something else than debian stable, you are doing the same thing as control files in deb packages do. Its at worst a trivial change or a force install away from working. At some point you have to do something to the source, that isnt anything specific to deb packages, and it only happens when you want to roll out new libraries...  And testing testing, pun intended, is not really hard to do...

Lets reccomend a sane development platform with languages and solutions that dont break backwards compatibility each time, i want to hurt myself when i see everyone jumping on the latest shiney hipster shit.

If you dont want to support pnd bandwidth, send it upstream to debian, they have bandwidth aplenty, problem solved [ ]   _OR_ Make a beta repo for unstable packages and drop stable stuff on users in the deb repo we already have when packages are deemed stable.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

All in all maybe someone wants to do offline binary files and are more comfortable with that, maybe even the documentation for it, but why does that have to mess up the whole system? A good deb base system makes it easier for other distros to shine where they should.

Speaking of which, how are all the pnd packages going to get ported? Hopefully not into another non-free / free without -src pretend everything works when the user sees it type ordeal.

Its ok to add intermediate hacks (like a chroot) to the state they are in, but if we can just do the heavy lifting over to deb with source once, and do it right, then no more ugly wizard sourcery.

Thinking of software as cartridges and the system like something you aren't supposed to touch is antiquated thinking, and it sure isn't Linux|gnu.

TL;DR The debian way, a new chapter will be written.

 

[Bosbeetle]

Thinking of software as cartridges and the system like something you aren't supposed to touch is antiquated thinking, and it sure isn't Linux|gnu.

 

??

Thinking of games in a plug and play manner might not be "linux/gnu" or "modern" it sure is damn handy. And for being an antiquated way of thinking it sure works on the pandora and things like the 3ds. Of course you don't want to close anything down, i.e. you can do what you like.

I see not going for a pnd-like solution (luckily it doesnt seem like an option for ED) as throwing the towel in the ring. Im confident that there are people in this community that can push forward and make the pyra system the best of both worlds. The only thing "we" have to keep in mind is that not all people using a pyra are comfortable handling linux in the way a developer is. Just to give an anecdote I managed to thether my phone to the pandora using some pdd (I think thats what it was called) script and changing a bunch of files  that seem to be core linux and setting some bluetooth. And however I managed I cant say I felt very confident doing it, and during an update something didnt work anymore and I cant find what it was.... 

 

[bzar]

comradekingu: Appdata overlay is what allows your PNDs to write to $PWD and have it go to /media/theSDCardYourPNDIsOn/pandora/appdata/yourapp/ while still keeping everything inside the PND squashfs filesystem available like it was in the same directory. It's what allows you to override specific files inside PNDs by putting files in appdata.

Please consider how your proposals would work when the software is on removable storage that may disappear at any moment and next time appear with a different set or other media present.

EDIT: Oh, and about the porting part, I do hope we'll use Cloudef's PUR and makepnd recipes for that so the issues you mentioned will be no more.

 

[vinipsmaker]

Nor did you explain why my approach is worse, unless you want to also do system dist-upgrades offline too

Your approach is not worse, only when it comes to offline installation:

PND-game Foobar needs libabc. It include libabc on the SD-card. libabc requires updated version of libdef. An old version of libdef was already included in the system, because maybe Firefox or any app was already installed. But your system included an updated libdef, because your system is brilliant. Now I update libdef and Firefox stops working? Why? Because libdef doesn't maintain ABI compatibility and packages need to be recompiled. Congratulations, you fucked my system.

.deb and offline installation can be done without fear under apt-offline, unless you include the whole repo on the SD card, which is pointless.

Deb dependencies arent fragile

Your approach to offline installation with dependencies is fragile. See above. .deb is fine.

"ship a more self-contained package that will not interfere with the rest of the system and will not break things? " I like how you put _more_ in there, as if you knew what you were saying wasnt quite self contained.

You're right. But it can be improved in PND-next. Remember the idea, not the man.

"that will not interfere with the rest of the system and will not break things"  this sure isnt pnds you are talking about, more like how they should have worked. They are inherrently broken and they break too. Try running something on saxxon b2 and even earlier.

PNDs don't break the system. Your solution to ship .deb on SD card is even worse. Besides, PND-next can be improved.

 

[sebt3]

EDIT: Oh, and about the porting part, I do hope we'll use Cloudef's PUR and makepnd recipes for that so the issues you mentioned will be no more.

Me too

 

[Neelix]

So PNDs are mostly for gaming right? If this assumption is valid...

Personally I see that as a flawed assumption.  I want to be able to set up my base system with the few programs that I consider essential, and those that are impractical to make portable, then use PNDs for anything and everything else.

The amount of internal storage is not going to be unlimited,  (we don't even know how much room we have to play with worst case)  and it sounds like we'll be filling a fair chunk of it up with libs.  If we assume that only games will be installed to SD cards then we are limiting the number of other applications that can be installed to what will fit on the internal storage.  That doesn't sit right with me.

 - Neelix

 

[WizardStan]

Personally I see that as a flawed assumption.

Code::Blocks, for example, is 1.3GB. It contains all the dev libraries and such. Now, you COULD install these via deb packages and they'd reside happily on the internal disk space but then they're sitting there taking up 1.3GB and the only time they're ever used is when I want to compile something, and should I ever find myself needing to reclaim space rapidly that's dozens of packages to uninstall and no guarantee of getting them all or their dependencies. If it sits on the SD card not only can I just remove it if it causes problems but I can also install additional packages into the appdata directory and plop that SD card into something else and BAM got all my dev tools and libraries.Don't box the format as being for one thing or another, it should be for anything that makes sense to be able to install/remove quickly, or benefits from being portable.

 

[ptitSeb]

I don't want to interfer with your debates, but just to mention that, even if codeblocks PND is 1.3Go of filesize, don't forget it's a squashfs ISO, so it is a compressed archive. The uncompressed "lib" folder alone make something like 2.4Go. The whole Codeblocks is more close to 6Go with the includes, docs, binaries, and so on...

If I take a smaller example, wxPython, wich include Python, a large buch of Python libs (and the native libs that goes with them) is 260Mo compressed, but at least 600Mo as plain files...

 

[WizardStan]

don't forget it's a squashfs ISO

I did forget that, thank you. That proves the point further, I think.

 

[vinipsmaker]

In this/self post I don't argue in favor or against the PND drag'n'drop easy-install use case.

I don't want to interfer with your debates, but just to mention that, even if codeblocks PND is 1.3Go of filesize, don't forget it's a squashfs ISO, so it is a compressed archive. The uncompressed "lib" folder alone make something like 2.4Go. The whole Codeblocks is more close to 6Go with the includes, docs, binaries, and so on...

Now, I wouldn't take this in account. I don't think it's a fair argument. Btrfs, the "next default Linux filesystem", may be here (stable) before the Pyra handheld. Btrfs supports transparent compression ( https://en.wikipedia.org/wiki/Comparison_of_file_systems#Allocation_and_layout_policies ).

The argument is true now, but might not be so relevant when the Pyra handheld become available.