Confliting requirements


vinipsmaker

Member
Joined
Jun 22, 2013
Messages
209
Location
Brazil
Website
vinipsmaker.github.io
As you can see from here: http://pandorawiki.org/New_PND_format#The_2014.2FPyra_effort

Some requirements conflicts and we can't have both. For example, some users ( http://boards.openpandora.org/topic/15530-the-software-side/page-7#entry310691/URL] ) want a format that can be run in both (Pandora and Pyra) and some others want a format that can list deb files as dependencies ( _wb_ ), but the SuperZaxxon doesn't support .deb packages.

You guys need to explain yourself more clearly.

Also, there is the:

PND is good, it let you to put all your apps on the SD card. You just put it on another Pandora and all your apps work, no effort.
But a semi-confliting thought also:

It should have .deb dependencies that are installed on-demand. (but won't work when you don't have internet access. It's not plug-and-play, it's plug-and-download).
These are just a few examples, but let's start with them, then I'll move to other problems.

Why spent effort with this? Because the system needs to be IMPLEMENTABLE. Design and format specs can follow later.
 
Last edited by a moderator:

comradekingu

Glowing ember
Joined
Apr 15, 2011
Messages
5,055
Website
portfolio.anotheragency.no
An effort to bring everyone more together on the issue at hand, because there is good reason to do things differently now,

The problem here is that pnd files are made dependant on a fixed environment (superzaxxon), and that system upon itself as a result, and it doesnt always change, but when it does, as even remotely feasible solutions do, it breaks backwards compatibility. Instead of having source code available to be able to do something about this, its down to a "possibly, some of the time-approach" to do anything about it.

Add to that a package manager that cant deal with dependencies, and the problem is systemic to pnds.  Opkg, the package manager in angstrom, which is ok enough, is no longer poiting towards angstroms own package tree in superzaxxon, because those are now incompatible with superzaxxon as a result of it not staying up to date with changes upstream.

So there is a super zaxxon maintained tree, which is very lackluster. The "firmware" happens behind the scenes with magic, and pnds are conjured up in no (symbiotic) relationship other than another layer of ugly hacks. Its a marvel and a beast, and i have great respect for the ones who can keep it running.

So two versions of super zaxxon, in varying relations to angstrom, and everything else as pnds, which compatibility with saxxon to suppersaxxon broke over time, the alternative is a fresh .next approach to doing it right, but losing compatibility for the same reasons. This time in bulk. What happens is you end up having to roll out new programs manually as pnds. That is really grim.

Thanks to debian, being debian, the debian way, much of this is in the past. For the reason pnds arent as good as deb files, the baggage of pnds with usable data is tricky to maintain. Doing something to angstrom, even the right way, is hard (.next), doing something ugly in debian is easy enough, like a chroot, its ugly in the sense the problem is there to begin with, in the pnd structure.

Before one goes about reimplementing anything, having the source code, available, in machine readable format, is a good thing. Before you get to a freedom aspect, its broken in a fundamentally technical way.

Having the devs available and not dead, isnt a fully working system, its a poor excuse to tax them, or anyone else with work they shouldnt be doing in the first place, ideally. That is the lesson to learn.

The debian approach to a stable system that stays sane, while no excuse to be lazy about the fact, is debian stable, then rolling release is testing, which is what the stable is made from at intervalls of a few years give or take, and if you want anything else, get it from unstable or experimental

pnds are born out of the nand space being limited, which no longer (presumably) wont be a pressing issue.

For a variation over the theme, enough space, you install / at nand, then have SD cards for whatever /home things you might have. (music, films, media, personal files etc etc)

But what about big games, (which otherwise might saturate the internal space) and to a lesser degree system apps?

Self-contained environment packages isnt much of a thing once the base system has shiny stuff available, and that means things can work together, whenever something becomes incompatible, updates, it can be dealt with in an automated (best-case) manner.

.deb files are great, but whatever symlink, unionfs, or custom, whatever approach you take, dpkg doesnt like stuff going missing. And having the system be aware of dependencies is something best preserved, which is back to the debian way.

Having binaries on SD cards, going missing, is the issue to be dealt with, with as little glue-hacks as possible, and maintainable for the forseeable future.  Being able to download offline, and put on sd cards, and run, is convenient, but i would argue a little bit more defunct now that wireless speeds will be super duper, always, without fail, some of the time, everywhere, when wind direction premits.

Pandian is available for the pandora, which is a nice way of maintaining a current approach to having debian do the heavy lifting, and compatibility on pandora with whatever stuff is made for the pyra, atleast as good as can be.

Spending effort on porting pnds to something that is maintainable is worthwhile. And its doubly effective since then one doesnt have to document a non-standard system.

For reasons sharing is good, giving back to debian by way of games and emulators is a  nice gesture for the use of the fantastic service and tools it provides.
 
Last edited by a moderator:

vinipsmaker

Member
Joined
Jun 22, 2013
Messages
209
Location
Brazil
Website
vinipsmaker.github.io
comradekingu, thanks for the background.

.deb files are great, and whatever symlink, unionfs, whatever approach you take, dpkg doesnt like stuff going missing.

Having binaries on SD cards, going missing, is the issue to be dealt with, with as little glue-hacks as possible, and maintainable for the forseeable future.
I don't think this is a good decision. We'd have to fork/workaround a Debian's core-component.

Pandian is available for the pandora, which is a nice way of maintaining a current approach to having debian do the heavy lifting, and compatibility with whatever stuff is made for the pyra, atleast as good as can be.
So you're saying... there is no need for PND (or a possibly successor)? Then we should start working on the Debian port for the ARM.

But before, is this the feeling for all developers? I know at least one comment which states the opposite: http://boards.openpandora.org/topic/15530-the-software-side/page-3#entry310412/URL] . We can solve it all by just creating the compatibility layer. I can start working on this layer as soon as enough developers agree on this decision.
 
Last edited by a moderator:

levi

Still fresh, damnit!
Joined
Oct 6, 2008
Messages
12,290
Location
Somewhere off the coast of the EU
Having proper dependencies doesn't solve many of the problems that not having the source for things creates. I believe those are separate problems, of differing significance. But ignoring that...


I agree, we can rely on being connected to the internet. Offline installation is largely solving a problem of the 1990s or early 00's, at least in the developed world. And frankly, online installation is so much more flexible and dynamic that I believe the effort to maintain a connection while you do an install/upgrade is worth it even if you have to rely on a 2G connection. And downloads shouldn't be massive if you do periodical updates anyway.


Game developers generally shy away from linux packages and just provide .tgz binary files, but that's because they have to provide multiple formats for debian-based, fedora-based, arch, etc. And they certainly never push them to the official repositories, as getting involved in all those different communities is too much like hard work. We wouldn't have those problems if we used .deb files for libraries and something similar for games and end-user programs or games (but which allowed them to be removed, and have their home folder data on SD, and maybe allowed users to override individual files as they can currently with PNDs).


That two-class package structure would make the pluggable packages sort of second-class. Probably not tracked by the system's dependency tracker, and thus not usable for other packages dependencies, but that's probably okay. We may find ourselves needing to release updated .deb packages ourselves, as debian stable doesn't keep up with less core stuff on purpose, like Python. But we can do that by hosting out own semi-stable repository, if that doesn't go against the Debian way, and is only used by this community.
 

WizardStan

Mega GP Mania
Joined
May 24, 2008
Messages
16,733
So you're saying... there is no need for PND (or a possibly successor)?
I can't in good conscience agree with that.Installing stuff to firmware via packages automatically assumes a fixed location for $HOME. Working around this so that PND applications always fine $HOME on the SD card they were installed to was fairly simple (with a couple kinks along the way) but I forsee deb installed packages having much more difficulty. I'm not sure if I can be comfortable with having no standard way for apps to know to store their stuff to SD card. If we have both deb and pnd we can at least say "pnd stuff writes to SD card home, deb stuff writes to nand home". If we just have deb packages we enter a realm of "ths deb is from the debian repository so it writes to nand home, this was a quick compile in the Pyra repo so it might write to nand home or not, this deb treats the first SD card as home, this deb asks the user where home is every single time, etc...." confusion.

I'm also worried about flash size. No matter how large it is someone is going to install enough stuff to run out, and I am afraid it will be a lot more common than anyone might suspect. Wesnoth, Ur-Quan, it'll add up over time. Being able to swap SD cards and get a different set of games is extremely useful and saves overal space, and if you want to free some space it's trivial to delete a game without losing your save game.

It seems like the chief complaint about PNDs is that bundling libraries means a lot of duplication but I'd rather have 2 copies of te same library on replaceable SD cards than 2 different versions of a library on a finite nand space, but that's my take on it.
 

vinipsmaker

Member
Joined
Jun 22, 2013
Messages
209
Location
Brazil
Website
vinipsmaker.github.io
Installing stuff to firmware via packages automatically assumes a fixed location for $HOME. Working around this so that PND applications always fine $HOME on the SD card they were installed to was fairly simple (with a couple kinks along the way) but I forsee deb installed packages having much more difficulty. I'm not sure if I can be comfortable with having no standard way for apps to know to store their stuff to SD card. If we have both deb and pnd we can at least say "pnd stuff writes to SD card home, deb stuff writes to nand home". If we just have deb packages we enter a realm of "ths deb is from the debian repository so it writes to nand home, this was a quick compile in the Pyra repo so it might write to nand home or not, this deb treats the first SD card as home, this deb asks the user where home is every single time, etc...." confusion.
You're overreacting. Software data (usually stored on the .config folder) takes little space. The most consuming data is user data (the roms, the music, the videos, ...).

If you're worried about transfering the data around, you can use dropbox or whatever and sync the .config folder. Or you can also create a link from .config to your sd-card.
 

WizardStan

Mega GP Mania
Joined
May 24, 2008
Messages
16,733
You're overreacting.
Wesnoth: 360MBUr-Quan masters: 140MB

Penumbra: 1.6GB

it can add up fast.

Software data (usually stored on the .config folder) takes little space.
/usr/share is substantially larger, I assure you.
Or you can also create a link from .config to your sd-card
I'm not worried for me, I'm worried for the average user. Standardized PNDs makes it easy for the system to keep all data and saves on an SD card, and remove that card the game disappears. Attempting to symlink .config and /usr/share onto an SD card forces you to have a fixed SD card, at which point you may as well install to a 128MB SD card. There are a lot of problems PNDs solved and those problems continue to exist so long as we have finite storage and a desire for ease of use.
 

vinipsmaker

Member
Joined
Jun 22, 2013
Messages
209
Location
Brazil
Website
vinipsmaker.github.io
I'm not worried for me, I'm worried for the average user. Standardized PNDs makes it easy for the system to keep all data and saves on an SD card, and remove that card the game disappears. Attempting to symlink .config and /usr/share onto an SD card forces you to have a fixed SD card, at which point you may as well install to a 128MB SD card. There are a lot of problems PNDs solved and those problems continue to exist so long as we have finite storage and a desire for ease of use.
Then you can just continue to use PNDs. The system is here and there are a lot of packages already available. For what I'm reading, developers are falling in love for .deb.

It's not possible to meet all requirements being made, and... with nobody wanting to give something up, I think no new format will succeed and we'll have .deb and .pnd only.

Btw, somebody on the IRC suggested that Pandian already has a Pandora/PND layer.
 
Last edited by a moderator:

SeeThruHead

Member
Joined
Dec 23, 2010
Messages
112
Is it not possible to make the storage large enough that we don't have to worry about removing software? Maybe spanning the root across the nand + sd storage?
 

thatgui

Advanced Member
Joined
Apr 2, 2009
Messages
3,048
I'm also worried about flash size. No matter how large it is someone is going to install enough stuff to run out
I do not want ot argue against the pnd or a similar system it has a lot of advantages, but is this really a problem ? People are used to space on their HDD/SDD/etc running out, especially nowadays as most phones offer only a limited amount of storage (and often no possiblity to expand it). So what is stopping them to deinstall something (other than convenience) ?But maybe I'm just still confused by ED stating that he has "problems" filling up his Desktop hdd with software installs (in the short discussion about eMMC size) while he has 60-70GB woth of pnds on his SD-Card (in the pnd or not discussion).
 

EvilDragon

Administrator
Staff member
Joined
Mar 4, 2003
Messages
29,473
Age
42
Location
Ingolstadt
The PND system needs to be kept, in my opinion, but enhancing it if need be.

Yes, at home my desktop doesn't need that much space - because I don't play games on it.

Games need A LOT of space (for music, videos, etc.), so they should really be kept on an SD Card.

And a normal SD Card is formatted in FAT32 or exFAT these days.

I doubt Debian could use FAT32 as normal storage space (permission issue) and it will certainly wreak havoc in case the card is being removed (which will happen - it's removable storage!)

We've got PNDManager, we've got the daemon, we've got the repo, we know that system works.

Where's the issue installing normal stuff like FireFox and Libreoffice from Debian repos and most games using PNDs?

SD Cards are cheap, where is the issue including a library with a couple of KB size inside a PND to make sure it works with future firmware versions?

Most standard libraries stay downwards-compatible with new versions, so it shouldn't even be a lot of them.

Until no one can provide a way to use standard .deb files that offers the same advantages as PNDs do (config and data on SD, removing SD Carrd doesn't break things, simply copy the PND onto the card using a normal windows PC, etc.), I really don't see a reason to give up PNDs.
 

Bosbeetle

Terminally lost
Joined
Sep 7, 2008
Messages
3,771
Age
38
Location
The Netherlands
Website
Visit site
I must say that the pnd system is one of the things I really like about the pandora, im not a linux guy (at least not beyond setting up a swapfile or some stuff like that) but the pnd system is very intuitive. It is one of the things I always show off to the people I show my pandora... "look here I have a system with word, powerpoint, imagej, R, etc... and now by changing the SD card I have a system with an msx, nintendo, sega, duke nukem 3d and super geometry dust. I totally love this system :)
 

PokeParadox

Founder of Pirate Games - Penjin Coder
Staff member
Joined
Dec 8, 2005
Messages
6,519
Age
36
Location
UK
Website
www.projectinfinity.org.uk
I agree it does seem silly to abandon one of the strengths of the system. Sure it might be hacky and ugly underneath but even a duck is paddling like hell under the water when it is calm and graceful on the surface... or something.
 

CrazyIvan

Member
Joined
Oct 10, 2011
Messages
124
SD Cards are cheap, where is the issue including a library with a couple of KB size inside a PND to make sure it works with future firmware versions?
It might break eventually. Try running some old binary games like UT2004 on modern linux - at least the sound won't work until you force it to use system SDL library instead of a bundled one.

Adding libraries to PND help in the opposite situation - when library in PND is newer than in system
 
Last edited by a moderator:

mcobit

Advanced Member
Joined
Jul 28, 2008
Messages
6,910
But you are able to do that. There is a maintainer for a pnd and he can fix stuff like libraties that become incompatible.


If he walks away somebody else can do it or even a user can just copy the right lib to the appdatagolder to fix it.


That's better than having to maintain it in a debian repo imho.
 

WizardStan

Mega GP Mania
Joined
May 24, 2008
Messages
16,733
Then you can just continue to use PNDs
You quite clearly missed the part where I was specifically responding to someone asking if we didn't need PNDs anymore. I can't "just continue to use PNDs" if PNDs (or a replacement) are no longer supported at all. I was explaining why, regardless of whatever else we might have, PNDs still solve a lot of problems that we will continue to have.
People are used to space on their HDD/SDD/etc running out
You're comparing hundred gigabyte hard drives to 16/32GB internal flash. If I were to treat a Pyra exactly like I do my desktop that flash'll be full in no time, even just keeping music and movies and such on a seaparate SD card.
So what is stopping them to deinstall something (other than convenience) ?
Phone install and desktop install are different. Android is very good about sandboxing applications. If we added a specification to the repo that deb packages cannot rely on other deb packages and have to work with what's on the firmware then it would be trivial to uninstall: this is basically how Android handles it.If an application requires a library though, that library gets installed but may not be uninstalled when you remove the aplication that used it. Over time these can add up. It isn't a "problem" per se, the size of these orphaned libraries tends to be small enough that you probably won't notice them but it's still a nuisance.
 

vinipsmaker

Member
Joined
Jun 22, 2013
Messages
209
Location
Brazil
Website
vinipsmaker.github.io
SD Cards are cheap, where is the issue including a library with a couple of KB size inside a PND to make sure it works with future firmware versions?
Does it mean that we can kill the idea of .deb dependencies? One requirement is down? I hope so, because if someone want to upload a PND to just force the system to download some .deb packages, someone can just offer you a .deb package.

If no one argue, I'd like to mark this requirement as decided with the solution "new pnd won't have dependencies".

You quite clearly missed the part where I was specifically responding to someone asking if we didn't need PNDs anymore. I can't "just continue to use PNDs" if PNDs (or a replacement) are no longer supported at all. I was explaining why, regardless of whatever else we might have, PNDs still solve a lot of problems that we will continue to have.
Don't worry, you exposed your point. And also, a compatibility layer will be available.

If an application requires a library though, that library gets installed but may not be uninstalled when you remove the aplication that used it. Over time these can add up. It isn't a "problem" per se, the size of these orphaned libraries tends to be small enough that you probably won't notice them but it's still a nuisance.
Package managers uninstall unused dependencies.
 

_wb_

Microbe
Staff member
Joined
Apr 5, 2012
Messages
5,387
Age
38
Location
Brussels, Belgium
No. PNDs without any dependencies would suck. It means you have to include A LOT of libraries (or statically link them), and A LOT of tools like LaTeX and ImageMagick and bash and whatever you need.

Having to include all of that is problematic for many reasons. Size is the least of them. The biggest problem is that having no dependencies forces devs to become packagers/maintainers of not only their own software, but also all of its dependencies. If I want to use a recent version libqt and pdflatex in my program, I need to include those in my PND. If someone else also uses those tools, he also needs to include them. Whenever there's a bugfix or security update or improved implementation of any library or tool, it takes A LOT of effort involving A LOT of people to make sure that it percolates to all software that relies on it. It means EVERYTHING has to be re-packaged all the time and re-downloaded by everyone.

So no, this is not a good option.

The Pandora approach of choosing a fixed base system and allowing PNDs to rely only on that base system is a slightly better option, and it makes sense if you have a very limited internal storage anyway. But it is not a very future-proof option: it essentially means that the base system has to be frozen or at least has very restricted options for updates.

I think it's much better to have a flexible base system and PNDs (or rather .pyra files) that have meta-data for their dependencies. From the user point of view there is barely any difference with the current PND system, especially because .pyra files will mostly be used for games, which tend to have few dependencies and those dependencies will probably be installed already in the default install anyway. The main difference is that instead of getting weird crashes, you get a pop-up box saying "You have to update your base system to run this PND, or else it will not work properly. Click OK to update".
 

comradekingu

Glowing ember
Joined
Apr 15, 2011
Messages
5,055
Website
portfolio.anotheragency.no
To make debs for the self hosted repo and in some cases upstream, and then make pnd's from that, in an automated way, deals with everyones issues, no?

I personally think spending time porting pnds over to something maintainable is the best way to assure compatibility with the pandora long term, and certainly for the future.
 
Last edited by a moderator:

vinipsmaker

Member
Joined
Jun 22, 2013
Messages
209
Location
Brazil
Website
vinipsmaker.github.io
No. PNDs without any dependencies would suck. It means you have to include A LOT of libraries (or statically link them), and A LOT of tools like LaTeX and ImageMagick and bash and whatever you need.
I thought that the system you propose was already served by .deb packages and that everyone agreed that we would only use PND for special-purpose software (like games and large-data anything).

Having to include all of that is problematic for many reasons. Size is the least of them.
I thought size was the reason to keep some software on the SD-card. You can't install .deb packages to the SD-card. You can keep the .deb packages on the SD-card. Any .deb dependencies will be installed on the internal NAND anyway. It's difficult to replace the internal NAND, but guess what... SD cards are cheap. So it's a vote to move things to SD-card, not the other way around.

The biggest problem is that having no dependencies forces devs to become packagers/maintainers of not only their own software, but also all of its dependencies. If I want to use a recent version libqt and pdflatex in my program, I need to include those in my PND. If someone else also uses those tools, he also needs to include them. Whenever there's a bugfix or security update or improved implementation of any library or tool, it takes A LOT of effort involving A LOT of people to make sure that it percolates to all software that relies on it. It means EVERYTHING has to be re-packaged all the time and re-downloaded by everyone.
makepnd will make your life easier. No big effort.

Also, I thought that everyone agreed that PND would be used for special software only (like games). You're forcing me to repeat myself.

So no, this is not a good option.
Seriously? I think I tackled your arguments on the comments above. But do you want to know what is not a good option? Your proposal to merge .deb and .pnd. They serve different purposes.

PND-game Foobar needs libabc. It include libabc on the SD-card. libabc requires updated version of libdef. An old version of libdef was already included in the system, because maybe Firefox or any app was already installed. But your system included an updated libdef, because your system is brilliant. Now I update libdef and Firefox stops working? Why? Because libdef doesn't maintain ABI compatibility and packages need to be recompiled. Congratulations, you fucked my system.

And yes, this issue is real. I saw proprietary or manually compiled packages stop working several times complaining about libpng or something else.

The Pandora approach of choosing a fixed base system and allowing PNDs to rely only on that base system is a slightly better option, and it makes sense if you have a very limited internal storage anyway. But it is not a very future-proof option: it essentially means that the base system has to be frozen or at least has very restricted options for updates.
Yes, I agree with you. We should keep the required set of dependencies to minimum and stable/serious APIs. SDL and SDL2 are great candidates. Also, any library that abstracts hardware should be considered, because it'll help keep the system future-proof.

A game that uses system SDL will be able to upgrade to our system in the future if we decide to replace the X system in favor of Wayland. That's why software abstracting some bits of the system is an interesting choice to be in the minimum set of requirements.

I think it's much better to have a flexible base system and PNDs (or rather .pyra files) that have meta-data for their dependencies.
We have both, PND and DEB serving different purposes. Let me repeat myself one more time:

PND-game Foobar needs libabc. It include libabc on the SD-card. libabc requires updated version of libdef. An old version of libdef was already included in the system, because maybe Firefox or any app was already installed. But your system included an updated libdef, because your system is brilliant. Now I update libdef and Firefox stops working? Why? Because libdef doesn't maintain ABI compatibility and packages need to be recompiled. Congratulations, you fucked my system.

And yes, this issue is real. I saw proprietary or manually compiled packages stop working several times complaining about libpng or something else.

From the user point of view there is barely any difference with the current PND system, especially because .pyra files will mostly be used for games, which tend to have few dependencies and those dependencies will probably be installed already in the default install anyway. The main difference is that instead of getting weird crashes, you get a pop-up box saying "You have to update your base system to run this PND, or else it will not work properly. Click OK to update".
What you proposed is not self-contained and need internet access. This is .deb, but some people don't want to waste internal NAND space.

There is a third way. What do you think to replicate apt-get to install software on SD-card?
 
Top