The software side

[Cloudef]

I don't like having to define a runtime now, knowing that we will be stuck with it forever. E.g. today it would have to include relatively old versions of GTK and Qt and SDL etc, because they are still used. At some point we may want to get rid of those, but it will be impossible because it will break stuff.

Yes, this is bad. But it's still the way for example Steam works.

There is also LSB (Linux standard base, thanks to Slaeshjag, I know about it)

If you are using SDL 1.2 and you can just specify that as a dependency, then even if in 5 years everyone has only SDL 2.4 and SDL 3.0 installed on their device (which could be the successor of the Pyra by then), things will still work because the explicit dependency will tell your device to download the legacy version of the library.

This is good idea I admit, however it brings the following problems.

1. The PND isn't plug'n'play (heh)

2. What happens if the old version of dependency conflicts package installed in user's system?

3. What happens if the person doesn't have anymore space in internal flash?

4. Is the dependency and it's depencies mirrored anymore?

5. Will the old dependency bring whole tree of other old dependencies?

I'm trying to envision a PND format that will still work nicely in a decade or even two. Not something that works fine now, is still OK when the Pyra is released, becomes "meh" a year later and a minor pain in the donkey one year after that.

Isn't this what we all are

 

[vinipsmaker]

I'm going to preface this post by saying that I am not a computer programmer, nor a linux user or even a pandora owner. [...]

Don't worry, I'm the one fighting for the user case you presented and I'm giving tech arguments for us. Just use flat PNDs.

But what does "totally self-contained" even mean?

It means it will contains the complete user-space and only will depend on the kernel and the architecture, like this: http://www.docker.io/

But I don't want this (well, I want, but let's leave my paranoic needs for myself and work on something that work for everyone and is easier to develop). The minimum set of dependencies and the fact that PNDs will mostly be used by games will force you to link to few external libraries and makepnd will make everything easier.

Want the PND with deps? Just use light PNDs.

I don't like having to define a runtime now, knowing that we will be stuck with it forever.

Pyra will have more NAND space. And the most important packages to have in the user space (Pandora is for games, mainly), SDL, don't take that much space.

We will analyze what are good choices. We won't run polls on this board.

 

[Ziz]

I don't like having to define a runtime now, knowing that we will be stuck with it forever. E.g. today it would have to include relatively old versions of GTK and Qt and SDL etc, because they are still used. At some point we may want to get rid of those, but it will be impossible because it will break stuff.

But this is the way most systems work today. Look at the modern consoles or Windows versions or even Linux, if we leave out rolling distros. Old games don't work with Windows 7 and new games don't work in Windows 95. Or if you want to use a modern feature in Windows XP (like .NET) you have to hope, that Microsoft gives you a package to install it.

If you are using SDL 1.2 and you can just specify that as a dependency, then even if in 5 years everyone has only SDL 2.4 and SDL 3.0 installed on their device (which could be the successor of the Pyra by then), things will still work because the explicit dependency will tell your device to download the legacy version of the library.

But this is a new device! Your old device is stuck with old versions, but your new device has the new version - maybe with the ability to install old packages, maybe not. Some things will just not work anymore, but other things will work, that didn't work before.

I'm trying to envision a PND format that will still work nicely in a decade or even two. Not something that works fine now, is still OK when the Pyra is released, becomes "meh" a year later and a minor pain in the donkey one year after that.

Hm, I think a total list of all needed libraries with version (maybe made with ldd or similar) in the PND would be quite useful. If the PND-start-script sees, that a lib or it's version is missing, it could then suggest to update/install this package + maybe making a symlink.Of course we have to assume, that no big API changes happen in minor-version-changes. If a PND maybe needs libpng.so.1.8 and we can install only libpng.so.1.9, which maybe have an API change, that crashes the PND, we have a problem. :\

 

[vinipsmaker]

Of course we have to assume, that no big API changes happen in minor-version-changes. If a PND maybe needs libpng.so.1.8 and we can install only libpng.so.1.9, which maybe have an API change, that crashes the PND, we have a problem. :\

It's even worse. Sometimes there isn't an API change, but the ABI changes ((de)inline a function, add a new member to a struct). It takes a lot of effort from the developer to make things right.

 

[Natsu]

IDK where to say this,but I think it should dual boot with android. Just for the sake of more accessibility.

 

[Granitehead]

.pnd-next with .deb dependencies:

-with Pyra being online: No problem, can be easily made completely transparent to the user.

-with Pyra being offline:

--impossible to just make a download manager download all dependencies of a .pnd-next. Would be too big for users with a system which isn't completely outdated.

--software manager on online computer. Software manager would mostly be a gui to make different already existing tools work together user-transparently. You use apt-offline to generate that information file on the pyra (with a gui which automatically places the file with an agreed upon filename in the root of all SD cards, gui could be called "set up SD card" or something like that)(generating that information could also happen automatically after each .deb update/upgrade/install and each SD-insert), put the SD card in the online computer. Software manager auto detects SD cards with that apt-offline info file. You browse for software with the software manager. You select what you want. When you click on "install", the .pnd-next files are downloaded, their dependencies (a list of those can be maybe separately pulled from the .pnd-next-repo to get them earlier) are fed to apt-offline which downloads them to the sd card (into a standard folder for .deb dependencies about which the Pyra-OS knows). Put SD card into Pyra. .pnd-next daemon recognizes there are uninstalled dependencies at the same time it detects the .pnd-next files and askes the user whether he wants to install those dependencies in background. If the user the tries to open a .pnd-next with unmet dependencies before they are finished installing the user gets a "please wait" message which isn't the same as when there really are unmet dependencies (which shoudn't happen that way anway).

Of course that's more work to implement than no dependencies .pnds, but it's future proof, less work for package maintainers and leads to smaller downloads.

Edit: I'd be willing to program that cross platform "software manager". Would need to look into cross platform GUI programming a little first, but I should be able to do it.

But someone else would first need to make a .pnd-next repo and libs to access it. That would be too big of a project for me. Probably.

 

[vinipsmaker]

[...] less work for package maintainers [...]

makepnd is less work for package maintainers and packages will support whatever makepnd supports.

 

[Granitehead]

[...] less work for package maintainers [...]

makepnd is less work for package maintainers and packages will support whatever makepnd supports.

This solution would be less work too even when used alone, but I think it could be used together with makepnd if makepnd is what I think it is. It's something to automatically build pnds from upstream source code according to a set of rules, is that a more or less right description?

 

[vinipsmaker]

This solution would be less work too even when used alone, but I think it could be used together with makepnd if makepnd is what I think it is.

What solution, PNDs with deps? How is that different from flat PNDs + light PNDs available from the repo for you? The only difference I see is that you want to force the other user to follow the pattern you like. Are you not happy enough having your use case covered?

It's something to automatically build pnds from upstream source code according to a set of rules, is that a more or less right description?

Yes, this is a good description.

 

[Cloudef]

It's something to automatically build pnds from upstream source code according to a set of rules, is that a more or less right description?

From today, it can build ipks too.. Though depends array items may vary between distros. I used it to build newer bash for dchrt to support associative arrays in makepnd, those will be used to map PND application localized titles and descriptions on recipe.

 

[Granitehead]

This solution would be less work too even when used alone, but I think it could be used together with makepnd if makepnd is what I think it is.

What solution, PNDs with deps? How is that different from flat PNDs + light PNDs available from the repo for you? The only difference I see is that you want to force the other user to follow the pattern you like. Are you not happy enough having your use case covered?

If I get right what you mean with "flat" and "light", it's actually the same, but I'm only talking about the "light" ones because those interest me more and, with suitable software support, could be better than "flat" in general.
So, I'm again going to define and then let you check whether he have the same definitions.

Light: No dependencies included, has .deb dependencies.

Flat: All dependencies included which could possibly not be available or available only in the wrong versions. This would make some .pnd-nexts huge, while it would work okay for others (most?). Could probably be automatically generated from the "flat" pnds?

 

[Ziz]

IDK where to say this,but I think it should dual boot with android. Just for the sake of more accessibility.

This has been discussed plenty times... ED doesn't have anything against Android. But someone has to do it! Notaz did the hacky Android-PND for the Pandora, but he already said, that he will not do this again for the pyra, it was just a proof of concept.So if you want Android: Do it yourself!

 

[vinipsmaker]

Light: No dependencies included, has .deb dependencies.

Flat: All dependencies included which could possibly not be available or available only in the wrong versions.

Yes, your definitions matches what I have in mind. Thanks for the contribution.

This would make some .pnd-nexts huge, while it would work okay for others (most?).

Yes, but each choice has its drawbacks and... just use whatever you want to use.

Could probably be automatically generated from the "flat" pnds?

In the perfect world everything will be generated from the makepnd's recipes. The advantages are bigger than support for both pnd types.

 

[Granitehead]

Could probably be automatically generated from the "flat" pnds?

In the perfect world everything will be generated from the makepnd's recipes. The advantages are bigger than support for both pnd types.

For this you would need to write two recipes? But maybe knowing the .deb dependencies there could be a way to auto generate the flat recipe (which the developer could then adjust if it didn't work perfectly) from the light one.
But now your last sentence makes me unsure regarding your position again, you are in favour of both flat/light together (which after all could work nicely, with flat being a boolean setting in pxml/whatever which in turn makes it forbidden to use dependencies) or just flat?

What I would like, to make it clear, is flat and light both. Then the developer/maintainer can choose what to support (ideally both, of course, but wouldn't be forced).

 

[vinipsmaker]

For this you would need to write two recipes? But maybe knowing the .deb dependencies there could be a way to auto generate the flat recipe (which the developer could then adjust if it didn't work perfectly) from the light one.

One recipe. You write the recipe explicitly stating the deps. The makepnd generates both PNDs.

But now your last sentence makes me unsure regarding your position again, you are in favour of both flat/light together (which after all could work nicely, with flat being a boolean setting in pxml/whatever which in turn makes it forbidden to use dependencies) or just flat?

The repo should have both PNDs for every app, then the user choose what he wants. It can work more-or-less the way you describe.

 

[Granitehead]

For this you would need to write two recipes? But maybe knowing the .deb dependencies there could be a way to auto generate the flat recipe (which the developer could then adjust if it didn't work perfectly) from the light one.

One recipe. You write the recipe explicitly stating the deps. The makepnd generates both PNDs.

Okay, that would be ideal.

 

[WizardStan]

How about this:  The PND includes .deb files for it's dependencies.   If they are already installed it uses the installed version.  If they aren't installed it prompts to install them.

This doesn't actually solve the proposed root problem, that of needing multiple copies of the same library in different PNDs. The only difference is that now they're multiple copies of uninstalled DEB packages.

What I don't quite get is why you seem to see these issues as insurmountable.   I've no doubt there's a good solution for most of the issues raised it's just a matter of finding it.

What do you think about a tool to create delta-packages? I can develop one

Binary diffs were experimented with about a year ago I think and found to work very very well but no one took the initiative to propose changes to the repo's JSON specification and how the repo should actually handle the storage of these delta packages.If Milkshake's repo and PNDManager could communicate just these delta packages you wouldn't have to redownload a full 5GB package (or even the multi-hundred MB packages of Wesnoth and whatnot) just because one tiny library changed or the executable was recompiled or whatever.

 

[Granitehead]

Binary diffs were experimented with about a year ago I think and found to work very very well but no one took the initiative to propose changes to the repo's JSON specification and how the repo should actually handle the storage of these delta packages.

If Milkshake's repo and PNDManager could communicate just these delta packages you wouldn't have to redownload a full 5GB package (or even the multi-hundred MB packages of Wesnoth and whatnot) just because one tiny library changed or the executable was recompiled or whatever.

I guess they could be a good tool anyway, if we go for flat and light .pnd-nexts. Both flat and light .pnd-nexts were the consensus of the discussion, but of course this is worthless unless someone actually implements it.

 

[sswam]

Let's stick with what we have now, simplify it, and improve it a little.

- put everything generally useful in the base system, based on Debian or whatever

- PNDs for large common stuff such as Java, if it's not in the base system

- PNDs can depend on other PNDs, or a new version of the base system

- every PND has a maintainer, who will rebuild or fix the fucking thing when it stops working

We don't need too much magic to pull in old libs and ensure that every PND continues working as it is forever.

We need source code, some effort at backward compatibility, and maintainers.

What I really want to see:

- source code for all open source-ish PNDs, and mirrored in a common location, preferably in git

- run tools / programs within a PND, without having to mount it first (i.e. a wrapper that auto-mounts it and execs your program passing on the arguments)

- simplify PND system very much

- update PNDs transmitting only the differences

 

[sebt3]

- source code for all open source-ish PNDs, and mirrored in a common location, preferably in git

In a way, current Cloudef's work is also targetting this.

- run tools / programs within a PND, without having to mount it first (i.e. a wrapper that auto-mounts it and execs your program passing on the arguments)

Could be done right now. Reading this, I think we could just add a :
/sbin/modprobe binfmt_misc
mount -t binfmt_misc none /proc/sys/fs/binfmt_misc
echo 'nd:E:nd::/usr/bin/pnd_run:' > /proc/sys/fs/binfmt_misc/registeron a current pandora at boot and have it right now (some path might be wrong, i dont have my pandy near hands).

- simplify PND system very much

What cluter would you remove ?

- update PNDs transmitting only the differences

That's a feature we also could have working currently.If ED setup and rsyncd on the repo server and B-Zar add support for rsync. It could be as simple as :

rsync -avz repo.openpandora.org:<path to file from the repo master list>.pnd /path/to/be/updated/file.pnd