Policy for PND maintainers re SuperZaxxon / .next handover

[freamon]

So, in the .next alpha image thread, ajs124 said:

Hm, I wonder what that library does? I only maintain one application in the repo (SpaceFM, it fails to start because of libudev), but what is the "policy" for application maintainers? What if fixing a pnd for .next breaks it for SuperZaxxon?

And I said:

My plan for my PNDs that need updating for .next was for there to a temporary period where the PND contains enough stuff for both SuperZaxxon and .next, and the startup script chooses between them.

So, internally, it would have:

libs_for_next/

libstuff.so.1

libs_for_superzaxxon/

libthings.so.1

and the startup script would do:

if [ OS == .next ]

then

LD_LIBRARY_PATH=./libs_for_next

else

LD_LIBRARY_PATH=./libs_for_superzaxxon

fi

That said, once .next is officially released, we don't have to support superzaxxon any more - anybody still using it for whatever reason can always download an archived copy of a PND.

And Askarus was all like:

That said, once .next is officially released, we don't have to support superzaxxon any more - anybody still using it for whatever reason can always download an archived copy of a PND.

Agree.

But then hmc came back with

I only partially agree.

There may be reasons for people to stay on SuperZaxxon for some time, even after .next is released.

Or will you only release .next, once EVERYTHING works flawlessly in it, that already worked flawlessly on SuperZaxxon? Then it may be okay to drop SuperZaxxon support in newer PNDs.

However, no matter how the answer to above question is, during the time when both firmware variants are around, these problems will occur:

1. It will create big efforts for PND maintainers to support both firmware variants in the way you describe, freamon

2. Many PNDs won't be upgraded during that time, or even not at all anymore, so they will never be comatible with .next, even if they could if only one lib was removed / replaced.

So I suggest rather finding another solution to make old PNDs also run in .next, e.g. by providing some "compatibility mode" (reminds you of Windows? Yes, sort of ;-).

I am not enough of an expert to guess, if something like that could work with reasonable efforts, but what about solving most PND lib problems with symlinks, and those that are not solvable with symlinks, solve with providing SZ lib version AND .next lib version in parallel in some way? If not both lib versions can coexist in one system, then some switching mechanism, that symlinks either version in place could be used maybe?

Anyways, so I thought I should create a new thread for it, 'cos it's the kind of thing we can have a big fun argument about.

 

[Linux-SWAT]

Don't forget other OSes as well.

For the Slackware case, approx. 75% of .pnds are working, without or with a simple symlink.

And most of the non-working ones, because of things like LD_LIBRARY_PATH=./ inside the pnd.

For the .pnd makers who want to identify Slack, here's how:


$ cat /etc/slackware-version
Slackware 14.0

So a simple


if [ -f /etc/slackware-version ]
Will do it.

 

[freamon]

There may be reasons for people to stay on SuperZaxxon for some time, even after .next is released.

Or will you only release .next, once EVERYTHING works flawlessly in it, that already worked flawlessly on SuperZaxxon? Then it may be okay to drop SuperZaxxon support in newer PNDs.

Firstly, I would say that there's no need to support PNDs that nobody is actually using. There's plenty of stuff on the repo that folks ported because they could, but is actually pretty rubbish (I include my own PNDs in that criticism, of course). So we can lower our sights in terms of supporting "EVERYTHING"

Secondly, if somebody is still on SuperZaxxon, they can mostly use an older version of a PND. Broadly speaking, the kind of PNDs that get updated a lot, where someone will really *need* the new features it provides, are also the ones that are dev's personal projects (e.g. DraStic). These are typically fairly OS-agnostic anyways.

However, no matter how the answer to above question is, during the time when both firmware variants are around, these problems will occur:

1. It will create big efforts for PND maintainers to support both firmware variants in the way you describe, freamon

2. Many PNDs won't be upgraded during that time, or even not at all anymore, so they will never be comatible with .next, even if they could if only one lib was removed / replaced.

1. I was only really saying it what I was doing, not what others should do. It's not even a general policy for me, thinking about it, because I have PNDs it would be impossible to do for.

2. If a PND doesn't work with .next, it's because it's not being maintained. It may seem brutal, but if a new maintainer can't be found for it, then really it should be deleted from the repo. (a "new maintainer" would often just have to repack the PND, rather than re-compile it, so if there's a PND that someone really likes, it doesn't work with .next and the original maintainer has disappeared, when then: congrats - you're the new maintainer for it).

So I suggest rather finding another solution to make old PNDs also run in .next, e.g. by providing some "compatibility mode" (reminds you of Windows? Yes, sort of ;-).

I am not enough of an expert to guess, if something like that could work with reasonable efforts, but what about solving most PND lib problems with symlinks, and those that are not solvable with symlinks, solve with providing SZ lib version AND .next lib version in parallel in some way? If not both lib versions can coexist in one system, then some switching mechanism, that symlinks either version in place could be used maybe?

Having 2 lib versions comes up against the problem of limited NAND space.

Also, I think aTc's idea behind .next is for a "fresh start", where everything is compiled using OpenEmbedded, and we're compatible with Angstrom. So, for a given library, we're often using a bitbake recipe provided from upstream. If we want to include the old library aswell, we'd have to add another bitbake recipe for it. It seems, to me, that we'd end up with a cluttered and hacky OE setup, that produces a cluttered and hacky OS.

Regarding symlinks, I'm starting to think that adding new ones wouldn't work.

In SuperZaxxon, you have something like:

/usr/lib/libSTUFF.so.1 => /usr/lib/libSTUFF.so.1.1.0In .next, they'll be:

/usr/lib/libSTUFF.so.1 => /usr/lib/libSTUFF.so.1.2.0

Binaries in a PND will be linking to libSTUFF.so.1. So they won't need any help in terms of finding the newer library, and if they're breaking when they try to use it, it's because of some fundamental change in the library that means the PND wants re-doing anyway.


----

To summarise: I think the goal of .next is to produce a clean, modern, upstream-compatible OS, and for PNDs to be made to work for it, rather than the other way around.

 

[sebt3]

Beside _wb_ points, so far the reason for a PND to not work on .next that work on superzaxon are :

- Missing symlinks for upgraded libraries (think libjpeg.so.6 -> libjpeg.so.8) Easy to fix, just need a good list (could be generated...) of missing links

- Dependency to an upgraded yet incompatible lib (probably not much, xfce come to my mind) ; you'ld want an upgraded version anyway.

- incompatible lib bundled, geez. Seriously bundling libdl in a PND is just plain dumb. if you dont trust the linker, you'ld better build a static bin anyway. So you should expect an updaded PND with these removed...

- maybe an c++ abi break ? not even sure

- and 3d, but that should be fixed (I should try leastest improvement in this field...)

and that's it so far.

 

[ingoreis]

I think that the Main Idea all PNDs more Combatible to other Pandora OS like Slackware is great,but can we not simply make a Tool who make PNDs more Combatible with Libs?


Or like a Angstrom Wrapper for Slackware and Bodhi with a Chroot is possible.


When then Anyone wants to run a PND that is not normaly working,can this Person use the Wrapper like Ginge

 

[ajs124]

Beside _wb_ points, so far the reason for a PND to not work on .next that work on superzaxon are :


- Missing symlinks for upgraded libraries (think libjpeg.so.6 -> libjpeg.so.8) Easy to fix, just need a good list (could be generated...) of missing links


- Dependency to an upgraded yet incompatible lib (probably not much, xfce come to my mind) ; you'ld want an upgraded version anyway.


- incompatible lib bundled, geez. Seriously bundling lidl in a PND is just plain dumb. if you dont trust the linker, you'ld better build a static bin anyway. So you should expect an updaded PND with these removed...


- maybe an c++ abi break ? not even sure


- and 3d, but that should be fixed (I should try leastest improvement in this field...)


and that's it so far.

Also "lowlevel" stuff, that tries to call udev or related things. I think SuperZaxxon is pre systemd udev merge and .next is after. This most likely doesn't concern as many packages (Android as a pnd, SpaceFM, …).

I think that the Main Idea all PNDs more Combatible to other Pandora OS like Slackware is great,but can we not simply make a Tool who make PNDs more Combatible with Libs?


Or like a Angstrom Wrapper for Slackware and Bodhi with a Chroot is possible.


When then Anyone wants to run a PND that is not normaly working,can this Person use the Wrapper like Ginge

The problem with this  is, that all of it has to fit on the NAND and your solution (chroot) sound like two separate operating systems.

 

[skeezix]

New OS versions should include most or all versions of libs used in prior versions, within reason..

- if its ABI compatible, symlink (sucks to know which are ABI compat or not, but such is maintainer life)

- if ABI incompatible, see about including both actual versions.. the 'last of one family', and the current one

- if a pnd has incompatible libs in the pnd, theres nothing you can really do about it .. but if we keep the first couple rules then most libs in pnds should still work

I doubt g++ ABI breakage is an issue for us; we didn't use that old a g++ for SZ

At the same time.. the next big firmware is an opportunity to break things that need breaking

jeff

 

[ingoreis]

I think that the Main Idea all PNDs more Combatible to other Pandora OS like Slackware is great,but can we not simply make a Tool who make PNDs more Combatible with Libs?


Or like a Angstrom Wrapper for Slackware and Bodhi with a Chroot is possible.


When then Anyone wants to run a PND that is not normaly working,can this Person use the Wrapper like Ginge

The problem with this  is, that all of it has to fit on the NAND and your solution (chroot) sound like two separate operating systems.

Can anyone that not simply put this into an PND?

Like a Wrapper PND with the Normal OS Inside for Chroot using or using the Libs from that.

Is just a Idea,because Ginge is in a PND too

We have bigger PNDs than that on the Repo

 

[hmc]

2. If a PND doesn't work with .next, it's because it's not being maintained. It may seem brutal, but if a new maintainer can't be found for it, then really it should be deleted from the repo. (a "new maintainer" would often just have to repack the PND, rather than re-compile it, so if there's a PND that someone really likes, it doesn't work with .next and the original maintainer has disappeared, when then: congrats - you're the new maintainer for it).

Okay, I get your point freamon.

Maybe new PND versoins don't need to be compatible with older firmware versions.

I like the idea of using archived PND versions for older firmware and the need to upgrade firmware to being able to use the latest PND version.

Then maybe an enhancement of the repo could be the solution: Categorize PNDs AND all versions of a PND as being compatible with firmware XY version Z. 

So when I want to use PND X with .next, it downloads the latest version, and if I want to use PND X with SuperZaxxon, it downloads the last  SZ-compatible version of the PND.

 

[freamon]

Beside _wb_ points, so far the reason for a PND to not work on .next that work on superzaxon are :

- Missing symlinks for upgraded libraries (think libjpeg.so.6 -> libjpeg.so.8) Easy to fix, just need a good list (could be generated...) of missing links

- maybe an c++ abi break ? not even sure

- and 3d, but that should be fixed (I should try leastest improvement in this field...)

- Isn't something like libjpeg.so.6 to libjpeg.so.8 incompatible by definition? I thought it was standard practise that if you're releasing a library, you change the 'minor' version number for internal improvements that don't affect the external API, but a change in the major 'version' signifies that the external API has changed, and existing methods of calling functions probably won't work. My concern is that if you don't have a symlink, then a PND will fail with the message that the library it needs is missing, but if you do, it may fail with an error that is hard to track down.

- Re: c++ ABI, there doesn't seem to be any issue with running C++ programs that were made for SuperZaxxon on .next. The library is a bit newer though, so it doesn't work the other way around, of course.

- The 3D problems weren't with the libraries themselves - it seems like both SZ and .next are using the same version (1.4.14.2616)

 

[_wb_]

To summarize: I think the goal of .next is to produce a clean, modern, upstream-compatible OS, and for PNDs to be made to work for it, rather than the other way around.

I think that is exactly the way to go. I would handle broken PNDs on an "on-demand" basis: if a PND breaks, it could be checked why it breaks and fixed. If it's something that can be fixed easily with a minor change in .next without having to change the PND (e.g. by adding a symlink or some kind of simple wrapper script), it would be nice to do it that way (especially if the same fix works for several PNDs), but in general, keeping all old PNDs out there working in .next should not be the main goal. It's the task of the PND maintainer to keep stuff up to date - we don't have space to keep an unlimited amount of legacy stuff for backwards compatibility around. If the current SZ contains some library that is outdated or judged "not important enough", by all means, throw it out, even if that breaks some PNDs. It is easy enough to repackage the PND to include some missing library.

Ideally, I would hope that .next could also be the basis for a Pandora successor (the only difference being things like the kernel and some of its modules), such that PNDs that work on both the Pandora and its successor are possible. Of course in the successor, more optional packages could be in the default install because there would be less space restrictions, but those would be mostly programs and not libraries that PNDs would rely on.

 

[milkshake]

There is already a pxml tag for required firmware why can't we just use that?

 

[hmc]

Oh great, so this can be used to match the PND version to the used firmware.

However, which field is this? I can't find it on http://pandorawiki.org/PXML

 

Also, is there some standard for versioning numbers for all the firmware versions?

E.g. how do we differentiate in that field between "Zaxxon", "SuperZaxxon" and ".next" with their own version numbers?

I suspect most PNDs will not have that field filled by now, so only packages updated in the future can be included in the PND versoin selection mechanism based on requesting firmware version.

 

If we have such a field, if I request a specific PND from the repo and in my request, I set the parameter "firmware" to version X, would then the repo decide, based on X and on the PND versions' "required firmware" fields, which version of the PND to deliver? 

 

Ideally, a warning should then be displayed, in case that not the latest version of the PND can be delivered...

 

Just a few ideas. Hope they make sense.

 

[sebt3]

- Isn't something like libjpeg.so.6 to libjpeg.so.8 incompatible by definition? I thought it was standard practise that if you're releasing a library, you change the 'minor' version number for internal improvements that don't affect the external API, but a change in the major 'version' signifies that the external API has changed, and existing methods of calling functions probably won't work. My concern is that if you don't have a symlink, then a PND will fail with the message that the library it needs is missing, but if you do, it may fail with an error that is hard to track down.

Usually, if a livrary version increment by a minor version number then, binary built using previous library version should still work. If the lib is breaking compatibility, you'ld expect a major number change. Most project work this way (like imlib2 isnt compatible with imlib, but imlib have always been backward compatible). In the example case, just do the symlink and test, you'll see it just "works"

- The 3D problems weren't with the libraries themselves - it seems like both SZ and .next are using the same version (1.4.14.2616)

I know. It didnt worked last time I checked. But it sound fixed thanks to your (and aTc) works. I realy need to tests my PNDs there now.

Oh great, so this can be used to match the PND version to the used firmware.

However, which field is this? I can't find it on http://pandorawiki.org/PXML

This one : http://pandorawiki.org/PXML#The_.3Cosversion.3E_elementEvryone so far used a 1.0.0 version in here (that's what genpxml put there and all the template pxml file i've seen use it too. Milkshake could confirm, but I even think the repo reject any PND without it). Using a 2.0.0 value here sound like a plan imho.

 

[thatgui]

Oh great, so this can be used to match the PND version to the used firmware.

However, which field is this? I can't find it on http://pandorawiki.org/PXML

The osversion element specifies the minimal OS version that supports the PND file. The PND file will not be loaded by an OS that has an older version than the one specified here. If unsure, just use a '0' in the respective field.

Edit: sorry, didn't update the thread

 

[milkshake]

I don't think the repo even checks this at the moment since it wasn't really useful before but it can be made to use it and display it on the app details page. On holiday at the moment though.

 

[freamon]

- Isn't something like libjpeg.so.6 to libjpeg.so.8 incompatible by definition? I thought it was standard practise that if you're releasing a library, you change the 'minor' version number for internal improvements that don't affect the external API, but a change in the major 'version' signifies that the external API has changed, and existing methods of calling functions probably won't work. My concern is that if you don't have a symlink, then a PND will fail with the message that the library it needs is missing, but if you do, it may fail with an error that is hard to track down.

Usually, if a livrary version increment by a minor version number then, binary built using previous library version should still work. If the lib is breaking compatibility, you'ld expect a major number change. Most project work this way (like imlib2 isnt compatible with imlib, but imlib have always been backward compatible). In the example case, just do the symlink and test, you'll see it just "works"

Ah, okay, thanks. I was misunderstanding which number constituted a 'major' version change - I was thinking that something like libjpeg.so.6 had a major version number of '6'.

 

[Rockthesmurf]

For what it is worth, I had users complaining that one of my PNDs failed to run on their device (on the default OS), with an error like:
 


./Application: /lib/libm.so.6: version 'GLIBC_2.15' not found (required by ./Application)
 
I resolved this by packaging up libm.so.6 from my tool chain into the PND libs directory; and I have the PND libs directory set up as an rpath at link time. This error started occurring I updated to a newer toolchain. I don't know how clever it is to have various libraries like this packaged up inside the PND (I also have an optimized OpenAL lib in the same directory), however I didn't see any forum responses as to how to fix the libm.so.6 issue without packing the library (except to use an older toolchain which was not an acceptable answer to me). After I packed up these libraries I received no more problems from users.

In my head, packaging up exact version dependencies like this inside the PND made sense; my application expects certain versions of libraries, so if I package them up inside the PND I know they will be available, and I won't be at the whim of the user and their OS to decide what lib ends up being used (if my application launches with a different lib it that might not mean it is okay, it might crash 10 minutes in when a specific library function gets called at the end of a level for example). I appreciate you end up with duplicated libraries inside a PND, but within reason I don't care about this, the price of storage and download speeds for the majority of users means an extra few MBs makes no odds.

I am posting this here as I am curious as to how this could negatively affect .next (I haven't tried any of my PNDs on .next).

 

[sebt3]

imo, there is 3 safes solutions :

1) fully self contain pnd (as in: if the run script might call cp, include cp too, etc)

2) Staticly linked binaries

3) Use system libraries and hope for the best.

All my pnds are in the 3rd category, and as I've finished testing all mines, results are :

- simply works in .next

- a few missing system libs (like libpng.so.3 and a few others)

- 3d not yet fully ready

The vast majority is in the 1st case by now (audacious, cavestory, the zelda's games etc... just works)

Personnally I dont think the 1st solution is a good one. For close source, I would recommand static : it will work for years

 

[comradekingu]

Number 3.