Pnds And Recipes

[Tramboi]

Hi guys, and happy new year!
may pandoras reach you if you ain't got yours yet.

I have installed my OS on a big flash card and I can't help to notice that few Pandora apps play nice with openembedded.
Why? Because of the emphasis on PND format.
Which is ok for people who have their OS in NAND but mounting temporary unionFS is a kludge when you just want to install an ipkg to your system.
I think this is a political problem : developers should be told by the team to make a bitbake recipe and then eventually a PND for their applications.
It would have many benefits:
* anybody could rebuild the apps
* it ensures the modifided free software source IS available somewhere
* it ensures applications don't strongly depend on PND mounting structure
* versioning and dependencies

What do you think about it?

Cheers,
Bertrand

 

[typs lik dis]

To me, Pandora is synonymous with PND. Without it it's just not the same system. Just my two cents.

 

[Pleng]

This argument has come around so many times before, but...

The Pandora is designed so that you can put an SD full of applications in it and be able to run them. ie All libraries that aren't already in the base OS are included in the PND itself.

Bitbakes and recepies, as far as I'm aware, are designed to be able to fuse applications into the OS, giving them the opportunity to add files all over the place.

The official stance is PNDs and, for the most part, it works very well. It also makes it a lot easier for developers to debug their apps, as reproducing a bugs should be pretty simple - all you need to know is the firmware/hotfix version being used.

Having an official voice saying 'yea bitbake it for those that wanna do it this way and PND it for those who wanna do it that way' give off a fragmented view of how things should be done and we'd more than likely end up with half the apps only available in bitbake, and half only available in PND format, as developers choose the way THEY preferred.

The Pandora is a very versatile system, of course, and some people prefer to run the entire OS from an SD card using OPKG. That's fine, but it's not officially supported. If you intend to do things that way, you're pretty much on your own, and have to rely on the help of other users who have gone that way too. Stukie's Debian extend should be useful.

 

[WizardStan]

PNDs work for both NAND and SD based systems (and even hard drive systems, if you want to build it up for your computer) so there's no actual benefit other than your perceived benefit of avoiding a "kludge".
What exactly are you trying to get them to do with openembedded that you think they don't "play nice"? They should be completely separate entities.
If you want to cut out the PNDs, having built your own OS for your SD cards, you can just opkg install anything you want from the Angstrom repository.
As far as depending on the "PND mounting structure", what mounting structure? What dependency? PNDs go into a directory on the SD card, but they don't have to go there, you can configure the PND system to look elsewhere. There is no dependency, just update the config as needed and place them wherever you want. They get mounted to /mnt/utmp, but there again, the PNDs don't care, the mount point can go anywhere you want and it'll still work.
I may be misunderstanding your reasoning completely, but it seems to me that you're asking OPT and more importantly the community developers to take on extra work for the benefit of a very very small group of people.

 

[Tempel]

PNDs were created for a specific purpose, and they're used on Pandoras because they meet its needs in a way that a traditional package manager does not. Therefore, where appropriate (ie most any end user appliction), many devs around here will make a PND for their application long before they'll consider making a bitbake recipe.

I agree though, it's unfortunate that devs don't create both, but until it's easy to make both, they won't do it; they'll only make the one that their users most want. But I had a thought: bitbake is designed to be able to output multiple different package formats, right? So, you might be able to make bitbake capable of creating PNDs; if it can do that, then devs could create a recipe for their application and immediately be able to create packages in all formats that the Pandora can use.

Unfortunately, I know almost nothing about bitbake and therefore couldn't do this myself. But I did do something similar with distPND, allowing standard Python distribution mechanisms to create PNDs (and if you're wondering, those distribution mechanisms can be used by bitbake, so recipes are easy to create). With this, it's easy for Python devs to create many different package types in a standard way. If you can do the same for bitbake, everyone will benefit.

 

[Tramboi]

PNDs work for both NAND and SD based systems (and even hard drive systems, if you want to build it up for your computer) so there's no actual benefit other than your perceived benefit of avoiding a "kludge".
What exactly are you trying to get them to do with openembedded that you think they don't "play nice"? They should be completely separate entities.

Yet the apps packaged in PNDs can access system-wide .so.
BTW: No "actual benefit", your "perceived benefit"... please try not to be judgmental for the sake of the discussion.

If you want to cut out the PNDs, having built your own OS for your SD cards, you can just opkg install anything you want from the Angstrom repository.

Yet a vast majority of the ported apps available as PNDs are not in the opkg repository, so there is no real alternative.

As far as depending on the "PND mounting structure", what mounting structure? What dependency? PNDs go into a directory on the SD card, but they don't have to go there, you can configure the PND system to look elsewhere. There is no dependency, just update the config as needed and place them wherever you want. They get mounted to /mnt/utmp, but there again, the PNDs don't care, the mount point can go anywhere you want and it'll still work.

If the source code ever refers to /mnt/utmp, you've got a dependency and the binary in the pnd doesn't work in the more general context of a full install.

I may be misunderstanding your reasoning completely, but it seems to me that you're asking OPT and more importantly the community developers to take on extra work for the benefit of a very very small group of people.

I think this would be beneficial for many people, keeping an unified database of applications and how to build them.
My point is that Pandora leverages Angstrom, so it would be good practice sticking with the host os style. And first-class packages...
When I use Debian, I use aptitude. When I use Fedora, I use yum
I understand the necessity for PNDs, but the underlying layer should not be swept under the rug.
If there ever was a statement of the team to push integrating apps as recipes on the .git repository, I'd be glad to contribute recipes for the great works of many contributors.

 

[WizardStan]

BTW: No "actual benefit", your "perceived benefit"... please try not to be judgmental for the sake of the discussion.

I meant what I said exactly as I said it: I believe what you want has benefit only in your mind, possibly because you believe something about either PNDs or bitbake recipes that isn't true. Keep in mind that when talking about your opinion, it is impossible not to be judgmental, especially if your opinion is based on flawed information. If you say you believe something, there is simply no way to impartially tell you your opinion is wrong. And yes, opinions can be wrong.

Yet a vast majority of the ported apps available as PNDs are not in the opkg repository, so there is no real alternative.

Why do you need an alternative? The PND system is fairly portable and robust. If you really can't or won't use the PNDs, you can simply extract the files it contains and run them from wherever you want.

If the source code ever refers to /mnt/utmp, you've got a dependency and the binary in the pnd doesn't work in the more general context of a full install.

The same is true of any program though. If the binary refers to a specific location and you want to put it somewhere else then you are going to have clashes. For the Pandora, best practice is to either use $HOME or . as the working directory: the PND system will work out the rest, allowing you to put anything wherever you want.

I think this would be beneficial for many people, keeping an unified database of applications and how to build them.
My point is that Pandora leverages Angstrom, so it would be good practice sticking with the host os style. And first-class packages...
When I use Debian, I use aptitude. When I use Fedora, I use yum
I understand the necessity for PNDs, but the underlying layer should not be swept under the rug.
If there ever was a statement of the team to push integrating apps as recipes on the .git repository, I'd be glad to contribute recipes for the great works of many contributors.

No one has swept anything anywhere. ipkg is there if you want it, and there are plans to leverage it for system updates, as well as an ipkg-like system which is smart enough to handle removable media.
I'm still not certain what that has to do with bitbake though. Bitbake isn't Angstrom, it's just that there's a recipe in openembedded for conveniently building an Angstrom distro. Not everything in the Angstrom repository necessarily has a recipe. And you've not even considered what to do about the programs that don't even have source.


Perhaps it is best to start from the beginning? What exactly do you think is the problem with PNDs that will be solved?

* anybody could rebuild the apps

A nice theory, but given how many people still find setting up the openembedded cross compiler daunting, I seriously doubt this is true. The only people that will be rebuilding apps are those that have the ability to setup their cross compiler, and honestly, if you're clever enough to do that, you should be capable of using it in conjunction with make, cmake, or whatever the developer has chosen.

* it ensures the modifided free software source IS available somewhere

Again, a nice theory, but it doesn't. Just because OPT says "make source available and create a recipe" doesn't mean it will get done. If a developer is feeling a little lazy at the moment and doesn't feel like posting the source just yet, they may just release the binary and get to the source when someone asks about it; if you try to force developers into releasing their source constantly, AND that they have to maintain a recipe to go with it, there is no argument you can make that will convince me the amount of releases we have wouldn't have been reduced.

* it ensures applications don't strongly depend on PND mounting structure

Again, as I said, applications shouldn't be depending on the PND mounting structure. If they've been written correctly, they'll use $HOME or . (usually $HOME). The same is true of any application as well: if it is designed to depend on a specific directory then you will be stuck with that directory.

* versioning and dependencies

How would it help with versioning? You may be in a niche of users that git pulls and builds new recipes every day, but I suspect most users don't have the time or interest in doing so: they want to download their game and play. If a new version comes out, they download that, overwrite the old one, and keep playing easy peasy, and once the automatic update system is finished it'll be even easier.
Dependencies are already handled by the PND system, at least.

The Pandora was designed so that you could do anything you wanted to with it. If installing your own OS to an SD card is what you want to do, so be it. However, as Pleng pointed out, the Pandora Team cannot support every possible thing you could ever want to do with it, so they have to focus their efforts on the group that will get the most benefit. They've made their choices to benefit end users most, and application developers second. Your suggestion is beneficial to people such as yourself, but is actually a hindrance to both end users and application developers: if they followed your suggestion, they would not be helping the largest group they possibly could.

 

[skeezix]

More to point ..

Many of the pandora ported apps depend on Pandora features, such as the keyboard or the nubs or the d-pads or the .. etc. Now, if coded using SDL or being very port-minded when coding stuff coudl work on all Linux machines pretty well, for the most part things are pandora-ified; its not the pnd-ing that makes them unfriendflt to OE, its the pandoraification.

ie: 'pnd' files work on nonPandora systems; you can port libpnd to a lot of OSes and have it work just find. You can get the pnd-contained binaries and run them fine --- it just implies that writes are to './' (current workdir) instead of to 'ramdom places' or homedir, so shoudl work fine elsewhere; it also implies lib and icon and so on packaged with the pnd, but if you just unpack or mount a pnd and run it on any other Angstrom (or other ARM based linux machine) you've got a fair shot of them running -- but the problem again is that they usually will want pandora-ified controls or screen resolutions or etc, and so wont' play nice.

Nothing says the app cannot be written to be workable everywhere, but they usually are done like that.

ie: You can use SDL, and be mindful of resolutionl; you can use X11, or even /dev/fb .. same on every linux; you can use the /dev/events and be careful to scan through them for the ones that look like joysticks, and monitor all the joysticks, and it'll work on every linux around. But most people are just reading the known-pandora stuff.

So the trick is .. it probably does work pretty well, or it was a quick port or no interested in working everywhere port; its not the pnd system which has anything to do with that.

The idea of pushign devs to bitbake it etc so its handier for OE, and then pnd'ing it, is nice.. but its a lot more work for people who just want to do a port (be mindfull of those Windows and OSX users who would find making recipies and so on a right PITA.) Right now you can just use Codesourcery (taking less than 10 minutes to set up, no joke, I've documented it), or do a build of gcc for ARM, and you're good to go on all major OSes.

*shrug*

The whole 'pnbd system' and 'pnd guidelines' and so on are _not perfect_ by any means, but they work well, and are enjoyed by many; I'd like to do a great many changes to the system over time (without breaking backward compatibility of course), but its best to let them machines get out and be hastle free before we go torturing our poor users right?

I woudl agree though -- if someone is doing a port, it would be nice to be mindful of non-pandoras and play nice, but thats really up to the porter; its not like PSP ports or dingux ports or gp2x ports or n900 ports etc etc are generally made portable to anything else; talking OE, talk n900 or Palm WebOS or various other Angstrom distros, and many of them have ports that don't play nice; its not us.. we just made a handy package format for SD based user

(And again, no one has to use it, and its not set in stone, and where it all goes is open to discussion. Heck, someone else can take the reins, Crom knows my hands are full

jeff

edit: I dont' think he's criticizing anything (and if he does, no biggy; thats what open forums are for , just doesn't want to ignore other OE platforms. I think its probabyl too much to ask of most porters (Remember, we have only a few of them, our community and especialyl those with actual pandoras, is small), but if we coudl do some subtle 'policies' to help ipk's or OE, thats cool. But with our goal that pnd's be like cartridges .. no install/uninstall and no data-loss with OS reflash and so on and on and on, its a different goal. We want to be easy on devs and easy on users, since easy to everyone makes for greater and faster adoption. Some standard is needed, so we wanted an easy one to go with day zero .. we can change later, but we needed things to fall into place quick, else we'd have fragmentation and death right off. I think we've done okay, but I agree.. maybe its a shame to not feed back upstream for pnd-games .. but thats not OPT's fault per se.

(We _do_ have recipies to build libpnd and such which could be fed upstream; I've always meant to, but have not gotten around to, separating out all the 'pandora specific' gunk from libpnd, and really just have libpnd be for 'pnd file stuff', so it woudl apply to OE or linux at large, and then have libpandora be the pandora specific junk; we really shoudl do all that, but lets get pandoras shipping first, shall we?

 

[Tempel]

Well WizardStan, I think you need to acknowledge that there are certain benefits to traditional package management over PNDs. Namely, there's automatic updating, no library duplication, and a consistent interface for all installed packages. However, I should note that automatic update of PNDs is being worked on.

But Tramb, you have to acknowledge that there are certain benefits to the PND system. This debate has been done before, even before libpnd was created, but it was ultimately decided that traditional package managers couldn't provide for the Pandora's needs. PNDs are great because they avoid all dependency issues (important with multiple SD cards), they need no installation (being manageable from any computer without special software), and they work no matter where the file is located (allowing it to be on SD cards instead of needing to be installed somewhere in the root). Well, WizardStan's own Java PND kinda breaks some of these benefits, but we'll figure that out eventually .

Most importantly, though, is that opkg is currently pretty much useless for anyone booting from NAND, as there's minimal space to install new packages. Not everyone boots from SD - I intend to, but it seems like most people around here don't - so if a dev makes something needing installation through opkg, many Pandora owners will not use it. Therefore, they are going to make a PND first (since all Pandora owners can use it), but they'll only make a bitbake recipe if they really want. But like I said earlier, if we can make it easy to do both at the same time, devs will do it. If you're at all familiar with the workings of bitbake, I encourage you to look into making it output PNDs.

 

[Tramboi]

Thanks for the discussion, guys.
I agree to the points that devs shouldn't be forced to write recipes, as PNDs is the condoned package method.
opkg users still can write recipes for their favorite apps without bothering the porter.

Basically, I'd just want to have word from the team that if recipes are submitted, they will find their way to openpandora.oe.git/recipes
Submitting recipes patches is one thing I (and presumably others) would gladly do, while hosting a full clone of the git repository with added recipes would be a bit of a PITA
Pushing them upstream to openembedded.git/recipes was not really a goal I envisioned, because there would be lots of support for various architectures, but, hey, these other platforms owners could steal our recipes and try them, after all

By the way, and this is totally off-topic, the HEAD of the git repositories are slowly rotting for modern Linux hosts, with the latest versions of make, gcc, m4...
Everything is quite easy to fix, but this can clearly discourage one trying to build his toolchain. I don't know what would be the correct approach:
- submitting temporary patches for op's git that will be overwritten when openembedded.git is merged again in openpandora.oe.git
- trying to cherry-pick the good fixes from openembedded.git
- merging openembedded.git in openpandora.oe.git (gulp, probably daunting)
- leaving it like this for now

PS: By the way, I agree we've seen this debate a long time ago, but now that I (and many others) have a Pandora, it makes everything more concrete

 

[Tramboi]

So I've decided to bite the bullet and I setup a clone of openpandora.oe.git at https://github.com/tramboi/OpenPandora-Community

I just created 2 recipes for Hatari and Vice, and I will make more.
Anybody who'd like to contribute recipe patches is welcome, as I'm new at this packaging method

Cheers,
Bertrand