(pardon the brevity here .. got a really poor connection to mny shell box right now)
Just noticed.. when ED pulled all the scripts out of libpnd git, he included pnd_make.sh and genpxml.sh (ie: the whle testdata/scripts directory.)
I will have to check the Makefile, but that probably breaks the 'make deploy' target a little, but I'm not sure if taking genpxml.sh and pnd_make.sh out of libpnd is right ..
.. they _can_ be used on the pandora (should be present), but they should be part of libpnd itself I think, since they're part of the SDK and libpnd system than overlay; they're not generally used for system operation, like the other scripts.
Also, if we want to properly support pnd_make.sh on pandora itself, we should include the iso and squash tools in firmware, so it can work right :)
Keeping all the scripts together is nice, but those two give me an itch.
jeff
-- If everyone would put barbecue sauce on their food, there would be no war.
On Fri, Feb 25, 2011 at 4:27 AM, Jeff Mitchell skeezix@skeleton.org wrote:
(pardon the brevity here .. got a really poor connection to mny shell box right now)
Just noticed.. when ED pulled all the scripts out of libpnd git, he included pnd_make.sh and genpxml.sh (ie: the whle testdata/scripts directory.)
Right, those should stay there, even I'm running pnd_make.sh directly from the libpnd git when I'm making PNDs. ED, please bring them back.
Am Fri, 25 Feb 2011 13:51:51 +0200 hat Grazvydas Ignotas notasas@gmail.com geschrieben:
Hi,
ahh, didn't know they belong there. Okay, I'm putting them back.
On Fri, Feb 25, 2011 at 4:27 AM, Jeff Mitchell skeezix@skeleton.org wrote:
(pardon the brevity here .. got a really poor connection to mny shell box right now)
Just noticed.. when ED pulled all the scripts out of libpnd git, he included pnd_make.sh and genpxml.sh (ie: the whle testdata/scripts directory.)
Right, those should stay there, even I'm running pnd_make.sh directly from the libpnd git when I'm making PNDs. ED, please bring them back.
[13:09:43] <maplesugarlover> skeezix: so could we have a discussion on amendments to the pnd format? I want to see a discussion happen
This comes up quite regularly, so here, a kick-off post.
History:
We banged around a lot of ideas back at the beginning of the project, and we did throw out a lot of possibly good ideas due to limits of the 'in development' kernel, problematic SD drivers, missing drivers for various filesystems we wanted to try, etc. Being a 'moving target', this was not a problem, but does mean that assumptions we made back _then_ may not apply _now._ Such is life :)
The pnd-file system we have now obviously works pretty well, but does have shortcomings (what doesn't?) Fortunately, by nature of how libpnd works, its _very_ easy to change the whole pnd system without breakin gminimenu or anything.
We had a _lot_ of ideas back in the day, but short of digging through old IRC logs, or the old wiki, they're lost to antiquity. People tend to think we're idiots and just did not think of things, or did not test performance of various combinations, etc etc; we're not, we did, but we had tight limits back then :)
Go go go:
So heres your chance .. if you've got an idea for a revised pnd-file format, let us know! The frameworks can be updated to work with existing pnd-files and new format ones pretty easily, as its all contained in a few tight little places. We've always had the goal of upgrading the pnd-file system (and the whole firmware across the board) as new ideas come.. its a hackers system, so lets hack :)
Requirements:
1) we don't punish the user 2) .. see (1)
Implications:
- upgrade path from old to new; preferably we support both formats for awhile (short, long time) - remember, pnd-files have many features; pepole tend to have their pet-idea, but it only offers a subset of what we have now .. we wanteds to ffer the world, so that necessarily limited our choices. If you ignore a lot of feature ideas, it widens the net considerably. (And what features are kept/added/lost is perhaps open for discussion, but I tend to like not to lose features :) - multi-user support -- remember, there might be multiple people on the pandora at the same time; future-proof, and really.. someone might have ssh open on there, with peopel running tools from the shell, etc. - multi-run -- a single pnd coudl be run many times simultaneously - union friendly -- aufs/union has to wrap around it nicely, for the appdata magic to occur - perform well -- a pnd cannot take forever to mount and run, regardless if its small or large; some formats strain when large files are used - reasonable file size (less of an issue nowadays due to large volumes, but matters to NAND-locked users) - multi-app friendly (one pnd may have multiple applications in it) - documentation friendly (one pnd may offer documentation out to the system) - automation friendly (automated rescan of apps, automatic publishing of documentation, automatic generation/cloning/referencing .desktop, etc.) - .. probably more.
Basic ideas that came/come up:
- binary header; ie: think 'table of contents'; we did rough up a TOC system, whereby a binary header could show where the appended bits are, without having to seek/scan anywhere. Coudl still use iso (or other) as the main payload, and the TOC just shows where the PXML, icon, etc are. Pro: you can now have multiple icons or ranom other bits appended, since the TOC defines where they are Cons: we did test this .. its _trivial_ to add this into libpnd (might even still be in there, would have to check), but I think it performed poorly or had some issue; vimacs might remember :) We tried it, it didn't fly for us back in the day, but it might now. I forget.
- payload filesystem change; could be zip instead of iso/cram/squash. (We can already handle any kind of filesystem since its really just pnd_run.sh that cares.. since we _append_ stuff right nowm the format of the payload is irrelevent!) -- at the time we couldn't get zipfs/fuse/etc working nicely in the in-development kernel, so it was not an option we could use. We had to get aufs working (thanks djwillis! :) at all, so fighting with every filesystem was a problem for us.
- appended format; could use iso/cram/etc payload, and then append a zip-file instead of raw appending PXML and icon; suddenly you have a tail-TOC (table of contents on the rear), and zip includes an index; could be uncompressed even. Suddenly it'd be easy to pull screenshots out of the appended zip without mounting the main payload. The risk is you end up with everything and the kitchen sink appended, and also in the iso/payload, but at least it'd be fast. Wouldn't use zipfs for that, just any old zip lib would let you pull stuff from an appended zip. -- we did mull this over, but I think we had some problem with it at the time, or else it was just os easy to do what we already do that we didn't do it. So needs re-thinking. (ITs still goofy to append stuff to an iso, though :)
- 'executable' bin loader; could do something drastic ("pnx" was how I referred to it), where you teach the kernel how to load a pnx file, so its just a new kind of executable. You just 'run' the "pnd", and the mount/run/unmount just magicly happens. Suddenly you don't need any fancy execution logic. It'd be goofy still to figure out how to generate .desktops automatically, or generate documentation from it, and impossible to host multiple applications inside one pnx, so we didn't pursue it much. But its a _neat_ idea.
- theres 'crazy like a fox' ideas too; detect pnd-files, union mount them all into the filesystem as if they were natively installed, and then they work like normal linux apps (just read-only.) Can really muck up the firmware with all these overlays, uses up all you rloopback devices, makes SDs un-ejectable and whatnot, but such a crazy neat idea; I've never looked into it myself, but I think JayfoxRox was mulling it over. Maybe the negatives can be overcome, so something to think about.
So .. the frontend can be changed (isn't anything now), the middle payload can be changed, the tail appended stuff can be changed. ITs all open, since libpnd doesn't really care.
Discuss, if you have any concrete good ideas, that don't screw the user :)
jeff
-- If everyone would put barbecue sauce on their food, there would be no war.