Panorama

[slaeshjag]

Nobody has stepped up with a replacement, that's why we're still stuck with it.

 

[Schnatterplatsch]

Stuck? Really, I love it.
From both sides, creating and using. Maybe I am stupid or what, but what are its major faults?

I mean, we have some brilliant devs here and I cannot remember them complain.


dflemstr, have you ever really used it? I cannot really imagine.
Seriously, I've got my apps installed and updated in seconds from PNDStore, no matter if I change SD cards or anything. Any faster would have to be telepathy or something like that.

 

[Esn]

I know this is off-topic, but just out of curiosity, what exactly is bad about it? It seems simple enough for the user, and is going to get simpler now that apps like PNDstore are being developed. If it does take up more room as the price for that simplicity, it's not a huge issue because most of us have a lot of space on SD cards anyway. Is the main issue that it's more complex for developers?

 

[dflemstr]

I know this is off-topic, but just out of curiosity, what exactly is bad about it? It seems simple enough for the user, and is going to get simpler now that apps like PNDstore are being developed. If it does take up more room as the price for that simplicity, it's not a huge issue because most of us have a lot of space on SD cards anyway. Is the main issue that it's more complex for developers?

I've ranted about this in the past, but briefly:

• The container format is undefined (iso9660 or squashfs or...?) and the most unsuitable one is preferred (ISO)
• The PXML file is arbitrarily stored as a stream at the end of the file, which drastically increases PND indexing and general loading times.
• The PXML specification contains a lot of redundancy, duplication and corner-cases. I mean, seriously, there are package, application AND root elements? An application is identified by package ID, application ID, appdata ID, package version, OS version, app version, and launch mode? A version is forced to be a combination of 4 basically arbitrary strings, PLUS a status string? REALLY? And title and description tags are duplicated across the board, internationalization is broken, etc... it's a MESS
• The system basically REQUIRES you to use awkward launcher scripts for EVERYTHING, because PNDs are launched by an awkward BASH script. Yeah, that sure is elegant, simple and efficient, right?

How a well-designed PND system would work:

panorama-1.2.3~prerelease~dflemstr.pnd
                                   ^^^ PND suffix
              ^^^^^^^^^^^^^^^^^^^^ Tags of package (you can install the same package with different tags at the same time, eg. panorama~prerelease~dflemstr and panorama~full etc
                                                    and the package system will only upgrade to a new package version with the same tags automatically, so you wont accidentally
                                                    go ~full → ~beta or ~dflemstr → ~bzar)
         ^^^^^ Version of package, numbers and periods only
^^^^^^^^ Name of package

Contents of PND (normal squashfs file system without anything special). Note that the contents are laid out exactly like the "/usr/" directory.

panorama-1.2.3~prerelease~dflemstr.pnd
├── bin -- Actual executables bundled with the PND, just like how /usr/bin works
│   ├── panorama
│   └── panorama-debug
├── lib -- Libraries that the applications in the PND require, or that this PND should provide for other PNDs
│   ├── libpanorama.so
│   ├── libpanorama.so.0
│   ├── libpanorama.so.0.1.1
│   └── panorama -- Other binaries etc., just like /usr/lib works
│       └── plugins
│           └── libapplications.so
└── share -- Shared data, like /usr/share
    ├── applications -- Application shortcuts for applications this PND provides, just like /usr/share/applications
    │   ├── panorama-debug.desktop -- Contents: "[Desktop Entry]\nExec=panorama-debug\nIcon=panorama\nName=Panorama\nName[de]=Panorama\nComment=A launcher\nComment[de]=Ein Launcher ......"
    │   └── panorama.desktop       -- See the FDF .desktop spec for more info http://standards.freedesktop.org/desktop-entry-spec/latest/
    ├── icons -- Icons that this PND provides. This can contain multiple icons depending on context, resolution and current theme (just like /usr/share/icons)
    │   └── hicolor
    │       ├── 32x32
    │       │   └── apps
    │       │       └── panorama.png
    │       └── 64x64
    │           └── apps
    │               └── panorama.png
    └── panorama -- Data files, just like regular /usr/share
        └── interfaces
            ├── Colors
            │   └── ui.qml
            └── Tabbed
                └── ui.qml

All PNDs would then be overlay-mounted (We've demonstrated multiple times that this is possible) over /usr/pandora, and that directory would be treated like /usr/local (/usr/pandora/bin would be added to PATH, /usr/pandora/share/applications would be scanned for .desktop files, etc etc).
PNDs don't get write access to their directories (Just like an app can't normally write to /usr/*); instead, the FDF XDG standard is followed, and the preferred data dir is stored where the env var XDG_DATA_DIR points to (which can be on the SD cards or wherever) http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
Voilà, all problems with PNDs solved! There's a solid spec, no ambiguities, never any miscommunication. If a tool needs information from inside the PND, just mount it! (That's what we have to do today anyways because there are no tools for reading ISO files via C/C++)
This system uses the existing Linux Desktop standards, and is what I've been trying to push the PND system towards for the last 2 years.

 

[CPUnltd]

sounds good to me... so what's stopping the movement toward this PND system?

 

[dflemstr]

sounds good to me... so what's stopping the movement toward this PND system?

Bureaucracy, lack of publicity, conservatives (the Latin meaning), lack of interest.

 

[sebt3]

I've ranted about this in the past, but briefly:
The container format is undefined (iso9660 or squashfs or...?) and the most unsuitable one is preferred (ISO)

The prefered format is squash....

The system basically REQUIRES you to use awkward launcher scripts for EVERYTHING, because PNDs are launched by an awkward BASH script. Yeah, that sure is elegant, simple and efficient, right?

I have loved to read that....

 panorama-1.2.3~prerelease~dflemstr.pnd

Nobody stop anybody to use that filename nomenclature. yet there is nothing set in stone. Is that realy requiered ?

Contents of PND (normal squashfs file system without anything special). Note that the contents are laid out exactly like the "/usr/" directory.

all my pnds are this way. in fact most PNDs out there are this way.....

All PNDs would then be overlay-mounted (We've demonstrated multiple times that this is possible) over /usr/pandora, and that directory would be treated like /usr/local (/usr/pandora/bin would be added to PATH, /usr/pandora/share/applications would be scanned for .desktop files, etc etc).

This is the first divergeance between your proposition and current reality.
Would you mind to enlighten me how will we separate the configuration directory for each PND as this was a request on the drawing table since day 1.

PNDs don't get write access to their directories (Just like an app can't normally write to /usr/*); instead, the FDF XDG standard is followed, and the preferred data dir is stored where the env var XDG_DATA_DIR points to (which can be on the SD cards or wherever) http://standards.fre...pec-latest.html

Sadly many open-source apps/games doesnt respect XDG definition. Yet we all want to play games that are not XDG aware (that's 99% of all existing)

Voilà, all problems with PNDs solved!

So your only issue is that we want to have a separate config directory for each apps ?

 

[dflemstr]

I really don't want to discuss this and I don't like your condescending attitude.

You can take a horse carriage and a car and say "both have wheels, both are used to transport people; the only difference is that the car features a cigarette lighter." The point is moot if you can't make an objective assessment.

Preferred format is squash, yet much of the documentation tells us to use mkisofs.

The bash scripts might be a pleasure to read, but the problem is that it exists; a proper solution would just let the application be launched like usual via .desktop files. The bash scripts, pndnotifyd, environment variables, LD_LIBRARY_PATH hacks, etc, are all redundant.

The point with the nomenclature is that package info is unrelated to the PND file, and should be external. I didn't want to demonstrate a naming scheme, but instead a packaging principle and what's supposed to be stored in external package metadata databases.

So what how PNDs are organized right now internally; the point is that the internal structure should *actually matter*, be non-weird, and replace the "<application>" and "<icon>" entries in the PXML

Since each PND application can store a config file wherever it wants within the XDG_DATA dirs (think of it like ~/.config/ and ~/), the apps chooses where it stores which config files. This isn't enforced by a PXML file.

If an open source app doesn't respect XDG, just modify it. You must modify it anyways for the current PND system.

 

[sebt3]

If an open source app doesn't respect XDG, just modify it. You must modify it anyways for the current PND system.

most my PNDs are from the original source without any code modification....

 

[dflemstr]

If an open source app doesn't respect XDG, just modify it. You must modify it anyways for the current PND system.

most my PNDs are from the original source without any code modification....

Then, since they work with the PND system, just do 'cwd=`dirname "$0"`; cd "$XDG_CONFIG_HOME/myprogram"; eval "$cwd/myprogram"' as a launch script and get the exact same behavior, if you really can't be bothered (You have to make the launch script after all for anything moderately advanced).

 

[Schnatterplatsch]

I'd really love to hear skeezix opinion on this. Not to bash anyone, no, but under all those people criticising I cannot remember anyone complaining about the PND system. It maybe hacked together in some ways, but in the end it works really, really nice for the user.
Do you still have a pandora preorder, dflemstr? Maybe you would think different if you could try it out for real.

Heck, they say that PocketSNES is hacked together and it's a miracle it does even work on my WiZ, but hell, it does and I do not care how it does it.

Maybe you really should discuss this with skeezix or ED(or anyone else). If there are design flaws, they are fair guys and will take them into consideration, I think.

 

[bzar]

There seems to be a lot of discussion on the PND system here, maybe enough to warrant a separate thread? I don't mean to step on any toes or point people to go elsewhere. It would just be easier to follow the two separate- though somewhat intertwined -topics in two threads.

(Aaaand now I sound condescending. Not my intention.)

 

[dflemstr]

I'd really love to hear skeezix opinion on this. Not to bash anyone, no, but under all those people criticising I cannot remember anyone complaining about the PND system. It maybe hacked together in some ways, but in the end it works really, really nice for the user.
Do you still have a pandora preorder, dflemstr? Maybe you would think different if you could try it out for real.

Heck, they say that PocketSNES is hacked together and it's a miracle it does even work on my WiZ, but hell, it does and I do not care how it does it.

Maybe you really should discuss this with skeezix or ED(or anyone else). If there are design flaws, they are fair guys and will take them into consideration, I think.

Oh I've talked to them, I've offered ready-to-go finished replacement code that didn't address all of the problems with a system like this, but offered a gradual transition. They didn't want anything to change.

Now let's leave this topic in this topic (see what I did there? ) and go back to stuff that's actually not futile.

 

[skeezix]

Someone just pointed this ot me, and no time to read but..

As always -- pnd is version 1.0; It (and libpnd) is very much by design to work well, and be replaced; a lot of it can be replaced under the hood, without breaking anything for the user, to improve over time. (Historical note -- we devs got dev-pcbs to work on and figured we had a couple of _weeks_ to build all of this and get a working firmware going. We actually got dev boards after the date the units were supposed to be shipping to people We had an unstable kernel, a lot of bad drivers, and a lot of stuff didnt' work or didn't perform well enough.. we've talked about this in IRC and message boards to death for anyone wanting to know more history, but suffice to say .. pnd system works great, despite the constraints we had. See all our happy users But as always, day one, it was planend a lot of it could be changed up. It works great, and it has problems. Gosh darn, a 1.0 thign has issues, news at 11

v2.0 is not yet defined, and as always, we have firmware-dev mailing list and so on to discuss things. Coming out of the blue in random board threads isn't going to make anything happen
http://pandorawiki.org/Firmware_governance

Historically we have very few peopel coming out with patches or ideas; you were one of the great ones, and you actualyl put in some patches (thanks!) .. but you took off Stick around, help make thigns better

To wit -- the requirements of users are vast, and one of the big ones is not to break everything thats already there; further, most people like to toss out 'this is the best' without considering all the implications. (A lot of ideas commonly break when you start discussing multiple apps in one file, multiple simultaneous runs of the same file, mutliple users on one pandora runing the same file, etc. Ask sebt3 how hard its been to rewrite pnd_run.sh.. dozens of subtle oddities pop out

I still like the binfmt idea as one interesting option (though it fails many of the requirements I just listed) -- make a pnx format related to pnx, where the file is just executable outright

--

laying out a pnd like /usr/.... etc hierarchy is an interesting new twist idea. Mount and overlay and xfce just detects it .. hmrf.

Some issues mind you -- auto documentation while the app is unmounted wouldn't work there; categorization wouldn't work in a standard way, so you'd haev to do it in filenames as you say, whcih is pretty guly. (What if its in several categories.. getting really messy with your tags there. But an interesting idea anyway.

--

Hey dflemstr, you're around, cool Join firmware-dev and kick thigns up again, its not too happening, we need more action

Your idea above is very different than your previous 'pnd sucks, it shoudl be like _this_" ideas. Its obviously a hard problem, since even you come up with multiple different 'bests'.

--

Let me make a quick summary -- it is a _hard_ problem to come up with an ideal solution, since ideal varies by person. The whole system _is_ up for change, with gatekeepings being our community; no one person can just come up with 'this is best' and be right, it takes discussion and debate; a lot of good ideas will just not be workable, but a lot of ideas shoudl go in for v2, and ideally, can be back compatible with v1 so as not to break everything for the user.

I'm sick of seeing a few people coming out once in awhile (not sticking around to actualyl achieve change) and saying 'this sucks, heres some patches, lets discuss it and come up with something'. Its _easy_ for any topic in the world to say 'this government sucks, I'd do better!", but to actually say "this and this, lets discuss it over the next month or two, maybe we can all come together" is hard. Usually its "this is a great idea, but anyone who disagreess with me is wrong" (#openpandora I'm looking at you

Anyway, the whole stage is open .. firmware 2 ("Y") is not yet well defined, now is the time to discuss these things.

HF5 is nice and stable; HF6 is more patches; after than, Firmware 2 .. the sky is the limit!

So join the mailing list, and lets have positive discussions..

jeff

And I'm having twins in a couple months, so the process is more important than ever .. we have a few more bodies helping with the firmware now, and ownign the commitership.

 

[skeezix]

I'd really love to hear skeezix opinion on this. Not to bash anyone, no, but under all those people criticising I cannot remember anyone complaining about the PND system. It maybe hacked together in some ways, but in the end it works really, really nice for the user.
Do you still have a pandora preorder, dflemstr? Maybe you would think different if you could try it out for real.

Heck, they say that PocketSNES is hacked together and it's a miracle it does even work on my WiZ, but hell, it does and I do not care how it does it.

Maybe you really should discuss this with skeezix or ED(or anyone else). If there are design flaws, they are fair guys and will take them into consideration, I think.

Oh I've talked to them, I've offered ready-to-go finished replacement code that didn't address all of the problems with a system like this, but offered a gradual transition. They didn't want anything to change.

Now let's leave this topic in this topic (see what I did there? ) and go back to stuff that's actually not futile.

This is blatantly untrue my friend

I've always been very welcoming (and begging for!) submitters to the system. For the record, _every_ patch you have submitted to me was accepted. I found a few bugs in the code, but not many, but every patch of yours went in (you pretty much redid the xml parser, remember?) We got few other patches from people, and they went in too. sebt3 and Ivanovic have signed up and done lots of work too.

There was your 'other libpnd' that you'd worked on; in your own git, you came and mentioned it a couple times to me .. it was not a full rewrite as you liek to recall, most of it was just runing libpnd through pretty printer to your style of indents, so it was _undiffable_; but ass always, I'd asked you to provide patches for chunks you thought were good, and It hink that was arodn when you ragequit before. I never received any patches from that libpnd of yours to mine.

(I've no time to go out looking for random bits of code to integrate into a stable working tre;e you want it in, you submit a patch

So do _not_ lay anything on my feet and say I was in your way. I've tried very hard to always be helpful and keep thigns open (though of coruse I'm human, I may have failed or hurt someones feelings along the way, I try to be pretty decent.) I've also handed out the commiter keys to other people, I'm not the ogre

You're a great coder, and a great designer; sometimes you have a"my way is the best way" blinders on (and I'm sure I and everyone else does to, to some extent); but thats why we have a process. Submit patches, let folks take a look and test, to make sure we dont' rbeak everything, and good to go.

jeff

 

[skeezix]

(I also have kept every email and IRC entry of mine since about 1985 when I was running BBSes; its a tonne of work to comb through all that, but dont' make me go dig out all those IRC logs

 

[skeezix]

Wait..

You're suggesting if the guy has 1000 apps, to mount them all on boot, unmoun on shutdown, and live with massive memory and performance issues the whole time? (ie, so the .desktop and icons are visible to menus); if not then you still need somethong to seek through all the pnds and find the files, extract them and make them available. If so, how does that system then run things.. Doctored .desktop or binfmt or rtc etc as discussed for pnd now? Even if you do it per category/tag, each mount takes a half second..

Either the is nasty or not much different to what we have

Okay im done catching up, sorry for replying 3-4 in a row

Jeffphone

 

[dflemstr]

No skeezix, you take this the wrong way, I'm not saying that *you* weren't cooperative. You have always been very much so, and I'm not complaining about that
It's not the *code* that isn't well written; it's the idea behind it that I think is broken. (And you never saw the rewrite code - that was in an entirely different code base separate from libpnd)

The issue with stuff rather lies with other people that haven't been very cooperative, in my opinion (And that's all that matters for justifying my own decisions ). It has been people like CraigIX and EvilDragon that caused me trouble; CraigIX for sticking to an old version of the PND standard for a *very* long time, causing correct PNDs to misbehave on the Pandora Appstore thing (which caused a whole story on its own, because *that* had bugs too), and EvilDragon for not really having enough time to actually care about the problem, having declined multiple times when I offered to help, saying "what we have is good enough."

The solution I presented here isn't new; it was discussed at length in the Pandora Overlays thread from 8 months ago. Because it was discussed at such length, and with **so many people participating (not just 3-5 people..)**, I can claim that it's well-designed. And of course there are multiple solutions to the problem. The thing is that one needs to find a *good* solution and **stick to that solution** to get a high-quality system. There is no optimal solution, but there are *correct* solutions and *incorrect* solutions, judged by some standard (and my standard is: does the system actually help more than it causes difficulties?).

Now remember, I didn't come here to pull this up again. I could work on a new system if people are willing to cooperate, but I didn't come 'out of the blue in random board threads' to talk about this. I chose to make this less of a concern to me, and if I have any say in the matter, it will stay that way.

Misc comments:
Documentation: We have man, info and html-help pages that would be used just like for every other package; if there should be a menu entry for documentation, the dev can just add it in bla.pnd/share/applications
Category: Tags aren't for categories, just for compatibility (remember when you introduced appdata fields? Yeah, same tags = "same appdata". This is also how things work in e.g. the Haskell and Ubuntu worlds). We've observed that categorization doesn't matter once you actually have a PND file in your hand (Does libpnd treat a PND differently depending on its categories? No!), so just specifying categories when the package is uploaded somewhere is enough, à la Android apps (The actual app .desktop files still contain categories as usual)
Performance: See how PuppyLinux does it. It works, it's elegant, it's efficient. 'Nuff said. See "SFS modules":http://www.puppylinux.com/development/package-management.htm

 

[Schnatterplatsch]

CraigIX for sticking to an old version of the PND standard for a *very* long time, causing correct PNDs to misbehave on the Pandora Appstore thing

Just in case you aren't aware: We have two great things now. The repo.openpandora.org and PNDStore, pretty much TheBox if I remember correctly. The only one who talks about his appstore is craig himself.
The repo and pndstore work much better since a lot shared ideas and found solutions

I am sure there have been misunderstandings, but we can solve that out. I hope the discussion continues and we win dflemstr back. You are, no doubt, a marvelous dev.

 

[skeezix]

I still like just using zip (a la jar) but fuse-zip eats up all ram :/ with .desktop inside instead of pxml..

Okay end - I can appreciate backing out of it, but everyone has with the twins comin I'm having to slow down a lot too

Looking fwd to Panorama

Jeffphone