Clearing Up Some Misconceptions

[dflemstr]

This post should just serve as a small shout-out to all developers who plan on releasing some kind of application for the Pandora.

I've taken a look around at various implementations of PND-related software, and they all seem to have one thing in common: their PXML (a file type related to the PND format) file parsing is really buggy, and/or they provide a faulty PXML file.

So in order to prevent a PND compatibility hell once the Pandora is released, I beg all developers who plan on ever touching something PND-related: Read the PXML specification. It will do you good, I promise.

Now, I know that things related to this specification have been messy before (and I still see EvilDragon's version of it lurking in the file archive!) but I'm telling you: this is the real deal, this is the spec you're supposed to follow.

I hope that this post will have some effect; that I won't see "gameemulator" as a category or "*.myfile" as a "file association". There's no way that a launcher will be able to pick up your PXML file if you don't follow the spec, and if a launcher itself doesn't follow the spec... chaos.

I've cleaned up the specification a little over the last few days, to make it easier to understand, and I will continue to do so in the future. Nothing will be added/removed though.

It also seems like skeezix has added some extensions to the PXML format that are undocumented, but us mortals shouldn't have to care about those yet so I won't add them to the specification.

All clear? OK, dismissed

 

[WoD]

Is it worth me building a Pandora PND provider, an online service which helps developers generate the correct XML file via a GUI and handles, to a great extent, the generation of unique IDs?

I haven't searched to find if such an undertaking is already being made, it seems like a sensible addition to a Pandora package repository.

 

[gruso]

There's no way that a launcher will be able to pick up your PXML file if you don't follow the spec

This sounds like a potentially good thing for quality control. Will a PND with non-compliant PXML still appear in the menu? If so, can this be prevented? From a user point of view, nobody will want to run .PNDs that don't magically pop up in the menu like all the others, so devs will be compelled to comply.

Regarding non compliant launchers, maybe they could go through an "app approval" process before going up on the archive? (ED has to manually approve all archive submissions anyway, so it would just be an extra step).

 

[skeezix]

If it doesn't parse, it won't get a .desktop and won't return in app-discovery. Hopefully people will test their pnds before publishing

I have included stubs and stubtools to validate apps, though mostly those don't check too closely as we've chased a moving target, but ever-tightening is good. Thanks dflem for the spec rewrite and parser redo.. Only a few additions and fixes have been done, though we're forever tweaking here or there. I'll do a few updates to the wiki soon.. Figured the wiki would be the best place to publish and attempt to keep up t date

I'll write more tomorow.. Typing on a phone is hell

jeff

 

[skeezix]

Apps should use libpnd for parsing.. At least for consistency?

We should make a good xsd validator..

Vimacs did include a sh script that emits properly formed default blank pxml's including useful semirandom uniqueids, as well, for during the build process for unixheads

Jeffphone

 

[WoD]

http://www.gadgetoid.com/pxml/

I made a start, damned if anyone else is doing it but it's not going to do any harm if I use it as an excuse for a bit of out-of-hours PHP honing.

 

[torpor]

Can I just say that having undocumented features sucks and that this is a big turn-off for using PND? Please don't ask one side of the PND developer pool to conform to standards while the other side is off playing malarky with their own thoughts.

Either its a standard, normal, documented feature, or its out.

 

[GuyMontag]

http://www.gadgetoid.com/pxml/

I made a start, damned if anyone else is doing it but it's not going to do any harm if I use it as an excuse for a bit of out-of-hours PHP honing.

I started a similar Projekt some month ago, but discontinued because the specification was really unclear to me.

On the new Wiki site I read something about a XML schema. Where can I find that schema?

 

[benjymous]

One simple solution to ensuring that pxml files are valid is to include a data hash in a tag at the end (this can simply be a CRC or MD5 of the rest of the xml data)

So:

• team make a "PXML Signer" app - just a simple command line tool that takes a PXML file, does rigorous validation, and adds the hash on the end if it's happy. This could easily be integrated into the build process for any dev, so wouldn't be anything to worry about
• team add a "validate" function to the pnd library - this simply loads the pxml, regenerates the hash, and compares it with the one in the file. If it's missing, or doesn't match, the file is ignored.

This won't stop bad launchers ignoring this, but it will mean bad PND files won't break/confuse good launchers, since they can simply pop up a "bad pnd" message when they see the file

 

[dflemstr]

On the topic of validation:

There are some things that cannot be easily validated, like for example the mime-type for file associations, the "command line args" associated with those associations, the "category" part etc.

And Skeezix; libpnd just parses the text of a PXML, it doesn't check it's validity; e.g. "pnd_pxml_get_standalone" returns "char *" and not true/false, and "pnd_pxml_get_clockspeed" doesn't return an uint32 etc. And if we want it to be a true combinatoric parsing library, we'd maybe want to return a "char *" containing PNG bytes instead of a file path from "pnd_pxml_get_previewpic#" (or if not, at least a function that returns an array of paths would be in order, but we've talked about this before).

So libpnd doesn't really improve the standardization situation; it only fixates the XML format in a way, but nothing more. This leads to e.g. a bug in pmenu ("main.c:125"):

if((strcasecmp(d -> main_category, "emulators") == 0) | (strcasecmp(d -> main_category, "emulator") == 0))
{
	tmpSection = EMULATORS;
}
else if((strcasecmp(d -> main_category, "games") == 0) | (strcasecmp(d -> main_category, "game") == 0))
{
	tmpSection = GAMES;
}
else if((strcasecmp(d -> main_category, "applications") == 0) | (strcasecmp(d -> main_category, "application") == 0))
{
	tmpSection = APPLICATIONS;
}

(Please note that none of {emulator, emulators, games, applications, application} are valid categories according to the spec ("Game" IS valid, but that's coincidental in this case), so PNDs that DO follow the spec won't even show up correctly in pmenu.)

EDIT:

On the new Wiki site I read something about a XML schema. Where can I find that schema?

You can't. The old schema seems to have been screwed up as it got committed into the pandora-libraries repository (It uses UTF-16 encoding I believe) so it's broken, and it's out of date by now anyways. I thinkt hat producing a new schema would be in order.

This won't stop bad launchers ignoring this, but it will mean bad PND files won't break/confuse good launchers, since they can simply pop up a "bad pnd" message when they see the file

It would be as easy to integrate a validator into libpnd, so that no validation process would have to occur.

 

[Rockthesmurf]

I love the idea of using PNDs as an end user, but if you are relying on people reading the relatively large amount of information in http://pandorawiki.org/PXML_specification in order for this thing to be successful you might be having yourself a few problems. It is pretty much guaranteed developers won't always read the document either at all, or in full. Yes I know it wouldn't take long. Yes I know it isn't really too much to ask. But still, lots of people just won't - the easier this process can be made, and the more fool proof it can be made, the better!

Just want to once again make sure you understand that I really like PND, I just am all too familiar about the lazyness levels of people!

Steve

 

[dflemstr]

Steve, you're absolutely right.

So what I'll do next is that I'll develop a simple tool, that will check a PXML file and tell the user exactly what's wrong with it. E.g. "You're missing an icon element" or "That language code doesn't exist" and so on. Do you think that people would use something like that?

 

[dflemstr]

Sorry for the double post; just wanted to say that I've made an almost complete validator for PXML files.

Get it while it's hot.

Example output:

 

[Rockthesmurf]

Steve, you're absolutely right.

So what I'll do next is that I'll develop a simple tool, that will check a PXML file and tell the user exactly what's wrong with it. E.g. "You're missing an icon element" or "That language code doesn't exist" and so on. Do you think that people would use something like that?

If the user runs a script to make the PND package, and your tool is run as part of that process, then they wouldn't have much choice but fix the problems - unless they don't use any scripts to create the PND and do it manually. The tool output looks good to me by the way!

 

[bzar]

Sorry for the double post; just wanted to say that I've made an almost complete validator for PXML files.

:blink: wow. That looks very informative. Also funny at times

 

[dflemstr]

New version with better error detection, + source and sparse documentation just for the heck of it/if anyone wants to hack it.
(My 30-minute-job last night contained some really ugly code as you can see in the previous zip, which contains my source code too)

Get it here

..Oh, right, new features list:
- Tests are ran in parallel (so e.g. the previewpics and exec tags are checked simultaneously)
- Better file type checks
- Better URL checks
- More cowbell
- Er... that's it.

 

[Jan-Nik]

Nice job.

- More cowbell

 

[dflemstr]

I uploaded an updated version (aka: the same thing that's in the Wiki right now) of the PXML specification to the Pandora File Archive, to prevent confusion.
(It is my worst nightmare to get a PND compatibility hell as you might have figured out by now)

Get it here.

I'll give the LyX source to anyone who asks, and I will update the file in the archive whenever someone gives me an improved version.

EDIT: I uploaded an updated version; waiting for it to be approved by an admin or w/e...

 

[zhasha]

I'm reworking libpnd's pxml loader to handle all that your validator handles, and also work properly with all features integrated. I'm mostly done, but it will not be included in the launch image as I broke the API in order to make it less horrible. I could write a small wrapper if there's enough interest.

EDIT: also, I'm using a validating XML parser in C called parsifal.

 

[dflemstr]

I'm reworking libpnd's pxml loader to handle all that your validator handles, and also work properly with all features integrated. I'm mostly done, but it will not be included in the launch image as I broke the API in order to make it less horrible. I could write a small wrapper if there's enough interest.

EDIT: also, I'm using a validating XML parser in C called parsifal.

I wish that I had broken the API back when I adapted to the new format... Whose idea was it to use e.g. pnd_pxml_get_associationitem1_name?

I'd really like to see your code sometime. Also, I don't think that libpnd may have external dependencies so you better include it in the source tree.