Ok, Guys

[dflemstr]

So, I felt like downloading some PNDs from the Pandora AppStore, right? You know, just some random files to be used for testing purposes in my Box webapp.

So I uploaded them to the app, and can you guess what I discovered? (Hint: the app validates all PNDs when they are uploaded)

Some of the uploaded PNDs aren't even remotely valid. Some are missing <version> tags, some are even missing PXML files, some contain versions like '<version ... minor="b">' (what the heck? the spec clearly states that only integers are accepted), etc.

I would have expected this from absolute newbies, or developers that are new to the scene, but most of the PNDs come from Official Pandora Developers.

So now I ask of you: PLEASE fix these errors, they are probably the reason why so many PNDs can't be discovered (see the "I Need Help" subforum).

List of affected files:

• CodecPack.pnd contains invalid PXML markup.
  The error was:
  There must be exactly one version tag in the PXML file!
• Dosbox.pnd didn't contain PXML data.
  The package didn't contain any PXML metadata.
• fennec.pnd contains invalid PXML markup.
  The error was:
  Invalid release, must be an integer!
• Hatari131d.pnd contains invalid PXML markup.
  The error was:
  There must be exactly one version tag in the PXML file!
• Heretic.pnd didn't contain PXML data.
  The package didn't contain any PXML metadata.
• Hexen.pnd didn't contain PXML data.
  The package didn't contain any PXML metadata.
• HotFix1.pnd contains invalid PXML markup.
  The error was:
  There must be exactly one version tag in the PXML file!
• Stella312a.pnd contains invalid PXML markup.
  The error was:
  There must be exactly one version tag in the PXML file!

If the error "The package didn't contain any PXML metadata." but the file *does* have a PXML block, it might mean that the preview picture of the PND file is way too large (I accept a maximum of 1 MiB) and that should be seen as a bug, as well.

Cheers,
Your PXML advocate

 

[Pickle]

hexen and heretic might have been because of missing description text, other than that dont see a problem.
dosbox did have issues with the help file, but i fixed that in the latest version.

can you give better descriptions other than bad meta data?

 

[dflemstr]

hexen and heretic might have been because of missing description text, other than that dont see a problem.
dosbox did have issues with the help file, but i fixed that in the latest version.

can you give better descriptions other than bad meta data?

I didn't say 'bad metadata', but 'missing metadata'. I'll have to check whether this is a bug in my system, or something random (ie. that the PNG image contains '<PXML' randomly), but there must be a reason why the "<PXML" isn't found in exactly that file. Anyways, Pickle, your PND files are OK in general so you shouldn't have to worry... I'll see whether this turns out to be a bug on my side or not (which would be strange since I'm using exactly the same scanning algorithm as in libpnd, just ported to Scala and optimized)

 

[JayFoxRox]

I'm surprised that you didn't find more than that. I also wrote a similar tool getting the files from EDs dl page and a lot of them failed because they used an "&" symbol in the description without escaping it. However, I also noticed what you found. Besides that the icon sizes vary between 16x16 and 256x256, some PNDs use the author tag for nicknames or who did the port, others for real names and the original devs and some are using the homepage tag for the projects page, others for the authors page and so on.

It's a mess

I said it on the IRC channel before: Drop PND now, improve PXML or switch to some other standard and only keep the PND loader code etc, but make a completly new format like dflemstr suggests for the whole package.
I also had some PXML files which named invalid files as preview pics, others had: "todo" as the german description so it wouldn't even display the english fallback.
I would also like to have some features in the next PXML and PND format.

- So there should definitly be some room to discuss this further which should also be offical (linked from the offical page and make it clearly visible) because it's a change made to the OS which affects everybody.
Possibly a new project in the bug tracker? (- personally I'd like to have at least the projects "Hardware" [for hardware problems - long overdue] and "PND / PXML" [also overdue imo])

 

[Pickle]

hexen and heretic might have been because of missing description text, other than that dont see a problem.
dosbox did have issues with the help file, but i fixed that in the latest version.

can you give better descriptions other than bad meta data?

I didn't say 'bad metadata', but 'missing metadata'. I'll have to check whether this is a bug in my system, or something random (ie. that the PNG image contains '<PXML' randomly), but there must be a reason why the "<PXML" isn't found in exactly that file. Anyways, Pickle, your PND files are OK in general so you shouldn't have to worry... I'll see whether this turns out to be a bug on my side or not (which would be strange since I'm using exactly the same scanning algorithm as in libpnd, just ported to Scala and optimized)

well my stuff is usually just a copy of files from a previous pnd to another, so its possible I miss changing something. But all the tags and formatting should be correct.
.

 

[dflemstr]

JayFoxRox - Look through my posts from the last year or so, I've said exactly that from the very beginning, that PXML needs to be reworked. Everything from the storage format to the PXML syntax. For an "official discussion", please refer to http://pandorawiki.org/New_PND_format

EDIT: but this is no place for such a discussion...

 

[JayFoxRox]

Yup, I have seen that wiki page and would like to change it, but I can't seem to login at the wiki anymore - it instantly logs me out again.
However, it I wouldn't have searched for the PND file format description or didn't dislike the store this much, maybe I'd have never come accross it - it should be on some offical platform.

 

[Sphinxter]

Eeek! A few data errors, let's go all scorched earth and start over completely!

 

[dflemstr]

Eeek! A few data errors, let's go all scorched earth and start over completely!

Do you have a better idea? That sounds perfectly reasonable to me, considering the large value that "A few" has in this case (accumulated over the last year), and considering how easy it would be to rewrite the format.

 

[Sphinxter]

Eeek! A few data errors, let's go all scorched earth and start over completely!

Do you have a better idea? That sounds perfectly reasonable to me, considering the large value that "A few" has in this case (accumulated over the last year), and considering how easy it would be to rewrite the format.

Yes, fix and validate data in the offending packages.

 

[dflemstr]

Yes, fix and validate data in the offending packages.

Well, yeah, that was what I said in my first post, but JayFoxRox (and me, in my latter posts) refers to problems other than those discovered in this small group of PNDs, that cannot be fixed as easily. (Examples: loading times, fuzzy metadata format, encoding problems)

 

[fusion_power]

Sounds messy. ^^"
Is ther no input tool, that foces a correct input of the PND Data, otherwiese it will not generate a PND? Should be pretty idiot-proof this way.

 

[slaeshjag]

That should be possible to hack together in a evening.

 

[skeezix]

"version" is required, indeed; I think in absence, assume "0.0.0.0" I suppose

jeff

ie: Your system has to work with reasonable, if flawed, PXML.xml, or it ain't going to fly (rejecting on upload might help, but more than likely, it'll piss people off; if the PND-file works, with flawed PXML.xml, then your index should handle it as best it can. *shrug*)

Only have 10 mins near a keyboard today, but I threw a version tag into Stella and Hatari, for whenever I do another build of the pnd. Life has been way too busy, forever, damnit

 

[Exophase]

With the scale of the app store would it really be a bad idea to have review and validation required to prevent non-conforming PNDs from even making it in? After reviewing so many after the fact would you be willing to do something like that yourself dflemstr? Then everyone would be forced to learn how to do things right and the app store will be kept tidy.

 

[1magus]

I agree even though I don't have a Pandora but I did notice in the app store that not only are there a lack of screenshots there is a lack of any explanation at all in the notes area when a PND is uploaded.

 

[overjupiter]

I agree. Validate PNDs during the upload process. If it doesn't validate, report it to the uploader like so:

 

[bzar]

Or:

 

[cosam]

So is there any easy way for packagers to validate their PXML? An XML schema or something? Folk are more likely to use such tools if they're easy to find and use. Even more so if said tool makes appropriate suggestions as to how the PXML should be altered to make it compliant.

Your system has to work with reasonable, if flawed, PXML.xml, or it ain't going to fly (rejecting on upload might help, but more than likely, it'll piss people off; if the PND-file works, with flawed PXML.xml, then your index should handle it as best it can. *shrug*)

Usually I'd agree with being flexible in what you accept but when you're talking about a repository, I think it's reasonable to enforce the spec. Sure, a half-correct PND will work on a default Pandora installation, but what about other tools? Catch the errors at the source, and you'll save a whole lot of grief further downstream.

 

[multiplex]

Usually I'd agree with being flexible in what you accept but when you're talking about a repository, I think it's reasonable to enforce the spec. Sure, a half-correct PND will work on a default Pandora installation, but what about other tools? Catch the errors at the source, and you'll save a whole lot of grief further downstream.

Agreed. The robustness principle is nice where it applies, but accepting invalid data and then forwarding it to every client isn't robust at all.