Ok, Guys


dflemstr

It's a ball.
Joined
Jul 31, 2008
Messages
2,514
Location
Stockholm, Sweden
Website
Visit site
So, I felt like downloading some PNDs from the Pandora AppStore, right? You know, just some random files to be used for testing purposes in my Box webapp.

So I uploaded them to the app, and can you guess what I discovered? (Hint: the app validates all PNDs when they are uploaded)

Some of the uploaded PNDs aren't even remotely valid. Some are missing <version> tags, some are even missing PXML files, some contain versions like '<version ... minor="b">' (what the heck? the spec clearly states that only integers are accepted), etc.

I would have expected this from absolute newbies, or developers that are new to the scene, but most of the PNDs come from Official Pandora Developers.

So now I ask of you: PLEASE fix these errors, they are probably the reason why so many PNDs can't be discovered (see the "I Need Help" subforum).

List of affected files:
  • CodecPack.pnd contains invalid PXML markup.
    The error was:
    There must be exactly one version tag in the PXML file!
  • Dosbox.pnd didn't contain PXML data.
    The package didn't contain any PXML metadata.
  • fennec.pnd contains invalid PXML markup.
    The error was:
    Invalid release, must be an integer!
  • Hatari131d.pnd contains invalid PXML markup.
    The error was:
    There must be exactly one version tag in the PXML file!
  • Heretic.pnd didn't contain PXML data.
    The package didn't contain any PXML metadata.
  • Hexen.pnd didn't contain PXML data.
    The package didn't contain any PXML metadata.
  • HotFix1.pnd contains invalid PXML markup.
    The error was:
    There must be exactly one version tag in the PXML file!
  • Stella312a.pnd contains invalid PXML markup.
    The error was:
    There must be exactly one version tag in the PXML file!

If the error "The package didn't contain any PXML metadata." but the file *does* have a PXML block, it might mean that the preview picture of the PND file is way too large (I accept a maximum of 1 MiB) and that should be seen as a bug, as well.

Cheers,
Your PXML advocate
 

Pickle

Mega GP Mania
Joined
May 30, 2006
Messages
5,512
Location
Detroit, Michigan
Website
Visit site
hexen and heretic might have been because of missing description text, other than that dont see a problem.
dosbox did have issues with the help file, but i fixed that in the latest version.

can you give better descriptions other than bad meta data?
 

dflemstr

It's a ball.
Joined
Jul 31, 2008
Messages
2,514
Location
Stockholm, Sweden
Website
Visit site
Pickle said:
hexen and heretic might have been because of missing description text, other than that dont see a problem.
dosbox did have issues with the help file, but i fixed that in the latest version.

can you give better descriptions other than bad meta data?
I didn't say 'bad metadata', but 'missing metadata'. I'll have to check whether this is a bug in my system, or something random (ie. that the PNG image contains '<PXML' randomly), but there must be a reason why the "<PXML" isn't found in exactly that file. Anyways, Pickle, your PND files are OK in general so you shouldn't have to worry... I'll see whether this turns out to be a bug on my side or not (which would be strange since I'm using exactly the same scanning algorithm as in libpnd, just ported to Scala and optimized)
 
Last edited by a moderator:

JayFoxRox

Member
Joined
Aug 3, 2008
Messages
779
Age
31
Location
Hanover, Germany
Website
jannikvogel.de
I'm surprised that you didn't find more than that. I also wrote a similar tool getting the files from EDs dl page and a lot of them failed because they used an "&" symbol in the description without escaping it. However, I also noticed what you found. Besides that the icon sizes vary between 16x16 and 256x256, some PNDs use the author tag for nicknames or who did the port, others for real names and the original devs and some are using the homepage tag for the projects page, others for the authors page and so on.

It's a mess

I said it on the IRC channel before: Drop PND now, improve PXML or switch to some other standard and only keep the PND loader code etc, but make a completly new format like dflemstr suggests for the whole package.
I also had some PXML files which named invalid files as preview pics, others had: "todo" as the german description so it wouldn't even display the english fallback.
I would also like to have some features in the next PXML and PND format.

- So there should definitly be some room to discuss this further which should also be offical (linked from the offical page and make it clearly visible) because it's a change made to the OS which affects everybody.
Possibly a new project in the bug tracker? (- personally I'd like to have at least the projects "Hardware" [for hardware problems - long overdue] and "PND / PXML" [also overdue imo])
 

Pickle

Mega GP Mania
Joined
May 30, 2006
Messages
5,512
Location
Detroit, Michigan
Website
Visit site
dflemstr said:
Pickle said:
hexen and heretic might have been because of missing description text, other than that dont see a problem.
dosbox did have issues with the help file, but i fixed that in the latest version.

can you give better descriptions other than bad meta data?
I didn't say 'bad metadata', but 'missing metadata'. I'll have to check whether this is a bug in my system, or something random (ie. that the PNG image contains '<PXML' randomly), but there must be a reason why the "<PXML" isn't found in exactly that file. Anyways, Pickle, your PND files are OK in general so you shouldn't have to worry... I'll see whether this turns out to be a bug on my side or not (which would be strange since I'm using exactly the same scanning algorithm as in libpnd, just ported to Scala and optimized)

well my stuff is usually just a copy of files from a previous pnd to another, so its possible I miss changing something. But all the tags and formatting should be correct.
.
 
Last edited by a moderator:

dflemstr

It's a ball.
Joined
Jul 31, 2008
Messages
2,514
Location
Stockholm, Sweden
Website
Visit site
JayFoxRox - Look through my posts from the last year or so, I've said exactly that from the very beginning, that PXML needs to be reworked. Everything from the storage format to the PXML syntax. For an "official discussion", please refer to http://pandorawiki.org/New_PND_format

EDIT: but this is no place for such a discussion...
 

JayFoxRox

Member
Joined
Aug 3, 2008
Messages
779
Age
31
Location
Hanover, Germany
Website
jannikvogel.de
Yup, I have seen that wiki page and would like to change it, but I can't seem to login at the wiki anymore - it instantly logs me out again.
However, it I wouldn't have searched for the PND file format description or didn't dislike the store this much, maybe I'd have never come accross it - it should be on some offical platform.
 

dflemstr

It's a ball.
Joined
Jul 31, 2008
Messages
2,514
Location
Stockholm, Sweden
Website
Visit site
Sphinxter said:
Eeek! A few data errors, let's go all scorched earth and start over completely!
Do you have a better idea? That sounds perfectly reasonable to me, considering the large value that "A few" has in this case (accumulated over the last year), and considering how easy it would be to rewrite the format.
 
Last edited by a moderator:

Sphinxter

Says What?
Joined
Oct 1, 2006
Messages
2,897
Location
Silicon Valley California, USA
Website
fullsack.com
dflemstr said:
Sphinxter said:
Eeek! A few data errors, let's go all scorched earth and start over completely!
Do you have a better idea? That sounds perfectly reasonable to me, considering the large value that "A few" has in this case (accumulated over the last year), and considering how easy it would be to rewrite the format.

Yes, fix and validate data in the offending packages.
 
Last edited by a moderator:

dflemstr

It's a ball.
Joined
Jul 31, 2008
Messages
2,514
Location
Stockholm, Sweden
Website
Visit site
Sphinxter said:
Yes, fix and validate data in the offending packages.
Well, yeah, that was what I said in my first post, but JayFoxRox (and me, in my latter posts) refers to problems other than those discovered in this small group of PNDs, that cannot be fixed as easily. (Examples: loading times, fuzzy metadata format, encoding problems)
 
Last edited by a moderator:

fusion_power

Advanced Member
Joined
Dec 25, 2005
Messages
13,173
Location
germany
Website
Visit site
Sounds messy. ^^"
Is ther no input tool, that foces a correct input of the PND Data, otherwiese it will not generate a PND? Should be pretty idiot-proof this way.
 

skeezix

Internal Development
Joined
Mar 11, 2003
Messages
8,064
Website
www.codejedi.com
"version" is required, indeed; I think in absence, assume "0.0.0.0" I suppose :p

jeff

ie: Your system has to work with reasonable, if flawed, PXML.xml, or it ain't going to fly :) (rejecting on upload might help, but more than likely, it'll piss people off; if the PND-file works, with flawed PXML.xml, then your index should handle it as best it can. *shrug*)

Only have 10 mins near a keyboard today, but I threw a version tag into Stella and Hatari, for whenever I do another build of the pnd. Life has been way too busy, forever, damnit :)
 

Exophase

Nothing good will ever come of Exophase.
Joined
Sep 21, 2006
Messages
10,308
Age
38
Location
Cleveland OH
With the scale of the app store would it really be a bad idea to have review and validation required to prevent non-conforming PNDs from even making it in? After reviewing so many after the fact would you be willing to do something like that yourself dflemstr? Then everyone would be forced to learn how to do things right and the app store will be kept tidy.
 

1magus

Member
Joined
Jun 29, 2007
Messages
462
I agree even though I don't have a Pandora but I did notice in the app store that not only are there a lack of screenshots there is a lack of any explanation at all in the notes area when a PND is uploaded.
 

overjupiter

Still Fresh
Joined
Nov 9, 2005
Messages
54
I agree. Validate PNDs during the upload process. If it doesn't validate, report it to the uploader like so:

error.gif
 

cosam

Active Member
Joined
Sep 1, 2008
Messages
703
Location
Netherlands
Website
www.cosam.org
So is there any easy way for packagers to validate their PXML? An XML schema or something? Folk are more likely to use such tools if they're easy to find and use. Even more so if said tool makes appropriate suggestions as to how the PXML should be altered to make it compliant.

skeezix said:
Your system has to work with reasonable, if flawed, PXML.xml, or it ain't going to fly :) (rejecting on upload might help, but more than likely, it'll piss people off; if the PND-file works, with flawed PXML.xml, then your index should handle it as best it can. *shrug*)
Usually I'd agree with being flexible in what you accept but when you're talking about a repository, I think it's reasonable to enforce the spec. Sure, a half-correct PND will work on a default Pandora installation, but what about other tools? Catch the errors at the source, and you'll save a whole lot of grief further downstream.
 
Last edited by a moderator:

multiplex

Member
Joined
Aug 20, 2008
Messages
189
SteveM said:
Usually I'd agree with being flexible in what you accept but when you're talking about a repository, I think it's reasonable to enforce the spec. Sure, a half-correct PND will work on a default Pandora installation, but what about other tools? Catch the errors at the source, and you'll save a whole lot of grief further downstream.
Agreed. The robustness principle is nice where it applies, but accepting invalid data and then forwarding it to every client isn't robust at all.
 
Last edited by a moderator:
Top