Hi everyone.
With Ivanovic's recent updates to the PXML spec, I've made some changes to the PND repository spec that's being used in milkshake's repo. Currently, milkshake and I are the only people using this - his site as a producer, my client as a consumer - but I thought it'd be good to get some feedback before nailing it down. You can see my proposal on the wiki: http://pandorawiki.org/PND_repository_specification
I designed this so that it's capable of holding just about all the information that's in a PXML, but can still be very functional with much less. This way, it'll be easier for a site to implement even if it doesn't read PXMLs (like Craig's app store, or the file archive).
The downside is that it can get quite large. PNDs commonly hold just one application, which will have the same id, version, author, title, description, and icon as the package as whole; in this case, all that information will be included twice. This can be worked around if the repository skips the application list in such cases, but that could be getting needlessly complicated.
Also, I'm not sure about the "approved" list or the "rating" value. I threw them in to get some discussion on them, but might want to remove them for version 2.0. They could be re-added for version 2.01+, as long as they're optional so as to avoid issues with backwards compatibility.
Thanks in advance for any feedback.
Randy Heydon Tempel
On Wed, 23 Mar 2011, Randy Heydon wrote:
# With Ivanovic's recent updates to the PXML spec, I've made some changes # to the PND repository spec that's being used in milkshake's repo. # Currently, milkshake and I are the only people using this - his site as # a producer, my client as a consumer - but I thought it'd be good to get # some feedback before nailing it down. You can see my proposal on the # wiki: http://pandorawiki.org/PND_repository_specification
gzip is discouraged? why say that? (for most implementations I'd imagine its invisibly handled by webserver and framework)l or is there some reason?
The 'trust' approval thing sounds fishy; ie: how will a client know anything about these people, or how to trust them? or if an account gets decommissioned over time and a new guy with the same name comes along .. sort of begs a whole other REST API to inquire about users by their own unique-id and so on.
I like to fall back to specs being 'minimal' -- that sounds 'site specific', so keep it out of the spec; a site is free to add _Additiona_ values to its own API withou tbreaking any consumer .. maybe that shoudl be codified in the spec -- the assumption that consumers will parse for known values, and not consider unknown key/values an error.
But sometimes I do go too minimal, so I could well be wrong :)
Same question for 'rating'; is that too specific? ie: I could see one site uses 5-star, another using 100% .. they could map to some value system using this spec though, so it sounds doable; but just not sure its appropriate.
I guess the question is .. is this for information culled from pnd,s or site generated? and if both.. shoudl that be two separate feeds.. a pnd repository feed, and a related pnd site specific feed? That might be too loose or krufty, but I think I prefer hybrid .. let sites augment.
But at the same time .. if you have each site augmenting with their own thing .. that can get goofy. In other specs, its common for you to have a minimal core everyone can agree upon, and then domain specific extensions; if at some point the domain specific guyds can agree that yeah, this is really good we should all do it, then the spec goes to v4 and everyone adopts.
But i the need is present at the onset build it in..
So I'd guess..
-- drop the trust stuff -- keep the rating
Sites can always add their own rating system in, or map it to something native. The trust thing sounds really too fragile and specific.. sites might do something very different entirely.
Licenses -- I forget offhand, but does that map to how we have the PXML licenses? ie: license and source are tied there, but not in this proposal;
You may want to define what RESTY operations are available specifically; can an app enquire specifically about an app in the repo by itself to get additional details, etc?
jeff
-- If everyone would put barbecue sauce on their food, there would be no war.
gzip is discouraged? why say that?
I don't know! dflemstr must have written that when he first made the spec, and I didn't really notice it. I've removed the "and is discouraged".
The 'trust' approval thing sounds fishy; ie: how will a client know anything about these people, or how to trust them? or if an account gets decommissioned over time and a new guy with the same name comes along
I figured the user would know at least a few trustworthy people (eg. through the forums) and would tell the client program to trust them. Granted, there's no indication that a forum username would match a site username, or that usernames would stay constant; this problem even extends to the "vendor" field.
This wouldn't be a problem for sites that use a more centralized approval system (both can work the way I've layed it out here). Still, I agree it's pretty wacky, and I'm fine putting off validation stuff for a little while. I've heard of no malicious PNDs yet, and hopefully they'll hold off for a while longer.
I like to fall back to specs being 'minimal' -- that sounds 'site specific', so keep it out of the spec; a site is free to add _Additiona_ values to its own API withou tbreaking any consumer .. maybe that shoudl be codified in the spec -- the assumption that consumers will parse for known values, and not consider unknown key/values an error.
Yeah, I think sites should be able to add site-specific fields of the form "x-sitename-field", so as to avoid conflicts. I've added a note on this to the preamble.
Same question for 'rating'; is that too specific? ie: I could see one site uses 5-star, another using 100% .. they could map to some value system using this spec though, so it sounds doable; but just not sure its appropriate.
I figured mapping to a 100-point scale isn't too difficult. That's what Metacritic does. And repo maintainers can choose how to do the mapping themselves (or not include a rating at all).
-- drop the trust stuff -- keep the rating
I'm leaning towards that. Let's wait a bit for any further comments, then I'll make that change.
Licenses -- I forget offhand, but does that map to how we have the PXML licenses? ie: license and source are tied there, but not in this proposal;
Yeah, it's not an exact match. In the PXML, each license element can have a source attribute. I couldn't figure out how to make a clean mapping from the PXML license element to here without having a lot of unnecessary data, so I simplified. Multiple licenses make sense to me, but multiple source code URLs don't, especially with one tied to each specific license. So I figured the repository should pick one link and make that the source link. But I just realized I could make it a list of source links; they still wouldn't directly correspond to individual licenses, but at least they would all be there. How does that sound?
You may want to define what RESTY operations are available specifically; can an app enquire specifically about an app in the repo by itself to get additional details, etc?
I hadn't considered it. I figured a client would get the whole thing once, then parse it offline. This is how all Linux package managers I've seen work, so I thought it would suffice. Is there any real use to having specific inquiries?
Thanks a lot for the feedback Jeff. Anyone else?
Randy Heydon Tempel
On Wed, 23 Mar 2011, Randy Heydon wrote:
# Yeah, I think sites should be able to add site-specific fields of the # form "x-sitename-field", so as to avoid conflicts. I've added a note # on this to the preamble.
Perfect; a 'defined' extension mechanism.. win :) I didn't check the wiki, but make saure it has its own bullet point, so its not lost.
# > -- drop the trust stuff # > -- keep the rating # I'm leaning towards that. Let's wait a bit for any further comments, # then I'll make that change.
Woot!
# > Licenses -- I forget offhand, but does that map to how we # > have the PXML licenses? ie: license and source are tied there, but # > not in this proposal; # Yeah, it's not an exact match. In the PXML, each license element can # have a source attribute. I couldn't figure out how to make a clean # mapping from the PXML license element to here without having a lot of # unnecessary data, so I simplified. Multiple licenses make sense to me, # but multiple source code URLs don't, especially with one tied to each # specific license. So I figured the repository should pick one link and # make that the source link. But I just realized I could make it a list # of source links; they still wouldn't directly correspond to individual # licenses, but at least they would all be there. How does that sound?
Could it get by with just a list of known licenses across source and not list source links; or if the source references should be in there, then make it map to the PXML so we don't run into chaos. If too hard, then just a list of licenses, and a list of source references, and milkshake can note the consmer has to figure out which are which.
"App is under BSD and GPL licenses" "Sources are: (1) ... (2)..."
# > You may want to define what RESTY operations are available # > specifically; can an app enquire specifically about an app in the # > repo by itself to get additional details, etc? # I hadn't considered it. I figured a client would get the whole thing # once, then parse it offline. This is how all Linux package managers # I've seen work, so I thought it would suffice. Is there any real use # to having specific inquiries?
Put it into the spec; RESTy often implies GET PUSH etc, so should define which are supported.
# Thanks a lot for the feedback Jeff. Anyone else?
Right back at you for working on it :)
jeff
-- If everyone would put barbecue sauce on their food, there would be no war.
Could it get by with just a list of known licenses across source and not list source links; or if the source references should be in there, then make it map to the PXML so we don't run into chaos.
I'm not sure what you mean here. Could you clarify?
If too hard, then just a list of licenses, and a list of source references, and milkshake can note the consmer has to figure out which are which.
"App is under BSD and GPL licenses" "Sources are: (1) ... (2)..."
Yep, this is what I was saying. Better than my current single source link, but still doesn't capture all of the nuances of the PXML.
# > You may want to define what RESTY operations are available # > specifically; can an app enquire specifically about an app in the # > repo by itself to get additional details, etc? # I hadn't considered it. I figured a client would get the whole thing # once, then parse it offline. This is how all Linux package managers # I've seen work, so I thought it would suffice. Is there any real use # to having specific inquiries?
Put it into the spec; RESTy often implies GET PUSH etc, so should define which are supported.
The spec mentions that GET and HEAD requests must be available on the main repo data. There's currently no other data to be accessed, and no way to modify the data through the spec (ie. no need for PUT, POST, or DELETE), so I don't think there's a need to say anything more on them. Then again, my knowledge of REST is entirely from some quick reading early this morning, so I might not know what I'm talking about.
I hope that's about it. I want to firm this up quickly so milkshake can get to implementing it and I can get back to work. I thought my thesis was almost done, but it turns out my advisors disagree, so I'll be strapped for time for a few more weeks. But maybe I shouldn't be complaining about how busy I am to Jeff :P
Randy Heydon Tempel
On Thu, 24 Mar 2011, Randy Heydon wrote:
# Yep, this is what I was saying. Better than my current single source # link, but still doesn't capture all of the nuances of the PXML.
It doesn't really have to capture all of PXML imho; it serves a different process.
ie": If it does have to capture all of PXML, just paste the PXML in there.
(Hell why not .. store the PXML in the DB as raw, and include it in the repo spec. Or have another GET tht gets the full PXML.)
ie: The PXML is the PXML; the repo spec is for repos .. it should include enough detali to get repos going and other useful apps, but for apps that want the full monty.. let 'em go to the PXML; thats what its for. Why duplicate it?
# The spec mentions that GET and HEAD requests must be available on the # main repo data. There's currently no other data to be accessed, and no # way to modify the data through the spec (ie. no need for PUT, POST, or # DELETE), so I don't think there's a need to say anything more on them. # Then again, my knowledge of REST is entirely from some quick reading # early this morning, so I might not know what I'm talking about.
OKay; I like to be clear ..
"the APi includes the following methods"
GET -> blah
HEAD -> blah (what does head to different than GET)
"Other operations are an error."
# I hope that's about it. I want to firm this up quickly so milkshake
Seems okay, pretty much as you thought :)
# can get to implementing it and I can get back to work. I thought my # thesis was almost done, but it turns out my advisors disagree, so I'll # be strapped for time for a few more weeks. But maybe I shouldn't be # complaining about how busy I am to Jeff :P
Feel free to until 6 months from now, where I have twins showing up. I'm busy now, but my sense of busy is about to be redefined :)
jeff
-- If everyone would put barbecue sauce on their food, there would be no war.
With no more comments forthcoming, I've finalized version 2.0 of the repository spec. As per Jeff's suggestions, I dropped the "approved" field, changed "sources" to a list of URIs, and added a few clarifications. You can see it at the same place as always: http://pandorawiki.org/PND_repository_specification
I'll be coding my client to this spec, and milkshake will do the same for his site. Anyone else who might want to use it, now is your chance (like ED and the file archive maybe, hm?).
Feel free to until 6 months from now, where I have twins showing up. I'm busy now, but my sense of busy is about to be redefined :)
Remind me to buy you a beer before then :)
Randy Heydon Tempel