Two-Way Pndstore

[Esn]

How feasible would it be for PNDstore to not only transfer PNDs from the Repo to your Pandora, but to do it the other way around as well? Two-way communication. This would create a truly complete database of all PNDs, even obscure ones.

As a side benefit, this would further reduce the need for Craig's Pandora Apps site, which right now despite its many flaws still holds the title of having the most complete collection of files in one place (the wiki has a more complete collection of links to files, but links can sometimes go dead... also, Pandora Apps only ever updates after the wiki does, but I digress).

I'm thinking of two potential routes:

1. A central server to keep all PNDs, which is updated from PNDstore. Upon opening PNDstore, a scan is made of the user's PNDs and checked against the database on the site (PNDstore already does this). If a PND is found that is not on the site, it is uploaded from the Pandora onto the site, and is then available for others to download when they use PNDstore.

1.b (alternative). A central server that doesn't keep all PNDs (just because this would require a lot of bandwidth, and some like Wesnoth are 300mb), but only the ones that it recognizes as being absent from other sites that adhere to the PND Repo spec (so only milkshake's repo, at present). So a kind of "backup". If it sees that an identical PND is uploaded onto the Repo, it is deleted from the site, to keep costs low.

2. A peer-to-peer method, like torrents. A central database is still kept somewhere, but the actual downloading takes place not directly from a server but from other Pandora users, if they're online. The database keeps track of which Pandora has which PNDs. Maybe the database itself has no fixed location, but is updated peer-to-peer.

This doesn't have to be PNDstore of course, but could be another program. And of course, I'm not a dev, so there are probably lots of things I haven't thought of. But how's the overall idea?

 

[bzar]

Isn't milkshake's repo pretty much the option #1? Why wouldn't you upload the PND normally through the repo site? How would authentication work?

EDIT: I mean, I get what you are getting at: getting all the PNDs to one spot by automatically sending unknown ones to the repo, but I think this is a bad idea. I know I have a bunch of in-development PNDs on my SD cards that are not ready for circulation. When they are, I'll upload them myself. Milkshake's repo AFAIK scrapes PNDs from the other sites already (at least I remember it was supposed to), so the only ones missing are the ones only on some users pandoras. Would this really be necessary to get those?

 

[Esn]

I know I have a bunch of in-development PNDs on my SD cards that are not ready for circulation. When they are, I'll upload them myself.

So you could be able to opt-out of uploading any info from your Pandora then, or mark certain PNDs as "off limits" in the program. Maybe have a certain folder that you can put any off-limits PNDs into.

By this rationale, peer-to-peer file sharing is a bad idea because you might accidentally share something that you didn't want to. Yes, it does happen, but the benefits of information exchange far outweigh the negatives. Especially in open-source communities.

I think that most Pandora users will not have any PNDs that they don't want to share, because they're not devs. So even if all devs opt out, the program will still have plenty of users.

Milkshake's repo AFAIK scrapes PNDs from the other sites already (at least I remember it was supposed to), so the only ones missing are the ones only on some users pandoras. Would this really be necessary to get those?

It does not do this, and as far as I know it is impossible for it to do this. How would it "scrape" a PND that was uploaded to a private site, or Mediafire, or a forum attachment? Yet some people release their PNDs precisely this way (release, not private). It doesn't scrape the File Archive or Pandora Apps either.

PNDstore currently gives a checklist that tells if a PND is on the repo, or if it's only on my Pandora. Well over half the PNDs on my Pandora are not currently on the Repo.

What my idea will accomplish if implemented is that any PND released publicly will be available on one list. Private in-development PNDs should stay off-limits of course, but if a dev releases his PND to the public, I think it is clearly beneficial to make sure that it is not forgotten just because it was released in some obscure place. That's what I've been trying to do with the wiki software lists over the past year - it's the same thing.

How would authentication work?

I don't know; I'm not a dev... How's it done on the repo?

 

[bzar]

So you could be able to opt-out of uploading any info from your Pandora then, or mark certain PNDs as "off limits" in the program.

Okay, I'll give you this one. This would solve the issue.

It does not do this, and as far as I know it is impossible for it to do this. How would it "scrape" a PND that was uploaded to a private site, or Mediafire, or a forum attachment? Yet some people release their PNDs precisely this way (release, not private).

I see your point.

PNDstore currently gives a checklist that tells if a PND is on the repo, or if it's only on my Pandora. Well over half the PNDs on my Pandora are not currently on the Repo.

What my idea will accomplish if implemented is that any PND released publicly will be available on one list. Private in-development PNDs should stay off-limits of course, but if a dev releases his PND to the public, I think it is clearly beneficial to make sure that it is not forgotten just because it was released in some obscure place. That's what I've been trying to do with the wiki software lists over the past year. It's the same thing.

Uploading to the repo if you're not the original packager or developer is currently forbidden (check the terms that come up when trying to upload a PND) for reasons debated in earlier threads.

Basically I can't at this point provide a good argument against the idea except for a few anecdotes about badly (not) maintained PNDs, polluted PNDs and such, but they could be easily turned to apply to the current system as well, so I won't bother making them

I'll wait 'til other opinions surface

 

[Esn]

Uploading to the repo if you're not the original packager or developer is currently forbidden (check the terms that come up when trying to upload a PND) for reasons debated in earlier threads.

I'd like to see a link to those discussions; I'd like to know the reasons. I'm thinking maybe it has something to do with being able to control the description, and being given due credit?

A thought: once a PND has been packaged up, is it possible for someone who's not the original developer to reopen it, change it, and package it up again? If that was possible, that would be a potential problem. Although it would be a problem in the current system as well, because someone who does that would be called a packager and could upload another version of a program to the Repo.

 

[CPUnltd]

generally sounds interesting... will read more thuroughly when I make time to do so. I KNOW I will have input and questions on feasibility...

 

[Tempel]

There's a serious problem with redistribution rights. If you have a PND, you don't necessarily have the rights to share that PND with anyone else. Most currently are redistributable under FOSS licences, but not every one will be.

Uploads to the repo should only be made by the packager so as to establish ownership. On a technological level, I understand that the repo has no way to transfer ownership from one user to another; whoever uploads it first therefore owns that package. But more importantly, it is in fact trivial to open a PND to make a new, slightly modified version of it; only the package creator should be uploading so that you know who to blame if it steals your credit cards and kicks your cat.

And on a related note, the repo did have the ability to scrape the appstore and file archive, but this was disabled on its official launch for the above reasons.

The wiki avoids all of these issues by only linking to PNDs, but it has other limitations which you mentioned.

 

[streak]

This could be usefull as long as pndstore will not suck my PRAWN collection into pnd store repo..

 

[milkshake]

There's a serious problem with redistribution rights. If you have a PND, you don't necessarily have the rights to share that PND with anyone else. Most currently are redistributable under FOSS licences, but not every one will be.

Uploads to the repo should only be made by the packager so as to establish ownership. On a technological level, I understand that the repo has no way to transfer ownership from one user to another; whoever uploads it first therefore owns that package. But more importantly, it is in fact trivial to open a PND to make a new, slightly modified version of it; only the package creator should be uploading so that you know who to blame if it steals your credit cards and kicks your cat.

And on a related note, the repo did have the ability to scrape the appstore and file archive, but this was disabled on its official launch for the above reasons.

The wiki avoids all of these issues by only linking to PNDs, but it has other limitations which you mentioned.

what he said...
the original intention was to parse the app store .. still have the code for that BUT the more I thought about it the worse the idea sounded see tempels quote for my shared reason why this would not be a good idea in practice.

 

[Esn]

The wiki avoids all of these issues by only linking to PNDs, but it has other limitations which you mentioned.

Wait, haven't US courts decided that linking is the same thing as distributing, and that therefore links are illegal as well? Or something to that effect...
http://en.wikipedia.org/wiki/Hyperlinking#Legal_issues

Linking to illegal or infringing content can be sufficiently problematic to give rise to legal liability.

That's why we can't link to ROMs on this board. I don't see why the wiki should, in principle, be any different. The same copyright rules apply.

I guess there's nothing in the PND that says what it's licensed under? I know that the File Archive asks me to agree to a license before I can download some PNDs, which the Repo never does... how does that work for the File Archive; is the licensing information entered separately from the PND?

Because if the license was in the PND, the app I'm proposing could simply ignore any PNDs whose license doesn't permit them to be distributed. Or for those that require you to click "yes I agree" to the license, have a dialogue before you can download them, like the Archive. Problem solved.

 

[milkshake]

I think only direct linking is illegal right? Not linking to the app page or whatever, dunno.

 

[Tempel]

Linking to infringing content may be disallowed under US law; there's nothing illegal about linking to a PND that the creator has deliberately made available.

But now that you mention it, there is indeed a licence element that can be optionally included in a PXML. In theory, an uploader program could check this licence, and only upload if it's a licence known to allow redistribution. But I don't think this element is widely used, and the breadth of available licences could make it hard to implement it.

I just had a better thought: we don't need Craig to implement the repo spec for the app store. Scrape the site to collect info on the PNDs it holds, then produce a repo file; have each URL point to the download URL on the appstore. This avoids potential copyright infringement by making users download directly from the source (presuming, of course, that Craig's app store has the rights to redistribute in the first place). Anyone should be able to do this, though I imagine milkshake has a head start on making it happen. Seem feasible?

 

[milkshake]

Seem feasible?

I guess it could be done but the parser I wrote only locates the download URL, that is all my site needs to download the pnd and process it.
The problem is, the information on the app-store is sometimes incorrect or missing i.e. bad version numbers etc

So in essence using Tempels idea of not downloading but creating a db and or JSON output based on "scraped" data this will just lead to issues in the client itself because it cant verify the app id or even trust the version number, so until the client were to get the file and parse it with libpnd or the clients backend on the pandora it wouldn't know if it was an updated app or an old app etc, also it cant tell pre-download if the PXML is valid so even if it can download if the PXML is screwed what will the client do with it?

also in regards to the repo, its implemented in a way it is to encourage correct use of PXML specs which is what we want in the long run, I know apps are missing but not all apps in the app-store follow the specs, and craig doesn't seem to mind plus most apps devs cant upload and correct it because they didn't upload it (issue after issue).

I still feel strongly about only letting the dev upload their own apps, however if someone has left the scene and the license doesn't prevent it, I can't see any reason why someone couldn't re-package the PND with up to date valid PXML data and upload it to the repo, but making it clear from maybe the description or "additional info" that its a repackaged game that if it offends the original Dev it can be removed.

I believe that if someone creates a game or an app, they should be able to choose how it is distributed, craigs app-store doesn't do this and I feel thats why alot of devs do not upload there.

 

[torpor]

Meh.

I'm in the "lets create a Pandora-owners VPN that every single Pandora can access, with private and public areas" camp.

There is more that this system could be used for than just plain ol' piracy of dodgy roms.

Besides that, there *ARE LEGAL ROMS* and it is perfectly acceptable for me, for example, to share my big fat pile of cool Oric-1/Atmos ROM's, with authors permission and all rights to share granted.

 

[Tempel]

Ah milkshake, you misunderstand me. What I'm suggesting is that a server downloads all the PNDs in the app store, and extracts information from them, exactly as your site does. Then, that server forms the repo file, exactly as your site does. The trick is that the server does not send those PNDs to users; it sets the "uri" field for each package to point to the original download location from the app store. That way, end users still get the PND from the original source.

Yes, there could easily be issues with PXML validity, and the repo really is better. But this helps act as a stop-gap, encouraging people to use the repo spec and getting them used to the idea of installing and upgrading packages in a consistent manner. I think this will help make people gravitate towards your repo, since it lets them use both, but yours provides the better experience. Also, it would let us gloat about building an auto-updater for Craig's own site before he does .

And milkshake, I think you're in the best position to make this happen since you already have code that can scrape the app store, parse PXMLs, and generate repo files. But of course, you don't need to do this if you don't want to. Maybe someone else would be willing to step up.

torpor: I'm not quite sure what you're getting at. We are talking about PNDs here, not ROMs. But in both cases, there will be some you can legally share, and some you can't, but it's not always easy to tell the difference. Could you clarify your suggestion?

 

[Esn]

Linking to infringing content may be disallowed under US law; there's nothing illegal about linking to a PND that the creator has deliberately made available.

So there's nothing illegal about linking to that PND, but it is illegal to send someone that PND? I'm trying to figure out why you see no problem with the wiki but you do see a problem with the proposal in the first post.

But now that you mention it, there is indeed a licence element that can be optionally included in a PXML. In theory, an uploader program could check this licence, and only upload if it's a licence known to allow redistribution. But I don't think this element is widely used, and the breadth of available licences could make it hard to implement it.

How does the File Archive handle the many different licenses? It has already been done, so there must be a way to do it. For that matter, the File Archive seems to be more within the law than the Repo is, because it forces you to agree to a license for certain PNDs that require it, which the Repo never does. So if we're talking about staying within the law, it doesn't seem like you guys are perfect at the moment.

As for not being "widely used", if someone doesn't care enough to properly fill out the license, then they obviously don't care what is done with the PND, that's how I see it. If the program I propose becomes widely used, anyone who does care will start filling that part in. I imagine that people who sell commercial PNDs would care, for example.

I still feel strongly about only letting the dev upload their own apps,

I believe that if someone creates a game or an app, they should be able to choose how it is distributed

There's that copyright issue again. Let's get a few things clear. The vast majority of PNDs are ports from other sources, and so the dev who ported it does not have primary say over the license. Whether it's one person or another who uploads an app - I don't see how the law disallows it as long as whoever ported it follows the terms of the original license.

If a dev makes a program available, and its license doesn't say anything about only being allowed to download it at the original site that it was uploaded to, why should there be artificial restrictions put in place? I just don't get it.

I'm trying to figure out what law or code of ethics you guys basing your views from.

torpor: I'm not quite sure what you're getting at. We are talking about PNDs here, not ROMs. But in both cases, there will be some you can legally share, and some you can't, but it's not always easy to tell the difference. Could you clarify your suggestion?

Indeed, we're talking about PNDs here. However, milkshake and Tempel brought up the issue of copyright law, so I used the analogy of ROMs. We can think of infringing PNDs in the same way that we think of ROMs.

 

[milkshake]

How does the File Archive handle the many different licenses? It has already been done, so there must be a way to do it.

because in the file archive you select the license from a drop down box, but not all licenses are available especially if it is a custom license.

the file archive doesnt make u accept a license for everything, it depends on whether one was selected when it was uploaded.

 

[Tempel]

I'm certainly not a lawyer, but here's my understanding of copyright law. It's based on my knowledge of Canadian and US law, but I think the basics of copyright law are fairly consistent in most of the developed world.

So there's nothing illegal about linking to that PND, but it is illegal to send someone that PND?

That's correct. Copyright holders reserve all rights unless they explicitly release those rights. That includes distributions rights. Free software licences explicitly give redistribution rights to anyone, but standard copyright does not. That's why sharing a PND is illegal unless explicitly allowed.

But if the copyright holder makes a PND available for anyone to download, it's perfectly legal for anyone to direct anyone else to that download location. From a technical perspective, this distinction is important since the copyright holder still controls what people are getting. The copyright holder could modify or remove the download at will.

For that matter, the File Archive seems to be more within the law than the Repo is, because it forces you to agree to a license for certain PNDs that require it, which the Repo never does. So if we're talking about staying within the law, it doesn't seem like you guys are perfect at the moment.

I'm unsure about this. When an app specifies its licence, the repo does show that licence information in the package contents. But unlike the File Archive, it doesn't force you to acknowledge the licence. I think that a licence still applies even if not explicitly agreed to, but I'm not certain. Regardless, the Linux package managers with which I'm familiar do the same: the licence is there to be seen, but you're never explicitly asked to agree to it.

As for not being "widely used", if someone doesn't care enough to properly fill out the license, then they obviously don't care what is done with the PND, that's how I see it. If the program I propose becomes widely used, anyone who does care will start filling that part in. I imagine that people who sell commercial PNDs would care, for example.

Unfortunately, that can't legally be assumed. Copyright is granted to a creator immediately on creation, and all rights are reserved unless explicitly released. I'm sure there are many creators who wouldn't mind having their content shared, but if they haven't clearly said so, you can't share it without opening yourself to infringement liability.


So there are the legal issues I see. I still think that a repo file for the app store is better since Craig and co. try to get every PND they can on there, and they can deal with copyright and licencing (or maybe even a repo file generated from wiki contents?). But as a passing thought, I'd like to mention that the US at least has safe-harbour provisions: if a user uploads infringing content to a site, the copyright holder must request that the site remove the content before the site can be held liable. So maybe we don't need to err on the side of caution here. But once more, I'll express the fact that I'm not a lawyer.

 

[torpor]

A PND is no different than any other file format you upload and share on the Internet. If you put a PNG file up on Imageshare, your agreement with them says they can do things with the file. Same goes for any other media-sharing web sites. So, the agreement is always between the uploader and the site, however that occurs. If it doesn't occur - i.e. its completely open-ended, thats one thing - but it really is up to the site and the uploader what the license will be, and what will occur if the copyright is found to not belong to the uploader (i.e. piracy).

For the PND Store direction, I'd personally like to see 'our app store' be free, easy and open, as well as supporting *real* commerce between people who are extremely happy about the software being written for them, by those who are happy to be writing extremely cool software. We should have some way to reward our heroes. Whatever PND Store includes the most successful ways to reward heroes of this small community, gets my vote. I've seen plenty of things so far that I'd like to pay for, on Pandora, to keep the effort happening and see new, regular, releases .. like I see on other platforms.

 

[WizardStan]

Linking to infringing content may be disallowed under US law; there's nothing illegal about linking to a PND that the creator has deliberately made available.

So there's nothing illegal about linking to that PND, but it is illegal to send someone that PND? I'm trying to figure out why you see no problem with the wiki but you do see a problem with the proposal in the first post.

Some people will upload their PNDs saying "this PND can only be downloaded from my site." Lots of reasons they might do this: maybe they want to keep an accurate count of who is using it; maybe they get ad revenue when someone visits their site; maybe they like having the control to throw updates or pull things down at an instant. Whatever the reason, they want all downloads to be through their site.
My understanding of how the courts have interpreted copyright law is that linking directly to the PND is effectively the same as hosting it yourself, which in this case with the original author's request would be bad. Linking to the site where the PND can be found however should be OK, otherwise Google would be in a lot of trouble.