My ideas for changes to the way apps work.

[PokeParadox]

I agree with making things easy and simple.


I disagree with reinventing the wheel.


As Skeezix mentions there are ideas to make scanning quicker and these ideas could be applied to the current systems and progress what we have rather than starting from scratch...


Sure the scanning system probably should get some hints from the the filename, but that's all the should be "hints". If a user renames a file, then BAM your versioning system would be broken. I think imposing that anything that doesn't have a PND file extension is ignored by the scanner would be doing plenty to speed things up if it is not already doing that.


The current tools just need tying together and you will have your simple update system... make sure these tools are in the next hotfix or OS release and BOOM, we're good to go.

 

[Tempel]

This is preposterously complicated.


All you have to do in minimenu is the following and we all win and it's the most efficient AND optimised solution:


Scan the root and one level deep for PND files.


If the app name contains a source, call the URL and see if the file is the latest version (you can of course have an option to disable this).


That's it.


I don't understand this obsession with wanting to make this over complex. It is what is killing the user friendliness of the Pandora and seriously - it's upsetting me to see this situation becoming an issue for new users.

I don't understand what makes Jeff's suggestion complicated (other than the fact that he tends to use more words than he needs to ).


First off, the issue of which directories to scan is distinct from the issue of automatic updates. Scanning an SD card's root directory is fine, I don't care.


But when it comes to automated updates, I don't see how the repository specification is any more complicated than your suggestion. To the end user, it's all the same: the system checks for updates and lets you know if they're available. The repository spec is openly available, well-documented, and already working well, so why not use it? Do you not like the way PNDstore works? That's fine; since the spec is well-documented, anyone could make a new one. It could be integrated with minimenu without trouble. Power users could have a command-line app to handle it. And all of these tools can work together since they all follow the same specification.


In conclusion: You're trying to make a specification to help automatically update PNDs. There is already such a specification, and it works well. Please don't reinvent it unnecessarily.

 

[WizardStan]

If the app name contains a source, call the URL and see if the file is the latest version (you can of course have an option to disable this).

No system should ever tell me how to name my files unless it truly is absolutely necessary. I can rename my ROMs to anything I want, I shouldn't be restricted in the filenames of my PNDs either.

 

[dsleaf67]

While I don't agree with polluting file names either, I'd like to clarify that I think the main part Craigix wants in file name is what repo it came from (as well as version). However that would mean possibly only looking up that website it came from, even if there's a newer version on a different site. Oh and that different site could have different file names so even more to watch out for.


I agree that the way things are going with the standards it would soon be closer to link repositories for pnds which I think would be great.


Only gripe I have is, I wish there was easier way to open (unpack etc. on windows) for editing fixing or just learning from ports. I have tried getting squash on windows with no luck.


PS Skeezix, can you fix (or did you fix bug when typing letter to go to first app/folder) that if folders exist that start with letter, it never checks apps just cycles folders (of course when you have chosen option to display apps in category as well as folder).

 

[skeezix]

Done!


Checked a change into libpnd, so that root (and onesubdir of root) will be scanned; this is in libpnd, and invisible to applications using libpnd, it 'just works'.


(needs to be tested, but thats why we have betas of hotfixes before they release, right?


It is configurable; so now a searchpath is a list of dirs to scan separated by colon (defaulting to recursing down each piece), it now has optional <n> argument for each piece; so I've prepended to the _menu_ searchpath "/media/*/<1>:" which means to look in root, and no more than 1 subdir deep a scan, and then it goes onto the rest of the searchpath (/media/*/pandora/menu, and so on)


Thats the one piece, down


jeff


dsleaf67 -- was not aware of that little buglet; seems true. Can you open it up on the bugtracker, for record keeping? http://bugs.openpandora.org/

 

[meandu229]

Would changing the file name not break .ovrs? I use them alot so I would have pidgin-p1v.pnd and pidgin-p1v.ovr. ohhh and update pidgin-p2v.pnd now I have to go and rename the ovr file? doesnt sound useful especially if there are quite alot of updates at once

 

[centus]

Just one comment beyond the already too long discussion.


I hope we all agree that there should never be automatic application update, it has to ask the user if they want to update (even iTunes asks if you want it to be updated and just notifies that an update exists).


By the way, great work Jeff... I only wish I had my Pandora to test it out...

 

[Scntrblob]

Done!


Checked a change into libpnd, so that root (and onesubdir of root) will be scanned; this is in libpnd, and invisible to applications using libpnd, it 'just works'.


(needs to be tested, but thats why we have betas of hotfixes before they release, right?


It is configurable; so now a searchpath is a list of dirs to scan separated by colon (defaulting to recursing down each piece), it now has optional <n> argument for each piece; so I've prepended to the _menu_ searchpath "/media/*/<1>:" which means to look in root, and no more than 1 subdir deep a scan, and then it goes onto the rest of the searchpath (/media/*/pandora/menu, and so on)


Thats the one piece, down


jeff


dsleaf67 -- was not aware of that little buglet; seems true. Can you open it up on the bugtracker, for record keeping? http://bugs.openpandora.org/

Just for clarification: I have a directory "genesis" in my root with a "few" files in there. Does this now make the time longer everything needs?

 

[centus]

Sounds like a folder exclusion option would be handy...

 

[meandu229]

Sounds like a folder exclusion option would be handy...

Well I assume anyone who would want to exclude would already be using /pandora/menu etc so would just revert the config back to that thats what ill be doing I have no issue with it being set to root + 1 directory but I hate a messy SD card so I will be using pandora/menu still and disabling the new change,


If I have a pnd in the root where will it go? menu/desktop?

 

[bzar]

^ With skeezix's latest patch, wherever you set it to go in /etc/pandora/conf/desktop (IIRC the file). By default it goes to the menu.

 

[milkshake]

@craigx


I dont know if you have been following the progress on the boards but, what skeezix what saying in laymans terms is that we almost have an "apple app store" of our own and it could soon be part of the core firmware meaning it will be easy for "new"/"man on the street" users to get apps and not worry where the files are saved because the PNDstore (or alternative) will take care of this for them.


heck skeezix was even saying it could be implemented directly into mini-menu (once the software is out of "testing") making it even easier.


I don't mean to sound rude but, you ether don't "get" whats happening in the community right now (which is understandable because you are really busy making the best handheld in the world), dont understand what skeezix/pickle/ESN are referring to, or you are just not listening.


I like to think its one or both or the first 2.


As far as i'm aware the people working on new things for the pandora are not in the habbit of trying to make things harder for people to use (even if they live in the terminal); I mean why would they do that?


The stuff thats happening in the background from the community members is going to make getting apps so much easier and user friendly (for the man on the street), which in turn will make the console much more inviting for people who might be scared of linux, and in the end hopefully will net you guys a profit which you deserve.


hope you don't think i'm bashing you because i'm not.

 

[pmprog]

Pandora Apps adds a key to the file names which are downloaded eg:


'Snes9x.pnd' would become 'Snes9x.Pv1.PND', doing this removes the need to 'repack' the PND, but marks it so minimenu can see where the app has come from 'P' (for pandora apps) and the version of the app (v1). It's quick and simple.

Could you not use a CRC or Hash of the file to identify versions? Otherwise you potentially have several "version 1"'s that came from different sites, but are the same app

 

[milkshake]

Could you not use a CRC or Hash of the file to identify versions? Otherwise you potentially have several "version 1"'s that came from different sites, but are the same app

yes but as many others have said why do stuff that is not needed? The Pandora can already get the app version from the embedded PXML (thats what the spec was created for, not for fun and giggles).


Also if a CRC hash was used that doesn't rule out the possibility that the app may have come from another repo as the files could be identicle, anyway the repo spec deals with this by also sending an MD5 hash of the file.


As said before the wheel does not need to be re-invented, all this stuff is already there ready to be used.

 

[Emnasut]

Could you not use a CRC or Hash of the file to identify versions? Otherwise you potentially have several "version 1"'s that came from different sites, but are the same app

yes but as many others have said why do stuff that is not needed? The Pandora can already get the app version from the embedded PXML (thats what the spec was created for, not for fun and giggles).


Also if a CRC hash was used that doesn't rule out the possibility that the app may have come from another repo as the files could be identicle, anyway the repo spec deals with this by also sending an MD5 hash of the file.


As said before the wheel does not need to be re-invented, all this stuff is already there ready to be used.

If scanning the files takes too long and thus Craig wants a filename based scanning, maybe we could use something like a lookup table for caching? I.e. we save the names/path of all PNDs on the card together with its MD5 (which is already in the PND?) and the version number in a single file.


Then:


- Looking for newer versions in any repo is done every hour/day/when a internet connection becomes available/whenever but only for files in the table.


- When inserting an SD card the whole card is scanned but only a DIR scan with filenames which - according to Craig - only takes microseconds. If a filename is not in the table the table checks if the MD5 is in it and updates either the filename (if someone only renamed a pnd) or inserts the new entry.


- If someone renames a pnd it's just like a "newly" found pnd and the table gets updated.


- Different versions of the same app can exist as the MD5 & version in the table are different; If someone copies the same pnd to different locations duplicates will be found


This way, filenames can be whatever the user wants, the pnds can be wherever the user wants, scanning is done in microseconds most of the time as only updating takes a bit longer (but doesn't have to be done a lot) and the info in the embedded PND is used for identifiying (and everything else) instead of having to change the filename.

 

[Tam Toucan]

Without getting into the whole debate, something that's not really been mentioned is how user-friendly can the Pandora be made? I've been using computers for decades and was under no illusion that this is not a plug-n-play device. Yes it would be great to have all the stuff discussed, but after that you've still got the problem that it's running software that's not "man-on-the-street-friendly". Basically if someone is going to have problems following a readme sitting on the desktop or displayed when they first run mini-menu then is the Pandora ever going to be for them?


Do people here think that the Pandora can by made Apple-easy? (See here)

 

[skeezix]

The directory listing bit is done now, not an issue (ie: scannign a whole 64GB SD does take awhile, but now we're clamping to top two levels, we'll see; maybe just top level, if its a pinch.) (plus the existing /pandora hierarchy of course.)


Craigs idea to mark up filenames, then do http post for each file, is not a great one imho for a lot of reasons; but the basics are pure performance --


- given say 1000 apps


- at scan time (be it automatic or scheduled or on-demand) do 1000 http posts, to see if they're good; even at 4/second (who knows if this woudl be 1/second or 10/second) thats 250 seconds -- several minutes just to check for updates.


What we have now (ie: we've already written, I'm not really a fan of writign somethign again just to be slightly different


- given 1000 apps


- one pull of repo data -- not sure how long this takes on pandora; (gah, sounds like it currently includes the kitchen sink, so is overbig, shoudl strip it down a little); still, even now, the entire collection fo apps on repo is 244K for a couple hundred apps.. so calling it 1000 apps there would be saya MB of data (and the spec shoudl be trimmed a little imho). Pulling 1MB on Pandora's not the fastest connection shoudl still take maybe 5-10 seconds.


- check all the apps against it -- instantly


Difference --


- so current method should be faster, assumign user has a few dozen apps or more


- does let the pandora show a list of new apps, updated apps, other apps of interest etc


- doesn't depend on marking up filenames


ie: more features, usually (but not always for small cases) faster, and a community driven standard


--


The root here is trying to make user life easier; both of these ideas (one already exists!) will facilitate the same 'user experience', just a different way of doing it; the existing way is more robust and clean imho.


jeff

 

[WizardStan]

Difference --


- so current method should be faster, assumign user has a few dozen apps or more


- does let the pandora show a list of new apps, updated apps, other apps of interest etc


- doesn't depend on marking up filenames

- current method is user triggered; making it part of the regular scan can leave a user with several seconds of "why is my Pandora suddenly going so slow?" at best.


Current method is also based on every Linux repo spec out there. Most repo's do have a file naming convention which includes the version, but it's not ancillary, it's just convenience; you can download and rename the package and it'll still run because the actual package information is stored in the package itself.

 

[skeezix]

yeah, but even easier --


- Trigger is irrelevent; it could be automated or not for either approach, so no need to discuss for comparison


- Version isn't really relevent, since its already known -- we're mostly talking mmenu here anyway, which especially knows all details all the time its up (from the PXML); so marking up the filenames is totally unnecessary there.


-> ie: why put it in filename, when mmenu already knows it


-> linux distros do it because it can be handy, but in those distros, the user never sees the package; its just one of 50,000 files on his hdd; we're talking about files visible (now possibly at root of) SD; which might well have filename length limit on a FAT16 card even


mmenu calls update-tool and says 'tell me current versions of stuff on repo', then internally compares (instantly) and shows 'updatable' badge on updatable apps (think 'apt-get update', and mmenu then showing the badges, and then doing apt-get for the desired app(s))


For xfce, have a pop up come up listing updates.. a la pndstore now.


jeff


Not sure if a debate will go anywhere, thats my worry


We shoudl just build it, and go

 

[Dead1nside]

I think we should work with what we've got. There's already improvements in people's minds to the PND system (which I think is excellent). Really some would say that getting PNDs set up is no different to putting music on an mp3 player in MSC mode, but then again some people prefer everything hidden from them. I'd posit that what might attract some to the Pandora is its configurability. There is a solution to this problem, there can be multiple solutions even. I think with repo.openpandora.org and the PNDstore, that is so far the best idea in automatic updates, and it can be improved on. Being adamant there has to be a revolution, and ignoring the developers' opinions, those who already contributed so much on the software side, is a poor idea. Why can't you act more collegiately? The App Store is a great example, implementing the repo spec would be a wise move, in the vein of ease of use, that you're trying to accomplish.


I'll leave this up to smarter people, and the ones who've already done so much for us.