Release PNDstore


Tempel

Active Member
Joined
Dec 30, 2008
Messages
670
Age
37
Website
randy.heydon.selfip.net
Cross-posted from gp32x


Dear everyone,


I present to you PNDstore, a Pandora application for managing PNDs from remote repositories!


The recent open beta of milkshake's repository prompted me to get in gear and finish up this client for interacting with it. PNDstore can manage all PND files on your system, including installing and updating them from any site that implements version 2 of the PND repository specification (of which milkshake's site is the only current example). Source is available under the LGPL.


The current version 1.0 mostly includes extra backend stability. The biggest change was updating to version 3.0 of the repository spec, allowing for more efficient list updating. Bug reports on this are the best help you could give!


But there's another way you can help, even without a Pandora. Though the backend is mostly complete, the GUI frontend is rather weak. I intend to continue improving it with PyGTK and Glade, but I know nothing about good GUI design! Show me how you think the interface should look and behave. Be aware that all information in the repo spec and PXML spec are available to the client and can be displayed to the user. Also consider that it should be able to work with any number of repositories (maybe ED or Craig will add support to the file archive or app store, or maybe someone will resurrect the Box). Consider what kind of options should be available and how they should be presented to the user. Finally, consider how to handle errors like checksum mismatches, or repositories and PXMLs that don't match the spec. Oh, and keep screen size in mind! Photoshop mockups would be absolutely great, Glade mockups would be even better since I could integrate them directly, and working code would be amazing (but I don't expect that level of effort from you yet).


And another! The current icon is ripped from the GNOME icon set. Make something cool that can be used instead!


One last note: there's also a command-line frontend, though it's not usable from the PND. But to use it you'll have to download the source and work around the ctypes bug yourself. Once you've extracted the source, you can run "python pndst" from the source directory, or you can install it to NAND with "sudo python setup.py install", then use it as "pndst". But that won't install it through the package manager, so it won't be easy to remove if you do that.


Thanks in advance for feedback :)


Old content:

The current

version 0.3 adds threading to the GUI, so it will let you know what's happening as it downloads applications. It also allows package upgrades to be selected individually. It's a small improvement, but they'll keep coming. As always, if it works well, let me know; if it doesn't work well, let me know and also post the contents of /tmp/pndrun_pndstore.out.

Version 1.0 has been released, though it's not all that different to the end-user.


The current version 0.2 includes a real GUI with the ability to install and remove apps in addition to updating them. The GUI is not yet threaded, meaning that long-running operations like installation and updating will appear to make it lock up; once the download or whatever finishes, it'll be usable again. Also, it isn't yet able to remove appdata directories, but that will come in later versions. So try it out! If it works well, let me know; if it doesn't work well, let me know and also post the contents of /tmp/pndrun_pndstore.out.


Version 0.3 is out, so try that instead!


Or rather, that's what I think it'll do. Though I've tested it here, it's never been tested on a real Pandora. In particular, I have no idea if my workaround for the missing ctypes bug will actually work. But that's where you come in, my intrepid beta testers! Please run the program, click "Okay" through all the windows (be patient, it can take some time for each window to pop up), and if it crashes (or doesn't run at all!), post the contents of /tmp/pndrun_pndstore.out in spoiler tags here.


Thanks to some tireless testers, particularly IsSuE and mindlord, all the show-stopping bugs have been ironed out.
 
Last edited by a moderator:
hi can't get it to run


and there is no associated file in /tmp?
 
here is what I get:



Code:
kiwidora:/tmp$ cat pndrun_pndstore.out 

sudo /usr/pandora/scripts/pnd_run.sh -m -p .///PNDstore-0.1.pnd -e PNDstore -a --working-dir=. -b pndstore

not mounted on loop yet, doing so

LoopMountedon: 

/dev/loop4

Filetype is Squashfs

Mounting PND (mount -t squashfs) :

/dev/loop4 on /mnt/pnd/pndstore type squashfs (ro)

Filesystem is vfat

Mounting the Union FS using /media/BIG/pandora/appdata/pndstore as Write directory:

mount -t aufs -o exec,noplink,dirs=/media/BIG/pandora/appdata/pndstore=rw+nolwh:/mnt/pnd/pndstore=rr none /mnt/utmp/pndstore

none on /mnt/utmp/pndstore type aufs (rw,si=67124f77,noplink)

none on /mnt/utmp/pndstore type aufs (rw,si=67124f77,noplink)

[------------------------------]{ App start }[---------------------------------]

Traceback (most recent call last):

  File "./PNDstore", line 6, in <module>

    from pndstore import options, database_update, packages

  File "/mnt/utmp/pndstore/pndstore/options.py", line 7, in <module>

    import libpnd

  File "/mnt/utmp/pndstore/pndstore/libpnd.py", line 7, in <module>

    p = c.CDLL('libpnd.so.1')

  File "/usr/lib/python2.6/ctypes/__init__.py", line 353, in __init__

    self._handle = _dlopen(self._name, mode)

OSError: /usr/lib/libpnd.so.1: undefined symbol: __cxa_pure_virtual

[-------------------------------]{ App end }[----------------------------------]

cleanup done
 
IsSuE: Thanks! That was reported in the other thread and now it's been fixed. Download it again and see what new errors appear.


meandu: Woops, apparently that syntax is only valid in Python 2.7, not 2.6. I'll push a fix for that in just a moment.
 
Any chance for screenshots for those of us who are curious but don't have a pandora yet?
 
^ There's no GUI to show yet. As described in the OP it's just a series of zenity popups. (These are basically the standard OS dialogues you see when opening files, clicking 'ok' to confirm, that sort of thing.)


My progress:


Update available package lists - success


Update installed package lists - fail



Code:
[------------------------------]{ App start }[---------------------------------]

/mnt/utmp/pndstore/pndstore/database_update.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead

  import options, libpnd, urllib2, sqlite3, json, md5, ctypes

Traceback (most recent call last):

  File "./PNDstore", line 27, in <module>

    database_update.update_local()

  File "/mnt/utmp/pndstore/pndstore/database_update.py", line 436, in update_local

    update_local_file(libpnd.box_get_key(node))

  File "/mnt/utmp/pndstore/pndstore/database_update.py", line 262, in update_local_file

    pxml = etree.XML(pxml_buffer.value[:-6])

  File "<string>", line 85, in XML

SyntaxError: not well-formed (invalid token): line 16, column 37

[-------------------------------]{ App end }[----------------------------------]

cleanup done


Would there be any way to indicate that the app is alive and doing stuff? Right now it's the SD light. Perhaps a taskbar icon?
 
Thanks Gruso. It looks like you might have a poorly-formed PND there. I've changed the code so it'll just skip bad PNDs, then log them in the output. So try again with the latest version, and you should be able to get all the way through. But once you're done, check the output and let me know which PNDs failed. I'll need to make the code more robust against them.


And I added a tray icon that should stay up as long as the program is still running. Thanks for the suggestion.


Klaue: make me a screenshot of what you think it should look like! There's a chance that's how it'll look eventually.
 
^ There's no GUI to show yet. As described in the OP it's just a series of zenity popups. (These are basically the standard OS dialogues you see when opening files, clicking 'ok' to confirm, that sort of thing.)
Ah, I read over that while skimming trough the post, thanks.


@Tempel:


Personally I would love something like synaptic


Not as feauture-rich, of course (unless you want to work eternities on it ;) ). Even "just" the main guy with installed/not installed/upgradeable with a simple repository adding/removing/priorising dialog would be great
 
Last edited by a moderator:
I think something like what ubuntu has would be nice, the sofware market.


or something similar to androids marketplace or apples app store.
 
^ Yeah :)





I grabbed Glade in the hope that I could throw something together. I think I'd better find a tutorial...


[edit] Tempel, I'm without Pandora internet right now, but I've grabbed the latest pnd and will test when I get home.


[edit2] Eyyy, it's updating my packages B) Not sure how much luck it will have on this wifi connection, but the SD light continues to flicker away.
 
Last edited by a moderator:
Though the backend is mostly complete, the GUI frontend is currently just a bunch of zenity popups, and it's entirely awful; [...] Show me how you think the interface should look and behave. Be aware that all information in the repo spec and PXML spec are available to the client and can be displayed to the user. Also consider that it should be able to work with any number of repositories (maybe ED or Craig will add support to the file archive or app store, or maybe someone will resurrect the Box). Consider what kind of options should be available and how they should be presented to the user. Finally, consider how to handle errors like checksum mismatches, or repositories and PXMLs that don't match the spec. Oh, and keep screen size in mind!
About 2 years ago (wow, that long already?), I started a project called "beauty-get". However, while I have refined and reworked the concept numerous times [2], nothing is implemented since I don't develop for a device if I can't test the software on at least a nearly-finalized prototype of the device itself.


At one point it became clear that we would eventually need some PND package managers, but typical package managers (and many specialized ones too) are not very well suited for entertainment purposes (especially not for finding interesting games). Basically, most package managers are as follows:

  • Install, remove and manage packages: Yes
  • Browse to find applications: Sure, why not?
  • Browse to discover new games: Not so good.



That's fine (or even desirable) in many scenarios, but in my humble opinion that's not the case if the package manager is meant to run on a portable machine that has gaming controls. For instance, if you look for tuxracer in ubuntu center/centre (which is one of the more entertainment-focused alternatives in the linux world), it will merely give you a single screenshot and a description that's rather useless for determining whether you may or may not like the game (so it's basically like synaptic). This means that you typically have to install all games that sound somewhat interesting to you in order to try them, visit the respective website or search the internet for more information. In Nokia's "Ovi store", Google's "Android Market", Sony's "PlayStation Store", Nintendo's "Shopping Channel"s for both Wii and DSi, Microsoft's "Xbox LIVE Marketplace" and to some extent even Apple's "App Store" (though they are a special case) there are opportunities-that-wouldn't-hurt-were-they-implemented abound.


The key ideas that I designed "beauty-get" around are as follows:

  • It should be well suited for finding new games.
  • It has to have as few perceivable loading times as possible and be responsible at all times.
  • It must enable developers to offer a lot of data related to their game/application without stressing the internet connection and/or server.
  • It has to have a very good information per interaction ratio.
  • It must be fun to use and not at all confusing while having lots of power user features.



...in addition to that, I always apply the principle of "It must be secure and \"sane\", easy to use by default while optionally being very powerful, and future-proof by design" to nearly everything I do, so that is a "key idea" also (except that the graphics are not scalable, but that is a minor problem).


------


I once prepared a first look at beauty-get [1] that includes a mockup screenshot and some text snippets about how the user interface should work, as well as thoughts on how to improve the user experience and incarnate the key ideas. (Warning - the text behind the spoiler is probably a bit hard to read and/or understand. A the very least it's not comprehensive and probably odd.)


Basically, you can scroll the list up/down using kinetic scrolling or an analog nub. By default, the list is randomly sorted and contains all PNDs that are available online as well as locally in one single list (based on an idea by xyta [3]). For an explanation on how sorting (which includes displaying freedesktop categories) is achieved, see "filter buttons" in [2]. By swiping left/right on an entry, you are moving the screenshots that the entry is displaying left/right. If you move the screenshots to the left, more screenshots will come in from the right, if you move them to the right, a video will slide in from the left, without the need to open any kind of menu, without stressing the server or internet connection, with minimized perception of loading times (as well as minimized loading times) and with sophisticated guidelines for PND developers that are meant to prevent clutter and confusion [1]. You can also rate PNDs with one click from within the list (or let the sane automatic voting feature do some basic voting for you). By opening the "detailed view" of an entry (which is currently represented by the non-ideal solution of a (misplaced) "More" button in the mockup), you can get all information that is available for the PND, browse the screenshots and videos, manage the PND (install, remove, move the PND file, "insta-play" (see below)), view rating information, add comments, suggest screenshots and maybe some more things (I'm still contemplating about some ideas there). (I'm not sure whether the detailed view should just "expand" within the list entry so that you would not leave the PND list at all, or consume the whole screen).



Code:
[     [VideoVideoVideoVideoVi]                                                         ]

[     [VideoVideoVideoVideoVi]    Next Video:                                          ]

[     [VideoVideoVideoVideoVi]    The Battle for Wesnoth's fantasy theme               ]

[     [VideoVideoVideoVideoVi]                                                         ]

[ Battle for Wesnoth, The                                                              ]


[ [ScreenshotScreensho][ScreenshotScreensho][ScreenshotScreensho][ScreenshotScreensho] ]

[ [ScreenshotScreensho][ScreenshotScreensho][ScreenshotScreensho][ScreenshotScreensho] ]

[ [ScreenshotScreensho][ScreenshotScreensho][ScreenshotScreensho][ScreenshotScreensho] ]

[ [ScreenshotScreensho][ScreenshotScreensho][ScreenshotScreensho][ScreenshotScreensho] ]

[ Tuxracer                                                                             ]

Slide screenshots to the left to get more screenshots, to the right to get videos - tap either and a fullscreen version gets displayed.


Settings as well as filter buttons are hidden behind the panel at the top that can be opened like a drawer (the mockup is missing all panel's child elements). Pressing any alphanumeric key on the Pandora opens up a text box for searching (that knows what regex is) inside the panel (but the drawer will not open up).


Any problems with a PND are indicated by a huge warning symbol overlay on the PND's list entry, with a text next to it that explains the issue in short. Thorough information as well as possible solutions are then displayed when you open the detailed view. While installing an application, you can choose whether or not it should be automatically updated in the future. If an installed application can be manually updated, a text is being displayed at upper right hand corner of its list entry that says "New version."


Based on an idea that xyta had on forum.gp2x.de (the idea to display installed and non-installed applications alongside each other [3]), I also thought about ways of including an "insta-play" function where, instead of having to "install" a PND, you could just click the "run" button on an entry of the list, a prompt would appear telling you that you are about to execute a PND that could harm your system and asked you if you would wish to proceed. If you chose to do so, beauty-get would be suspended and the PND would be run. After you'de be done using the PND, beauty-get would NOT ask you if you would like to install the PND, and instead the "run" button would be replaced by an "install" button (the "run" button would still be available next to the "install" button in the detailed view of an entry). I believe that such feature would greatly improve the user experience and could make a significant difference.


------


Keep in mind that I had started this project more than 2 years ago. As you probably can imagine, I had no idea that mass production of the Pandora would be so slow. Since currently there is quite a bit of movement in the PND management software area, I have no idea when or even if I will ever implement beauty-get (probably never, since as a late preorderer I don't even have my Pandora yet, but ultimately that decision will depend on the quality of upcoming PND management software releases). Maybe any of these interface ideas and considerations ('x-reponame-previewpicsthumbnailbunch01preparedbyreposerverforafluidscreenshotcentricuserexperiencewithanawfullylongcustomfieldname'?) are helpful for you or any other creator of a PND management application - "do with them whatever you want". After all, they should be put to good use.


Please tell me if you have any ideas or questions, thank you.


------------------------------------------------------------------------


[1] http://www.gp32x.de/board/index.php?/topic/50747-appstore-look/page__view__findpost__p__793248


[2] http://boards.openpandora.org/index.php?/topic/1470-new-idea-for-a-pandora-gui/page__view__findpost__p__28567


[3] http://forum.gp2x.de/viewtopic.php?p=140997#p140997 (German)
 
Last edited by a moderator:
I've made a slight update that should allow it to better handle malformed PXMLs, so a few more of your PNDs might see updates coming in. If this tests out well, then I'll post it in the repo and file archive and start working on the GUI. Fun fact: once I put the next version in the repo, PNDstore will be able to update itself!


Christoph.Krn: That is ambitious! There is an immediate limitation that neither the repo spec nor milkshake's site support demo videos or voting remotely. Those are certainly things that can appear in the future (especially voting/commenting without having to go to the site), but it means your vision can't be achieved quite yet.


Still, as I understand it, the central concept driving your design of beautyget is a package manager that is tailored to games rather than any old applications (your other design considerations are the kinds of contradictions inherent to computer applications in general, and they'll all have to be balanced against each other). This makes a lot of sense, considering the Pandora is rather tailored toward games itself. However, the Pandora was also designed for general computing, and there are plenty such applications, so I'll have to strike a balance between the two. But it should definitely lean more towards the games than most package managers do.


But I doubt I'll ever quite realize beautyget's vision. When it comes to the GUI, I'm adopting a somewhat worse-is-better approach, producing it quickly rather than well. This is why it has the awful zenity-based interface now, and why I'll be using GTK later. I don't even know if GTK is capable of the things you envision! But I'll try my best. Also, the backend is intended to be robust and easy to use specifically so that other GUIs can be made easily; we'll see if that happens.


In conclusion: interesting stuff and thanks for the notes :) .
 
Christoph.Krn: That is ambitious! There is an immediate limitation that neither the repo spec nor milkshake's site support demo videos or voting remotely. Those are certainly things that can appear in the future (especially voting/commenting without having to go to the site), but it means your vision can't be achieved quite yet.
I know. I also know that it's very ambitious. I also know that neither milkshake's site nor the current PND repository specification support videos, bundled thumbnails as well as some other things that would be necessary to fully implement the idea. Keep in mind that when I started to design this, there was hardly any sign of any PND management application (as far as I know). It was a vision from the beginning, and I'm ambitious enough to make such things become a reality (given enough time, which I had back then, but probably not when I'll receive my Pandora).

Still, as I understand it, the central concept driving your design of beautyget is a package manager that is tailored to games rather than any old applications (your other design considerations are the kinds of contradictions inherent to computer applications in general, and they'll all have to be balanced against each other). This makes a lot of sense, considering the Pandora is rather tailored toward games itself. However, the Pandora was also designed for general computing[...]
Almost. beautyget should indeed make it easier to find interesting games, however that must not come at the expense of much less usability for applications or legacy things. The basic idea is simply to make the user experience secreenshot-driven with a high information per interaction ratio (i.e. no constant going back and forth between different application states/views). The considerations I shared are probably not very well suited or very well tailored for communicating the idea behind the concept, since that idea is rather extensive.

But I doubt I'll ever quite realize beautyget's vision.
Well, I didn't expect you to do so. :) But maybe some of the considerations are useful for anyone.


Anyway, thank you for your feedback!
 
I posted a comment over at Milkshake's site. Worked great updating Firefox for me.


Edit: actually just ran it again and it updated a few games.
 
Last edited by a moderator:
i mis the function to select what upgrades you want, the wesnoth one is a bit big to do via pandora wifi. and a progress bar would be welcome too :)


nice work though
 
Last edited by a moderator:
Well I ran it, it updated 3 of my PNDs, and now subsequent runs report nothing to upgrade. I suppose that means its working. :)


One thing I did notice, is that the zenity notification on the task bar disappears if you click on it. I made the mistake of doing this after confirming the upgrade, and then the only indication I had that it was still running was that the PND was still mounted. I eventually realised that the seeming lack of any other activity was because it wasn't getting enough wifi bandwidth to download the packages terribly quickly... moving closer to the router fixed this and I started seeing other signs of activity. (SD light coming on regularly and PNDStore showing up in top)


May I suggest replacing the notification icon with a zenity progress meter?
 
Rather than screw around with zenity some more, I've instead built a real GUI!


This one also supports app install and removal (appdata removal will come later, though). It's not threaded yet, so any large operations (like installation and updating) will make it appear locked up while it's working (a threaded GUI with a statusbar and stuff will also come later). But PyGTK seems good at swallowing errors, so it shouldn't break easily. But if it doesn't do what you expect, or it somehow manages to crash, let me know what's in the .out file. If it manages to behave exactly as you expect, let me know that too so I can quickly finalize this release and get on to a fancier one.
 
Back
Top