Release PNDstore


Oh no, a segfault. I was so glad to have not seen any of those so far, since those will be tougher to debug. Just to be certain, was it on "Updating local package list" when it crashed, or was it still in the middle of "Updating remote package list"? Most likely you have one PND that's breaking it; if the problem persists, try removing any recently installed PNDs to figure out which is causing the fault. Then, if you can find a culprit, please let me know.


hi it happened when updating remote list. not happened again so far though
 
Might be, that you updated, when milkshake made changes to the repo, so that the data could not be found?
 
If that were the case, there would be a warning printed, and it would keep running. Segfaults should only appear in external, non-Python code. So it could be a bug in, say, PyGTK, but it's more likely an error in how PNDstore interfaces with libpnd, which it only does when it needs to update the list of installed packages. That's why I suspected it might be one bad PND causing it. But since it hasn't happened again, I'll conclude it was gremlins. peelie, you need to keep those gremlins away from your Pandora.
 
Segfaults should only appear in external, non-Python code. So it could be a bug in, say, PyGTK, but it's more likely an error in how PNDstore interfaces with libpnd, which it only does when it needs to update the list of installed packages.
Unless you are generating your own python bytecode at runtime (which is fun, but I know you are not doing that), the most likely sources are c-libraries and python bindings of those such as e.g. PyGTK or especially ctypes interfacing with libpnd. Personally, I would start looking at the latter one.


I've been working a bit on turning XBMC into a full featured Pandora front-end application and as the first step in that direction I have plugged PNDStore directly into XBMC through its addons. Whenever I perform a local database update (i.e. database_update.update_local()) it seems to (consistently) crash XBMC. This is probably unrelated and most likely caused by an improper setup on my end. Last time I checked out your source was about a week ago or so, so if you recently fixed something there then let me know. I'll try to keep an eye open for that segfault but it is not my first priority now.


[edit]My error seems to occur when finding the PNDs on the searchpath in the line:


search = libpnd.disco_search(searchpath, None)


[/edit]


[edit2]pndrun_xbmc.out reports:



Code:
[--------------------------------]{ App start }[--------------------------------]

sh: lsb_release: not found

sh: lsb_release: not found

ALSA lib pcm.c:7234:(snd_pcm_recover) underrun occured

/usr/pandora/scripts/pnd_run.sh: line 24: 2370 Segmentation fault     "./$EXENAME" $ARGUMENTS

[--------------------------------]{ App End }[--------------------------------]

This occurs exactly on the invocation of libpnd.disco_search(searchpath, None). So it seems that is causing the segfault (at least in my case).

The value of searchpath was: "/media/*/pandora/apps:/media/*/pandora/desktop:/media/*/pandora/menu:/usr/pandora/apps"

[/edit2]



[edit3]

Hmmm. I moved all pnds of the searchpath except for a freshly downloaded xbmc.pnd to see if there was a corrupt pnd present.

This changes the error to the following (putting the pnds back generates the segfault again):



Code:
[--------------------------------]{ App start }[--------------------------------]

sh: lsb_release: not found

sh: lsb_release: not found

ALSA lib pcm.c:7234:(snd_pcm_recover) underrun occured

ALSA lib pcm.c:7234:(snd_pcm_recover) underrun occured

*** glibc detected *** ./lib/xbmc/xbmc.bin:: double free or corruption (out): 0x018c9500 ***

/usr/pandora/scripts/pnd_run.sh: line 24: 3806 aborted        "./$EXENAME" $ARGUMENTS

[--------------------------------]{ App End }[--------------------------------]

The error still occurs on the same line, i.e. invocation of libpnd.disco_search(searchpath, None).


[/edit3]


-Caine.
 
Last edited by a moderator:
Caine: Thanks for the investigation! Unfortunately, I don't know what to do about your results. In disco_search, I've only ever been able to induce a segfault by setting searchpath to None. Even an empty string works fine. Since you say that searchpath is set to the libpnd default, I'm not sure where the segfault could be appearing. I could perhaps read through the libpnd code to see if there could be a problem in pnd_disco_search, but I hardly know C and probably wouldn't be very effective at bughunting. So sorry Caine, I have to leave you on your own for now!


But out of curiosity, how are you interfacing with PNDstore? Your .outs seem to show calls to pnd_run.sh, yet you mention calling specific functions rather than the command-line interface. What magic are you performing in there? It probably has nothing to do with the segfaults, but since I can't figure out the segfaults, who knows.
 
So sorry Caine, I have to leave you on your own for now!
No problem. I'll track it down sooner or later :)

But out of curiosity, how are you interfacing with PNDstore? Your .outs seem to show calls to pnd_run.sh, yet you mention calling specific functions rather than the command-line interface. What magic are you performing in there? It probably has nothing to do with the segfaults, but since I can't figure out the segfaults, who knows.
No magic at all. I simply replaced your GUI with a python file of my own. I.e. I have a single python unit to implements a plugin for xbmc (which itself is launched by pnd_run). This python unit imports database_update.py and packages.py to do the work quite similar to __init__.py in pndstore_gui.


So, in a sentence, source level integration where I only replace the gui.


[edit]Did some more testing. I enabled the logging in libpnd by calling pnd_log_set_buried_logging as well as pnd_log_to_stdout (By the way, Tempel, you didn't expose any of those functions yet in your libpnd.py). Now I can see which pnd file was last encountered by pnd_disco_callback.


Some PND files are overrepresented in the crashes which suggest malformed PNDs or some other corner case. E.g. i'm seeing audacity very often before the segfault. When scanning /usr/pandora/apps/, I see consistent crashes on op_tvout.pnd. The strange thing is, PNDstore itself doesn't seem to experience those crashes even though it is calling exactly the same code.


I recall that XBMC actually uses a builtin python interpreter, maybe that one is causing some differences...


[/edit]
 
Last edited by a moderator:
New beta here! I'm calling this one version 1.0 because you need to empty the pndstore appdata directory before using (no need anymore)!


Get the 1.0 beta here! Despite the version number jump, there are very few user-facing changes here. But there are some significant backend changes, so I need it abused as best as you can.

  • Updated to fully use repo spec version 3.0. This should make list updating faster and more bandwidth efficient.
  • Made remote list updating much more robust. Not that's it's needed for milkshake's repo, but now I say bring on the hastily-coded alternate repos!
  • Now searches all the directories it should for PNDs, including /media/*/pandora/mmenu.
  • Ctrl+Q can be used to close the GUI.


The repo v3 change required a lot of changes to the backend code (but very little of the API, so you should be clear Caine). But though I've tested it as best I can, I can't be fully confident in it until it's used for real for a little while. So please keep an eye out for any inconsistencies in the package lists, like if new packages don't appear or installed packages aren't aware of upgrades, since those would be critical failures here.


And to remind you, you need to empty the pndstore appdata directory before using! Expect me to say that a lot in the near future.
 
Last edited by a moderator:
New beta here! I'm calling this one version 1.0 because you need to empty the pndstore appdata directory before using!


Get the 1.0 beta here! Despite the version number jump, there are very few user-facing changes here. But there are some significant backend changes, so I need it abused as best as you can.

  • Updated to fully use repo spec version 3.0. This should make list updating faster and more bandwidth efficient.
  • Made remote list updating much more robust. Not that's it's needed for milkshake's repo, but now I say bring on the hastily-coded alternate repos!
  • Now searches all the directories it should for PNDs, including /media/*/pandora/mmenu.
  • Ctrl+Q can be used to close the GUI.


The repo v3 change required a lot of changes to the backend code (but very little of the API, so you should be clear Caine). But though I've tested it as best I can, I can't be fully confident in it until it's used for real for a little while. So please keep an eye out for any inconsistencies in the package lists, like if new packages don't appear or installed packages aren't aware of upgrades, since those would be critical failures here.


And to remind you, you need to empty the pndstore appdata directory before using! Expect me to say that a lot in the near future.


hi anychance next release install location can be chosen? or have defaults to /sdcard/pandora/menu/apps


; /sdcard/pandora/menu/emulators; /sdcard/pandora/menu/games?


cheers
 
hi anychance next release install location can be chosen? or have defaults to /sdcard/pandora/menu/apps


; /sdcard/pandora/menu/emulators; /sdcard/pandora/menu/games?


cheers

It already lets you pick from the default locations that you would expect... i.e.


/sdcard/pandora/apps/


/sdcard/pandora/desktop/


/sdcard/pandora/menu/
 
Feature request, the location to save is set as a default for new time (if available)


E.g I always install things to /media/PANDY/pandora/menu


but the 2 partitions on my boot card are always before that in the drop down box,, (yes I am that lazy)


Love the app makes my life easier.
 
hi anychance next release install location can be chosen? or have defaults to /sdcard/pandora/menu/apps


; /sdcard/pandora/menu/emulators; /sdcard/pandora/menu/games?


cheers

It already lets you pick from the default locations that you would expect... i.e.


/sdcard/pandora/apps/


/sdcard/pandora/desktop/


/sdcard/pandora/menu/

yeah but awkward me has his menu directory split into apps, games and emulators :)
 
hi anychance next release install location can be chosen? or have defaults to /sdcard/pandora/menu/apps


; /sdcard/pandora/menu/emulators; /sdcard/pandora/menu/games?


cheers

It already lets you pick from the default locations that you would expect... i.e.


/sdcard/pandora/apps/


/sdcard/pandora/desktop/


/sdcard/pandora/menu/

yeah but awkward me has his menu directory split into apps, games and emulators :)
I do that aswell, Apps, Emus, Games, System, would it be possible to have a config to make it go into the folder from the 1st category from pxml?
 
New beta here! I'm calling this one version 1.0 because you need to empty the pndstore appdata directory before using!

The package list is not updated on my OP. When I click on Update lists, the message Updating remote package list... appears in the bottom but nothing happens.


I deleted the folder pndstore in appdata, and I can see that a new one has been created, containing 2 files (app_database.sqlite and pndstore.cfg), but the time stamp hasn't changed since the time when I executed PNDstore.


I have noticed that there is a pndstore at /mnt/utmp. Should I delete that folder first?


Thanks a lot.
 
Last edited by a moderator:
Nice, I'll check this out later.


Tempel, don't hesitate to make API changes for my sake. It's not a lot of code and easily adjusted.
 
New beta here! I'm calling this one version 1.0 because you need to empty the pndstore appdata directory before using!

The package list is not updated on my OP. When I click on Update lists, the message Updating remote package list... appears in the bottom but nothing happens.


I deleted the folder pndstore in appdata, and I can see that a new one has been created, containing 2 files (app_database.sqlite and pndstore.cfg), but the time stamp hasn't changed since the time when I executed PNDstore.


I have noticed that there is a pndstore at /mnt/utmp. Should I delete that folder first?


Thanks a lot.

you do have to wait a bit on initial update
 
Now waited for almost 30 minutes, nothing changed (still says Updating remote package list...). Files in both /mnt/utmp/pndstore and /media/PANDORASD1/pandora/appdata/pndstore have timestamp from 14:33 (that's when I launched PNDstore).


Internet is working, because I can surf with Midori.


Any ideas? Thanks a lot.
 
OK, somehow worked. I don't know, I just closed and tried again a couple of times.


Anyway, thanks for the help. And it may help somebody else who may have the same issue.
 
I was just about to post the same. For me the first remote update stalled but closing down the application and retrying worked.
 
I don't suppose you have the .out files from when it stalled, do you? I'm not seeing such stalls on my desktop. It could just be network flakiness, which I understand to be prevalent on Pandoras, but if it's something more serious, it should show up in the .out, I think.

Tempel, don't hesitate to make API changes for my sake. It's not a lot of code and easily adjusted.
Don't worry, I'm not giving you special treatment :p . Just haven't needed to change it.
 
Back
Top