Pandora PNDManager

[fusion_power]

Is it normal that PND Manager loads so slow and takes ages to update the lists? Seems to take longer and longer, even if only a few new entries are changed in the repo.


However, the program itself is pretty fast in things like scrolling and displaying stuff, only loading things is pretty bad, at least for me.

 

[mcobit]

The loading time is because it needs to load the whole qt stuff from the pnd as it uses a newer version than there is on the pandora.

 

[bzar]

Upgrade all > Behaves strange if large files are among upgrade session

Is there any chance you somehow managed to activate the download multiple times? Short of a bug in libpndman/curl I'm really quite baffled how those kinds of situations can happen. Anyone else experienced the same issues?

Is it normal that PND Manager loads so slow and takes ages to update the lists? Seems to take longer and longer, even if only a few new entries are changed in the repo.


However, the program itself is pretty fast in things like scrolling and displaying stuff, only loading things is pretty bad, at least for me.

The loading time is because it needs to load the whole qt stuff from the pnd as it uses a newer version than there is on the pandora.

mcobit: The base startup time, yes. However this does not account for the "longer and longer" and syncing.


Most of the sync time (after download) is spent recreating the package model (new info for new packages, checking for updates, checking for removed packages, etc...) and updating all the views to display the correct info (this can sometimes take time, since most views have custom filtered versions of the entire package list). It could be streamlined to be smarter about updating the packages, but that would introduce a whole new minefield for bugs and take another month or so of active development.


For progressively increasing sync/load times I'd suspect something to do with the package database, but that's not really my area of expertise.

 

[porg]

Upgrade all > Behaves strange if large files are among upgrade session

Is there any chance you somehow managed to activate the download multiple times? Short of a bug in libpndman/curl I'm really quite baffled how those kinds of situations can happen. Anyone else experienced the same issues?

I pressed Upgrade-All (Y), the other upgrades worked seemlessly, the troublemaker stalled at some point.


But it was not that I observed the stalling from reading a value 0b/sec near the progress bar, but rather the package and its detail panel was shown as if it was not downloading at all.


I then triggered it individually again (with Y), and then somewhen in the process those strange values came.


And sometimes I even quitted the whole application, and triggered the download again.


But pressing Y twice in one session, besides what mentioned above, I cannot recall.


But still it could happen of course, but in that case the backend must be clever enough to recognize that the download is already running, as a user may press 2-3 times, if s/he was unsure wether s/he pressed the button firmly enough (hardware), or that the software was somehow unresponsive, or simply because s/he forgot that s/he already pressed the button.

 

[fusion_power]

For progressively increasing sync/load times I'd suspect something to do with the package database, but that's not really my area of expertise.

It also could be caused of the WiFi maybe, my Pandora is not the fastest when it comes to WiFi. Maybe it has trouble with alot of small packages? ^^ If there are alot of small packages of course, I'm not an WiFi Expert or know what's going on during updating the PND lists.

 

[_wb_]

As far as I understand, syncing means downloading this file: http://repo.openpandora.org/client/masterlist and parsing it. I guess it could be made faster by doing it in an incremental way, since that file is sorted by update time anyway - in other words, you could stop downloading it when you reach a timestamp that's already in the local copy, and do a local db update instead of a complete local db refresh. This of course requires some coding, but at some point it will be necessary to do that to keep things sufficiently smooth.

 

[porg]

That incremental download would be cool for both for client and the server efficiency!


1) Check HTTP etag and date header. If same as local copy, you are up-to-date, no sync required.


2) Enable GZIP HTTP transfer to save bandwidth.


3) Pipe download stream through | unzip | parser for "modified-time": $lastTimeStamp. As soon as reached, abort download, merge with local file.


Or another approach: Sort the masterfile from oldest to newest.


1) Store byte-count of current cached masterfile.


2) HTTP download only from that offset.

 

[bzar]

Much of the above is libpndman domain (everything regarding the connection to repo and storing package states) and many of those are probably already implemented. If someone wants to fork libpndman and improve on it I'd gladly integrate that version into pndmanager.

 

[EvilDragon]

Has subcategory-browsing been removed, btw?


AFAIR you could select Games subcategories on an earlier version (the one with the old graphics).


Is that something that could be added?

 

[bzar]

You mean the old MilkyTest UI? Besides being based on Panorama that package manager was an entirely different application. MilkyTest UI and PNDManager UI share no code, as don't the package management panorama plugins they use (milky-plugin and pndmanagement-plugin, respectively), and neither do the backends (libmilky and libpndman).


I opted for simplified metacategories (which almost map to the freedesktop main categories) to keep the UI clean, but you can get the apps in a specific freedesktop category or subcategory by using search. subcategory filtering in a category view would be implementeable, of course. Some work, but implementable.

 

[ekianjo]

B-Zar, this seems like a potential bug report :


http://boards.openpandora.org/index.php?/topic/9411-pandora-clock-release/page__st__100#entry191868


Let's wait a little and confirm if we can reproduce it first.

 

[porg]

Yes, that person reports the VERY SAME BUG.


If the bug is confirmed we should link to our thread there as well.

 

[porg]

Would be fine if the database-sync would have a progress-bar and/or download speed display as well.


Within the bottom-right positioned box, whose border is animated while sync, you could write the download speed


3 characters wide over 2 lines should suffice.


Decimal precision anyhow only interesting for small or large numbers.


Examples


700


K/s


8.5


K/s


1.3


M/s

 

[porg]

Bug report: Download stalling issue again! Trying to report as precise as possible this time:


This time not with the upgrade-all function, but a distinctive download, namely that of the Pandora SuperZaxxon Updater.


App detail > start install (=download) process > you get to the download process screen > After some time the progress bar gets stuck and its numbers falsely show as complete (i.e. 11.09/11.09 MiB although it is 15.32 MiB in reality), then the screen fades back to the app detail page.


Luckily the Pandora community now has the excellent system monitoring & logging tool Pandora System Info , which proved that there was indeed no downloading going on anymore, a mere wrong info shown in PNDManager's GUI can be likely excluded.


Going to "Installed" > section "Downloading" shows the app, but its progress bar disappeared.


Restarted the download several times from within app's detail page.


The progress bar then changed from the false numbers (i.e. 11.09/ 11.09 MiB) to the real one (11.09/15.32 MiB) but after some time again got stuck and shows the wrong number (12.08/12.08 MiB).


Maybe this is not at all a PNDManager bug, but a WiFi issue (something like the kernel reporting "network link down" to PNDmanager possible). Seems plausible to me, as WiFi sadly is way slower (~ 80 instead of 800 K/sec) and possible also less stable, after I got it my Pandora back from repair (broken shoulder button + purple screen) with a new board.


If the progress bar numbers can be trusted under that circumstances,


it seems that PNDManager resumed the download at each attempt,


that's at least some relief, to not start from zero each time! (Dunno however wether PNDManager had discarded the temporary download, if I had quit PNDManager inbetween...)


Anyhow, even with having a workaround, it is frustrating to need about 20 manual resume actions and 1 hour or more until a 15 MiB file is downloaded completely!

 

[milkshake]

As far as I understand, syncing means downloading this file: http://repo.openpandora.org/client/masterlist and parsing it. I guess it could be made faster by doing it in an incremental way, since that file is sorted by update time anyway - in other words, you could stop downloading it when you reach a timestamp that's already in the local copy, and do a local db update instead of a complete local db refresh. This of course requires some coding, but at some point it will be necessary to do that to keep things sufficiently smooth.

I'm pretty sure this is how it already works, as that's why I added the etag header in the first place.


Cloudef could confirm this.

 

[Cloudef]

Cloudef could confirm this.

Merge syncs are there, PNDManager doesn't use them AFAIK.


There were problems?

 

[porg]

There were problems?

No bug or crash, but efficiency concerns.


People observed quite long sync times, to then realize that only 1-2 entries changed, and asked themselves whether the sync is total or incremental.

 

[Cloudef]

No bug or crash, but efficiency concerns.
People observed quite long sync times, to then realize that only 1-2 entries changed, and asked themselves whether the sync is total or incremental.

No, I was pointing out that the merge syncs in PNDManager were took out since there were some sort of problems with them AFAIK..


Though, I haven't seen any problems with them when developing milkyhelper.

 

[quartercast]

Is this where we post bug reports?


I don't know if this is a feature or not but if you try to download a package when the wifi is off or in wifi power saving mode (my unit is v slow wifi and sometimes I forget to sudo iwconfig etc etc..), PNDManager goes into endless grey screen with busy cursor.


Then I have to force quit and start again.

 

[bzar]

PNDManager currently has no idea about the wifi status. Some kind of timeout could be added, but meanwhile the workaround is simple: don't start PNDManager without a wifi connection