On Tue, 1 Mar 2011, Anders Petersson wrote:
# There are various benefits and various hurdles, I'll try to outline the ones # I see below.
Thats what need more.. the pro/cons comparison, and relative to existing.
For historical perspective -- we did try a binary "TOC" very early on, but I don't recall why we discarded it. Perhaps it was problematic with the loopback mount, or didn't perform well enough, or any number of things .. the earlier kernels causes us lots of headaches thaqt changed our decisions :)
The TOC we did try took all of about 20 mins to add into libpnd; fragments of it might still be in there, or old gits; we had just put in a header 4bytes to define the PND-ness, and then a number of entries in the TOC, and for each entry an offset and size; thus we could read 4b and get the PNDness, then read the TOC-size and jump straight to the icon or PXML with a seek.
We did also test speed for SD -- knowing that all SD reads of certain sizes get blocked up to SD-chunk-size; so reading 1b is same speed as reading 30k.
One con that comes to mind: a binary header of any complexity means its harder for the devs to bundle up; nolonger just a make-this, make-that, cat 'em together.
# argue we do want to keep the pxml as the definition format and still # bundle it with the filesystem, benefit being that it's easy to edit with # human-readable tools and application dirs still work unchanged.
My well formedness rule require 'appended' tyep data (icon, PXML.xml, sound clips, whatever) to always be inside the main iso/whatever, so that it is not lost during unpack/repack, and there during a mount, etc. IMHO it is an error to omit the PXML.xml from the iso but we don't enforce it.
# Why change at all? # # Well first off the current system is _slow_, not sure if the primary # bottleneck is the back-seeking for the pxml or the xml parsing and dom # scanning but by cutting both out and caching (on disk mind you) I was able # to cut the mmenu load time in half with 172 pnds (and only 125 of those were # be cached as some exist on the nand). Not to mention the fact this was # despite all the static startup costs etc nor did cache any of the png # loading (which did shave another 0.5 seconds off the startup).
As noted in IRC; caching works for sure; I put on a cache at one poiint in mmenu, but pulled it out as in my implementation it cached the app scan result list, and thus the user had to hit 'refresh cache' (you'll notice rescan is still in the mmenu select menu.)
It did speed things up for sur; your approach of just caching the PXML and image surface can work without operator trigger, but a few caveats .. (ie: what if the guy changes themes, etc.) But all stuff we can handle easily.
So once HF5 goes out, submit the diff and I'll take a look, and we cna merge her in; we can slap a 'enable cache' (or possibly 'disable' if it seems reliable enough all the time), so user can manage it.
I've not tested in a year or more, but if can reduce mmenu start up in half, thats a big win ;)
Another option I toyed with was more drastic -- make mmwrapper maintain the app cache; so when mmenu exits and restarts, it pulls or shares the data from mmwrapper, and is _instant_ statr time. That was a bit dirty fealing but worked really well in testing, but I've not had time to work it over.
So an 'app pool' in mmwrapper might be the way to go right out, but it turns mmwrapper from a tiny little job into a bigger thing.
# First it's always prudent to begin with a file type identifier, I like four : # The header payload could simply be an index of offsets into the for the pxml # and png and we could call it a day, I have some thoughts regarding that as # well but I'd say we can evaluate that separately once we've pondered what we # have here.
When we fiddled with this before, we allowed for >1 icon and other bits too; we had always intended to have up to 1 icon per sub-app, and optional screenshot/sound-clips for the menus to try (you'll see this in PXML spec, but only ever worked for dir-apps of course, since the pnd-format we went with is overly restrictive.)
So theres definately an average of 2 attachments (PXML and icon), but quite a few apps would have 3-4 attachments (multiple sub-app icons.)
# Oh, and we could append this header and maintain some additional compability # but it would make testing for complete pnds more difficult as we'd have to # back-seek for the PXML tag if we don't get the full file and find the # appended PND 0xff. Also, if I understand correctly part of the reason for # appending is that mounting loopback devices didn't support offsets when it # was first brought up, this is no longer an issue as all loopback mounts # support both and offset and a max_length option these days.
A fun alternative is..
zip (uncompressed) everything. zip contains iso, PXML.xml, icon(s). Since its uncompressed, asking the zip tailer for the offset to the iso is trivial, and can use loopback to mount it (since everythig is uncompressed.) (the iso itself could be squash or crammed or whatever still, of course.)
So no custom binary header at all, just a zip of the whole lot.
(I've often brought up iso+zip, but I think zip-all would be the way to go)
# Any thoughts so far? Am I thinking too hard, making things overly # complicated or missing the obvious? ;)
We're coders, our job is to think way too hard about how to do simple things :)
jeff
-- If everyone would put barbecue sauce on their food, there would be no war.