Release pnd_utils

[freamon]

I think dash should be okay in appdata; I don't see why not. We definately don't want special globbing characters like * or ? or \ or / or the like or "The result is undefined" .. but dash should be okay I'd imagine; why call that one out, instead of the more scary characters? Not sure who put that in

jeff

Eeek - it was me. In my defence, it was a long time ago (when I was creating my first PND, and it didn't work when had 'get-iplayer' as the appdata. I wasn't doing proper systematic testing or anything, it was just something I noticed). That was when we had a different pnd_run.sh script though, so it no longer applies (I've just created a PND with a dash in the appdata, and it works fine).

 

[ssokolow]

I think dash should be okay in appdata; I don't see why not. We definately don't want special globbing characters like * or ? or \ or / or the like or "The result is undefined" .. but dash should be okay I'd imagine; why call that one out, instead of the more scary characters? Not sure who put that in


jeff

Eeek - it was me. In my defence, it was a long time ago (when I was creating my first PND, and it didn't work when had 'get-iplayer' as the appdata. I wasn't doing proper systematic testing or anything, it was just something I noticed). That was when we had a different pnd_run.sh script though, so it no longer applies (I've just created a PND with a dash in the appdata, and it works fine).

Thanks. I've removed the warning.

 

[ssokolow]

If we now have support for file type associations, I guess we also need MIME type definitions. 

E.g. many of the file types LibreOffice uses are not known yet to the system. and thus they can't be associated, as far as I know. (Or can they? If so, how?)

I'm not sure, but I think these two forms of pseudo-mimetypes might be treated specially. Association editors which generate them don't need root privileges and every attempt I've made to grep for some kind of mimetype database overlay in my homedir which contains definitions for them has failed.

application/x-extension-foo

x-scheme-handler/http

We'd definitely want to support proper mimetypes if possible, since that would probably make more sense to people writing PXML definitions, but it's something to explore.

 

[hmc]

What do you mean by pseudo-mimetype and proper mimetype?

 

[ssokolow]

What do you mean by pseudo-mimetype and proper mimetype?

 

[skeezix]

I've not researched it much, but on the Pandora it should be fairly typical.. (a little different perhaps, but typical.)

ie:

Drop an XML file into /usr/share/mime somewhere (we have /usr/share/mime/application) and then invoke update-mime-database (I don't know the args offhand, but I think it just scans all the xml files and updates its database accordingly.) Not sure if we need to kick the desktop env after this or not. Anyway.

So more or less the process would be libpnd (pndnotifyd at startup or user login or app install/removal/update) would drop out .xml files, and when done, fire off the update-mime-database.

This sounds like a reasonable amount of work and testing, so I don't want to hold up the current firmware drop (I think we're planning a release candidate tonight or tomorrow, or something soon), which includes the so-far file association work. Once this firmware is out and stable, we can work on building and testing this for the next cycle (and I imagine ED has no worry over releasing another firmware update if its significant to do so.)

So next steps would be..

- we need a PXML specification change request and review (I can make a recommendation, or you guys can; it doesn't have to all be me

- we can build it (I can do the libpnd work, no problem there)

- we build and test some apps

Now, that said .. since theres a lot of XML _stuff_ in these xml files (a lot of different kidns of info, and they can be quite lengthy.. or very short.) .. I'm not sure we want to have PXML thats 200b or whatever, and then a 5K xml chunk wrapped up inside of it.. especially if theres many of these XML bits.

So my no-thought-at-all proposal would be something like..

<associations>

  <association ... />

  <association .../>

  <mimetype file="./foo.xml" />

  <mimetype file="./bar.xml" />

</assocations>

Thus the actual xml files are _in the pnd_, and the PXML merely refers to them. This would be ideal since it keeps all the details out of PXML, excepting one thing.. mounting pnd's is slow work, especially if theres 50 apps that are doing this.. imagine burdening the system with mounting/unmounting all those in a row; our boot time would go down the toilet.

This means we either stuff it into PXML (sigh), or do some serious caching (better than we do for icons and so forth); no storing it in /tmp but instead storing it in flash (say, since SDs come and go), and checking the package or app versioning info in the cache, to know when to update it without doing scans. ie:

Concerns: if a pnd is removed (ie: say the SD is ejected), do we drop the new mimetypes? could be painful for rescans when SD comes back.. be better to just leave them and let the user break thigns.. if he pops out an SD while a mimetyype is needed, his own fault, should know better (to wit, right now.. if you eject an SD with a file assoc'd app, how elegant is that? xfce will see the association vanish, but it will probably still keep the preferred application reference and just not do anything. Thats the best you can really do when the user is dropping apps live.. not a typical linux scenario

Likewise .. when a pnd(s) is newly found that has <mimetype..> references, it will have to be mounted to extract the xmls .. thats slow going.

--

So do we instead just put all that xml, shovel it right into the PXML? Then libpnd just vacuums it up, and writes it out, and no mounting malarky.

--

Last option .. clumsier:

App on first run installs the XML files.

ie: the app by this time is mounted, so the info is all there.

You simply do..

- assume you've got a 'mimefiles' directory say

if [ ! -e /usr/share/mime/application/myapp.xml ]

  gksudo cp ./mimefiles/* /usr/share/mime/application # gui pops up sudo box to user

  gksudo update-mime-database --foo --bar  # no pop up this time, since they did it last time already

fi

That is probably the easiest; trivial to do in app, no crazy specification or mounting business.

Lets do it this way, and you can do it right now

--

So, yeah, _this_ is why I didn't want to go into it right now Big mess

jeff

 

[hmc]

Hi Jeff,

I considered that "install XML on first application launch" approach, too.

But I think this is very unelegant. User needs to enter the sudo password then, no way to automatically remove the MIME type definition, if the PND is removed. More work for the develope / PND packager. Easily forgotten and error-prone.

The "reference in PXML / PND mounting" option is not optimal either, as you already explained.

I don't think it will be a big problem putting all the MIME type definition stuff directly into the PXML. Usually it will be simple stuff I guess, and this will give best overview, easiest implementation for the developer / packager, and most potential to automate things.

So I clearly vote for the PXML solution, as you suggested.

As an example, for NoteCase Pro this could look like:

(the "<mime-info...>" section is 1:1 the same as would appear in the XML file put into /usr/share/mime):

<associations>
<association ... />
<association .../>
<mime-info xmlns='http://www.freedesktop.org/standards/shared-mime-info'>
<mime-type type="application/notecase-plain">
<comment>NoteCase Pro document</comment>
<magic priority="80">
<match type="string" offset="1" value="!DOCTYPE NoteCase-File"/>
</magic>
<glob pattern="*.ncd"/>
</mime-type>
<mime-type type="application/notecase-enc">
<comment>NoteCase Pro encrypted document</comment>
<magic priority="80">
<match type="string" offset="0" value="NOTECASE"/>
</magic>
<glob pattern="*.nce"/>
</mime-type>
<mime-type type="application/notecase-db-plain">
<comment>NoteCase Pro database document</comment>
<glob pattern="*.ncdb"/>
</mime-type>
<mime-type type="application/notecase-db-enc">
<comment>NoteCase Pro encrypted database document</comment>
<glob pattern="*.ncde"/>
</mime-type>
<mime-type type="application/notecase-zlib">
<comment>NoteCase Pro compressed document</comment>
<glob pattern="*.ncz"/>
</mime-type>
<mime-type type="application/notecase-license-key">
<comment>NoteCase Pro license key</comment>
<glob pattern="*.nckey"/>
</mime-type>
</mime-info>
</assocations>
Question: Should the <mime-info> section go into the <associations> section, or should it be an own section? This is a semantic question I guess. Will the MIME type definitions be good for anything else than file type assiciations? If so, it should probably go into an extra section.

Of course we could restructure the XML for the /usr/share/mime XML part, so that it becomes shorter and more readable, e.g.

<mime-type type="application/notecase-db-plain">
<comment>NoteCase Pro database document</comment>
<glob pattern="*.ncdb"/>
</mime-type>
could become

<mime-type type="application/notecase-db-plain" comment="NoteCase Pro database document" glob_pattern="*.ncdb"/>
pndlib must then restructure it back to the needed structure when writing the XML into /usr/share/mime.

Automatic removal could also be handled easily by libpnd then: If the PND is removed, pndlib would delete the corresponding XML file and call update-mime-database again.

 

[mcobit]

Maybe a stupid question, but what happens when two pnds have thecsame mime definition?

 

[ssokolow]

Maybe a stupid question, but what happens when two pnds have thecsame mime definition?

Not a stupid question. I can imagine three types of things that could happen:

1. The records could get merged somehow.
2. Whichever gets processed later could override whichever gets processed first.
3. The mime database updater could fail to gracefully deal with receiving two definitions and bug out.

 

[mcobit]

Would be good if libpnd could catch a doubledefinition and pop up a window asking wich program should be used to open the filetype with.

 

[skeezix]

hmc or ssokolow .. can you remind me in a few days, after this next firmware release comes out? then I can do a quick patch to libpnd, and you gyus can test a private build.

ie:

I'll just roll up everything that occurs between the two tags, and spit it out to a file, in 'order by application discovery'. ie: I don't see it beneficial to wrap up someone elses XML into a different kind of XML for us, meaning that I'd have to parse and validate and store all that, then reformat into proper XML.. when I can just swallow it and trust it.

It would be ideal if one of you can test it in advance -- make up some xml as per above, and drop it into /usr/share/mime/applications and see if that makes it work. If so, let us know, and I can build it into libpnd.

As to tags..its probably only useful for associations, but I think you are right that that is not conceptually ideal; they _could_ be used for something else, I just can't think of it off the top of my head. But declaring a mimetype and no associations is just as valid as declaring only associations or both..

So I think perhaps something like:

<mimetypes>

  <mimetpye name="what its for" filename="foo.xml">

      ...

  </mimetype>

  ...

</mimetypes>

Is probably best. Then libpnd (pndnotifyd) can just spit them out and wash its hands of it.

jeff

 

[Binky]

Would be good if libpnd could catch a doubledefinition and pop up a window asking wich program should be used to open the filetype with.

 

[ssokolow]

Would be good if libpnd could catch a doubledefinition and pop up a window asking wich program should be used to open the filetype with.

Quote++

Something like:

Your pandora is currently set up to open <mimetype> with <pnd1name>,
but has recently found some other application(s) that can handle <mimetype>.
Please choose which application you would like to open <mimetype> files
with by default.
If this application later becomes unavailable, the setting will
revert to your previous choice.

[    pnd1name       ]
[    pnd2name       ]
[    pnd3name       ]


   [ OK ]     [ Ignore ]

 

[Askarus]

Would be good if libpnd could catch a doubledefinition and pop up a window asking wich program should be used to open the filetype with.

Quote++

Something like:


Your pandora is currently set up to open <mimetype> with <pnd1name>,
but has recently found some other application(s) that can handle <mimetype>.
Please choose which application you would like to open <mimetype> files
with by default.
If this application later becomes unavailable, the setting will
revert to your previous choice.

[ pnd1name ]
[ pnd2name ]
[ pnd3name ]


[ OK ] [ Ignore ]

Can't it simply take the first application in the list.

If it's not the right one: Right klick > open with other application > choose you application and set it as default.

Isn't this the easiest way to do it?

 

[skeezix]

Don't confuse the two issues..

- if >1 pnd wants to register a mimetype == a problem; and no, pndnotifyd is not going to pop up some UI at boot time before we even have a GUI system available More to point, I iamgine if multiple apps create image/png say, that thats fine.. apps register to that type.. if theres 5 apps creating image/pnd, fine, as long as one of them works.

- if >1 pnd want to associate to a type, thats fine, and all DEs already handle that (xfce included.) You pick a file (a .png image say), and you can open it with the default app, or hit 'open with...' and pick another app that associates to that type; it'll show you a list. you can do that once, or check the box that marks that as the new default. --> done, and I don't have to worry abotu that at libpnd level at all.

So yes, apps fighting over mimetype creation is goofy, but not really a problem (ie: each has their own likely unique file, and the mime-database update tool will take care of it, same as it does for all linux distros.) And multiple apps trying to associate is entirely normal. ie: you probably have 20 apps that can read textfiles

jeff

Remember.. we're not inventing anything here

 

[Askarus]

Don't confuse the two issues..

I was confused .

 

[hmc]

Hi Jeff,

I'll just roll up everything that occurs between the two tags, and spit it out to a file, in 'order by application discovery'. ie: I don't see it beneficial to wrap up someone elses XML into a different kind of XML for us, meaning that I'd have to parse and validate and store all that, then reformat into proper XML.. when I can just swallow it and trust it.

So you really want to mount each PND on boot time, in order to find the MIME type XML inside it?
Really?  :wacko:  

 

[skeezix]

the quote you quoted is actually saying I'm leaning towards dumping all the XML into the PXML, and spitting it out; I have zero interest in mounting 100 pnd's during boot time   What I was implying was not transforming the MimeType XML into somethign more like PXML xml; instead, just plop the XML files in 'as is'. It'll sort of suck to have really long PXMLs full of XML definitions like that, but better than trying to mount them all, or anythign else I can think of right now.

If we do a pnd-format-2 and extend the format not to be goofy, then it coudl be easire (say a pnd was an iso+zip, then could stuff the icon(s) and XMLs and audio clips and such into the payload zip, so much easier..) .. but these days, with most or many pnds being compressed, theres no easy way to get at the stuff inside.

But remind me in a few days, once the SZ1.54 goes out the door in final form, and I'll pop out a test for you guys to work with.

jeff

 

[Linux-SWAT]

 .. but these days, with most or many pnds being compressed, theres no easy way to get at the stuff inside.

jeff

Even with some append, like for the icon ?

 

[ptitSeb]

 .. but these days, with most or many pnds being compressed, theres no easy way to get at the stuff inside.

jeff

Even with some append, like for the icon ?

Just to remind that both icon and PXML are appended.