A possible PND-system replacement

Discussion in 'Pyra OS (Debian GNU/Linux)' started by slaeshjag, Jul 28, 2014.

  1. slaeshjag

    slaeshjag ¯\_(ツ)_/¯

    Joined:
    Apr 8, 2010
    Messages:
    2,687
    Location:
    ~Stockholm, Sweden
    So, for the past week I've been trying to hide from some real life stuff and forget about this 30°C summer heat. What better way to do it, if not writing software nobody will care about?
     
    So far, it is still very early in development, but I've tried really hard to not use more hacks than I need to. There's plenty of time this time. For reference in this post, I will refer to it as the DBP system. This is not in any way final, and for any PND-system successor, it'd be mandatory to have at least 10 polls about its name. I just want something to call it when comparing it to the PND-system, k?
     
    The currently implemented features are not final, and they are not the final list of features that will be implemented.
    All dependencies so far exist in Debian Jessie, and I intend to keep it that way.

    Design goals
    * As few hacks as possible. No Zenity. Doing a system()-call instead of just properly implementing it in code is forbidden.
    * Correct the meta-data situation with PND's (raw append, wtf?)
    * Keep it extendable
    * Allow for terminal integration with DBP-installed programs
    * Potential for multi-user support
    Probably more stuff I can't remember

     
    DBP-files
    DBP-files consists of an optional SquashFS file, and an appended ZIP-file with meta-data. The core meta-data consists of at least one .desktop-file. All .desktop-files in a DBP are exported if they contain a "Desktop Entry" section. DBP-files themselfs have some core package data inside the "Package Entry" section inside "meta/default.desktop". These .desktop-files are exported mostly unchanged, with the only exception being that the Icon path is changed to an absolute path into the icon cache.

    Apart from desktop-files, executables are also exported, via a shellscript template, that invokes run-dbp with the arguments preserved. This template will also set up enviroment variables, if needed. The idea is that executables can be exported without having their enviroment mangled to redirect writes to appdata. As a result of executables being exported separatly, it means that if the .desktop-files themself only refer to executables already present on the system, and no other data needs to be included, it's possible to create a working package without a SquashFS image. This is unlikely to become a norm though, as the idea itself is a bit silly. It's merely a side-effect.

    DBP-system
    The DBP-system consists of two main parts.
    * A daemon running as root, that does package crawling and mounting.
    * A user program (dbp-run) that runs a program using package ID and executable path within the DBP as argument

    The daemon is configured via one configuration file, that sets stuff like icon/exec/image/union/appadata directories. It also depends on a shellscript template being available, that is used for emitting the executables.

    The daemon and user program communicates over dbus's system bus, so for this reason, dbus needs an additional config-file.

    Currently implemented features
    * Executable exports
    * .desktop and icon installation
    * Automatic crawling on media mount/umount
    * Very, very rudimentary executable launcing (requests to mount the aufs, does a system()-call, requests a umount.)
    ** Supports multiple instances of a program running without trying to umount when one quits

    * Directory watch for removed/added/modified DBPs

    * Proper application launching

    * Proper error logging (like pndrun_*.out) when launching DBPs.
    ** Needs to split stderr/stdout so that terminal stuff still works properly.

    Features I want to implement Soon™
    * Modifying the exec shellscript to set up the enviroment, if needed

    * Automatic mimetype association for packages
    * Other stuff I've forgotten

    Features I'd like to see
    * run-dbp spitting out error messages with full gettext translation support
    * Some form of dependency tracking
    * Allow for some packages to show on the desktop
    * Multi-user support (with stuff like per-user appdata)

    * Maybe perhaps possibly sometime PND backwards compatibility
    * Other stuff I've forgotten


    Contributing
    Preferably, this thread should focus on this specific implementation, rather than the general direction a PND-replacement should go in. For general talk about a PND-replacement, there's already a thread about it.

    Code is currently hosted on github: http://github.com/slaeshjag/dbp
    The issue tracker there is probably the best place to file serious feature requests at. Anyone wanting to help is welcome to do so, and may ask on IRC what current issues are the most relevant.
     
    Last edited by a moderator: Feb 8, 2016
    Tags:
    ClockworkCoder, Trip, KickAss and 9 others like this.
  2. rygD

    rygD Nihilistic Mystic

    Joined:
    Feb 28, 2014
    Messages:
    7,260
    Location:
    Everywhere
    Not really related to the topic:

     
    Some days that is the temp inside my house with the AC on, and in most places I have lived that temp outside would make for a cool summer.
     
  3. _wb_

    _wb_ Microbe Staff Member

    Joined:
    Apr 5, 2012
    Messages:
    5,387
    Location:
    Brussels, Belgium
    Looks good!


    The metadata .desktop files is a good idea. How do you plan to do the global metadata? Package name and description for the repo, maybe some other things...


    I think there should also be some way to specify dependencies on particular .deb packages (and .dbp packages?), and the launcher should check that the dependencies are met.
     
  4. slaeshjag

    slaeshjag ¯\_(ツ)_/¯

    Joined:
    Apr 8, 2010
    Messages:
    2,687
    Location:
    ~Stockholm, Sweden
    Currently, the idea is to keep that data under a separate section in default.desktop. Like this: http://p.rdw.se/default.desktop.txt

    A longer description should probably be placed in a separate file though, with bigger freedom for line breaks, etc. 
     
  5. milkshake

    milkshake Advanced Member

    Joined:
    May 18, 2009
    Messages:
    3,727
    Location:
    Rotherham, UK
    Why not keep the xml data but inside the zip?

    Or even better json file :)
     
    ryan27968 likes this.
  6. slaeshjag

    slaeshjag ¯\_(ツ)_/¯

    Joined:
    Apr 8, 2010
    Messages:
    2,687
    Location:
    ~Stockholm, Sweden
    Because the desktop files needs to be created anyway. Why not just let the desktop files be and put the required additional information inside them, rather than using some other Cool and Hip format just to generate new files at run-time?
     
  7. milkshake

    milkshake Advanced Member

    Joined:
    May 18, 2009
    Messages:
    3,727
    Location:
    Rotherham, UK
    Because json or xml has more support in more languages than the dot desktop file so it's win win for developers.
     
  8. slaeshjag

    slaeshjag ¯\_(ツ)_/¯

    Joined:
    Apr 8, 2010
    Messages:
    2,687
    Location:
    ~Stockholm, Sweden
    .desktop files are hardly difficult to parse anyway. The time it takes to write a parser is about the same as it takes to google up a library supporting them. They're just ini-files with locale-support, pretty much. With that in mind, I couldn't justify using some convoluted multi-level data description format for something that will only result in flat key-value pair files...
     
    Last edited by a moderator: Jul 28, 2014
    dimag0g, DrHAX, moxie and 1 other person like this.
  9. milkshake

    milkshake Advanced Member

    Joined:
    May 18, 2009
    Messages:
    3,727
    Location:
    Rotherham, UK
    It allows for nested data like multi language title, description collections as well as things like collections of screenshots locations.


    It's much better to have the flexibility of a format that is easy to work with and not hard to create, than it is write a new parser with every different language u use.
     
  10. sebt3

    sebt3 homebrew player (P. & C.)

    Joined:
    Sep 9, 2008
    Messages:
    4,740
    Location:
    France
    So that's the reason you were bugging about d-bus on IRC ?

    You should have told :p

    Anyway great work :)
     
  11. notaz

    notaz Certified Guru

    Joined:
    Aug 23, 2005
    Messages:
    4,904
    Location:
    Lithuania
    XML may have lots of libraries but is rather painful to write and maintain, I wish we hadn't used it..

    I think Exophase once said that one of harder parts of making a pandora release was writing the PXML.
     
    error, DrHAX and sebt3 like this.
  12. milkshake

    milkshake Advanced Member

    Joined:
    May 18, 2009
    Messages:
    3,727
    Location:
    Rotherham, UK
    There can be tools made to generate pxml, foxblock wrote one and an online one could be done too.
     
    myNewUsername and pmprog like this.
  13. Neelix

    Neelix Insecticidal Maniac

    Joined:
    Jan 8, 2011
    Messages:
    3,195
    Location:
    Melbourne, Australia
    You could do that...   or you could use a format that is easily readable, and easy to construct in any text editor. 


    I think slaeshjag made the right choice.

    - Neelix
     
    DrHAX likes this.
  14. levi

    levi Still fresh, damnit!

    Joined:
    Oct 6, 2008
    Messages:
    9,645
    Location:
    Somewhere off the coast of the EU
    There are always corner cases that if you're writing your own parser you'll decide on differently to the next person, making compatibility a mess.  So writing your own parser when there's already one out there is very rarely the right choice IMO.

    I'm as anti-XML as the next man, but at least it covers concepts like escaping the syntax characters when you actually want to use them in your text.  Ideally you want something that has a defined way to do stuff like that, but doesn't go to the lengths that XML goes to to acheive that.  Whether that's .desktop files, JSON or something else.  I used YAML in a recent project, but that seems to be missing documentation on how to escape commas in lists, or referencing lists, and while I worked out how pyYAML did it I have no confidence that that's the proper way.
     
  15. milkshake

    milkshake Advanced Member

    Joined:
    May 18, 2009
    Messages:
    3,727
    Location:
    Rotherham, UK
    Yeah it's nice being able to write it easy but when it comes to parsing that data that's a different story.
    Anyway I'm not that bothered which way he does it I was just thinking ease of use for developers to take advantage of gathering the meta data, not just users or the packager.
     
  16. _wb_

    _wb_ Microbe Staff Member

    Joined:
    Apr 5, 2012
    Messages:
    5,387
    Location:
    Brussels, Belgium
    The easiest way to find PNDs on the Pandora is not by extracting the PXML from PND files, but by grepping the .desktop files in /usr/share/applications.

    For flat key-value pairs, nothing fancy is needed. Parsing can be done easily enough from shell scripts using simple tools like awk and grep if needed.

    It should be easy enough to convert between plain key=value lines, XML and JSON when needed. The important thing is that the discovery daemon and the launcher have minimal overhead, and that things are easy to use from shell scripts -- preferably without needing hacks but by using some standardized tools as proposed: at least dbp-run (probably not that necessary if executable names can also be used directly), but maybe also things like dbp-mount and dbp-umount to just mount the squashfs (dsp-mount would output the path), dbp-info which returns metadata like version numbers and file location, dbp-get to download and install a new dbp package (like apt-get) including its dependencies, and so on.
     
  17. milkshake

    milkshake Advanced Member

    Joined:
    May 18, 2009
    Messages:
    3,727
    Location:
    Rotherham, UK
    For example which ever way you do it, it should be pretty easy to update PndAid to get access to the information it needs... assuming your little project was taken on as the defacto format for the Pyra.

    Afaik this is not how it works anyway, the PXML is only for the meta data not pnd detection.
     
    Last edited by a moderator: Jul 29, 2014
  18. _wb_

    _wb_ Microbe Staff Member

    Joined:
    Apr 5, 2012
    Messages:
    5,387
    Location:
    Brussels, Belgium
    I mean if you want to do things with other PNDs from your PND, like ptitSeb's LibreOffice launcher PNDs, or like my Game Soundtracks: it finds installed games and audio players not by looking at PXML but simply by looking for .desktop files. That is a kind of hacky way to do it, so it would be better to have standardized tools for such tasks (dbp-info, dbp-run, dbp-mount etc). But I think this shows how irrelevant it is what file format is used for the metadata -- the only thing that needs to look at that format are those standardized tools/libraries (and maybe the repo server, although it could also use such tools/libraries instead of doing its own parsing).
     
    Wally likes this.
  19. Wally

    Wally TrashyMG's worst nightmare Staff Member

    Joined:
    Jan 31, 2006
    Messages:
    2,870
    Location:
    Melbourne, Australia
    Admit it Milkshake, you don't want to change because you have the repo as you want it ;)
     
    sebt3 likes this.
  20. sebt3

    sebt3 homebrew player (P. & C.)

    Joined:
    Sep 9, 2008
    Messages:
    4,740
    Location:
    France
    I could wrote a .desktop file parser in every language i know in half an hour. Xml on the other hand (hint bash is one of them ;) )
     

Share This Page

Loading...