Minimenu Tab Plugin Spec .. Discuss Before I Code :)

[skeezix]

Minimenu doesn't really need a plugin architecture, but it both could be handy (depending what/if-any plugins get built) and is fun to work on, so I'm making this proposal. If I/we go ahead with it, it will have almost no impact on current usage -- it will not slow anything down or change anything, but will make available some new options to scripters and developers. A user could enable the plugin system, and explore plugin options if they wish .. but by default, plugins would be disabled. (And hence no startup time cost, no new secuerity concerns, etc.)

Pardon my typing.

Scope:

- this is not a hard and fast spec; this is just some ideas noted down to illustrate where I'm going. If everyone seems happy with it, then I/we can proceed and during the coding nail down the hard spec that plugin developers would use. So this is for illustration and discussion.

- this specification is not comprehensive -- I'll illustrate a number of API functions below, but it is certainly not all of the planned API, and certainly much more API would become available over time as we anticipate more requirements or make more available to plugins. Again, I'm always a fan of designing small first, and designing expandability in, and going baby steps.

- this specification is for a specific set of plugin extensions, not a comprehensive plugin system; ie: under this spec, plugins could create tabs or add some basic effects (audio, display) and whatnot, but not entirely replace the whole rendering engine for example. It is better to start with specific achievable goals and get that tested and out, and then bite off larger and larger pieces. Going for the big enchilada day one means it'd never happen. But this does not mean I've not considered it!

- in designing this (where 'designing' is too big a word; these are things just rolling around in my head on occasin for a few weeks) I'm picturing a few basic plugin ideas; one sample plugin might be a weather app, where it opens up a single tab and then shows a picture of the sun for a sunny day. Another plugin sample could be a file explorer that opens up one or more tabs based ont he number of SD cards it finds (say), capturing user input to move a selection around, etc. Another plugin sample would be one that makes a wooshing noise when tabs change, but doesn't have a UI of its own. Another plugin sample coudl be somethign that renders a datestamp on the normal UI, and has a configuration panel sasking wehat date format to show in.

- in the future, plugin options that may be useful are such ideas as.. plugin 'application sources'; right now minimenu scans for pnd-files, but we coudl extend the plugin system so a plugin could offer a new pool of applications (say, regular unix executables found on various devices.) I decided not to present that in this specification, because its actually a complex topic -- consider if two plugins want to both scan for some file type and offer it up; then we need mm to offer a 'which plugin to run the selected app?' type function; and what if one plugin wants to be a file browser, does it need to talk to a app-source plugin to negotiate something? I'm specifically avoiding inter-plugin-communications and other heady topics right now

- in the future, we may have inter-plugin communication if needed; that is certainly not discussed now

Security:

- by default, plugins will be disabled

- by default, plugins will only be able to effect their own tabs; if a plugin wishes to be able to render on top of something else, it will have to ask for permission to do so, which could be a configurable user thing. (ie: a weather plugin coudl open a weather tab, and draw it, but it cannot draw over some other plugins tab.) Some operationks are global -- a plugin can request any tab to 'move to front', so that a plugin could somehow cycle tabs based on some event, say, but I don't consider bringing tabs to front a destructive operation. Obviously it is abusable, if a plugin randomly switches tabs every half second. (If abuse becomes commonplace, tight secueity coudl be implemented without changing the API .. limit the number of operations per time or something, but let us not tie developers hands together in the spec; let us see how it goes and offer a reasonable 'catch mistakes' form of security.)

- rather than build (for now) an API for a plugin to ask for permission for various activities, let us just make assumptions -- a plugin can do whatever it wants to its tab(s), and not much of anythign to anything else.

- a plugin is a standalone application, so can really do anything it wants; this is a security concern, when running any foreign application and so out of scope. But remember -- a plugin could play audio, capture keystrokes, or take over the framebuffer on its own. In general, it will take effort for plugins to be so badly behaved.. a naively written little plugin will use the API and crash itself or whatever, without really bothering the user; but a malicious plugin can go nuts!

- resources; when a plugin requests loading a graphic or audio or whatever, the searchpath mm will use is _implied_; ie: it is the plugins path, plus the system path, notwhere else. The plugin is not going to be pully artwork willy nill across the device and making a mess, unless it goes out of its way to do so. (Possibly we may relax this, or make a API function so the plugin can attempt to extend the searchpath for loading resources, but we'll see.)

- plugins will have a specific searchpath; akin to other searchpaths in libpnd, I figure we'll search something like: /etc/pandora/mmenu/plugins:/media/*/pandora/mmenu/plugins -- something like that; ie: a user could deposit plugins onto SD or NAND (and some coudl be shipped with firmware if desired). Plugin searchpath would be defined in minimenu conf file, so could be editted by clueful users (not average joes.)

- plugins will have to be enabled/disabled on a per-plugin basis; ie: user enables plugins _at all_, and then the plugin conf panel would list all known plugins (default set to 'not run'),and flip them to 'run' if they wish. (ie: similar to the tab yes/no and app yes/no conf panels already present.) Maybe this is annoying, but seems safe -- user drops new plugin onto SD, and it won't magicly just run until they enable it.

Stability:

- plugins will be standalone applications that communicate with the host (minimenu presumably); if they crash, minimenu will not be impacted.

- ex: If a plugin opens tab(s), and then crashes, minimenu can detect the crash and close those tabs implicitly or notify the user or whatever; it will not bring the system down. (Which is important, since mm coudl be someones launch desktop; if mm is made crashing, and thus making the device hard to boot, this is BAD. Recoverable, but bad!)

- minimenu reserves the right to not run plugins at all; ie: if mmenu crashes, comes up, crashes again, we may build in logic to flip skin back to default and turn off plugin mode, automatically, so that when it comes up again, it has a higher chance of not crashing.

Architecture:

-in general, many plugin systems use shared libraries for the plug; the host scans its searchpath and finds plugins and links them live, and then the plugins API is called back and forth. This makes coding easy, but means if the plugin crashes, everything crashes.
- instead, I'm going for the approach where a plugin is a separate application, communicating via pipes with the host. ie: plugin stdin is from minimenu, and plugin stdout goes to minimenu. All dialog is via this -- if minimenu has 10 plugins, it has a pile of pipes open. (or sockets, but pipes make it much more convenient .. you end up with perl, python, shell scripts, all the way to C or whatever, being able to make plugins. Very handy!) IF the pipe breaks (due to plugin crash), mm just gets a 'pipe broken' error and drops the plugin and tabs, piece of cake.

- communciation across the pipes will be a lightweight form, essentially a slimmed JSON. Every language can parse JSON, and some 'just do it' (ie: python). IT is easy and fast to parse, without being 'heavy' like XML usually is. The specification for functions is below/.

- synchronous; for this spec, the host or plugin can make one call, and shoudl not do anything until a response (or timeout) is hit. Currently there is not to be batching of instructions, though perhaps in the future we may permnit this (especially for rendering?). For now, make one API call, then wait.

- It is expected that plugins will have an intermediate wrapper do the work. ie: If you're writing a plugin in python that really just wants to be a MVP (model view) thing, with the python fetching weather data and then having minimenu do the rendering etc, then you don't really want every python plugin writng its own api. I fully expect we (and the public) will make up wrappers that we can hopefully all share. Consider:
python plugin <-> python wrapper (pyMMPlugin or something) <-> minimenu
C plugin <-> C library ( libmmplugin.so or whatever) <-> minimenu

- I debated a few different philosophies; be it plugin is doing its own rendering and handing rendered data over to minimenu for display, versus minimenu doing all the work with a heavier API, so the plugind tend to be more simplistic, etc. After some debate, I've decided to opt on the 'expected use case' -- a plugin will probably be a simple 'model' and have mm do the heavy lifting (loading assets, rendering, playback, etc.) ie: It is doubtful many plugins will be writing fiull games or heavy animations, but instead offering things like file browsers. Do correct me if my assumption seems incorrect. By moving the heavy lifting to mm, it makes heavy graphical plugins hard to make, but makes simple plugins easy to make. Example -- a weather app ro file browser is rendering once, or perhaps redrawing after key events, so is not heavy rendering, so mm doing the work is fine. For a game with 50fps or something, using proxy-rendering in mm would suck.

- In the future, we could extend the protocol; if we need better graphic APIs in the interface, or the ability for a plugin to do its own rendering and hand the ready to use artwork to mm (over shared mem say), or even have host supply cvoords to the plugin and the plugin just render to the box specified using its own rendering code, then we could build that in. We'll just have to see where it goes. Yes, I know, by mentioning it here, I pretty much require someone to make the send plugin ever to be a Pacman or Space Invaders game, and thus break the whole plugin concept day one. I'm ready, bring it.

-- for a video player say, you'd want mm telling the playback tool where to draw;p its just not possible with this spec to receive high speed wholesale screens and show them; for that, we'd extend the spec

- We're talking about an event and call model; host (minimenu) will post events to the plugin, and the plugin can make calls to the mm hosting API. There can be some give and take where the plugin negotiates features with the host, so that we can default some behaviours off and let the plugin ask for them (which is another way of saying, we can extend the protocol over time, and the plugins can ask for features to see if present or set things up, and thus know if they're being run on too old a minimenu version or whatever.) (I'd liek to avoid versioning issues, but such is life.)

- the plugin code will presumably be a layer in minimenu; possibly other applications could be a plugin host using the same code. (ie: as a debug harness for one, or perhaps just to re-use plugins. A new host could ignore rendering calls or honor them, could be neat, or useless? But keeps the mmenu code neat.) ie:
minimenu <-> mm plugin host code <-> plugin(s) so the end pipeline looks like:
minimenu <-> mm plugin host <-> pipe <-> plugin wrapper/lib <-> plugin

- one idea I've had with the layered host-code, is that a plugin interface could in theory be made for xfce or other environs .. a mini-minimenu; the idea being that perhaps a useful mm plugin is made, that someone might want it on xfce as a widget or something. But yes, plugin/accessories/panels APIs are a dime a dozen .. shoudl we be making another one, or doing this at all in mmenu? hmmrdf

- most common needs shoudl be covered by the spec, but uncommon (by expectation) APIs may need ot be done by plugin. ie: We're not going to have mm pass lid open/close to plugins, or have timer functions. A plugin is welcome to cruise through the pandora API for lid open/close, or to open its own timers. A plugin _is_ a standalone app after all, so it can do a lot of its own screwing around. But since most plugins will not (likely?) be games, we don't need ot offer high granularity timers to plugins.. they can do it on their own. But for catchign button UI, a lot of plugins will want to do that, so we make an API for it.

- note that mm will maintain tab state; if tab is fronted (and rendered by plugin) and then backed, and then fronted again, mm will ensure the tab content is still showing (and not blank.) The plugin thus could decide to render on first fronting, and not again,m if it doesn't need to.)

Protocol:

- I will leave the actual JSON protocol out of this; it is not critical for specification, but in essence, something like ..
{ func: NAME, args: { arg1: value, arg 2: value } }
with response being aninteger, a string, a hash of values, whatever, as defined below.

TBD as coded, really.

Setup:

- host mmenu, when plugins are active etc etc, will iterate the searchpaths and pull up a list of plugins. For each found, and for those the user has enabled (see Security above), mmenu will then pass a startup event to the plugin.
- host, on close, will pass an event to the plugins so they can do cleanup

Events:

- Startup event (mmenu has loaded the plugin and hooked up the in/out pipes): eStart ( plugID ) -> ACK response only, responding with plugin name. It is not defined if mmenu will be running all plugin startups concurrently or in queue .. plugins should not care, and leave it to host to decide

- Shutdown event (mmenu needs to close, only happens on controlled exit and not crash): eShutdown ( plugID ) -> ACK response only. -- this in general shouldn't do anything, since not many will be start/stopping their desktop environment. Instead, I would suggest plugins do their work during the Startup event, and then save state during config changes or during interaction, for now.

- Tab change event -- by default, plugins do not receive this, but as noted below a plugin could request it. eTabChange ( tabID from, tabID to ); if this is registered, the event order relative to tab-activated/deactivated is not defined, so pick one or the other to operate on.

- Tab activated -- when a plugins own tab(s) become the active ('on front of stack'), they get this notification. If a plugin has 2 tabs in a row, it could well see 'tab deactive'tab active' in a row, as one loses focus and the next gets it. The order of events it not defined, so play smart. eTabFocus ( tabID ) -> expects ACK

- Tab deactivated -- when a tab loses focus. eTabUnfocus ( tabID )

- Key Down -- when a 'button' is pressed; by default no notification for this, to save on bandwidth, but plugin can ask for it. Technically, plugins could just open up /dev/event and do their own IO for keys, and may have to for lid type events etc, but to keep interface clean for 'most plugins'., we're offering button interface here. Buttins include face-buttons or keyboards, but not touchscreen, USB joysticks, etc.
Security: Plugins to receive input events for their tab when focussed;
eKeyDown ( key )
eKeyUp ( key )

- Pen Down -- similar to buttons, but for touchscreen; again, by default this will not happen unless requested, and the request documents the scope .. ie: only when plugins tabs are fronted, or for other tabs or the root-window or what
ePenDown ( x, y, pressure )
ePenUp ( x, y, pressure )

Plugin samples -
- the 'weather plugin' for instance would handle Startup event and request to open a tab; then it would poll some weather server to get info, and request loading some resources in mmenu, and finally, on eventFocus-thing, render the tab.
- for a plugin that makes swooshy noises on tab change, it needn't open its own tab, and instead just request and monitor tabchange events, and play fart noises as it sees fit.
- for a plugin that shows a battery indicator or date-stamp, it could just request permission to render on the root window (outside of its own tab, and may not have a tab at all), and then on tab change (or perhaps on its own timer event) request rendering.

 

[skeezix]

When I say ACK or NACK, it means 'to be defined'; ACK may be nothing, or maybe its a true or 1 value, for instance. NACK could be nothing, or maybe a false value, or whatever. I'm just using it to clarify intention here.

Host API that plugins can call:

for Tabs:

- TabNew ( tab name ) -> receives ACK and tabID back
- TabDelete ( tabID ) -> receives ACK or NACK (fails if deleting another plugins tab
- TabList () -> receives ACK and a list of all tabs in the application
tabID : tabName : owner pluginID
Security: it is assumed the plugin and tab listing are public info; or if we need to have a request-permission model here, we can .. should plugins only get a list of their own tabs, unless they've asked to be able to see all tabs? I dont' want to get totally ouut of control here
- TabFocus ( tabID ) -> request to flip tab to front; receives ACK or NACK. This would trigger events for tab-change and tab-unfocus and tab-focus, etc, with order of course being undefined.
- TabQueryCurrent () -> gets ACK and tabID of current tab or NACK if not permitted (TBD)
- TabFocusNext () -> request go to next tab, whatever it is, receives ACK
- TabFocusPrev() -> request to go to prev tab, receive ACK
- TabMove ( tabID, afterTabID or 0 for first ) -- do we need this? should never really reorder tabs on a user I imagine, but thought maybe someone wantd to open tabs, then change their names, and sort them
- TabRename ( tabID, name )
- TabShow ( tabID, true falase ) -- do we need the option to be able to hide an existing tab? ie: so a plugin could oipen 4 tabs, but only show 2 of them, and late reveal two more? or should it just open them then?
- TabInfo ( tabID ) -- returns tab details or error (ie: get the tab drawing size and location)

for Events:

- EventRegister ( eventname, true/false, option ) -- expect an ACK/NACK value. ie: a plugin may wish to ask to be notified for tab change events, and mm may accept or refuse this request.
ex: EventRegister ( eTabChange, True )
ex: EventRegister ( eKeyDown, False )
ex: EventRegister ( ePenDown, True, AllOverScreen ) -- some plugins may need to watch events outside of their tab .. they might not even have a tab. mm reserves trhe right to refuse to offer this, depending on user security or level of mm code completion

for Resources:

- AssetLoad ( file basename ) -> receives ACK and assetID, NACK; searchpath is as noted above, implied, and full paths are not permitted here. (or maybe we will allow some, but not many. TBD.) Note that mm/host is free to return an already loaded asset's ID, so do not assume it is unique.
- AssetFree ( assID ) -> requests free of asset; receives ACK. Note mm/host is free to just say 'all good' and not ditch the asset, since maybe it doens't know how, or maybe its being used by another plugin with reference counting, etc.
- FontLoad ( font file basename ) -> receives ACK and fontID

- maybe some convenience routines; ie: instead of doing live-render scaling, could load asset and then pre-scale it, if expecitng to draw and undraw it a lot
- AssetScale ( assID, scale function, new width, new height ) - return ACK and new assID (client shoudl free previous ID if it wishes)
- AssetRotate ( assID, degrees ) - return ACK and new ID, or NACK

for Rendering:

Note -- a plugin can request mm to render various things, and note that mm is free to not render antying it doesn't want to, or to render on top. ie: IF a tab is open, but the user brings up a mm menu, the menu may well be alpha-drawn over top of the plugin window. Plugins should not assume anything.

Note -- at this time, plugins requets all rendering by API to the host; they do not render themselves or load artwork assets themselves. If needed for high speed or craazyness later, we can consider it.

Note on target -- each render operation has a target, so host knows where this is relative to; be it relative to tab, to root window, or to conf panel, or even another plugins tabs (future possibility.)

- RenderText ( target, string, x, y, fontID, size, color, alpha )
- RenderAsset ( target, assetID, x, y, offset Y, alpha, optional clip W, clip H, offset X, option to scale type or leave too-small or clipped? )
- RenderLine ( target, tl x, tl y, br x, br y, color ref, alpha )

- PlaybackAsset ( assetID ) -- could be mod or mp3 or ogg or whatever; not sure if I want to do this, or leave it to plugin to play on its own ... ie: then we end up with mm needing to have API to list known media types with loadable playback software, and events for end of song, and api for restart, change song, etc. Sort of heavy feeling, but not sure.

- do we want to have API for working box .. ie: so plugin can say 'set origin here' and 'clip me at this (w,h)'; thats all well and good for toolkits, but don't want to get really heavy here.

example: for weather type plugin sample, it'd just render some text and sun/clud/rain type artwork, piece of cake
- for file browser, plug would just render some lines, some text, some folder or file art, maybe some buttons, no big deal

for Configuration:

-ConfNew ( config entry name )-- receives ACK or NACK; requests that a new Config Menu panel be added for this plugin. Note that config panels in current mmenu are full screen for this reason So user may hit the menu, then hit Config, then hit Plugins, and see a list of plugins, and then pick one and get a config panel option.
- conf events not yet defined, but consider the above -- we'll probably have to add ConfFocus and ConfUnfocus type events, so that config rendering can occur

 

[skeezix]

Page 3 - reserved

 

[mindlord]

Are you planning on including a way for the user to input textual infomation? textbox, combobox, listbox. You know the standard UI stuff, or will the script writer have to massage a way to build UI elements out of lines, labels, and assets?

 

[sebt3]

Hi bro,

If that is a baby-step for you, then mine are micron-step (or even smaller), realy.
I mean, there is already a lot's to code there.
I'm not sure that this is needed in anyway. But you're still free to hack the thing you like
I'm confortable with most your spec. 2 things bug me :
- mm stop when startng an app, and start when the app is done. So the plugin will have to do the samee. It will have a significant impact on performance and usability 9imho.
- if mm have to manage the drawing plugins like weather will b harder to write and wont draw fast.

cheers

 

[skeezix]

Right off first thing I think I say -- its basicly not needed, and probably shoudln't be added (and may well not be.) But if I do go ahead, figured shoudl have a spec for it

Its a fair amount, but its not a huge job; couple days to get a basic plugin going, tops

mindlord -- I've not worried about it yet; it could be constructed by a plugin, but would be some word; if desirable, we coudl try to roll something; mmenu does not have any text entry widgetry yet for itself, you will notice It is all implied -- tab names from categories, etc. But if you're thinking of an IRC client or something like that as a plugin, you'd want an entry widget. We could spec something in as a very rudimentary widget perhaps, but stuff like buttons I figure is just .. plugin tells mm to show a png on the screen, and watches for a mouse event, not an actual 'button widget' per se. (Do we think plugins wil need a full UI widget set?!) -- I've specifically avoided going too far, since really, I shoudl just drop a webkit frame there and let the pluginds render in HTML and tie html events back to the plugin, but thats way too heavyweight for minimenus design

sebt3 -- there will be an option in mm to remain up when launching an app; that was requested (in other minimenu thread) by Pleng, so no problem But for normal users where mm drops down, yes. But thats by design -- mmenu doens't take up any memory footprint, for instance, by dropping out on run. If plugins start up quick, it shoudl be okay .. plugin users woudl be aware of the cost of enabling 20 plugins, so plugins shoudl be very frugal on startup. A plugin could just, on startup, open a thread and ACK, and let the other thread sleep for 20s, and then start loading its assets, to avoid slowing down the mmenu load.. or mm could load and get up and going, and then start loading plugins, so that mm can come up fast. The side effect of that, is sluggish user response while plugins loading though, so not sure. Concern is noted though :/

-- the trade off for rendering shoudl be noted above; its hard for a third party app to render into another app. I imagine usual case for plugin is not a lot of rendering -- for weather sample app, it draws _once_ and then never again (or maybe once every hour or something ; file browser woudl render only on button event.. so not a problem. Its only a problem if you want to write a game or something as a plugin, and doign a lot of graphics via proxy would suck. I mentioned that, in that we could try to extend the aPI to allow plugin to 'render' to a buffer, then hand the buffer to minimenu to display. Another option is mm to hand coordinates to the plugin, and let the plugin render to the screen directly, inthose coords, but thats sorty of goofy.. but works, and is high speed. I figure thats v2.

 

[mindlord]

I just figure that the less rendering that the plug in has to do the better things are. UI widgets in general will allow plug-in writers to quickly mash together tools and tiny apps, like SAMBA mounting scipts, Shoutcast radio-station listing apps, or any manner of useful bits and bobs that go a bit farther than just displaying info.

 

[skeezix]

Ahh, dev threads, never get replies

jeff

 

[JayFoxRox]

Please note that I changed the order of arguments etc

- this specification is not comprehensive -- I'll illustrate a number of API functions below, but it is certainly not all of the planned API, and certainly much more API would become available over time as we anticipate more requirements or make more available to plugins. Again, I'm always a fan of designing small first, and designing expandability in, and going baby steps. 

- this specification is for a specific set of plugin extensions, not a comprehensive plugin system; ie: under this spec, plugins could create tabs or add some basic effects (audio, display) and whatnot, but not entirely replace the whole rendering engine for example. It is better to start with specific achievable goals and get that tested and out, and then bite off larger and larger pieces. Going for the big enchilada day one means it'd never happen. But this does not mean I've not considered it!

I agree.

- in designing this (where 'designing' is too big a word; these are things just rolling around in my head on occasin for a few weeks) I'm picturing a few basic plugin ideas; one sample plugin might be a weather app, where it opens up a single tab and then shows a picture of the sun for a sunny day. Another plugin sample could be a file explorer that opens up one or more tabs based ont he number of SD cards it finds (say), capturing user input to move a selection around, etc. Another plugin sample would be one that makes a wooshing noise when tabs change, but doesn't have a UI of its own. Another plugin sample coudl be somethign that renders a datestamp on the normal UI, and has a configuration panel sasking wehat date format to show in.

I think that a file-browser should be a core-element of minimenu, it shouldn't just be a plugin - Or it should be a core plugin included with minimenu.
Not only does a filemanager / browser make it possible to manage (move, delete, rename, ..) your files, but also to run, view and listen to them.
The wooshing noise and the date renderer should also be a core functionality of minimenu OR it should be seperated into a skinning engine for minimenu [with a seperate API]. In my opinion it would bloat the plugin interface to provide an entire rendering and / or audio library. It wouldn't be used much probably.

- in the future, plugin options that may be useful are such ideas as.. plugin 'application sources'; right now minimenu scans for pnd-files, but we coudl extend the plugin system so a plugin could offer a new pool of applications (say, regular unix executables found on various devices.) I decided not to present that in this specification, because its actually a complex topic -- consider if two plugins want to both scan for some file type and offer it up; then we need mm to offer a 'which plugin to run the selected app?' type function; and what if one plugin wants to be a file browser, does it need to talk to a app-source plugin to negotiate something? I'm specifically avoiding inter-plugin-communications and other heady topics right now :)

Doesn't libpnd already provide the options to add search paths? I think a functionality for virtual pnd handlers shouldn't be hard to do using a selfmade filesystem based on FUSE for example. The filesystem would generate a PND wrapper in realtime for simple unix applications. - However, if a file manager (as previously mentioned) would be added to minimenu this is a redundant feature in my opinion because the filemanager could also be used to start applications.

- in the future, we may have inter-plugin communication if needed; that is certainly not discussed now

Not necessary in my opinion

- by default, plugins will only be able to effect their own tabs; if a plugin wishes to be able to render on top of something else, it will have to ask for permission to do so, which could be a configurable user thing. (ie: a weather plugin coudl open a weather tab, and draw it, but it cannot draw over some other plugins tab.) Some operationks are global -- a plugin can request any tab to 'move to front', so that a plugin could somehow cycle tabs based on some event, say, but I don't consider bringing tabs to front a destructive operation. Obviously it is abusable, if a plugin randomly switches tabs every half second. (If abuse becomes commonplace, tight secueity coudl be implemented without changing the API .. limit the number of operations per time or something, but let us not tie developers hands together in the spec; let us see how it goes and offer a reasonable 'catch mistakes' form of security.)

Plugins should not have the option to confuse the user - like randomly jumping tabs would do. Besides that it could possibly break the plugin loosing the focus or result in race conditions where the tabs are changed back and forth. There should be a queue though in which plugins could request to be shown or announce that something changed, much like blinking of tasks in most modern taskmanagers.

- rather than build (for now) an API for a plugin to ask for permission for various activities, let us just make assumptions -- a plugin can do whatever it wants to its tab(s), and not much of anythign to anything else.

Sounds ok.

- a plugin is a standalone application, so can really do anything it wants; this is a security concern, when running any foreign application and so out of scope. But remember -- a plugin could play audio, capture keystrokes, or take over the framebuffer on its own. In general, it will take effort for plugins to be so badly behaved.. a naively written little plugin will use the API and crash itself or whatever, without really bothering the user; but a malicious plugin can go nuts!

...

[b]Stability[/b]:

- plugins will be standalone applications that communicate with the host (minimenu presumably); if they crash, minimenu will not be impacted.

- ex: If a plugin opens tab(s), and then crashes, minimenu can detect the crash and close those tabs implicitly or notify the user or whatever; it will not bring the system down. (Which is important, since mm coudl be someones launch desktop; if mm is made crashing, and thus making the device hard to boot, this is BAD. Recoverable, but bad!)

Running a plugin as a standalone application doesn't feel right in my opinion and will probably result in a somewhat more complex API. So it might be enough to split it in a second thread and try to add signal handlers to help plugin developers. Plugins, for usability and security should also be "approved" and checked using our own TCR (technical certification requirements). Plugins which would follow the guidelines would be marked, those who don't follow them should not be provided officially and developers would have to distribute them on their own.

- minimenu reserves the right to not run plugins at all; ie: if mmenu crashes, comes up, crashes again, we may build in logic to flip skin back to default and turn off plugin mode, automatically, so that when it comes up again, it has a higher chance of not crashing.

Sure, absolutly necessary

[b]Architecture[/b]:

-in general, many plugin systems use shared libraries for the plug; the host scans its searchpath and finds plugins and links them live, and then the plugins API is called back and forth. This makes coding easy, but means if the plugin crashes, everything crashes.
- instead, I'm going for the approach where a plugin is a separate application, communicating via pipes with the host. ie: plugin stdin is from minimenu, and plugin stdout goes to minimenu. All dialog is via this -- if minimenu has 10 plugins, it has a pile of pipes open. (or sockets, but pipes make it much more convenient .. you end up with perl, python, shell scripts, all the way to C or whatever, being able to make plugins. Very handy!) IF the pipe breaks (due to plugin crash), mm just gets a 'pipe broken' error and drops the plugin and tabs, piece of cake.

As mentioned before there might be other ways to handle this. I'd also be interested if and how much more CPU and memory demanding the stand-alone application solution is.

- communciation across the pipes will be a lightweight form, essentially a slimmed JSON. Every language can parse JSON, and some 'just do it' (ie: python). IT is easy and fast to parse, without being 'heavy' like XML usually is. The specification for functions is below/.

- synchronous; for this spec, the host or plugin can make one call, and shoudl not do anything until a response (or timeout) is hit. Currently there is not to be batching of instructions, though perhaps in the future we may permnit this (especially for rendering?). For now, make one API call, then wait.

Sounds fine, although I have no idea how JSON works, nor can I see how this would be used except for plugin initialization.

- It is expected that plugins will have an intermediate wrapper do the work. ie: If you're writing a plugin in python that really just wants to be a MVP (model view) thing, with the python fetching weather data and then having minimenu do the rendering etc, then you don't really want every python plugin writng its own api. I fully expect we (and the public) will make up wrappers that we can hopefully all share. Consider:
python plugin <-> python wrapper (pyMMPlugin or something) <-> minimenu
C plugin <-> C library ( libmmplugin.so or whatever) <-> minimenu

The wrappers should be included in the plugin - it's probably a simple task to slap an elf infront of a python script to do all necessary wrapping, turning the plugin into an actual native plugin.

- I debated a few different philosophies; be it plugin is doing its own rendering and handing rendered data over to minimenu for display, versus minimenu doing all the work with a heavier API, so the plugind tend to be more simplistic, etc. After some debate, I've decided to opt on the 'expected use case' -- a plugin will probably be a simple 'model' and have mm do the heavy lifting (loading assets, rendering, playback, etc.) ie: It is doubtful many plugins will be writing fiull games or heavy animations, but instead offering things like file browsers. Do correct me if my assumption seems incorrect. By moving the heavy lifting to mm, it makes heavy graphical plugins hard to make, but makes simple plugins easy to make. Example -- a weather app ro file browser is rendering once, or perhaps redrawing after key events, so is not heavy rendering, so mm doing the work is fine. For a game with 50fps or something, using proxy-rendering in mm would suck.

- In the future, we could extend the protocol; if we need better graphic APIs in the interface, or the ability for a plugin to do its own rendering and hand the ready to use artwork to mm (over shared mem say), or even have host supply cvoords to the plugin and the plugin just render to the box specified using its own rendering code, then we could build that in. We'll just have to see where it goes. Yes, I know, by mentioning it here, I pretty much require someone to make the send plugin ever to be a Pacman or Space Invaders game, and thus break the whole plugin concept day one. I'm ready, bring it.

-- for a video player say, you'd want mm telling the playback tool where to draw;p its just not possible with this spec to receive high speed wholesale screens and show them; for that, we'd extend the spec

Minimenu should work as a simple GUI handler, nothing more. It should provide a number of signal sounds and graphics etc which could be used by applications for appropiate actions.
If a plugin needs more than that it would either have to use an independent picture box [or possibly child X window to do the other things you suggested?] to render stuff like new weather icons and request for it to become a standard for themes to include that icon. If it needs more sounds it would also have to provide it's own player for that and make minimenu aware that it uses custom sounds.

- We're talking about an event and call model; host (minimenu) will post events to the plugin, and the plugin can make calls to the mm hosting API. There can be some give and take where the plugin negotiates features with the host, so that we can default some behaviours off and let the plugin ask for them (which is another way of saying, we can extend the protocol over time, and the plugins can ask for features to see if present or set things up, and thus know if they're being run on too old a minimenu version or whatever.) (I'd liek to avoid versioning issues, but such is life.)

The version number check should be provided by minimenu itself by checking the plugins plugin-protocol-version. That way backwards compatibility could possibly be provided in the future.
Besides that it's basicly what I just mentioned for audio [telling minimenu that the plugin uses custom audio].

- most common needs shoudl be covered by the spec, but uncommon (by expectation) APIs may need ot be done by plugin. ie: We're not going to have mm pass lid open/close to plugins, or have timer functions. A plugin is welcome to cruise through the pandora API for lid open/close, or to open its own timers. A plugin _is_ a standalone app after all, so it can do a lot of its own screwing around. But since most plugins will not (likely?) be games, we don't need ot offer high granularity timers to plugins.. they can do it on their own. But for catchign button UI, a lot of plugins will want to do that, so we make an API for it.

- note that mm will maintain tab state; if tab is fronted (and rendered by plugin) and then backed, and then fronted again, mm will ensure the tab content is still showing (and not blank.) The plugin thus could decide to render on first fronting, and not again,m if it doesn't need to.)

I agree.

[b]Protocol[/b]:

- I will leave the actual JSON protocol out of this; it is not critical for specification, but in essence, something like ..
{ func: NAME, args: { arg1: value, arg 2: value } }
with response being aninteger, a string, a hash of values, whatever, as defined below.

TBD as coded, really.

[b]Setup:

[/b]- host mmenu, when plugins are active etc etc, will iterate the searchpaths and pull up a list of plugins. For each found, and for those the user has enabled (see Security above), mmenu will then pass a startup event to the plugin.[b] 
[/b]- host, on close, will pass an event to the plugins so they can do cleanup[b]

Events:

[/b]- Startup event (mmenu has loaded the plugin and hooked up the in/out pipes): eStart ( plugID ) -> ACK response only, responding with plugin name. It is not defined if mmenu will be running all plugin startups concurrently or in queue .. plugins should not care, and leave it to host to decide :) 

- Shutdown event (mmenu needs to close, only happens on controlled exit and not crash): eShutdown ( plugID ) -> ACK response only. -- this in general shouldn't do anything, since not many will be start/stopping their desktop environment. Instead, I would suggest plugins do their work during the Startup event, and then save state during config changes or during interaction, for now.

- Tab change event -- by default, plugins do not receive this, but as noted below a plugin could request it. eTabChange ( tabID from, tabID to ); if this is registered, the event order relative to tab-activated/deactivated is not defined, so pick one or the other to operate on.

- Tab activated -- when a plugins own tab(s) become the active ('on front of stack'), they get this notification. If a plugin has 2 tabs in a row, it could well see 'tab deactive'tab active' in a row, as one loses focus and the next gets it. The order of events it not defined, so play smart. eTabFocus ( tabID ) -> expects ACK

- Tab deactivated -- when a tab loses focus. eTabUnfocus ( tabID )

- Key Down -- when a 'button' is pressed; by default no notification for this, to save on bandwidth, but plugin can ask for it. Technically, plugins could just open up /dev/event and do their own IO for keys, and may have to for lid type events etc, but to keep interface clean for 'most plugins'., we're offering button interface here. Buttins include face-buttons or keyboards, but not touchscreen, USB joysticks, etc.
Security: Plugins to receive input events for their tab when focussed;
eKeyDown ( key )
eKeyUp ( key )

- Pen Down -- similar to buttons, but for touchscreen; again, by default this will not happen unless requested, and the request documents the scope .. ie: only when plugins tabs are fronted, or for other tabs or the root-window or what
ePenDown ( x, y, pressure )
ePenUp ( x, y, pressure )

Plugin samples -
- the 'weather plugin' for instance would handle Startup event and request to open a tab; then it would poll some weather server to get info, and request loading some resources in mmenu, and finally, on eventFocus-thing, render the tab.
- for a plugin that makes swooshy noises on tab change, it needn't open its own tab, and instead just request and monitor tabchange events, and play fart noises as it sees fit.
- for a plugin that shows a battery indicator or date-stamp, it could just request permission to render on the root window (outside of its own tab, and may not have a tab at all), and then on tab change (or perhaps on its own timer event) request rendering.

I think it's too early to think about the plugin API at the moment. But it sounds ok-ish so far, even though I think that it provides too much functionality already.

When I say ACK or NACK, it means 'to be defined'; ACK may be nothing, or maybe its a true or 1 value, for instance. NACK could be nothing, or maybe a false value, or whatever. I'm just using it to clarify intention here.[b]

Host API [/b]that plugins can call:
 
 for Tabs:
 
 - TabNew ( tab name ) -> receives ACK and tabID back
- TabDelete ( tabID ) -> receives ACK or NACK (fails if deleting another plugins tab :) 
- TabList () -> receives ACK and a list of all tabs in the application
tabID : tabName : owner pluginID
Security: it is assumed the plugin and tab listing are public info; or if we need to have a request-permission model here, we can .. should plugins only get a list of their own tabs, unless they've asked to be able to see all tabs? I dont' want to get totally ouut of control here
- TabFocus ( tabID ) -> request to flip tab to front; receives ACK or NACK. This would trigger events for tab-change and tab-unfocus and tab-focus, etc, with order of course being undefined.
- TabQueryCurrent () -> gets ACK and tabID of current tab or NACK if not permitted (TBD)
- TabFocusNext () -> request go to next tab, whatever it is, receives ACK
- TabFocusPrev() -> request to go to prev tab, receive ACK
- TabMove ( tabID, afterTabID or 0 for first ) -- do we need this? should never really reorder tabs on a user I imagine, but thought maybe someone wantd to open tabs, then change their names, and sort them
- TabRename ( tabID, name )
- TabShow ( tabID, true falase ) -- do we need the option to be able to hide an existing tab? ie: so a plugin could oipen 4 tabs, but only show 2 of them, and late reveal two more? or should it just open them then?
 
 for Events:
 
 - EventRegister ( eventname, true/false, option ) -- expect an ACK/NACK  value. ie: a plugin may wish to ask to be notified for tab change  events, and mm may accept or refuse this request.
 ex: EventRegister ( eTabChange, True )
 ex: EventRegister ( eKeyDown, False )
 ex: EventRegister ( ePenDown, True, AllOverScreen ) -- some plugins may  need to watch events outside of their tab .. they might not even have a  tab. mm reserves trhe right to refuse to offer this, depending on user  security or level of mm code completion :)  
 
 for Resources:
 
 - AssetLoad ( file basename ) -> receives ACK and assetID, NACK; searchpath is as noted above, implied, and full paths are not permitted here. (or maybe we will allow some, but not many. TBD.) Note that mm/host is free to return an already loaded asset's ID, so do not assume it is unique.
- AssetFree ( assID ) -> requests free of asset; receives ACK. Note mm/host is free to just say 'all good' and not ditch the asset, since maybe it doens't know how, or maybe its being used by another plugin with reference counting, etc.
 - FontLoad ( font file basename ) -> receives ACK and fontID

- maybe some convenience routines; ie: instead of doing live-render scaling, could load asset and then pre-scale it, if expecitng to draw and undraw it a lot
- AssetScale ( assID, scale function, new width, new height ) - return ACK and new assID (client shoudl free previous ID if it wishes)
- AssetRotate ( assID, degrees ) - return ACK and new ID, or NACK

for Rendering:

Note -- a plugin can request mm to render various things, and note that mm is free to not render antying it doesn't want to, or to render on top. ie: IF a tab is open, but the user brings up a mm menu, the menu may well be alpha-drawn over top of the plugin window. Plugins should not assume anything.

Note -- at this time, plugins requets all rendering by API to the host; they do not render themselves or load artwork assets themselves. If needed for high speed or craazyness later, we can consider it.

Note on target -- each render operation has a target, so host knows where this is relative to; be it relative to tab, to root window, or to conf panel, or even another plugins tabs (future possibility.)

- RenderText ( target, string, x, y, fontID, size, color, alpha )
- RenderAsset ( target, assetID, x, y, offset Y, alpha, optional clip W, clip H, offset X, option to scale type or leave too-small or clipped? )
- RenderLine ( target, tl x, tl y, br x, br y, color ref, alpha )

- PlaybackAsset ( assetID ) -- could be mod or mp3 or ogg or whatever; not sure if I want to do this, or leave it to plugin to play on its own ... ie: then we end up with mm needing to have API to list known media types with loadable playback software, and events for end of song, and api for restart, change song, etc. Sort of heavy feeling, but not sure.

- do we want to have API for working box .. ie: so plugin can say 'set origin here' and 'clip me at this (w,h)'; thats all well and good for toolkits, but don't want to get really heavy here.

example: for weather type plugin sample, it'd just render some text and sun/clud/rain type artwork, piece of cake
- for file browser, plug would just render some lines, some text, some folder or file art, maybe some buttons, no big deal

 for Configuration:
 
 -ConfNew ( config entry name )-- receives ACK or NACK; requests that a new Config Menu panel be added for this plugin. Note that config panels in current mmenu are full screen for this reason :) So user may hit the menu, then hit Config, then hit Plugins, and see a list of plugins, and then pick one and get a config panel option.
- conf events not yet defined, but consider the above -- we'll probably have to add ConfFocus and ConfUnfocus type events, so that config rendering can occur

Now this is where stuff gets bloated..

- by default, plugins will be disabled

- resources; when a plugin requests loading a graphic or audio or whatever, the searchpath mm will use is _implied_; ie: it is the plugins path, plus the system path, notwhere else. The plugin is not going to be pully artwork willy nill across the device and making a mess, unless it goes out of its way to do so. (Possibly we may relax this, or make a API function so the plugin can attempt to extend the searchpath for loading resources, but we'll see.)

- plugins will have a specific searchpath; akin to other searchpaths in libpnd, I figure we'll search something like: /etc/pandora/mmenu/plugins:/media/*/pandora/mmenu/plugins -- something like that; ie: a user could deposit plugins onto SD or NAND (and some coudl be shipped with firmware if desired). Plugin searchpath would be defined in minimenu conf file, so could be editted by clueful users (not average joes.)

- plugins will have to be enabled/disabled on a per-plugin basis; ie: user enables plugins _at all_, and then the plugin conf panel would list all known plugins (default set to 'not run'),and flip them to 'run' if they wish. (ie: similar to the tab yes/no and app yes/no conf panels already present.) Maybe this is annoying, but seems safe -- user drops new plugin onto SD, and it won't magicly just run until they enable it. 

- one idea I've had with the layered host-code, is that a plugin interface could in theory be made for xfce or other environs .. a mini-minimenu; the idea being that perhaps a useful mm plugin is made, that someone might want it on xfce as a widget or something. But yes, plugin/accessories/panels APIs are a dime a dozen .. shoudl we be making another one, or doing this at all in mmenu? hmmrdf :)

I think that minimenu shouldn't even be the plugin host.


The plugin host should be provided by the Pandora OS and act as some simple database where every PND application which is currently running is registered.

The application would then decide wether it tells the OS (or PND manager) to act as a priority-module [getting exclusive permission to some sysfiles etc] or if it should run as a background-module [which is basicly what would be necessary for multi-tasking etc] or handler-module [not doing anything except for adding video/audio/etc handlers to the system].

A game would be started as a priority-module (instantly taking over the entire system, possibly asking the OS or user to unload some plugins), an emulator or the weather plugin as background-module (which could register a rom handler or no handler at all. It would take over when a rom is started then)
All of these PNDs would then register possible options in the PND manager.

The PND manager could have a simple GUI front-end using Gtk or something similar to work under a full desktop OR it could be controlled using minimenu which would poll for the options of a plugin, then render the GUI in it's own style. If a plugin requests a canvas its up to the application (minimenu in this case) currently connected to the PND manager to provide or block that.

A video player would therefore be a background-module which adds all necessary handlers. If a video file is selected it would request a canvas and temporary priority mode - if a game (or another priority application) runs at the same time this could fail instantly if the game blocks that action. If no other priority-module is currently running minimenu would switch to another view and provide a canvas and set up a new GUI. Below, next, or on-top (PIP style / controls overlay) it.
In the full desktop the request for a canvas could be handled in a similar way.


That way plugins would not only be limited to minimenu, drivers could be implemented in PNDs (by disabling the background-module which started it) and a unified API could also bring down the memory usage.
Plugins could also provide a lot of art on their own for the canvas mode by including them with the PND.
They would also manage their settings using the existing PND code.


In that case it would a lot more sense to add a skinning engine to minimenu which handles the original menu, provides standard icons, etc.

 

[skeezix]

I dont' think we're at the stage yet where we're ready to overhaul the whole OS (as a minimenu update -- we've not the manpower to pull it off. Certainly some changes in the future would be really nice (I've been leaning for a long long time towards having pnds as iso+zip for instance), and the whole pnd-running system could be redone as flagging the pnd as executable with a binloader, or your idea of a fuse filesystem is clever, etc and so on. But thats all 'firmware 2' or 'firmware 3' type overhaul level.) Here we're talking about relatively incremental bump. (But obviously you're thinking of avoiding doing useless work now, and fodling it into a future method of doing it, I know, but thats a long ways off imho.) So, keeping the scope relatively small and specific here..

(and thanks for your input, lots of good ideas there)

I do think plugins have to be separate, just to keep stability high. Making them as shared libs sure simplifies a lot of things mind you, but it does mean a rogue or buggy plugin coudl ruin the experience; if plugin is separate, it means that at worst, plugin crashes and the tab vanishes, and user says 'fucking plugin' and moves on.

That said, the big stuggle in my mind is where to place the frendering emphasis, and there are several all correct approaches.

1) plugins are lighter -- mm or host does the rendering, and plugin just relays messages back and forth. That weather plugin becomes a piece of cake .. py script sucks down weather info, tells mm to render an aimage and done. A little blip of cpu at startup once per invocation per hour, and then nothing.. and no graphic code or dependancies in the plugin,.

2) pluginsd are flexible -- you want to make a video player as a plugin? or write games as plugins? it jsut doesn't make sense for mm to offer such an API level. Hence I was leaving this for future spec.

I do think I'll trim out a bunch of crap from the spec above and reduce it more, yes.

But what I think will happen (if it happens at all of course is --

Two apis:
1) lightweight aPI, very basic render API in mm/host. ie: draw text, draw lines, draw a graphic, thats it. Good enough for a lot of basic but useful plugins to show some text, show the time, whatever.
(and yes, a lot of plugins would be 'core' and supplied with mm.)

2) a flexible api, which really just means a plugin coudl ask to take over a tab, and mm would just say 'draw to root -window (x,y,w,h) and let it go, with some rules. Some tricky stuff like a video player .. if mm needs to open a menu over that area, it needs to be able to pause the foreign app and draw the menu etc.)

All told, this is stuff that could be slipped into existing mm (after som other proposed changes), to provide some utility, but without drasticly changing the system, and nil impact.

(As to performance, with plugins enabled, it should not be a realproblem; if a plugin is using a lot of resources, its basiclty being a game then USers would tend to adopt or dislike based on system impact I'd imagine.)

The 'app sources' plugin idea mentioned (as future) is the idea that.. mm knows all about pnd's right now (it also has a disabled file browser, since I didn't finish debugging it yet ; a user coudl conceifvably not want to use pnds at all, or want to use pnds plus file browser, or use file browser for running mp3s and videos, or whatever, or do esoteric things like use some andrew-fs styled plugin that goes over an ftp and lets oyu run dshit from there without mouting it. Whatever. Coudl take the 'pnd' out of minimenu and move it to a core app-source plugin, so users coudl have pnds or not. Likewise then the same api coudl offer a file browser for other file types or the like. But its a bit complicated so nothing to go near now (or ever perhaps.)

jeff