Pui

[Sphinxter]

Not affiliated with the project in any way other than loitering on the gp2x side for a couple years, just finished a game, re-emerged my world with a new hard disk and had some bandwidth, few thoughts on an interface for a small device such as the Pandora and thought I'd share my notes. Not exactly poetry in public but almost as annoying so I'll apologize in advance. First I looked at the real estate and the sketches floating about I liked and came up with a butcher chart.

All areas are to be flat faceless unanimated buttons that hold an image. Thinking on first experiment of a gtk frame with areas 1-10 as gEvas canvases or hbox and eventbox and a central clutter stage for 11. Much, much experimentation to be done.
A, 'skin', is a collection of bitmaps sized to fit the canvases in the /skin/<Skin Name>/ directory. Here is a rather crappy reference set from /skin/Default/;
http://pandora.fullsack.com/images/pui/active.png
http://pandora.fullsack.com/images/pui/inactive.png
http://pandora.fullsack.com/images/pui/deactive.png
http://pandora.fullsack.com/images/pui/top.png
http://pandora.fullsack.com/images/pui/bottom.png
http://pandora.fullsack.com/images/pui/left.png
http://pandora.fullsack.com/images/pui/right.png

The data to drive such an interface and how to organize it with room for customization is a problem I leave for XML, it's text, easy to edit and instantly leaves hooks for third party applications to join the fray. Daniel Veillard's excellent libxml2 for C makes it a snap. Sample XML;

CODE

<directory>
<skin>Default</skin>
<page>
<tabicon/>
<item>
<displayname/>
<image/>
<fullpath/>
</item>
<item>
<displayname/>
<image/>
<fullpath/>
</item>
</page>
<page>
<tabicon/>
<item>
<displayname/>
<image/>
<fullpath/>
</item>
<item>
<displayname/>
<image/>
<fullpath/>
</item>
</page>
</directory>




EVENT FLOW

INIT
1. Wake up, look for a default /skin/directory.xml on nand.
2. Load the default directory,xml from nand.
3. Load images from /skin/<name> directory pointed to by the style tag.
4. Set the stage according to it's items. These will be system tools in firmware.
5. Interface displays three tabs, the populated active Pandora tab and one inactive tab for each sd card slot.


TOUCH SD CARD TAB 2-3
1. Check/mount sd card.
2. Check root of sd card for directory.xml.
3. If not found offer to scan and write one.
4. If don't write one then in memory use.
5. As needed scan findfirst findnext for *.box with matching *.png in the same directory and optionally write it.
6. Load images from /Skins/<name> directory pointed to by the style tag.
7. The Pandora tab will be inactive, the other sd card tab will no longer shown as sd card tab chosen is made active.
8. Add tabs for each page tag found in the directory.xml with pages icon, stop scanning after fifth tab.
9. Set stage of first tab according to items in the first page, change icon to icon from first page.
10. Pandora icon/tab is always available and only changes state.


TOUCHING INACTIVE TAB OF OPEN SD CARD 2-6
1. Set current tab inactive.
2. Set chosen tab active.
3. Set stage according to the items in chosen page.
4. The five tabs of the open sd card are indexes into the array of actors, clicking them jump scrolls and switches tab art.


TOUCHING EDGE AREAS 7-10 AROUND CENTRAL STAGE 11
1. Scroll stage actors in the appropriate direction using different effects depending on which canvas hit.
2. This is the only animation except for the changing active/inactive/deactive tabs art.


TOUCHING ITEMS ACTOR ON STAGE 11
1. Launch fullpath from directory.xml replacing itself in memory to be re-launched atexit by app or as a child or
2. possibly in the background, reachable from a window list item in the Pandora tab. The corner areas at the ends of the top and bottom canvases could be used for such things.


USE CASE
User picks up cheap sd card, downloads a ton of games and applications, plugs it into Pandora, unit sees it, writes a directory.xml and starts using it immediately. Possibly pop up something like a bookmark organizer and arrange the items scanned in the future or an editor, use of xml supports third party apps and expansion.

 

[yosh64]

hey

I'd rather focus on a more flexible design, one where each component of the interface can be upgraded or read from an SDCard, and one where you can move each component around wherever you like.

So basically your screen is like a canvas where you can place and connect different components. So on firmware all you will need is the canvas, and then people can copy whatever components they would like onto their SDCard and place them however they like, the stylus is made for this sorta thing .

So if you want a time component, you just program your own (or download from somewhere) and place onto your SDCard.

I guess the core components would be things like a File browser, Application browser, Wallpaper background, Clock display, Battery display, and System Information display.

The canvas basically enables the user to define and place graphical items such as static and animated sprites, text, and the ability to customize fonts and cursors and global skins. It should also handle file associations and contain an application database for automatically building menus. There should also be functions for handling input and even raw graphics and sound output.

This is just a rough idea for now, so I might do a little bit more research into it and await feedback from others. Also, sorry for kinda hijacking ya thread.

Right now I think the inside pandora developers are taking care of the interface and such, and I doubt anything we make now will be in the initial firmware or whatever.

edit
I do think this canvas could be made to run both outside and within a linux environment.

cyas

 

[Yamara]

I'd rather focus on a more flexible design, one where each component of the interface can be upgraded or read from an SDCard, and one where you can move each component around wherever you like.
[...]
This is just a rough idea for now, so I might do a little bit more research into it and await feedback from others.

I think a lot of people enjoy that kind of flexibility. I'm happy as long as it's still easy to make it look pretty. I would appreciate something as simple as the iPhone icon moving (using this example because I don't recall others right now), where the icons are locked in place but can be quickly moved when, for example, an icon is held for 3-5 seconds. Being able to extend this to other parts of the GUI would be slick.

QUOTE

I do think this canvas could be made to run both outside and within a linux environment.

I would love to be able to keep the same UI both in whatever comes stock and in whatever extra linux-ness I might install.

That said, I had to rewrite that last sentence 3 times, as I realized I'm not entirely certain what will be "stock", despite my regular forum reading. I have vague memories that there will be a linux kernel, with core libs on the system, so in that regard there wouldn't really be an 'outside linux' unless you installed BSD or something on it.

 

[Sphinxter]

I'm not trying to stop anyone from spending their time editing mime data and building a shiny desktop based on the intimate knowledge of the content on your sd cards that will be instantly reduced to a pretty dysfunctional pile of broken links as soon as you pop one out.

The basic point I want to make is, "Document Based Browsing Does Not Work", your file of mime data is in nand with the file browser and points to mplayer for mp3 files, your sd card has the songbird app on it you just downloaded, double click and nothing happens, not a good user experience, double click the songbird icon that just popped up by itself and it knows what file types it likes, happy user experience. See, you've only had it for 30 minutes and you're already doing it my way, "Application Based Browsing Is The Only Way". Maintaining synchronization between nand and sd is something nobody should ever have to screw with. The pandora should leap to it's feet and be usable without any cards.

 

[Sphinxter]

hey

I'd rather focus on a more flexible design, one where each component of the interface can be upgraded or read from an SDCard, and one where you can move each component around wherever you like.

One should always stay focused. You can have a unique skin and style associated with any sd card, use the top element in the xml file, by changing the order, image data in it you can make the icons appear in any order on any tab you like. Graphical drag and stylus editors will no doubt be along later, that's a bit out of scope for what I'm starting on.

So basically your screen is like a canvas where you can place and connect different components. So on firmware all you will need is the canvas, and then people can copy whatever components they would like onto their SDCard and place them however they like, the stylus is made for this sorta thing .

So if you want a time component, you just program your own (or download from somewhere) and place onto your SDCard.

I guess the core components would be things like a File browser, Application browser, Wallpaper background, Clock display, Battery display, and System Information display.

I disagree, desktop elements on an sd card would make it near impossible to switch or update except through the pandora. May be what you want but I'm trying to jump the shark and create a simple yet fancy interface that just works when you shove in a card full of apps and let's you find and execute them quickly

The canvas basically enables the user to define and place graphical items such as static and animated sprites, text, and the ability to customize fonts and cursors and global skins. It should also handle file associations and contain an application database for automatically building menus. There should also be functions for handling input and even raw graphics and sound output.

This is just a rough idea for now, so I might do a little bit more research into it and await feedback from others. Also, sorry for kinda hijacking ya thread.

Cool, but a canvas is an actual X widget in several flavors, all hold an array of bits, they are not a container class. I'd like to keep this as technical as possible, let's try not to re-use terms and avoid confusion.

Right now I think the inside pandora developers are taking care of the interface and such, and I doubt anything we make now will be in the initial firmware or whatever.

My code is golden, these hands are touched by God.

edit
I do think this canvas could be made to run both outside and within a linux environment.

cyas

Really don't want to re-hash the choice of operating systems, let's stick to what's on top and what happens when you power up your new pandora before you even buy an sd card.

 

[yosh64]

hey

I'm not trying to stop anyone from spending their time editing mime data and building a shiny desktop based on the intimate knowledge of the content on your sd cards that will be instantly reduced to a pretty dysfunctional pile of broken links as soon as you pop one out.

I don't understand.

The basic point I want to make is, "Document Based Browsing Does Not Work", your file of mime data is in nand with the file browser and points to mplayer for mp3 files, your sd card has the songbird app on it you just downloaded, double click and nothing happens, not a good user experience, double click the songbird icon that just popped up by itself and it knows what file types it likes, happy user experience.

Firstly, why can't you also store mime data on the SDCard also? and so nothing is stopping you from changing the file associations.

Now I assume by "Application Based Browsing" you mean that all file navigation is done within each program, such that the person launches the pogram and then opens the file from it. In such a case then what is stopping people from still doing this even with a "Document Based Browsing" component?

Also, right off the top of my head I can think of three advantages to "Document Based Browsing". Firstly it unifies the whole process (think windows explorer), it gives people a choice, and it's second nature too most computer users (again think windows and even linux).

Again, you must remember that "Document Based Browsing", or rather a document based browser is just one component by itself, you can make things how you want. I see no sense in not providing some sort of file association database just because some don't want todo "Document Based Browsing", and a file association database can be used for more than just this purpose.

Although this is a portable device, I think the way people interface with it will reflect that of a regular computer. I mean, it's more than just a gaming system, it's a device that is open to whatever we want to use it for, and so I don't want to mimic the interfaces of any gaming devices. I don't mean to say your idea mimics a gaming device either, this is just something I wanted to point out.

Cool, but a canvas is an actual X widget in several flavors, all hold an array of bits, they are not a container class. I'd like to keep this as technical as possible, let's try not to re-use terms and avoid confusion.

Actually I just made the word up as I thought the word represented my idea well. I actually think of it more in terms of a paint program, hehe.

I disagree, desktop elements on an sd card would make it near impossible to switch or update except through the pandora. May be what you want but I'm trying to jump the shark and create a simple yet fancy interface that just works when you shove in a card full of apps and let's you find and execute them quickly

I don't understand what you are saying here, how do desktop elements on an sdcard prevent you to switch or update through the pandora? It just gives the users the choice todo what they want, sure you could provide a default theme or whatever along with the canvas on nand memory.

To be blunt, I can't fully understand your idea and find it hard to comprehend the way you have presented it. But based on what I think I understand of your idea I don't think it's the way forward, it's just another static interface (layout wise). On the same note, I have no problems with you going ahead and proving me wrong. Again, I find it hard to comprehend, and from what I think I do understand is that it's a static interface that allows some customization of only the appearence via XML. Hmm, I don't understand why you need to write a directory.xml file either. As such I tend to think such an interface is already in development, so you might be reinventing the wheel so to speak, but I can't be sure.

But yea, I'm actually a bit disapointed as it seems to me you have assumed alot of things about my idea and shot things down. I don't mean to say my idea will work, but I just want it to be considered and thought about fairly before letting it slide.

My main points of interest are (and I do think others will agree) are flexibility, an interface where you can easily program and add new components and move them around to change the layout, where you have a unified skinning system that all applications may use, and where you can change the fonts, cursors and other such things, where you have an application database for automatically building menus and a file association database for a unified file management, where people don't have to wait on new firmware for new components or whatever, and one that is also verstile such that it can be use both within or outside a linux environment or X.

When you mention everything here together here it looks to be quite a large thing, but I think the canvas is a solid foundation for such a project and I think this can be done . edit: Hmm, to be up front I'm not sure if I have the time, energy or motivation todo this at this time by myself, but I think it's what we should aspire to acheive and I do think it's possible. Hmm, maybe this will change if/when I get a pandora, hehe.

Finally I don't want to argue about whose idea is better, as I don't really know at this point. I do think it's nice that you bring up some issues you saw with my idea, but at the same time you made alot of assumptions to disprove and that don't reflect my vision and so I ask you to keep an open mind.

Yamara, you bring up a good point and it makes me wonder if the quick boot will also run through linux. Hmm, if this is the case then it makes me wonder if the quick boot will use X. It would be nice if anyone in the know could answer this.

edit
I just wanted to mention that if your idea/concept turns out to be a step ahead of what I think is already in development then that is cool and I would be eager to check it out. But again, I don't think it's the ultimate solution, and certainly doesn't do everything I'm looking for from an interface.

It might even be best to wait and see what the official interface has on offer and build upon it.

cyas

 

[Sphinxter]

I see mime data as a data entry task, mime types only work on the windows pc because all your apps are known by the os at install time, you do the final data entry by selecting an app from a list and checking a box. Half the time it doesn't for me anyway, I click on html and word opens or worse. You wind up wading and searching through unrelated files not even meant for a user to see. Searching you start typing, game over. You can't go fullscreen 1024x1280 in explorer and see 700 files at a time either. Mime types is useful for icon association only, applications are switched in and out on an sd card. The difference is mime and document centric browsing are made for the pc, not for a handheld unless it's the kind of device where you only have one choice with the application embedded too. You will have to maintain a database of every application on earth for it and what types of documents they like and roam the sd card seeing if any of those are available. A nightmare scenario to me, grind grind grind, is one of these the app you want? Look at what the pandora knows, what you can tell it and the amount of effort to keep that in sync, I don't see it working efficiently for long. You will most likely have file based browsing through thunar or midnight commander, firefox in firmware but I would not wish those for a startup app. Although putting the entire app in a gtk frame wrapping the mozilla widget executing functions in javascript in response to hardware signals really worked for me once, so I guess that would depend on the homepage with firefox as a may be acceptable.

I have 5 music players on my pc, which should I associate? What should it do if the associated application is not on the sd card, grope for another, how the hell does pandora know a music player from a text editor, you just downloaded it to an sd card a moment ago and plugged it in.
Whatever clicks the mime association saved you were eaten twice by the maintenance of it.
The use of the directory.xml is to speed up load, avoid wearing out your sd card and wasting time groping 8gb of slow card every time you hit a tab plus it lets you control the grouping, images and placement on the stage but only if you want to edit things, it will generate everything for you, no data entry ever.

My intention is to boot up show the embedded apps, file browser, wi-fi tools, configuration editors and such, push in an sd card and be instantly presented with all the applications on it ready to rock at a touch. No hassle, no browsing even, just insert and tap. Granted it does not look too hot at the moment but that may change when the clutter stage is rippling like a pond with every touch and your icons are doing the wave like an iphone. Widgets such as clocks and such are just any linux program written to the freedesktop spec, trivial to add but not of any concern at this early stage. "Form Follows Function", until it staggers to it's feet. I have not wasted any time on the skin for there are much more talented than I in that department. Candy is extra and will also likely arrive in time.

The problem with desktop components on an sd card is they break when you remove the sd card. Did you every have an ibm pc or a system that ran entirely on floppies? It's a bit of a different environment. You will have to maintain a database of available widgets and disable them when necessary. Delete all the docs linked to your windows desktop and think if the clock toolbar or calendar were also in that condition. Or you could just keep your desktop stuff all on one card and have only one to remove and use but I see one trading both cards quite frequently and having them as separate applications by functionality, the war dialer card, the game console card, internet play card. How will your desktop appear with no cards loaded is my question.
Ed. on second thought maintaining this would not be that hard with FAM

Hey maybe we will do some tiny little part of these that the development team see and think is worth stealing eh? More than one way to influence, be included in a program or device. Maybe just reading our discussion will combine with knowledge of things completely unknown to us and reveal a feature desired or save a dead end path from being traveled down. I don't usually do my thing in the public eye nor solicit input outside of a small group but for this what the hell and why not, maybe something good will come out as a side effect or just stir someone else like I seem to have stirred you into exploring another path, question is will you remember to give me credit when yours is the most successful interface in history. b-)

 

[yosh64]

hey

edited
Bah, decided to remove my old post.

Basically I don't think file based navigation is such a pain as you make it out to be and to put things in perspective it's only one potential component, but as I think you are trying to point out is wether any of this is appropriate for a quick boot program. I also think all of those problems you mentioned can be solve (or when looked in another way they may not even exist), but I come back to thinking the whole idea of mine might be a bit too ambitious (biting more off than can chew) and something more simple might be better.
edit: But yea, the application launched from double click can be user defined or based on the most popular program used, and you always have an 'Open with' submenu for times you want to use some other program.

I'm quite unsure about things atm, so if I do get into something like this I would have to make a prototype or whatever to see how it would go and how it works in practise.

Umm, as for your idea it seems pretty strait forward and something I would probably hope from the official interface. But I think even this is saying to much, I'd actually have to see your interface in action to be sure.

cyas

hey

Well I think I done too many quotes in my last argument. edit: I now wonder why I wrote argument instead of post? maybe I have not been thinking of your responce with clear thought.

Anyhows forget about my other idea for a minute, I was thinking it would be cool if the user could bring up information about each program, much like you see on the Nintendo Wii when you select a channel and it goes into the channels title screen where you can choose to play or move onto another application.

But yea, I can see where you are coming from when you say file based navigation or whatever is too much and the whole concept could lead to some headaches for a quick boot program, and I think I need to think about this more. edit: Although I'm not sure if these headaches are something real or can be easily avoided as was my thinking not long ago.

cyas

hey

Me again, I kinda feel like I have derailed the thread. Umm, so it's probably best to ignore my idea and see what can be made of your concept and such.

edit
I wanted to mention that I seem to like your idea for what it's intended for, and I like the use of the directory.xml for fast directory access. But yea, I think these directory XML files will need to somehow be made (does the user do this?) automatically when something new is added to keep it in sync, otherwise it's kinda pointless if you have to read through the contents of a directory regardless to see if it's synchronized as you will end up doing more reads, hehe. Anyhows I'm not sure how you would go about this for now.

Something I have no clue about, but you might wanna find out is if it's really so important to save reads from the SDCard anyhows. I would think you would more have to worry about writes.

cyas

 

[raster]

(note.. i'm just lurking waiting for pandora to actually be purchasable...)...

but i can say i'm definitely planning on getting one... and doing a full x ui for it... including a fb based bootsplash - already have this on another embedded (linux arm) platform. definitely recommend using X just for the sake of covering as many applications as possible. you'll need a window manager in the end... so that's what i intend to throw on. already got launcher and all the bits needed... just need a theme that looks nice... so i'm lurking watching whats going on here... but i do have working code waiting to be used...

 

[yosh64]

hey

raster: Nice to hear, a lite window manager designed for the pandora sounds pretty cool. I hope it supports the .desktop standard or whatever so menus and such are automatically made based on the software you have installed . Well this is something I miss in IceWM and find it quite a pain to update my menus everytime I install a new prog. Hmm, well I just use XFCE now days as it supports such things , although I like in IceWM how the user controls what goes on the system tray, but nevermind.

cyas

 

[trooper]

@ yosh64

I don`t mean to get on your case, But could you please follow the forum rules about multiple consecutive posts, As it is generally frowned upon here. I will merge your last three post in this thread, But in future if you could please refrain from posting more than twice in a row, It would be greatly appreciated.

Thanks,

Trooper

 

[Sphinxter]

In the original plan it says create the directory xml on the fly for the user and optionally save it. Use case would be you pop in a card and don't see the new program you added, clicking on the scan utility would update the tree by removing nodes not found and adding new ones to the fifth page and refresh and one could then move them to the right page with some editor function as yet unknown.

You made me rethink whether the directory.xml is necessary at all, a database of categories and programs on the device and some way to maintain them and their sort order could replace it's function. Maybe I'm doing it backwards.
Experiments are in order.
I need some icons for the tabs which should be part of the skin. I'm starting to think the gdk/gtk image widgets will do just fine by themselves for the tabs and reduce the requirements. Been years since I played with gtk+, forgotten what a joy it was. Also thinking the clutter stage should cover the entire area to the right of the tabs and 7-10 should be it's clutter's tidy buttons instead, size and skinned same.

Just something to do between games, probably drop it if an idea hits or better yet use it for my next menu system but after three days without programming life becomes meaningless. I actually sell this sort of thing once in a blue moon, nothing is ever wasted. Programming is like running, whether you're a junky running from the man or an athlete running a marathon, your program turns gold or turns shit, it's all just plain good exercise.

 

[raster]

well it's not hyper-light... but then again.. you have a lot of ram and flash to play with, but its not heavy either... but it fully support FDO standards (xdg .menu standards, .desktop files etc.). it has its own theme/layout system - but it can be entirely re-themed to do whatever you like.. pixelart, animations etc etc.

already runs/ported for the nokia n800/810/770, openmoko freerunner/neo9173 - its on my zaurus sl-c860, so it's working and happy... so just waiting for the hardware to pop out and be purchasable... my credit card is ready... (as i have no system to build it for - i don't know what the base os will work like in terms of packages or 3rd party software... or if we just have to build whole firmware images for any addons etc.).. let's see.

hey

raster: Nice to hear, a lite window manager designed for the pandora sounds pretty cool. I hope it supports the .desktop standard or whatever so menus and such are automatically made based on the software you have installed . Well this is something I miss in IceWM and find it quite a pain to update my menus everytime I install a new prog. Hmm, well I just use XFCE now days as it supports such things , although I like in IceWM how the user controls what goes on the system tray, but nevermind.

cyas

 

[Sphinxter]

All things come to he who hustles while he waits.

 

[raster]

All things come to he who hustles while he waits.

already done my hustling. finished. already working on nigh-identical hardware many years of hustling already done.

 

[Sphinxter]

All things come to he who hustles while he waits.

already done my hustling. finished. already working on nigh-identical hardware many years of hustling already done.


What I'm looking for is more like a graphical version of the run command in enlightenment 17 only hung in an animated 3d space without any typing.

 

[raster]

All things come to he who hustles while he waits.

already done my hustling. finished. already working on nigh-identical hardware many years of hustling already done.


What I'm looking for is more like a graphical version of the run command in enlightenment 17 only hung in an animated 3d space without any typing.


aaah. well going to have to disappoint you - not doing 3d here as well... that gets nasty if u want to mix it with 2d and x... but.. this is enlightenment 0.17 i'm planning on throwing on... it has modules to make it "handheld" (give it a different layoutpolicy, launcher etc.). i've been meaning to get a device with real hardware openglES to play with... and pannie seems the ticket.

 

[TomCooksey]

Hay Raster, I was wondering when you'd show up here.

Trolls are also itching to get our hands on pandoras.

Hmmm, just need an OpenedHand representative now...

 

[raster]

Hay Raster, I was wondering when you'd show up here.

Trolls are also itching to get our hands on pandoras.

Hmmm, just need an OpenedHand representative now...

can't be trolls anymore! conversion into crazy fins is now complete!

 

[Sphinxter]

aaah. well going to have to disappoint you - not doing 3d here as well... that gets nasty if u want to mix it with 2d and x... but.. this is enlightenment 0.17 i'm planning on throwing on... it has modules to make it "handheld" (give it a different layoutpolicy, launcher etc.). i've been meaning to get a device with real hardware openglES to play with... and pannie seems the ticket.

Cool, definitely be my choice if I go that route, huge enlightenment fan here, daily user for about 7 years now, don't even have qt installed and only a third of gnome. Was thinking and may still use the EFL for parts, was leaning toward Direct FB instead of X if I can get clutter to work with it. Maybe a picture will explain it better. clutter