Pre Installed Programs

[skeezix]

Patching the OS should not destroy anything, with this design as well. Its only 'full firmware reflashes' that could be dangerous to user data in NAND, but that shouldn't be the norm. (Who knows at first release, but I would expect when things are settled there will just be patch sets released which modify the firmware in place, without impacting user data. Note that if you've installed things on the unionfs firmware, then patches will be against the real firmware, and the union will then kick in and cover up those patched. So worst case a patch runs, but the previously installed user installed packages cause issues. Not over likely, but possible. But then, most people will likely be using pnd apps instead of Angstrom or linux packages.. in theory

jeff

 

[Svartalf]

Well, first ..

You can of course do anything you want; your box, open source evreything, make it do what you like But let me discuss a bit from a normal user perspective (ie: not hacking in knee deep.)

Heh... He beat me to it with an even more detailed (the what as opposed to some of the whys...) explanation. Cool!

 

[Zeno Arrow]

The NAND storage has never been off limits, don't be so over dramatic. If you want to install apps to NAND you can do.

Trust me, I sometimes know what I'm talking about.

Clearly you don't, otherwise you wouldn't have to ask.

It was months before you even joined that it was said, and no one has said otherwise.

Installing apps to the NAND has been mentioned numerous times since I started lurking here.

We don't really want anyone to mess with the NAND in any way, this includes programs trying to install libs to it.

We don't want anything being installed on the NAND including non standard libs, we want programs to include any libs in their own folders so everything is neat and tidy and runs without issues (no looking or asking for libs from elsewhere), I hate the way windows becomes bloated with libs everywhere and we have a limited amount of NAND to work with.


This was, of course, about 4 months before we got the 512MB upgrade, so with twice as much space it may no longer be an issue, but I don't want to run up any speculation until someone says "yes, we have a plan for upgrading the OS without botching any of the programs you have installed to the NAND"

Look, it's your Pandora, you do with it as you wish, craigix may have offered some advice of a sensible way to install apps but you can do it other ways. Take your NAND storage and partition it to split it, what do you have now? You have an area where firmware/OS can be written to, and an area you can use without interfering with the firmware/OS. Simple. Why do you need to have some official word from the devs when the solution is staring you in the face.

NAND currently has some drawbacks, you don't want to write too heavily to it because it will wear out quicker (though the lifespan will still be years unless you go crazy with it). Other than that, use it how you want.

 

[x68000]

All sounds great Jeff! First time i've been excited about the homebrew scene since the old GP32 days!

 

[skeezix]

It is fun times for sure. The libpnd code is up in the GIT if anyone wants to look, but its not documented really well and is changing fast (some docs in private locations).. but it'll be settling down soon so I'll start working on docs for it. Anyway.

After that big post above, I expect *nerdgasm* since most of it no one has heard before.

Kudos to everyone in the pandora dev team.. everyone is working pretty hard. I love EDs videos .. he brings so much personality to it all I'd drop a bunch of kudos to everyone, but not sure who wants their name dropped or not. Kudos to vimacs for coming up with some of the key points above.

If only this was a couple years ago when I had infinite coding time

jeff

 

[Svartalf]

Look, it's your Pandora, you do with it as you wish, craigix may have offered some advice of a sensible way to install apps but you can do it other ways. Take your NAND storage and partition it to split it, what do you have now? You have an area where firmware/OS can be written to, and an area you can use without interfering with the firmware/OS. Simple. Why do you need to have some official word from the devs when the solution is staring you in the face.

A full flash update to the NAND will likely overwrite ANY partition re-work you might have done (since it resides in the beginning of the store...). Doing that won't prevent this issue, sorry. Fortunately, for everyone, it should be a rare occurrence that one would have to do that sort of thing. Most updates to the firmware should be .ipk/.opk files and you use opkg to install/update the new stuff on the NAND.

 

[cb88]

just some thoughts i have about the unioned fs

you should have an unmount/sdcard eject program... you shouldn't remove the card while you have the loops mounted and then updating settings pretty much your doing it backwards.. maybe you just said it that way and meant unmount loops first then umount card then remove last

a program to join .pnd files would probably be a good idea for commonly used programs since mounting say a hundred apps will probably have enough overhead to be considerable especially on a device this slow. I think it isn't as big an issue for cloop which doesn't have that good a compression since DSL allows like 256 cloop modules.. puppylinux usually limits to 5 total mounted loops (which is probably underkill)

thats just what experience I have with unionfs/aufs from making squashfs packages on puppylinux

will the menus use .desktop files which are the standard since the unioned fs route has been chosen for packages? .desktop files are pretty easy to make as well.

now my question is... what fs are you using my experience are mainly with squashfs both aufs and unionfs versions with and without LZMA

 

[Vimacs]

you should have an unmount/sdcard eject program... you shouldn't remove the card while you have the loops mounted and then updating settings pretty much your doing it backwards.. maybe you just said it that way and meant unmount loops first then umount card then remove last

Well, we cant prevent the user from removing the sd card at any possible time so we have to take care of it that way,
and haveing the users umount the card manually before ejecting the card would probably also be quite bothersome for many users.

a program to join .pnd files would probably be a good idea for commonly used programs since mounting say a hundred apps will probably have enough overhead to be considerable especially on a device this slow. I think it isn't as big an issue for cloop which doesn't have that good a compression since DSL allows like 256 cloop modules.. puppylinux usually limits to 5 total mounted loops (which is probably underkill)

nah, you got that wrong, .pnd files are not mounted directly, they are only mounted once you start them and automatically umount when the app closes.
Thats why the pxml/icon isn't actually inside the fs part but appended to the file.

now my question is... what fs are you using my experience are mainly with squashfs both aufs and unionfs versions with and without LZMA

.pnd files can be iso or zip currently, while zip is probably a bad choice to use and may be removed before release.
Its also quite possible that there will be other fs types supported in the future for pnd.

 

[Karel Jansens]

Well, first ..

You can of course do anything you want; your box, open source evreything, make it do what you like But let me discuss a bit from a normal user perspective (ie: not hacking in knee deep.)

<snip explanation of .pnd files because -- well, it´s long>

That is brilliant!

I just thought of something: If we can get reasonably compact .pnd packages for say an ebook reader (which has to be FBReader, obviously B) ), an MP3 player and a video player, one could simply copy those packages to every card that has those specific media on it, with no need for permanently installed packages. This is assuming the packages will be compact enough and rely mostly on the system libraries in the firmware.

 

[WizardStan]

I just thought of something: If we can get reasonably compact .pnd packages for say an ebook reader (which has to be FBReader, obviously B) ), an MP3 player and a video player, one could simply copy those packages to every card that has those specific media on it, with no need for permanently installed packages. This is assuming the packages will be compact enough and rely mostly on the system libraries in the firmware.

This was actually a bit of the concern I had over the conflict as to whether the internal NAND was officially usable or not. If you have multiple cards with the same reader, what happens when there's an update? You'd have to update all the cards. It's not a serious concern, just one of those mosquito like "that'd be annoying" things. I was thinking more with emulators (which are more likely to need updating than ebook readers or MP3 players) but the idea is the same.
This also means that you can stick certain customizations on the internal flash as well (I'm thinking things like custom scripts, or modified VI config, etc...) rather than having to keep the same SD card in one slot, or copy any custom scripts between cards. Not likely to be needed either, but still a good thing to have.
When one person in-the-know says "we don't want you to use the NAND" and then nothing for 6 or 8 months when another steps in and says "you can write to the NAND" it sets my curiosity to wondering how we got from one to the other.
And now, knowing this much of the plan, (which probably wouldn't have been mentioned if I hadn't asked I might point out ) I have to say: I like it!

 

[skeezix]

due note .. we have limited resources (a few people working in their free time), and the core guys working on the main device and all that.

So when talking about pnd-joiners and all this and that, well and good -- ideas are good, maybe keep yourself a log of some good ideas up on the wiki or something. But for first cut we cannot aim too high. We have to aim for something managable and that helps define future standards and directions.. aim for something that will be usable and solid, and yet suggests where things want to go in the future. Something like may or may not be a good idea i the end, but need to use the system first and see how performance is and all that before nailing it a priority; something like that is clearly a 'phase two or more'. For phase one, we need to be tightly focused.

(Aiming too high just gets you hung. Further of course, its better to aim low, and then let things evolve as more usage experience is garnered. ie: aiming high and running that way might get you to the wrong target; better to aim lower and hit that target, then aim for more little targets and alter them as you go, so you end up where you really want to be. Moving targets et al.)

(Yes I'm a pro dev in real life, hence how I think like that

jeff

Its still early, but as things settle more we can open up more; no point in folks fiddling now while things are in flux, but in a little while tech folks might want to fiddle if they like. We'll wing it

 

[cb88]

so.. what is it doing copying the file into ram? in any case i can`t see how this format could be at all efficient unless there is something I never heard of and im not not exactly too uninformed on these things

afaik if you can mount an ISO directly but that would be pretty silly IMO since there are better formats (cramfs squashfs and even cloop would probably be better) for this since there isn`t any compression support there

how is what you are doing different from say

persistent / and everything underneath
apps unioned under /user somewhere
and all changes to everything under / written to a separate partition on NAND or SD ie an overlay since mounted apps and the nand will me mounted ro?


Are you not making an SD eject app/trayapplet? I`m sure everybody will enjoy losing data when they remove their cards without syncing and unmounting.

nothing I have mentioned is difficult at all how is that aiming high? you mean you can`t write a quick gtkdialog (perhaps get permission to borrow the one from puppylinux or slitaz the slitaz one already supports multiuser afaik puppylinux is root only) app to umount drives surely someone around here could if you guys don`t know bash?

oh and it would probably be a good idea to include gtkdialog or zenity or something since it might be useable for creating launchers for games that require command line paremeters and such

@skeezix it seem to me like you guys have over complicated the whole matter from the git go where a simple /usr/pnd folder where the app`s extra informations could be accessed when mounted. every suggestion i have made about this sort of thing has built on another simple easy to use idea like app launchers from the rox desktop for example. It isn`t usablility here that is an issue it is simplicity and maintainability. It isn`t about aiming high it is finding the angle you need to aim to shoot the furthest distance

also from what I understand the .pnd files have thier icon embedded right? well wouldn`t that issue be completely circumvented if they integrated properly with the menu to begin with upon the card being mounted and .destop file appearing in /usr/local/applications and then just updating the menu and i guess the launchers on the desktop as well?

 

[skeezix]

Theres nothing saying a .pnd has to only be iso; the PXML can identify the format it is.. just as I say, we're aiming to support limited things up front.

We've also just working on this, so performance analysis and such will have to come. Perhaps I mention it all too early, but hell with it thought it might be interesting to show our goals (easy app installation etc.)

If you just have a regular filesystem with union onto SD, then you run into issues when switching SDs around, and end up with data spread willy nill all over it as the union definitionsd suggest, not under app or user control.

The system we came up with is certainly not the only one that could work, but it should work and be pretty simple to use, which is the main thing.

ie: Download a .pnd, use it, delete it, and everything is still clean.

jeff

 

[Vimacs]

Also the reasoning for choosing ISO as th "fs" is that its universally supported on next to any os and thus makes it easy to manage for users while sqash/cramfs etc would be more complicated on windows/osx, and it is faster for random access as well and has the least overhead.

 

[verm71]

Vimacs, skeezix, awesome design. This will do wonders to those who don't want to worry about the complexities and just want to have a simple traditional game-on-a-cartridge console-like experience. Apps-as-a-plug-in, whodathunkit?

Question, though. What happens if the SD card is removed or changed while the Pandora is shut down? Will the necessary .desktop and other cleanups and rescans still occur?

 

[skeezix]

sverm -- sure; next time it boots, or at least next time you stick an SD card in, the rescan will occur and that process will trim away 'auto generated and nolonger available' apps .desktop files.

jeff

 

[Zeno Arrow]

Look, it's your Pandora, you do with it as you wish, craigix may have offered some advice of a sensible way to install apps but you can do it other ways. Take your NAND storage and partition it to split it, what do you have now? You have an area where firmware/OS can be written to, and an area you can use without interfering with the firmware/OS. Simple. Why do you need to have some official word from the devs when the solution is staring you in the face.

A full flash update to the NAND will likely overwrite ANY partition re-work you might have done (since it resides in the beginning of the store...). Doing that won't prevent this issue, sorry. Fortunately, for everyone, it should be a rare occurrence that one would have to do that sort of thing. Most updates to the firmware should be .ipk/.opk files and you use opkg to install/update the new stuff on the NAND.


You should be able to restore partition information with something along the lines of mbrfix if the following conditions could be met:
1. You can disable wear levelling (just for full flash updates, you'll want it on the rest of the time).
2. The new version of your OS fits into the size of partition you allocated for it previously.

Part 2 isn't going to be an issue if you're sensible. Part 1 is the only thing I'm unclear on.

 

[sterling976]

afaik if you can mount an ISO directly but that would be pretty silly IMO since there are better formats (cramfs squashfs and even cloop would probably be better) for this since there isn`t any compression support there

He mentioned using fuse, which is a userspace filesystem that allows you, among other things, to define (script, really) handlers for various file types. You could, in essence, `cd` into a .tgz file or .iso file, then cat the contents of a file contained therein, without having to mount the .iso or untar the .tgz. I work with a guy who's really pushing this around our office, and from what I've seen, it's pretty cool stuff, and seems to be pretty snappy performance-wise.

CODE
http://fuse.sourceforge.net/


Jeff -- good reads, look forward to seeing this info on the wiki.


EDIT: The .iso mounting and .tgz unzipping happens in the background -- the user doesn't have to be aware of it. Just wanted to acknowledge that, yes, it does still have to happen.

 

[cb88]

hmm i see your point there about ISO... and I guess it still leaves your options open for supporting other formats

how would you get data spread around if the unions are mounted ro and you write back only to the union on NAND which is what you are doing now anyway right for settings in /home i mean?

 

[Karel Jansens]

Just one thing: Make sure the "scanner" part is carved in granite. Nokia uses a scanner (called "metalayer-crawler") to scan the tablet´s storage for media files and it truly is a POS. It will on occasion hang for no apparent reason, without the user knowing anything about it hanging, and continue to run until the tablet´s battery dies. May a night I´ve been woken up by the Tablet´s "Chant of Imminent Battery Death".

This was actually a bit of the concern I had over the conflict as to whether the internal NAND was officially usable or not. If you have multiple cards with the same reader, what happens when there's an update? You'd have to update all the cards. It's not a serious concern, just one of those mosquito like "that'd be annoying" things. I was thinking more with emulators (which are more likely to need updating than ebook readers or MP3 players) but the idea is the same.

It´s just replacing a file. Anyway, my intent was to keep "master copies" of all the cards on a hard drive, which would make the update (if and when necessary) rather more smooth. I thought this would be a good idea for people (like me ) who own lots of cards and want an easier way to use them in the Panda. Provided the player file can be kept reasonably compact, even on a 1GB SD card, the loss of a little space would be far outweighted by the convenience of Plug and Play (literally this time).