dflemstr
It's a ball.
Yes of course. I suggest that you read up on union-style file overlays.Dimacus said:
Last edited by a moderator:
Pandora Directory Structure Question
- Thread starter DaveC
- Start date
Jeffery Mewtamer said:
Mmmm... I'm a fan of apt: it does the job very nicely. But y'know what would be sweet? Embed the "repository" information in each PND, so it knows where it came from. A PND-manager daemon could occasionally look for updates by querying each PND for its repository, check the repo, modify the icon to signify an update is available and install the update via a right-click menu option on the icon. With options for auto-download, etc. Nifty-keen.
Last edited by a moderator:
dflemstr
It's a ball.
All of this has been discussed to death already... I made a complete working system for PND management (the pndmanager) but for some reason, there wasn't much interest in it, and the system is obsolete now because of PND standard changes.todd said:
Last edited by a moderator:
skeezix
Internal Development
dflem -- PXML.xml and PND's haven't changed much in the last 6mo; probably be easy to fix it up.
I never looked at it I'm afraid (busy enough with my own krufy
, but but keep pursueing it, if you think it a good system. I mean, people don't care much about a lot of things just yet, but once shipped, a lot of those thigns will suddenly be a twinkle in folks eyes. I mean, 6mo ago it was pulling teeth to get answers to some questions
jeff
I never looked at it I'm afraid (busy enough with my own krufy
, but but keep pursueing it, if you think it a good system. I mean, people don't care much about a lot of things just yet, but once shipped, a lot of those thigns will suddenly be a twinkle in folks eyes. I mean, 6mo ago it was pulling teeth to get answers to some questions jeff
Aninhumer
Guy with scary face.
How would that work if someone updated a PND, and it had a new version of a file that had been edited?dflemstr said:
I realise that's probably bad design.
EDIT:
Also, I agree with skeezix, there will be a lot more interest in a PND manager after people actually have their Pandoras, dflemstr.
At the moment it doesn't really mean much to most people.
Last edited by a moderator:
dflemstr
It's a ball.
Well, the reason is more like that the actual *libraries* I used for the system are obsolete now (I was using cutting edge pre-release stuff back then and I haven't bothered to port the code to the now stable libraries)skeezix said:
Aka, I managed to wire something together using different libraries, making the whole system extremely PND-format-specific, and then the libraries became obsolete, the format changed (enough for it to matter) and yeah.
Last edited by a moderator:
laurencevde
Member
If I understand correctly, the PXML.xml is appended to the filesystem-image that constitutes the pnd? hmmmm, ok....
Might it be an option to also be able to store that metadata into extended attributes? I guess that'll be quicker.
So, that'll prefarably become:
Check to see if the FS can handle extended-attributes
if so, check to see if the PND has its metadata stored in extd-atts
if so, use those
if no ext-att support, or no metadata in it, check for a PXML.xml appended to the image
if so, use that, and put the info into extd-atts if possible
if not, mount the pnd, optionally append the PXML.xml stored inside it to the pnd, and add the info into extd-atts if possible
extended atts are often lost when copying/sending files around, so being able to auto-regenerate it would be very welcome. And auto-appending PXML.xml means less work for the developer, but that breaks the idea of non-changing pnds...
Might it be an option to also be able to store that metadata into extended attributes? I guess that'll be quicker.
So, that'll prefarably become:
Check to see if the FS can handle extended-attributes
if so, check to see if the PND has its metadata stored in extd-atts
if so, use those
if no ext-att support, or no metadata in it, check for a PXML.xml appended to the image
if so, use that, and put the info into extd-atts if possible
if not, mount the pnd, optionally append the PXML.xml stored inside it to the pnd, and add the info into extd-atts if possible
extended atts are often lost when copying/sending files around, so being able to auto-regenerate it would be very welcome. And auto-appending PXML.xml means less work for the developer, but that breaks the idea of non-changing pnds...
fearofshorts
Member
My evidence on why we need a central appdata directory is below. Beware: MASSIVE wall of text.
I'm sorry to respond to this without having finished reading the thread, but this REALLY gets to me.
I came from the DS homebrew scene. Very interesting place there. They have basically four or five different conventions for storing data, and a lot of people don't follow them. It makes my life hell.
(disclaimer: the below holds thinly-veiled references to piracy and I want to state my values on this. I will pirate any game I choose to if, and only if, it is a big-name-brand. If it is independently produced (IE: anything that will be commercial on the Pandora) I will not pirate it unless there is no demo. If I really enjoy a game produced by a large company, I will purchase it when I can afford to. If I dislike it or find I value it at less than ~1/4 to 1/2 the RRP, I will not purchase it. If it is independently produced and I don't enjoy it, I will delete it and not run it again. Also, I use a lot of homebrew- not just pirated and emulated games. I tend to play good homebrew games more often than I do pirated games)
(TL;DR disclaimer: I'm a pirate with morals)
Whenever I run out of space on my DS I have to remove some games. To do so, I have to go into my games directory and delete particular files. This is not too annoying, as I can just sort it by file-type. It's sort of easy, as each rom is stored in it's own Read-only file with a specific format, with the same for its save file (but read-write, of course).
However, whenever I need to remove a lot of data or need to re-format an SD-card (and believe me- this can happen once every 5-6 months) I have to spend several hours placing files back into place very carefully. I have to configure my primary OS (AKAIO with my new AceKard 2i), my secondary OSs (Moonshell 2 for videos and for it's ebook-reader and DSOrganise for it's advanced file-system management and it's PDA-functions), my 'easy' homebrew (placed directly into a homebrew sub-folder, automatically storing it's data in the same folder or in /DATA in the root) and then the 'hard' homebrew.
Now, the 'easy' homebrew is hard enough when it comes to save files. I basically have to copy over my entire homebrew folder and the DATA directory. But the 'hard' homebrew (which doesn't conform to these standers) will either have to be in the root with data there (causing clutter) or somewhere else (causing a LOT of sub-directories) or in it's own directory somewhere else. For example, VNDS is an excellent piece of homebrew for the DS which lets you play visual novels on the DS. The problem is that it needs to all be in /VNDS (in VNDS under the root) with it's novels and other data-files inside it. This by itself is not a problem, but it is just one of twenty or thirty different pieces of Homebrew I maintain which have annoying standards.
It ends up meaning that I can never recover my save-files for any of my 'hard' homebrew games because they are just too hard to find. I have to start again.
The saddest thing is that these save and data files are all very small, but because of the lack of a universally-accepted standard it is impossible to find anything.
Now do you see why I love this new system for the Pandora?
All of your save and data files will go in one folder, created automatically. All of your games and pieces of software will go into another folder. Simple. Run out of space? Delete a few games. Want to share a game? Email it's PND. Screwed up in an emulator and mapped it's 'quit' button to the touchscreen? Delete the appdata directory.
The situations you offered (of quake and doom) are the exceptions, not the rule. These are some of the only pieces of homebrew that will ever require that amount of data. Think about it. Everything else will be included. Games? Just need a save file. Video-players? Just need a config-file. Emulators? Maybe nothing at all!
The only type of file that should not go in an appdata folder is a office document- and it probably won't default to saving in there anyway!
And then you have to consider the interchangeability of the SD-cards. In Windows it works because you always deal with fixed locations. You might have your music in "D:/Music", your games in "C:/Games" and your pictures in "B:/Pics". On the Pandora you will have separate SD-cards, forcing you to manage each one. With a non-appdata-folder solution you might have games-saves in "SD1:/mysaves/games" on one SD, then "SD1:/savegames" on another. What the hell will you do then?
My point is, if we allow people to start changing things away from the standard the community will split several times and everything will go to shit. We'll end up with some using the separated app-data directory, some using an appdata sub-directory, devs sticking to the PND-specs and other devs just releasing them as must-be-extracted messy-as-hell programs.
And if people do not accept the PND-system completely, people will not be able to use the AppStore (as it works around PND files). And if this happens... we end up completely and utterly screwed- worse off then even the DS homebrew community.
People keep arguing that this system is messy, but they are all failing to realize that this is in fact the cleanest possible system for the Pandora- the only one that makes sense for a portable computer/console that has interchangeable storage.
DaveC said:
I'm sorry to respond to this without having finished reading the thread, but this REALLY gets to me.
I came from the DS homebrew scene. Very interesting place there. They have basically four or five different conventions for storing data, and a lot of people don't follow them. It makes my life hell.
(disclaimer: the below holds thinly-veiled references to piracy and I want to state my values on this. I will pirate any game I choose to if, and only if, it is a big-name-brand. If it is independently produced (IE: anything that will be commercial on the Pandora) I will not pirate it unless there is no demo. If I really enjoy a game produced by a large company, I will purchase it when I can afford to. If I dislike it or find I value it at less than ~1/4 to 1/2 the RRP, I will not purchase it. If it is independently produced and I don't enjoy it, I will delete it and not run it again. Also, I use a lot of homebrew- not just pirated and emulated games. I tend to play good homebrew games more often than I do pirated games)
(TL;DR disclaimer: I'm a pirate with morals)
Whenever I run out of space on my DS I have to remove some games. To do so, I have to go into my games directory and delete particular files. This is not too annoying, as I can just sort it by file-type. It's sort of easy, as each rom is stored in it's own Read-only file with a specific format, with the same for its save file (but read-write, of course).
However, whenever I need to remove a lot of data or need to re-format an SD-card (and believe me- this can happen once every 5-6 months) I have to spend several hours placing files back into place very carefully. I have to configure my primary OS (AKAIO with my new AceKard 2i), my secondary OSs (Moonshell 2 for videos and for it's ebook-reader and DSOrganise for it's advanced file-system management and it's PDA-functions), my 'easy' homebrew (placed directly into a homebrew sub-folder, automatically storing it's data in the same folder or in /DATA in the root) and then the 'hard' homebrew.
Now, the 'easy' homebrew is hard enough when it comes to save files. I basically have to copy over my entire homebrew folder and the DATA directory. But the 'hard' homebrew (which doesn't conform to these standers) will either have to be in the root with data there (causing clutter) or somewhere else (causing a LOT of sub-directories) or in it's own directory somewhere else. For example, VNDS is an excellent piece of homebrew for the DS which lets you play visual novels on the DS. The problem is that it needs to all be in /VNDS (in VNDS under the root) with it's novels and other data-files inside it. This by itself is not a problem, but it is just one of twenty or thirty different pieces of Homebrew I maintain which have annoying standards.
It ends up meaning that I can never recover my save-files for any of my 'hard' homebrew games because they are just too hard to find. I have to start again.
The saddest thing is that these save and data files are all very small, but because of the lack of a universally-accepted standard it is impossible to find anything.
Now do you see why I love this new system for the Pandora?
All of your save and data files will go in one folder, created automatically. All of your games and pieces of software will go into another folder. Simple. Run out of space? Delete a few games. Want to share a game? Email it's PND. Screwed up in an emulator and mapped it's 'quit' button to the touchscreen? Delete the appdata directory.
The situations you offered (of quake and doom) are the exceptions, not the rule. These are some of the only pieces of homebrew that will ever require that amount of data. Think about it. Everything else will be included. Games? Just need a save file. Video-players? Just need a config-file. Emulators? Maybe nothing at all!
The only type of file that should not go in an appdata folder is a office document- and it probably won't default to saving in there anyway!
And then you have to consider the interchangeability of the SD-cards. In Windows it works because you always deal with fixed locations. You might have your music in "D:/Music", your games in "C:/Games" and your pictures in "B:/Pics". On the Pandora you will have separate SD-cards, forcing you to manage each one. With a non-appdata-folder solution you might have games-saves in "SD1:/mysaves/games" on one SD, then "SD1:/savegames" on another. What the hell will you do then?
My point is, if we allow people to start changing things away from the standard the community will split several times and everything will go to shit. We'll end up with some using the separated app-data directory, some using an appdata sub-directory, devs sticking to the PND-specs and other devs just releasing them as must-be-extracted messy-as-hell programs.
And if people do not accept the PND-system completely, people will not be able to use the AppStore (as it works around PND files). And if this happens... we end up completely and utterly screwed- worse off then even the DS homebrew community.
People keep arguing that this system is messy, but they are all failing to realize that this is in fact the cleanest possible system for the Pandora- the only one that makes sense for a portable computer/console that has interchangeable storage.
Last edited by a moderator:
skeezix
Internal Development
Laurencevde said:
We thought about it a year back when brainstormning, and it really came to..
1) Consistency and simplicity is ebtter than having things vary by container, and most filesystems can't do that very well
2) You'd have to mount to get that info then; consider 1000 apps (or 500, or 10,000
and mounting every one just to get the info is a lot of work on the device (in real time while user waits, potentially.) Just pulling off the last couple K of a file (one read on flash) and grepping it for the values is consistent, easy, and fast.(It may not be perfect, but one has to consider we at the time only had two months (!!) to esign and build everything, and then we've been extending and patching and fleshing out ever since as we never really knew we'd have an extra year to work on it
jeff
Last edited by a moderator:
skeezix
Internal Development
Pnd is a dice-roll; we built a lot of cool stuff; we didn't "over do" it, since when you super over tightne the shoes, everyone complains about that too; to become 'common usage' somethign has to be easy, useful, easy for developers to pick up, and well communicated and so on .. pnd can work really well, but devs might just not use it at all; a plain executable works well too, and would be just lke every other device out there. Nothing stops people from just saying 'heres a zip, unpack it, works fine' .. it just turns into that mess wherre ebery app works differently, files spread out all over the place, etc.
So we hope pnd takes off, it coudl become a really cool thing over time.... or total fizzle.
Either way, thats cool; we throw some cool stuff at the wall, see what sticks; we've tried
Its not all or nothing either; theres lots of neat stuff everyone has built. From the software side, we've got several menus, we've got the every-device libpnd stuff (common ways to set brightness, simple conf files, adjust clock speed, all that sort of tstuff), makes developers lives easier. (And I hope devs will keep fleshing out and sharing rather than making 500 custom ways of doing the same things and fighting with each other..) -- we've included a few basic OS icons in the device for everyone to use, rather than make their own; we've put an effort into making everything open source and out there.. its Open Pandora. All these little bits.. some of it will stick, some of it won't. Thats cool, that is the way of things
jeff
So we hope pnd takes off, it coudl become a really cool thing over time.... or total fizzle.
Either way, thats cool; we throw some cool stuff at the wall, see what sticks; we've tried
Its not all or nothing either; theres lots of neat stuff everyone has built. From the software side, we've got several menus, we've got the every-device libpnd stuff (common ways to set brightness, simple conf files, adjust clock speed, all that sort of tstuff), makes developers lives easier. (And I hope devs will keep fleshing out and sharing rather than making 500 custom ways of doing the same things and fighting with each other..) -- we've included a few basic OS icons in the device for everyone to use, rather than make their own; we've put an effort into making everything open source and out there.. its Open Pandora. All these little bits.. some of it will stick, some of it won't. Thats cool, that is the way of things
jeff
Alpha2
Heroic Autobot
Skeezix, question, all the on board applications built into the OS, are those set up as PNDs as well? if so where do the appdata for those get saved? The nand?
Aninhumer
Guy with scary face.
I'm undoubtedly going to get ninja'd by skeezix but...Alpha2 said:
The on board applications are installed as they would be normally in linux, so their settings are either in /etc or hidden files in /home/username.
The PND system is intended (at least initially) for developers to distribute new apps easily, whereas it's already very easy to install anything from the angstrom repos, as they're already packaged as ipkg.
There may be an argument to separate out some apps into PNDs of course, such as firefox, abiword etc., but that would likely require configuration changes to make them work properly with the appdata/ system.
Last edited by a moderator:
Mithrildor
I Haz Custom Title
All these complains will be solved when some builds a tool that can delete both PND and appdata folder at the same time.
Caine
Hardcore Member
Just a matter of reading the base application name from the PND and removing appdata/<appname>. One can easily write a simple script (probably a one-liner) that does this and link that to the context menu. Is the appname read from the PXML or does it match the PND filename?Mithrildor said:
Similarly, a simple uninstaller can be made that prompts what to remove:
- Remove application (PND only)
- Remove configuration and save data (appdata only)
- Full uninstall (remove both)
Last edited by a moderator:
dflemstr
It's a ball.
From the PXML, and remember that one PND can host multiple applications.Caine said:
Last edited by a moderator:
skeezix
Internal Development
Built in apps are regular installed apps urrently, nit pndified.. It'd mean we'd be coding and patching all Bose apps which would be a tonne of work, and we have enough
I've though about wrapping them all as pnds, and have an arg to pnd-run to temporarily set $home to an appdata, just for consistency, but seems a little fragile to me
jeff
I've though about wrapping them all as pnds, and have an arg to pnd-run to temporarily set $home to an appdata, just for consistency, but seems a little fragile to me
jeff
fearofshorts
Member
Oh yeah, quick question: will there be a unique identifier for each PND sold/downloaded at the appstore?
I'd hate to end up with duplicate appdata folders.
I'd hate to end up with duplicate appdata folders.
WizardStan
Mega GP Mania
- Joined
- May 24, 2008
- Messages
- 16,731
You mean if someone creates "AwesomeGame" and then someone ELSE creates "AwesomeGame", can it handle them both?fearofshorts said:
Or someone creates "AwesomeGame" as "AwesomeGame.pnd" and then updates it with "AwesomeGameV2.pnd" will they both use the same appdata directory?
Last edited by a moderator:
