What Should The Pandora File Archive Look Like?

[monkeyo2]

A couple of observations for those without WiFi or mad Linux skills...

Would it not be possible to let the Pandora directly access the internet (and the Pandora archive) via a USB cable connected to a PC?

Alternatively, could the Pandora have a Window, Mac or Linux application stored somewhere on either the NAND or one of the SD cards, that could be run on a USB connected machine, that would allow install a downloaded Pandora application from the PC, a bit like the application installer in Nokia PC Suite (wow that was a long sentence).

So you download a file from the Pandora archive onto your computer, plug in the Pandora and run "PandoraInstalla" from the "Removable Disk", drag and drop, seconds later you're ready to rock'n'roll (assuming you just installed a GNU Guitar Hero clone.....).

 

[Dutch_Cap]

On another topic? Why is everyone talking about dependency(libarary) hell and such. Dependency hell DOES NOT EXIST on debian/ubuntu, unless perhaps you try and treat it like a windows system and try and install some random program you downloaded with your web browser.

Two words: removable storage. To the best of my knowledge apt has no means of installing to removable storage. If you install program X onto your SD, then remove the SD card and insert another one, program X would still be in apt's database of installed programs. It then happily installs program Y, which depends on X, and leaves you wondering why on Earth program Y won't run. I think apt would have to be modified to deal with this somehow. I don't know how hard this would be.

Also, while software installation is usually pretty smooth with apt, I have had the occasional problem or catch-22. For example, a program I want to install might have a dependency on a library that isn't going to be installed because it's got a conflict with another program that's already installed. So "DOES NOT EXIST" is slightly too strong a statement for my taste. Don't get me wrong though, apt is a most excellent package manager.

 

[Zarneth]

On another topic? Why is everyone talking about dependency(libarary) hell and such. Dependency hell DOES NOT EXIST on debian/ubuntu, unless perhaps you try and treat it like a windows system and try and install some random program you downloaded with your web browser.

Two words: removable storage. To the best of my knowledge apt has no means of installing to removable storage. If you install program X onto your SD, then remove the SD card and insert another one, program X would still be in apt's database of installed programs. It then happily installs program Y, which depends on X, and leaves you wondering why on Earth program Y won't run. I think apt would have to be modified to deal with this somehow. I don't know how hard this would be.That's why I'm saying we need an apt repository to handle common libraries and the base system, which gets installed on demand to the internal storage. And then user created apps/games are installed separately to an SD card of the user's choosing using a simple custom installer program, which likely uses traditional one folder per app method and can trigger the download/installation of common libraries it needs. Any other dependences should be included with the user created package.

 

[Orkie]

I'm sure we could make the extraction simpler if a user really does want it done for them. Sort of a "Found blah blah blah_install.zip on your sd card, would you like to install it?".

Or I could fix up a tool to make extract cramfs images really easily (if I remove the file size limit of course) . Small, single file binaries that don't take up loads of space if there are many tiny files. Don't need to be 'installed', just copied into place (the package manager could simply download them) so ultra-quick and easy installs.

 

[Yrx]

Just please, something simple that won't cause problems. All this arguing is giving me a headache. I'm definitely with Squidge on this about self-contained programs.

 

[Squidge]

I'm sure we could make the extraction simpler if a user really does want it done for them. Sort of a "Found blah blah blah_install.zip on your sd card, would you like to install it?".

Or I could fix up a tool to make extract cramfs images really easily (if I remove the file size limit of course) . Small, single file binaries that don't take up loads of space if there are many tiny files. Don't need to be 'installed', just copied into place (the package manager could simply download them) so ultra-quick and easy installs.


I'd forgot about that, sounds ideal. No libraries, no dependencies, just a single file that you copy or download to your Pandora and run. People will complain about a bit of wasted space, but we have 32GB SD cards at our disposal, so I don't think thats an issue.

 

[jdh2550]

To those with no interest in the details - I agree there should be a one click install. I think a web based front end that gives the user the option of saving straight to the pandora or saving to any other media for later installation would work. (JakeK - you'd use this on your internet connected PC and then later install the apps after you'd copied them to your pandora by whatever means you choose).

To those interested in the shared libraries debate:

How about the following system (to deal with removable media):

1) Install the catalog in internal storage - this catalog is keyed by the location and the library.
2) Write a GUID to each SD card used (and use this in the above catalog key)

Now when I use the package manager it checks the catalog (which is in internal storage and always available) and it also checks to see which SD cards are currently inserted. If the catalog tells it that the dependencies are already available on one of the SD cards then it doesn't need to download them. However, if any dependencies are missing it can download them onto one of the SD cards.

Some thoughts on this:
1) I would imagine a lot of users will end up with at least one really big SD card that they don't swap in and out very often - so this would solve the "seven versions of python" for them.
2) It would likely require a customized version of a package manager (I confess I don't know how the current package managers store the catalog).
3) More thought is required to deal with the fact that there are two SDs and deciding exactly where to put the shared libraries. But as a concept I think uniquely identifying the SDs and storing the catalog in internal storage would work.
4) If one wanted to get fancy one could offer a utility to analyze the catalog and offer to move the most common libraries into internal storage.

Just a thought.

Given that this would be a bunch of work and would require coordination amongst the dev community I still think I'm in the "package all the dependencies with the application" camp. I don't much care the details of how they're packaged together - but I think that this is a more flexible and more loosely coupled solution (at the expense of being less efficient in storage space and download bandwidth).

People will complain about a bit of wasted space, but we have 32GB SD cards at our disposal, so I don't think thats an issue.

And this from the guy who says "don't ask for more than 128MB you don't need it" (sorry, couldn't resist!)

 

[Squidge]

I don't really want to write to nand unless there's a need (eg. os upgrade). I know the Pandora is very well protected against bricking, but if the nand dies, you still have a semi-bricked unit (only able to run apps from sd card). Why not put the catalog on the sd card if you really need it?

QUOTE
And this from the guy who says "don't ask for more than 128MB you don't need it" (sorry, couldn't resist!)


We only have library hell because people want to use SDL, Python, Fenix, etc. If people just used bog standard C with only the default C libraries, there would no need for any dependency/library resolving, thus reducing package size and meaning you could just download and run without worrying about "do i have program x installed". However, I know some people like to reuse code (idle gits ), so I'll let them off - they can statically link As for the people wanting to use Python and Fenix, well, it wouldn't hurt them to learn a proper language

 

[jdh2550]

We only have library hell because people want to use SDL, Python, Fenix, etc. If people just used bog standard C with only the default C libraries, there would no need for any dependency/library resolving, thus reducing package size and meaning you could just download and run without worrying about "do i have program x installed". However, I know some people like to reuse code (idle gits ), so I'll let them off - they can statically link As for the people wanting to use Python and Fenix, well, it wouldn't hurt them to learn a proper language

You wimp! Just write everything in handcrafted assembler - that way you don't even worry about linking to anything. And you can also ask for the special 8 MByte version of the pandora for the hard core developers!

Why not put the catalog on the sd card if you really need it?

Yeah, actually that would be a lot easier. Have a catalog per SD card and make the user specify the SD card (which I'm sure they'll have to anyway). That way if you put all your python games on the same SD card it will only have one copy of python. Also, the package manager won't get confused by libraries on other SD cards.

Hmmm - thanks Squidge - I'm starting to shift away from your camp and back to the "package manager" camp...

 

[scarath]

I'd forgot about that, sounds ideal. No libraries, no dependencies, just a single file that you copy or download to your Pandora and run. People will complain about a bit of wasted space, but we have 32GB SD cards at our disposal, so I don't think thats an issue.

Using a Windows style One-File-One-Program will be easier for noobs to get used to sure but its a pointless waste of space, 32 gigs is nothing. Having to manually install things file by file (e.g. not using a package manager) is a step backwards.

 

[Squidge]

You wimp! Just write everything in handcrafted assembler - that way you don't even worry about linking to anything. And you can also ask for the special 8 MByte version of the pandora for the hard core developers!

8MBytes?! Are you trying to encourage bloated software? 64KB is more than enough.

Hmmm - thanks Squidge - I'm starting to shift away from your camp and back to the "package manager" camp...

Damn it!

32 gigs is nothing

32Gigs is the size of my one and only harddisk in my main PC...

Having to manually install things file by file (e.g. not using a package manager) is a step backwards.

There is no installing, you download one file to your Pandora, run it - thats it. It doesn't extract or unpack anywhere, it just runs. Want another SD card with the same software on (or give your friend a copy)? Just copy the single file to your other/there sd card. Want to uninstall the software? Just delete the single file.

 

[Orkie]

Orkie: the wiki has thumbnail problems.

It is now working .

 

[atomicthumbs]

8MBytes?! Are you trying to encourage bloated software? 64KB is more than enough.

But we all know that real programmers do plenty with the 1kb of RAM in their ZX-80s.

 

[jdh2550]

But we all know that real programmers do plenty with the 1kb of RAM in their ZX-80s.

I missed the ZX-80 - I stepped straight up to the ZX-81 and spent my first programming efforts watching blocks scroll up the screen. In fact I have a US version sitting in my desk draw (given to me by a friend at work who'd heard my "ahh, back when I started..." stories one time too many!!).

I think it was a 16kb game - but the first game I bought was a dinosaur chasing you around a 3D maze...




(Does this date me?)

 

[Shiny]

How about the following system (to deal with removable media):

1) Install the catalog in internal storage - this catalog is keyed by the location and the library.
2) Write a GUID to each SD card used (and use this in the above catalog key)

Now when I use the package manager it checks the catalog (which is in internal storage and always available) and it also checks to see which SD cards are currently inserted. If the catalog tells it that the dependencies are already available on one of the SD cards then it doesn't need to download them. However, if any dependencies are missing it can download them onto one of the SD cards.

But then you have SD cards that depend on each other for libraries, which will be confusing because you'll need to remember that you must have SD A inserted to run program B (unless you have a copy of the library on each SD card, wasting space, but not as much as self contained programs), but this could be fixed by only allowing programs to load libraries from their respective SD card.

Will duplicate libraries included with self-contained programs even be a major contibutor to storage usage? Most of your storage will be used by files that don't need any libraries to operate (such as ROMs, music, videos). Gaining that extra 100MB or so of wasted space is not worth complicating things over when 8GB SDHCs are so cheap, with 16GBs getting there.

 

[whiskthecat]

http://212.202.219.162/
16kb should cover it.

 

[Coder_TimT]

How about the following system (to deal with removable media):

1) Install the catalog in internal storage - this catalog is keyed by the location and the library.
2) Write a GUID to each SD card used (and use this in the above catalog key)

Now when I use the package manager it checks the catalog (which is in internal storage and always available) and it also checks to see which SD cards are currently inserted. If the catalog tells it that the dependencies are already available on one of the SD cards then it doesn't need to download them. However, if any dependencies are missing it can download them onto one of the SD cards.

But then you have SD cards that depend on each other for libraries, which will be confusing because you'll need to remember that you must have SD A inserted to run program B (unless you have a copy of the library on each SD card, wasting space, but not as much as self contained programs), but this could be fixed by only allowing programs to load libraries from their respective SD card.

Will duplicate libraries included with self-contained programs even be a major contibutor to storage usage? Most of your storage will be used by files that don't need any libraries to operate (such as ROMs, music, videos). Gaining that extra 100MB or so of wasted space is not worth complicating things over when 8GB SDHCs are so cheap, with 16GBs getting there.


I believe the idea is for basically each card to have it's own catalog. But it won't necessarily be a copy of the libraries on the other card, it would just be a catalog of the dependencies needed for the programs installed on that card. Overall, yes it would most likely take more space than a single shared library install, but it should still be much less than full duplicate libraries.

As for the overall space gained, you are probably right, although depending on what is "core" and what isn't it could become quite large... I don't think we want to have a seperate Java or Python runtime for each dependent game installed...that could get quite sizeable...

 

[GoatZilla]

Ugh. This really sounds like reinventing the square wheel.

If you really want to go down this awful (non debian package repository) route, you might consider some sort of union FS type setup. Each "package" is a union filesystem that can reside on an SD card. It's just mounted/unmounted like a regular filesystem so it's fairly removable. A properly crafted system would probably solve most of your "problems" (aside from the obvious main problem).



Really though, this whole fear of dependencies and going to monolithic hand-crafted "file archive" stuff is practically a throwback to the stone ages. The rest of the modern world just goes, "oh, you want this? just run apt-get install wizbang_of_your_choice".

 

[squeakypants]

Ugh. This really sounds like reinventing the square wheel.

If you really want to go down this awful (non debian package repository) route, you might consider some sort of union FS type setup. Each "package" is a union filesystem that can reside on an SD card. It's just mounted/unmounted like a regular filesystem so it's fairly removable. A properly crafted system would probably solve most of your "problems" (aside from the obvious main problem).

Really though, this whole fear of dependencies and going to monolithic hand-crafted "file archive" stuff is practically a throwback to the stone ages. The rest of the modern world just goes, "oh, you want this? just run apt-get install wizbang_of_your_choice".



That sounds a bit extreme. I think the best option besides just sticking with .deb would be to do something similar to OSX applications. I know the files aren't that hard to track down, but the one thing I never liked about linux was how it handled applications. In OSX, the application is a single file, which can actually be opened like a folder to see all the separate files. This would make moving apps around SD cards simple, and also solve our software distribution problem. As for shared libraries, could you have an extra lib folder (or anything else) on an SD card, then have the OS use both the one on the flash storage and on the SD card together?

 

[Squidge]

Will duplicate libraries included with self-contained programs even be a major contibutor to storage usage?

Not really, but instead of duplicating libraries in the self-contained 'archive'. You could just statically link, which'll use up less space.

Really though, this whole fear of dependencies and going to monolithic hand-crafted "file archive" stuff is practically a throwback to the stone ages. The rest of the modern world just goes, "oh, you want this? just run apt-get install wizbang_of_your_choice".

Assuming you have Wifi and an x-term. Plus it makes it more difficult to copy applications elsewhere without knowing exactly what files it needs. Want to clean up your card? You'll have to wipe and start again as you've no idea what programs need what files. I agree that apt-get and the like make things a lot easier for the desktop, but not for a mobile device. Self-contained software just makes the whole thing a LOT easier for everyone (apart from the purists, maybe - "But I could have saved 200KB if the libraries were seperate!")