Rethinking PNDs and standard packaging approach

[FIQ]

Isn't 2GB internal storage a bit on the small side? What about also having an internal eMMC @ for example 32GB in addition, to give you a bit more room to install things to the base system? Or would this be too costly? I'm aware that an eMMC is significantly slower than a NAND, but what about having both (like for instance the N900 has - basically a 256MB NAND with the rootfs (ours would obviously be bigger) and the very basic things, plus a 32Gb eMMC for media (25GB) and installing stuff (2GB) (and a swapfile on 768MB).

 

[TrashyMG]

Isn't 2GB internal storage a bit on the small side? What about also having an internal eMMC @ for example 32GB in addition, to give you a bit more room to install things to the base system? Or would this be too costly? I'm aware that an eMMC is significantly slower than a NAND, but what about having both (like for instance the N900 has - basically a 256MB NAND with the rootfs (ours would obviously be bigger) and the very basic things, plus a 32Gb eMMC for media (25GB) and installing stuff (2GB) (and a swapfile on 768MB).

Well ED said the there were chances the eMMC would be either 8GB/16GB or 32GB depending on how much it cost when its ready for production.

 

[_wb_]

There are advantages of the PND system, and it makes sense to keep those advantages. "Cartridge-like" SD cards that contain software that becomes immediately available once you insert the card

It doesn't become immediately available if it have dependencies.

It does if you already have the dependencies installed, which should be how it is unless you manually removed dependencies or you're using the SD card on a different Pyra than the one you installed it from. Even in those cases we could in principle cache the .deb packages on the SD card so they can be automatically (re-)installed off-line if needed.

 

[_wb_]

Also I'm not in favor of redundantly including copies of general-purpose libraries inside the package (or using statically linked binaries), not so much because of the size overhead, but mostly because that makes it impossible to have an update/bugfix of such libraries without having to update all packages that depend on it. Imagine a crucial security bug in libpng or libssl or whatever -- if it can be fixed just by updating the library, it's not that big of a deal, but if you also have to hope that all authors of all packages that depend on it will repackage their stuff, then it may take ages before the problem is solved. Some of this could be somewhat solved by having an automatic compilation+packaging system (cf. what Cloudef is doing), but even then you still get the annoying problem of needing to re-download potentially a lot of packages each time a widely used library gets updated. Of course if you're using very specific libraries that are not worthwhile to have installed globally, it's something else.

How often will that happen in reality (I have no experience with debian) ? And I don't see a need for every package beeing updated with new lib version. If the whole package still works, why change something in it ? There could be security concerns, but how many applications/games will affect that?

That is actually part of my point. Maybe the new version of the library only gives a minor improvement, say it is 5% faster or fixes some minor glitch. Then the attitude of most package maintainers would be: "meh, it's not worth the effort to update my package and force everyone to re-download it, why fix what is not broken?". But all of these "why fix it if it ain't broken" can add up.

Finally I think not all of the metadata should be included in the package itself. Of course the textual metadata, the icon, and perhaps one screenshot should be inside the package. But the rest of the screenshots, and perhaps videos or whatever, they should not be included in the package itself. There simply is no point in that. You want to look at those screenshots (and videos and whatever) before you download the application, and once you've downloaded it, you don't need them anymore. They should be hosted only on the repo server, and it should be possible to update them without having to modify the package itself.

I disagree. While things like videos unneccesarily increase the size of a pnd, I don't see any reason to limit the amount of screenshots - what harm is done by including several kbs of pictures ?

Because I also want preview videos, which could take several dozens or hundreds of megabytes. And because screenshots will have a higher resolution so they will get bigger, maybe several megabytes if you have a dozen of them. SD card space is maybe not an issue, but download size is.

 

[Wally]

Gotta think of the people using 3G Perhaps the videos could be streamed from somewhere like YouTube or somewhere and in the Pyra Package Manager we have them only play if you touch them on the screen

 

[klapse]

Because I also want preview videos,

And I don't.

I had to delete the rest of my response for courtesy reasons.

 

[milkshake]

Because I also want preview videos, which could take several dozens or hundreds of megabytes. And because screenshots will have a higher resolution so they will get bigger, maybe several megabytes if you have a dozen of them. While SD card space is maybe not an issue, but download size is.

Maybe preview videos don't belong in the pnd but in the repository as separate entities which GUI package managers can handle that way the pnd is not inflated when it isn't needed to be.

 

[WizardStan]

Isn't 2GB internal storage a bit on the small side? What about also having an internal eMMC @ for example 32GB in addition, to give you a bit more room to install things to the base system?

According to ED, the current plan is to have an internal eMMC (the NAND) of some currently unknown size, and a microSD socket. The SoC can't use both at the same time, but if you find yourself needing more space you can insert a microSD card of significantly larger size (I believe 128GB are now available) and it will start using that instead of the built in eMMC.We should probably stop referring to the Pyra's proposed internal storage as "NAND": technically yes, eMMC is NAND flash, but it kind of has a slightly different meaning in this context and can cause confusion.

 

[_wb_]

Because I also want preview videos, which could take several dozens or hundreds of megabytes. And because screenshots will have a higher resolution so they will get bigger, maybe several megabytes if you have a dozen of them. While SD card space is maybe not an issue, but download size is.

Maybe preview videos don't belong in the pnd but in the repository as separate entities which GUI package managers can handle that way the pnd is not inflated when it isn't needed to be.

 

[milkshake]

I'm sure mini menu could be amended to allow slideshow of preview pics. If u think about devices like PSP 3ds psvita they all contain image meta data for when browsing your game collection on the device without an internet connection.

 

[_wb_]

I'm sure mini menu could be amended to allow slideshow of preview pics. If u think about devices like PSP 3ds psvita they all contain image meta data for when browsing your game collection on the device without an internet connection.

The question is: do you really want to view lots of preview stuff if it is already installed? You can also just launch it and try it out yourself.

 

[levi]

I'm sure mini menu could be amended to allow slideshow of preview pics. If u think about devices like PSP 3ds psvita they all contain image meta data for when browsing your game collection on the device without an internet connection.

 

[klapse]

Hosting videos for applications on the repo server is going to cost a lot of bandwidth. I find it doubtful ED will want to pay for that when everybody can just use youtube.

 

[TrashyMG]

^ correct, but I think what I think what Milkshake is referring to isn't hosting the video files, but giving the PND author the ability to insert a youtube link and it will create the appropriate html tags to embed it much like youtube videos are on these boards. 

 

[fahrstuhl]

How about the PNDs include all the dependencies that are not already in the base OS and it's possible to install the most recent libraries onto the SD card and have the PND use those? You won't fill up the internal storage, have self-contained packages and it's possible to use the newest libraries.

 

[comradekingu]

Im reading http://www.tinycorelinux.net/corebook.pdf thus far it makes all sorts of sense.

 

[_wb_]

How about the PNDs include all the dependencies that are not already in the base OS and it's possible to install the most recent libraries onto the SD card and have the PND use those? You won't fill up the internal storage, have self-contained packages and it's possible to use the newest libraries.

 

[fahrstuhl]

How about the PNDs include all the dependencies that are not already in the base OS and it's possible to install the most recent libraries onto the SD card and have the PND use those? You won't fill up the internal storage, have self-contained packages and it's possible to use the newest libraries.

But what if two PNDs share dependencies (say, they both use Lua)? Where do you install the dependency? What if the two PNDs are on different SD cards?

Also you are speaking about "the" base OS, but that means we have to determine once and for all what is in "the" base OS and what is not. Whenever the base OS changes, the definition of what is a valid PND also changes if you use that approach (the Pandora approach essentially). I don't like that.

Better require dependencies to be declared explicitly in the metadata instead of working with an implicitly defined "the base OS". It's the only sane way to make things work without forcing people to install stuff they don't want, or to force them to redundantly install several times instead of just once what they do want.

Two PNDs on different SD cards install the dependency on each SD card, because the SD cards wouldn't be self-contained otherwise. Two apps with the same dependency on the same SD card share the one on the SD card. The only disadvantage is a bit of wasted space on an SD card, which is incredibly cheap and flexible storage compared to the eMMC.

 

[TrashyMG]

Ideally with the larger eMMC the Pyra will should be initially be pre-installed with a wide range of up to date libraries.. leaving only oddball ones that would need to reside in PNDs.

 

[_wb_]

How about the PNDs include all the dependencies that are not already in the base OS and it's possible to install the most recent libraries onto the SD card and have the PND use those? You won't fill up the internal storage, have self-contained packages and it's possible to use the newest libraries.

But what if two PNDs share dependencies (say, they both use Lua)? Where do you install the dependency? What if the two PNDs are on different SD cards?

Two PNDs on different SD cards install the dependency on each SD card, because the SD cards wouldn't be self-contained otherwise. Two apps with the same dependency on the same SD card share the one on the SD card. The only disadvantage is a bit of wasted space on an SD card, which is incredibly cheap and flexible storage compared to the eMMC.

I still don't think it's a good idea to install debian packages on removable media, because the package manager cannot deal with stuff disappearing unexpectedly. You can use the internal microSD to get more room for installing dependencies.

Ideally with the larger eMMC the Pyra will should be initially be pre-installed with a wide range of up to date libraries.. leaving only oddball ones that would need to reside in PNDs.

Well yes the Pyra will obviously be pre-installed with a wider range of libraries than the Pandora. But still not an exhaustive set of libs and other dependencies. We may all have different definitions of "oddball". Maybe some people want to make PNDs that have a full LaTeX as a dependency, but I would not make LaTeX pre-installed. Maybe lots of stuff uses Java, but I still want to be able to de-install Java (if it would be pre-installed) without breaking stuff: when I remove Java and I try to install or run a Pyra PND that depends on Java, I want to get a nice dialog box that says: "This package depends on: java-runtime, [maybe some other stuff too]. Do you want to download and install these dependencies, or do you want to cancel?". That would be nicer than nothing happening with only some obscure error in /tmp/pndrun_*.out.

This should not be hard to implement (just some shell script magic), but it would be a much better solution than having one unmutable fixed set of pre-installed stuff and requiring anything else to be redundantly duplicated all the time with no method to globally update a library or other dependency.