Apt Repositories For Pandora

[AireTamStorm]

Real package management is difficult to do across multiple SD cards, and difficult to do across multiple operating systems if the user wants to download on PC and copy across to SD card(s) later. Also, we can't assume every Pandora user will have Wifi. So .zip's are much easier for a lot of people. Installable from PC, Pandora, other OSs and completely self-contained.

It doesn't stop the 58% of people in that thread however from using package management if they want to however, but it will not be officially supported.

Remember: One of the main aims of Pandora is "ease of use".

58%? are you counting the votes for "I dont care as long as it has a GUI" against package management? There should be "supported" mechanisms in place for both, at least, considering the clearly visible amount of demand for it.

 

[Etinin]

I think it's insane to make every package self contained. Packages that could as small as 200kb might take up 5MB because of this decision.
Why the hell does the user need internet in order to for a package manager to do dependency checking? It's not so hard for a package to link to its dependencies so the user can download the dependencies and put them in an SD card together with the package he wants to install.

I think that dumb zips would be a good solution for the average user for a common PDA/Gaming Device, but for the Pandora, I don't think that's really necessary.

 

[sebt3]

58%? are you counting the votes for "I dont care as long as it has a GUI" against package management? There should be "supported" mechanisms in place for both, at least, considering the clearly visible amount of demand for it.

On the flash (the 256M) the upgrade should be handled by ipkg (from angstrom) so the kernel and lib are keept up to date. the dev point is that the apps have to be on the SD card. this is where is the problem.

I'm the first here for package management. (debian user for 8years now )
But when we speak about easy to use, it should be bullet proof. Because linux newbie don't have to manually fix there repository to be still able to install/upgrade.
Now if someone have a good idea on how to create a package manager that :
- is bullet proof,
- that always know where are the file (to resolv dependency),
- able to handle that you can remove the sdcard in no time.

I'm able to code a package manager that choose 2 of the 3 goal but I'm not able to find a way to handle the 3 in the same package manager.

The Wii homebrew scene, have not that good but not that bad solution to the problem :
- every app/game is an archive fully self containt (as what will be supported on the pandora) but nowaday SDcard are very cheap, so duplication of lib/code won't be a problem. And main libary will be in the flash so there won't be much of it anyway).
- there is "homebrew browser" that manage you current SD-card by downloading and unzipping archive on your SDcard in /apps/

This is bullet-proof, simple, no need to resolv depency, and you don't need to think too much about it.

Package management is not a solution with removable media. but a "pandora homebrew browser" will be very easy to write (picking everything for archive.pandora.org) and will happend some time when the device will be available (if not already included by the dev inside the flash...)

I know (and so do the devs) this is not optimal in regard to a single point : saving storage.
but who f******g care ???

And this will be bullet-proof and newbie friendly (aka very easy).

How the apps in the iphone world do ? How the apps do in the Windows world ? How the apps does in the android world ? exactly this way. (ok windows don't have this tiny apps that download the archive for you...)

The pandora is _NOT_ a desktop computer.

So where's the point of a package manager here realy ?


edit: a few clarification...
edit2: more clarification

 

[Megamixman]

The solution would not involve just a package manager, but a package manager and a file system such as aufs. The package manager could then be further coded to make sure that dependencies and programs are kept together.

 

[Chip]

The package manager could then be further coded to make sure that dependencies and programs are kept together.
This can also be achieved by bundling the dependencies in with the executables, which is what we are doing

Storage is cheap and libraries aren't that big. If every app you download comes complete with the exact version of the libraries that it requires, then it will just plain work every time. It may not be the most efficient conceivable system, but nobody can argue that it isn't he simplest and most bullet proof.

 

[phrosty]

Hopefully someone will develop an apt-like app that is run from the desktop or the pandora. That would be the best solution.

 

[AireTamStorm]

Squidge,

It isn't really a problem with storage, it is a matter of good practices. That kind of irresponsible practice bloats software and makes everything difficult to upgrade or patch. You start needlessly seeing (to quote another poster) a 200kb application take up 15MB because you seemingly /have/ to package the libraries.

Why not package a zip file with two .ipkg or APT packages instead of a ton of files, so that the libraries and application are properly installed, and everything can be updated reasonably? Like others have said, package management can use local files just as easily as files from a network. Concerned about SD cards? ipkg contains the featureset to deal with removable media. Look it up.

We cannot assume that every user has WiFi just as much as we cannot assume that they have a computer of their own or can read english. I as well as others are not advocating proper package management without reason. I believe it will improve ease of use, the quality of the software, and the software base all in one. Package management could mean the difference between a niche product and a product that brings a real movement. The difference between being a headache or a familiar experience for the tech savvy user. You're catering to the potentially ignorant and neglecting the people who are buying that are most likely to use the hardware to its full potential.

I hate to be frank, but this is idiotic. Any reasonably knowledgeable Linux user would tell you you're insane to waste the storage resources and bloat the local filesystem with a mix of completely unmanaged applications and data, which are impossible to update and keep track of. Especially on a mobile device, with relatively limited resources. I am buying this device /because/ of Linux, as I'm sure many others are. Package management is a Linux feature, one many Linux users really care about. Do not neglect that.

I'm not demanding my way, I'm asking that the mechanism is in place and supported, just as your .zip files are. See which ends up being more popular and useful, then choose one down the road.

 

[bigtruck]

I am strongly in favor of a package system. With a lot of apps installed (emulators, homebrew, media players, OS files, etc.), it could be the difference between spending upwards of an hour checking which packages are out of date, downloading and manually upgrading them vs. a entering single terminal command or clicking an icon to upgrade the system.

Yes, removable storage does present a problem that is not usually a factor for most desktop package managers. It is not insurmountable, however. Each SD card could have its database of installed packages. When upgrading packages, the package manager could automatically install the upgraded package to the SD card the old one was on. When installing a new package, the package manager could give the user the choice of which SD card to install it to. Someone could make a version of the package manager for Windows (and OS X, etc.), where the program would function almost exactly like the Pandora version, upgrading and installing new packages directly to the SD card. I'm no programmer, but I don't see why this would be so difficult.

I hate to be frank, but this is idiotic. Any reasonably knowledgeable Linux user would tell you you're insane to waste the storage resources and bloat the local filesystem with a mix of completely unmanaged applications and data, which are impossible to update and keep track of. Especially on a mobile device, with relatively limited resources. I am buying this device /because/ of Linux, as I'm sure many others are. Package management is a Linux feature, one many Linux users really care about. Do not neglect that.

QFT.

 

[ShyGuy]

I think a lot of you are missing the point of the Pandora. I've been using Linux for years, and I've played with RPMs, Yellowdog Update Manager, Gentoo ebuilds and apt - I know my stuff. Problem is, the Pandora is not a desktop computer.

The way the archive system will work, if you want to install an application you simply unzip its archive wherever you want on any SD card. There's no confusion over where libraries should be extracted versus applications; if you want a specific set of applications and not another one, you can simply swap out a single SD card instead of worrying if you're leaving libraries behind.

With dual removable media this just makes sense. Now, if you're planning to use your Pandora more like a desktop computer (perfectly reasonable), you're absolutely free to install Debian, Ubuntu, or hell, NetBSD if you so choose. But for the grand majority of us this is a gaming system first, and package management simply creates too many logistics issues to justify the saved storage space.

 

[Gilrad]

So... Why not a little bit of both? So the problem is that in order to make sure everything works correctly with removable storage, you have to include the libraries with the packages, right? Downside to that being you have 60MB of redundant packages in different folders for different games.

So why not have a dedicated folder to put libraries into for each SD card? You have game A, which is 600kb of game, and 15mb of libraries. You then download game B, which is another 600kb of game, and 15mb of the same libraries. When you install game B, the game is sent to its own directory, and the packages are sent to their own directory. It then sees that the same version of the libraries already exist, so net result is you save 15mb of space.
End result (as an example)
/lib/Game A and B libraries (15 mb)
/apps/Game A (600kb)
/apps/Game B (600kb)

 

[ShyGuy]

It's an interesting suggestion, but what happens if you want to move Game B onto Game A's SD card? Then you have to go in and manually copy the libraries from Game B's original SD card to the other one. Not to mention the now-broken package manager database.

 

[bigtruck]

Ah, now I feel stupid for entirely missing the point about the problem with two SD cards. However, I still don't think that simple zips would be better than package management. The package manager could be configurable to deal with this problem.

Say GameA requires LibA. LibA is already installed on SD card 1, but you want to install the GameA on SD card 2. For people like myself, who are probably going to leave the same 2 SD cards in all the time, there could be the option to ignore the fact that LibA is on a different SD card than GameA is going to be installed to. For people who just want everything to work and change SD cards frequently, there could be the options to either warn the user about the situtation or automatically copy over LibA from SD card 1 to SD card 2. One of the package manager's operations should be moving packages between the two cards, which would also do the aforementioned dependency checking and notification/movement (or not, if you don't want it to).

Hopefully, even if zips are all that are officially supported, there will be a community-maintaned package manager and repo. Repo maintainers could manually convert zips created by whoever wrote/ported the app to whatever package format is used. I might even be willing to help out with maintaining such a repo, provided I have enough spare time.

 

[ShyGuy]

Now that's a possibility - an unofficial package manager designed to work with more than one removable disk (including USB hard drives/flash drives). If it had functionality for moving software between devices and automagically moving libraries, it'd probably work out okay.

At the same time, this would definitely be too complicated for the average Pandora user who just wants to install his emulators, copy over his ROMs and go (...for example). I'd definitely be interested in seeing it as an option, but I do think the current idea of just packaging dependencies with binaries is probably the most accessible route.

 

[Gilrad]

It's an interesting suggestion, but what happens if you want to move Game B onto Game A's SD card? Then you have to go in and manually copy the libraries from Game B's original SD card to the other one. Not to mention the now-broken package manager database.

What happens if you copy an installed Windows app or game from one folder to another? Often nothing, but its not uncommon for such an event to break the game or app, as there are now parts of it looking at an empty or nonexistent folder for the game.

But yeah, an automatic detection system would be best, but at the very least, a system of scripts with a GUI can be used as a sort of "App arrangement tool".

 

[Chero]

I guess people should really take a look at ipkg.
It has a feature ipkg-link which detects apps installed on SD and (re)creates symlinks. Running this when switching SDs can be a solution.

 

[scarath]

I hate to be frank, but this is idiotic. Any reasonably knowledgeable Linux user would tell you you're insane to waste the storage resources and bloat the local filesystem with a mix of completely unmanaged applications and data, which are impossible to update and keep track of. Especially on a mobile device, with relatively limited resources. I am buying this device /because/ of Linux, as I'm sure many others are. Package management is a Linux feature, one many Linux users really care about. Do not neglect that.

Exactly, I wager Linux is the only reason some people are getting this machine, it seems to be a case of "dont make the windows users have to think and/or use something new".

And as for the whole 'keep it simple for newbies' thing, APT is easy to use, the user just has to learn it and it's not an overly complex concept to get your head around.

I know the Linux nerds (me included) will be able to make apt-get work even if it is not the default package management system but it would just seem bad form for the Pandora not to have an apt-get system from the start.

 

[Chero]

I know the Linux nerds (me included) will be able to make apt-get work even if it is not the default package management system but it would just seem bad form for the Pandora not to have an apt-get system from the start.

ipkg would be a better choice : it's apt based, but has the ability to install files to external media (SD).
The base system with common libs and apps would live in nand, the bigger apps could be installed to SD.
"ipkg-link" makes sure everything works fine when you switch SDs. It was developed to work on handheld devices with low memory (which I guess the pandora is).
I've been using it on a Zaurus with 128Mb storage. I have kde3.5 installed on an SD, gPSP on another SD, xfce with gthumb, gimp, firefox and more on a third one. Switching them and running ipkg-link (which runs automatically when it detects a new SD inserted) and all is up and running.
ipkg = apt for handheld devices.

Have fun,
Chero.

 

[Squidge]

I have personally experienced database corruption myself, and possibly the side effect of unknown database corruption - apt saying its doing something, and then an hour later it's still on the same job (it had locked up solid). Using databases on removable media will make this even easier to happen. Do we expect users to fix there own databases? Some may have the knowledge to do this, but others not.

Yes, an ideal package management dedicated to the Pandora would be nice, but things like that take time and it's very unlikely that anything like that will be even started by the time Pandora is released, never mind finished and rock solid.

So we are taking a tip from the Windows world - bundle everything needed into the downloadable package, throw it onto the SD card from any OS without having to worry about "Do I have the installer for my OS?" and enjoy the download.

There *will* be dynamic libraries included with the OS, so its unlikely that you'll see many 300KB programs bloat to 15MB (and even then you probably find statically linking will generate much smaller binaries than including all the required libraries). Secondly, most of the X-Windows programs that are not Pandora specific will not be ported to the Pandora anyway, so will not need to be in the archive - you will install them from the usual places, and if they use package management for that, then so be it.

 

[Creature XL]

It seems like that Squidge has pointed out exactly what i was thinking while reading this thread. Ofcourse you are right, that a lib can bloat a small app from 600Kb to 15Mb. But did you even try to think about which libs that are? Its the libc and maybe openGL and SDL (with ttf, _gfx, _audio). But those will most probbably be in the NAND, THAT (the libs and the kernel) IS the Firmware (or at least a big part of it). Further, if your PSX emulator needs some lib hwich is 15Mb large... well the ISOs for PSx take up some Mbs aswell. Then the extra 15Mb (if they were there) won't matter too much.

I am not too familier with APT however I use fedora (RPM therefore) since years. I have lots of Mb wasted on my HD because one dev thinks he needs lib-xya-2.1.2 and another thinks he needs lib-xya-2.1.3. And in his package he specifies: 2.1.3 instead of 2.1.x so I have the same lib on my HD twice.

Since years I am happy with LiNUX, however, from time to time I like Windows for its easy way of install stuff. And please don't take out your argumetns that i am a noob or what ever. First, the Pandora SHOULD be handled by such "noobs" and second, in former times I had fun compiling each SW package for LiNUX, I bet some of the guys who advocating APT don#t even know there was a tiem were you HAD to compile everything yourself . But nowadays I just want things to work! I don't want to spend any minute on solving problems with a package manger. You can trust me that I am technically able to do so. I just do NOT want to do it.

 

[EvilDragon]

Believe me - we did think A LOT about APT and package management.

Most problems have already been pointed out:

- Remove the SD Card and use a different one and the database is not usable anymore.
- Have an SD Card crash or reformat it - this also leads to a corrupted database
- It's not as flexible: Some users might want to copy the stuff onto the SD Card where they want it. Some might want to use an autoinstaller.


Therefore, this is how we decided it to do:

- On the SD Cards, there is NO package managements. You either unzip it there yourself or some installer will download it and unzip it for you. The PXML files inside the archives make sure it's added automatically to the right places.
- The Firmware will have ALL basic libraries preinstalled. Therefore, there should NOT be any dependency problems. If some apps REALLY need to use some uncommon libs, these have to be linked statically. Shouldn't be the case for most apps though.
- As the firmware resides on the NAND, those updates will work via a package manager. If you're online, the firmware can be automatically updated using the package manager.


This system should work out fine for MOST users.
However, this will just be the basic Pandora system.
Everybody is welcome to port or release different firmware systems or distros for the Pandora - also including Package Management. Remember we're working on Ubuntu, which will also use package management. But for the normal user, who wants to use the Pandora to play games and easily reformat the SD Card without getting database problems, the above described way is the best one in my opinion.