Apt Repositories For Pandora

[Squidge]

Will it be possible to "spread" (??) the use of that package management onto the SDHC cards? I mean:

• 
  
  I don't see a reason why not, but the database would be on the firmware, so you would be writing to nand everytime you installed, updated, or removed a package.
  
  Even though you have far more experience of deb over pxml, are debs really going to be that much easier to use than clicking "Install", "Update" or "Remove" on the Pandora gui?

 

[BookTracker]

i think the current solution is rather interesting, but i would like to point out...
did anyone miss this post?
cause it does seem like it...

So... Why not a little bit of both? So the problem is that in order to make sure everything works correctly with removable storage, you have to include the libraries with the packages, right? Downside to that being you have 60MB of redundant packages in different folders for different games.

So why not have a dedicated folder to put libraries into for each SD card? You have game A, which is 600kb of game, and 15mb of libraries. You then download game B, which is another 600kb of game, and 15mb of the same libraries. When you install game B, the game is sent to its own directory, and the packages are sent to their own directory. It then sees that the same version of the libraries already exist, so net result is you save 15mb of space.
End result (as an example)
/lib/Game A and B libraries (15 mb)
/apps/Game A (600kb)
/apps/Game B (600kb)

someone mentioned the necessity of being able to drag-and-drop a folder from one sd card to the other...
well any windows minded fellow would understand that if you want your software 'installed' on another sd card you would need to 're-install' it there, just re-run, say -the file.pkg, and install it on your new place.
placing the needed libs in the local libs folder.

like evildragon mentioned this might not be necessary, be even if just for organizations sake, i would think this interesting.
plus, with this view you -can- have a package manager just fine, and everybody goes home happy. ^.^;

please correct me if im misplacing my words, but atm it does seem to me that everyone missed this post...


cheers,
and glad to see such an open discussion.
special thanks to evilD and co. for responding so nicely to some, rather aggressive, posts.

 

[Tor]

For the record, what has been described w.r.t. pxml files and the various mechanisms for keeping track of installed applications: That, to me, is package management. That is good, and very different from a just-copy-stuff-around approach.

Then there was someone saying something about 'package mangement + dynamic linking = bloat like windows'. I think someone doesn't know exactly what dynamic linking means for bloat.. it's not so much the dynamic linking, what's important is that such libraries on Linux are shared libraries. This means that if application A, B,C are running, and they all need library D, E, and F, and maybe some other, non-shared ones, it means that there's only a single copy of D,E,F in memory. This is pretty important on a system with just 128MB of RAM.

I remember a horrible desktop system which Hewlett-Packard was using, called HP-VUE. All the applications were statically linked (=they kept the libraries inside each application). Had to buy a lot of extra RAM, until we figured how to rip HP-VUE out and install a normal, shared-library X11 setup. This reduced the memory requirement to just a fraction. And this was only about the graphical applications.

Someone else mentioned the trend to self-contained applications for Windows, which sometimes makes life simpler. It does, at least for the particular case where you can just keep your thunderbird and firefox applications around on a USB stick so that you can use something sane on someone else's Windows box, without installing anything. But remember a couple of things: A: the only reason this is now even feasible is because the average Windows box now has lots and lots of memory. You can afford applications like this, and B: Most of the time you use only a single, or maybe a couple, such applications at a time. Imagine, if you can, if most of the applications installed on the system were like this. We know how much RAM the Pandora will have, and that is 128MB. The message from Nokia users is that the best you can say is that it certainly beats 64MB.. It's just barely enough and not more.

Finally, the 256MB NAND has been mentioned several times, always described as 'just 256MB'. I'm looking at a Nokia tablet which has the same 256MB NAND.
df -h tells me it has currently used 231MB. But
du -h tells me there's 414MB of files there.
The explanation? It's because the NAND uses 'jffs2', as will the Pandora (afaik). This is a compressed file system. You can stuff lots more in than just 256MB.

(And the above 414MB, btw, includes 175MB of data files that I lazily keep on the NAND instead of the card. With shared libraries there's a lot of space for a full operating system on the NAND.)

 

[cosurgi]

snip... (in short words):

• using statically linked apps eats a LOT of RAM (each app has its own copy of a library in memory) and we have only 128 MB. Using dynamically linked libraries saves a LOT of RAM because all apps use the same lib loaded into RAM.
• if we use jffs2 filesystem on 256 Pandora memory we can get about 414 MB of free space

Sorry, I made your post a bit shorter, because I wanted to say, that those are a very good points! Both of them. We have to use dynamically linked apps, because you never have enough RAM, especially if that's only 128 MB! And a compressed filesystem is a great idea too.

 

[Tor]

Actually it's more than 414MB.. 414MB takes 231MB of real NAND space in that example, there's still room for around 30-40MB more (my guess. Linux OS plus data files - things compress differently).

There aren't many filesystems to choose from for NAND, because you need wear levelling on the software level. Currently I'm only aware of jffs2, and the just-arriving UBIFS (sp?). Compression comes with both.

 

[sindbad]

There's also yaffs, which is in direct competition with jffs2.

 

[EvilDragon]

<screams and bangs his head onto the wall>

WHY doesn't ANYBODY seem to believe me when I tell that mostly all needed libraries are IN THE NAND FOR DYNAMICALLY LINKING?!

Only a few SPECIAL ones need to be included.

How big is the chance, that you want to run two multitasked programs at the SAME time that both need the SAME weird library which needs a BLOATED 15MB to run?
I guess chances are higher that you get eaten by a wild running tiger that escaped your local zoo...

 

[Bosbeetle]

/me gives ED a nice drink, and says "forgive them they know not what they do"

 

[Prometheus]

You just know that someone will find a way to run two applications at the same time, which both require the same obscure library, ED, just to annoy you...

... And then they'll get eaten by a tiger.

 

[TaG]

WHY doesn't ANYBODY seem to believe me when I tell that mostly all needed libraries are IN THE NAND FOR DYNAMICALLY LINKING?!
I believe you. I think the reasoning behind it is also understandable.

I'm looking forward to hearing/seeing how many packages are going to be made available this way. Linux fans tend to want to choose their favourite programs and this system sounds pretty good if the chioice is there - I think that is one of the reasons people want apt etc so that they can get at their favourite apps quickly. I'm less fussy about what is used than its effectiveness.

 

[Tor]

Relax, I do believe you.. up to a point. So, standard libs are installed in the NAND. Fine, that is as it should be. Got that the first time..

That's just like on those other omap-based, 800x480, 256MB NAND, 128MB RAM systems some of us keep referring to.

Now, on those same systems, the new applications coming out like to constantly start using the latest and greatest features of any library (say, gtk+, cairo and the like). That's just what developers tend to do (which I can understand, being one myself). So, when you install that application it must somehow make sure that you have actually updated your "firmware" (the NAND libraries) to the latest version that came out a week before. That's what a package manager takes care of, so let's hope the pxml+zip magic will handle that..

I think I've made my point(s) though, so I'll restrain myself from now on.

 

[cb88]

It might take half an hour or an hour or whatever, but over the lifetime of the Pandora you'd gain that time back many times over by not having to manually check for updates and install packages. Seriously, the

Which would most people prefer, spend 30 minutes to an hour learning to use a package manager, or just throwing it on the sd card and being up and running within minutes?

Pandora uses Linux, not Windows. Why would we throw out one of the main benefits of Linux just to accomodate people who are stuck doing things "the Windows-like way"?

The Windows-like way also happens to be the most popular for a reason - ease of use.

Wrong, the Ubuntu was just to show the Pandora's abilities.
The official distribution will be based on Angström.

Err, no. I'm one of the Pandora devs. I should know what distribution is being used

I would prefer learning a package manager anyway what are the chances that you can't fit everything on a single SD anyway the only thing that might spill over into multi SDs is multimedia.... and since when did Ubuntu become the default did I completely miss something? I *really* hope you guys go with debian instead ubuntu messes up soo muc.... init system bloated etc...

 

[cosurgi]

I would prefer learning a package manager anyway what are the chances that you can't fit everything on a single SD anyway the only thing that might spill over into multi SDs is multimedia.... and since when did Ubuntu become the default did I completely miss something? I *really* hope you guys go with debian instead ubuntu messes up soo muc.... init system bloated etc...

Honestly I hope that it will work if I just use in /etc/apt/sources.list
CODE
deb http://ftp.pl.debian.org/debian/ lenny main contrib


with ARMEL achitecture.

And that you will provide a Pandora specific linux-image.2.6.x-pandora.deb package.

Honestly that's all I need for full happiness on my side. EVERY package other than linux-image should work straight-off from debian repository.

 

[AireTamStorm]

There WILL be a package manager for the firmware, simply not for removable medias like SD Cards as it leads to a lot of problems.
This can be used for the SD cards with a union mount. It is literally as if the installs were happening on the root filesystem itself. If people use EXT3 instead of EXT2 or VFAT you'll get a journal which cuts down on corruption.

SD Cards DO corrupt. If there's a power loss or the user removes it before the cache is written to it, the database can be corrupted. How do you recover it?
APT Includes utilities to rebuild the database. You could translate that to a single button in a GUI somewhere.

AireTamStorm, i hope more devs follow this idea and put their software on a repo as well as in zip format on the Panda library site.
As said I will re-pack most popular applications and host them on an open repository. If that is later provided on OpenPandora.org, even better.

It should be possible to:
- Install the programs WITH and WITHOUT the Pandora. A lot of guys like to prepare the SD Card without having the handheld around. So it should be possible installing the programs onto the card by simply using a card reader, regardless of the OS.
A simple application can be written that creates shell scripts telling the package manager on the Pandora to install the programs. Even if not automatic, all the user would have to do is execute the script on the device itself. The desktop application doesn't actually do anything beyond fetching all the necessary files, putting them in the right place, and creating the script for the Pandora to take care of.

 

[Squidge]

So, when you install that application it must somehow make sure that you have actually updated your "firmware" (the NAND libraries) to the latest version that came out a week before. That's what a package manager takes care of, so let's hope the pxml+zip magic will handle that..

Sounds like you've not been reading this thread NAND is updated by a proper package manager, SD cards are updated via pxml+zip magic.

 

[EvilDragon]

Now, on those same systems, the new applications coming out like to constantly start using the latest and greatest features of any library (say, gtk+, cairo and the like). That's just what developers tend to do (which I can understand, being one myself). So, when you install that application it must somehow make sure that you have actually updated your "firmware" (the NAND libraries) to the latest version that came out a week before. That's what a package manager takes care of, so let's hope the pxml+zip magic will handle that..

There WILL be a package manager for the firmware, simply not for removable medias like SD Cards as it leads to a lot of problems.
This can be used for the SD cards with a union mount. It is literally as if the installs were happening on the root filesystem itself. If people use EXT3 instead of EXT2 or VFAT you'll get a journal which cuts down on corruption.

I guess most guys will still use VFAT, as Windows still dominates... :/

QUOTE

SD Cards DO corrupt. If there's a power loss or the user removes it before the cache is written to it, the database can be corrupted. How do you recover it?
APT Includes utilities to rebuild the database. You could translate that to a single button in a GUI somewhere.

Never heard of that - and I use apt since years. Do you mean apt-cache gencaches?
This regenerates the apt database (cache), but only if /var/lib/dpkg/status is still intact.
If that is also broken, you will have a nice time reinstalling everything.

QUOTE

AireTamStorm, i hope more devs follow this idea and put their software on a repo as well as in zip format on the Panda library site.
As said I will re-pack most popular applications and host them on an open repository. If that is later provided on OpenPandora.org, even better.

Well, you can be sure I will offer free hosting to any Pandora projects, so if you like, you can setup an open repository there, sure

QUOTE

It should be possible to:
- Install the programs WITH and WITHOUT the Pandora. A lot of guys like to prepare the SD Card without having the handheld around. So it should be possible installing the programs onto the card by simply using a card reader, regardless of the OS.
A simple application can be written that creates shell scripts telling the package manager on the Pandora to install the programs. Even if not automatic, all the user would have to do is execute the script on the device itself. The desktop application doesn't actually do anything beyond fetching all the necessary files, putting them in the right place, and creating the script for the Pandora to take care of.

And this runs on any system, any internet cafe, at work, etc.? Even with restrictions?

 

[AireTamStorm]

I guess most guys will still use VFAT, as Windows still dominates... :/
There are ways to get the other filesystems readable and writeable on Windows. Again, depends on the users needs, I guess.

Never heard of that - and I use apt since years. Do you mean apt-cache gencaches?
This regenerates the apt database (cache), but only if /var/lib/dpkg/status is still intact.
If that is also broken, you will have a nice time reinstalling everything.
Of course there are impossible to recover problems that may possibly occur- but you're assuring everyone that this happens often. It doesn't. If it did, nobody would use it.

Well, you can be sure I will offer free hosting to any Pandora projects, so if you like, you can setup an open repository there, sure
Much appreciated. As with the two other projects I'm working on at the moment, once I have something to show for, we'll talk. Right now I've gotta learn how to create packages and maintain a repo. Just to confirm: The PM software on the Pandora /is/ APT, right?

QUOTE
A simple application can be written that creates shell scripts telling the package manager on the Pandora to install the programs. Even if not automatic, all the user would have to do is execute the script on the device itself. The desktop application doesn't actually do anything beyond fetching all the necessary files, putting them in the right place, and creating the script for the Pandora to take care of.

And this runs on any system, any internet cafe, at work, etc.? Even with restrictions?
If coded in lets say, Python, as long as you included the interpreter, had access to necessary files, and had write access to the SD card, yes.

 

[cb88]

http://www.fs-driver.org/ windows ext2 support ... also reads writes ext3 w/o journal

also ... debian is using cowfs for versioning of liveusb systems... perhaps that could work for maintaining separate file trees?

 

[EvilDragon]

QUOTE

Never heard of that - and I use apt since years. Do you mean apt-cache gencaches?
This regenerates the apt database (cache), but only if /var/lib/dpkg/status is still intact.
If that is also broken, you will have a nice time reinstalling everything.
Of course there are impossible to recover problems that may possibly occur- but you're assuring everyone that this happens often. It doesn't. If it did, nobody would use it.

Don't get me wrong - I also use and like APT. But that's on a fixed harddisk, not a removable media. That's where I see the problems.

QUOTE

Well, you can be sure I will offer free hosting to any Pandora projects, so if you like, you can setup an open repository there, sure
Much appreciated. As with the two other projects I'm working on at the moment, once I have something to show for, we'll talk. Right now I've gotta learn how to create packages and maintain a repo. Just to confirm: The PM software on the Pandora /is/ APT, right?

Yes. As it looks like, the main OS the Panda will based is a specially enhanced xubuntu.

 

[paddy]

Im totaly new to Linux and have just started using Ubuntu and yes im very impressed with it but found getting used to installing apps,games and updates totaly bizzare at first,then once i got used to Aptget i found it refreshing and very defining and i now think of it as a linux thing and should be used when possible.

My problem with aptget is that i am a newbee to linux and i have had to reinstall Ubuntu 6 times in a week since i have been installing things that i guess i shouldnt have even tho the apps i had been installing are aparently for my version of ubuntu,but i guess this is more a linux problem in general unless the aptget only has trusted safe downloads for said distro,i can see pandora in future mabe having such problems unless its very strictly maintained.

yes linux is good but i still prefer windows simply becase im not scared of installing stuff then wondering will this boot up after i restart ?
takes time getting linux looking the way you want it then you reboot and bang no boot screen ,i personaly dont want to do the commands and other stuff to keep linux running when windows just runs and does what its supposed to do,well most of the time.

so theres a thought from a newbe to linux,i hope pandora will have the option to have a full linux distro on with aptget and all the other things that make linux,linux but please keep the pandora os,simple and user friendly as i will be hooking it up to a windows machine and i think most,mabe not on this forum but most people will be using windows.

ohh and as far as the package installer goes ,cant we have both ,or cant the important files just be kept in nand
and then only download the main program files without the bloat ? there must be a simple way to do this
what about a server update that if you insert a new mem card ,you click on server update and it will download
all the main fines needed by most/all software to a location on the mem card,then all your programs that you
have stored elsewhere like on pc can be transfered over to the card as normal.