Discussion on repository, packages, etc. (split from Some thoughts thread)

[benoitb]

So, why did I write all this? I don't know myself. I just thought, I had to write down my feelings about this place. It was awesome. It went meeh... I don't think the Pyra will fix this. Yes, we will still have some very Pyra related releases, probably emulators and some optimized games. However the most interest of the board these days isn't Retro anymore. We are talking about PSX, N64, Dreamcast and Nintendo DS. These were great console - no question - but not really Retro. Most of the users want 3d graphics. I know that, my games had bad game mechanics but simple 3d graphics, why they were quite successful. Furhtermore the Pyra will be a mini computer even more than the pandora. So new users with new use cases will join, who are not interested in some hacky homebrew anymore, if they can install the best worms clone ever with sudo apt-get install hedgewars. Maybe the Pandora discussions even killed gp32x (there were senseless keyboard and button threads, too!) in the very first place? Interesting theory. We will see. If a new Pyra board will be created like openpandora.org as reaction to gp32x, this board will probably die, too.

Honestly, having the Debian ARM library an "apt-get install" away is such a good thing. I fail to understand how this could be a negative thing.

Then the Pyra specific ports will probably be only things that erally add value, like CPU/GPU optimization (a mplayer build ? some emulators ?) or control pad configuration.

I guess people are talking about PSX, N64, Dreamcast, Nintendo DS and PSP because those are great systems that we couldn't take for granted before. I would be so happy to get Sega Saturn emulation for example as I have been waiting to play Panzer Dragon Saga and Shining Force 3.

 

[ptitSeb]

Honestly, having the Debian ARM library an "apt-get install" away is such a good thing. I fail to understand how this could be a negative thing.

The "apt-get install" thing is like a fastfood of software. Hi quantity of Generic software, compiled for the masses. It's fine, but it doesn't compare to the hand-maded ports tailored for the machine...

What do all of you think might happen if we removed the "Like This" system

The "Like this" should be used only when words are meaningless, or as a complement for some comments. Just pushing "Like this" is A click. If it's the only thing done to show a dev or poster some encouragements, well, just one 1 click is a bit cheap...

 

[_wb_]

Honestly, having the Debian ARM library an "apt-get install" away is such a good thing. I fail to understand how this could be a negative thing.

The "apt-get install" thing is like a fastfood of software. Hi quantity of Generic software, compiled for the masses. It's fine, but it doesn't compare to the hand-maded ports tailored for the machine... 

I don't really agree. At least not on the Pyra. On the Pyra we should be able to keep everything up to date. I think we should tailor applications to the machine (e.g. let its interface work better on a small screen, customize its keybindings, tweak the compilation options etc), but this time we should try to push more of that upstream. So in the end, you would still apt-get install stuff, but it would not be generic software, it would be software that supports many platforms well, not just normal PCs but also the Pyra.

Like how it was done with Battle for Wesnoth. Originally that game needed a higher resolution than the 800x480 of the Pandora. Instead of just taking some version of the game and tweaking it to the Pandora, then releasing it as a PND, the changes were pushed upstream, and the result is that Battle for Wesnoth now supports low-res/small-screen devices. So when we will do apt-get install wesnoth on the Pyra, it will probably work reasonably well.

I think that's how we should do it more often in the future (disclaimer: this is not what I did in the past, so I'll have to change my behavior). I think it's better for Pyra users in the long run (because they will get updates even after the original 'porter'/tweaker has lost interest), and it will also give more visibility to the Pyra in the FOSS communities, improving the reputation of the device ("no, it's not just gamers who want to run emulators"), drawing in more good devs to the Pyra community, in a nice positive feedback loop.

 

[ptitSeb]

Honestly, having the Debian ARM library an "apt-get install" away is such a good thing. I fail to understand how this could be a negative thing.

The "apt-get install" thing is like a fastfood of software. Hi quantity of Generic software, compiled for the masses. It's fine, but it doesn't compare to the hand-maded ports tailored for the machine... 

I don't really agree. At least not on the Pyra. On the Pyra we should be able to keep everything up to date. I think we should tailor applications to the machine (e.g. let its interface work better on a small screen, customize its keybindings, tweak the compilation options etc), but this time we should try to push more of that upstream. So in the end, you would still apt-get install stuff, but it would not be generic software, it would be software that supports many platforms well, not just normal PCs but also the Pyra.

...

Yes and No.

I agree with the upstream thigy,

but Compile-time IFDEF will not be applied on the standard Debian repo. Dynamic guessing of the plateform is much more work to implement than compile time define.

 

[sebt3]

^^

ptitSeb point was more about compilation flag and perfect CPU optimisation than anything else.

For compatibility, the ARMHF port of debian target ARMv7 and VFP3D16. (and we're lucky as the soft-float port support way lower-end CPU, so with way less optimisations)

There's no support for NEON, nor for OMAP5 specific optimisation gcc could add.

For stuff like audacious, most of us couldnt care less...But there's still stuff for which we'll be very happy to have specific compilation for us

 

[_wb_]

^^


ptitSeb point was more about compilation flag and perfect CPU optimisation than anything else.


For compatibility, the ARMHF port of debian target ARMv7 and VFP3D16. (and we're lucky as the soft-float port support way lower-end CPU, so with way less optimisations)


There's no support for NEON, nor for OMAP5 specific optimisation gcc could add.


For stuff like audacious, most of us couldnt care less...But there's still stuff for which we'll be very happy to have specific compilation for us

OK, fair point. In those cases though, I think it makes sense to have a Pyra-specific .deb repository, specifically compiled for the Pyra, but based on the same Debian source packages but with OMAP5-specific compiler flags. And that Pyra-specific repository should of course be added to the default apt sources, so you can still just apt-get install stuff.

Even for software that does not need OMAP5-specific optimization to run "full-speed" (or "snappy enough"), it might be worthwhile to do the optimized compilation anyway. In particular for standard libraries and common stuff that many people will have running in the background much of the time. It may not make a noticeable speed difference, but it will still help a bit to conserve battery life (and perhaps memory too).

 

[TrashyMG]

^^


ptitSeb point was more about compilation flag and perfect CPU optimisation than anything else.


For compatibility, the ARMHF port of debian target ARMv7 and VFP3D16. (and we're lucky as the soft-float port support way lower-end CPU, so with way less optimisations)


There's no support for NEON, nor for OMAP5 specific optimisation gcc could add.


For stuff like audacious, most of us couldnt care less...But there's still stuff for which we'll be very happy to have specific compilation for us

OK, fair point. In those cases though, I think it makes sense to have a Pyra-specific .deb repository, specifically compiled for the Pyra, but based on the same Debian source packages but with OMAP5-specific compiler flags. And that Pyra-specific repository should of course be added to the default apt sources, so you can still just apt-get install stuff.

Even for software that does not need OMAP5-specific optimization to run "full-speed" (or "snappy enough"), it might be worthwhile to do the optimized compilation anyway. In particular for standard libraries and common stuff that many people will have running in the background much of the time. It may not make a noticeable speed difference, but it will still help a bit to conserve battery life (and perhaps memory too).

Well we actually do have a repository already setup for pyra specific things. I'm sure committing new deb packages is restricted to only a few people. 

 

[EvilDragon]

Games in a Debian repo? How can I install them on a normal FAT32 SD Card without crashing the OS when I remove it?


We need a hybrid here. Standard Pyra stuff for the OS in a debian repo, games, etc. as dbp.

 

[edgex004]

Games in a Debian repo? How can I install them on a normal FAT32 SD Card without crashing the OS when I remove it?

We need a hybrid here. Standard Pyra stuff for the OS in a debian repo, games, etc. as dbp.

Would it be possible to have a script on the Pyra that could automatically pull a package from the Debian repo and package it as a dbp? This would allow users to choose certain programs to be installed to SD cards even if there isn't a Pyra specific port yet.

 

[_wb_]

Games in a Debian repo? How can I install them on a normal FAT32 SD Card without crashing the OS when I remove it?


We need a hybrid here. Standard Pyra stuff for the OS in a debian repo, games, etc. as dbp.

Would it be possible to have a script on the Pyra that could automatically pull a package from the Debian repo and package it as a dbp? This would allow users to choose certain programs to be installed to SD cards even if there isn't a Pyra specific port yet.

Yes, that would be neat. Not so easy to do, but neat. There are two main issues: how to handle dependencies, and how to deal with the fact that normal Debian packages are meant to be installed in the root filesystem (mostly in /usr).

 

[Wrath Of Khan]

One of the things that made the pandora fun was when someone ported or wrote some new software that enhanced its functionality.

You might take out of the box, full functionality for granted, if it was an android system etc, but there was something cool about seeing how far people could push the pandora, and how much they could use the hardware features.

I expect much of the same for the Pyra and hope to see the same community spirit at work. The Pyra will be novel because it has a real keyboard and is a gaming device, but mostly because it is made by ED and his team, it should have a heart and a soul. 

 

[Antartica]

Would it be possible to have a script on the Pyra that could automatically pull a package from the Debian repo and package it as a dbp? This would allow users to choose certain programs to be installed to SD cards even if there isn't a Pyra specific port yet.

Yes, that would be neat. Not so easy to do, but neat. There are two main issues: how to handle dependencies, and how to deal with the fact that normal Debian packages are meant to be installed in the root filesystem (mostly in /usr).

Knowing what packages are "the base OS" for the pyra, the "dependency problem" is not very hard (it is a matter of pulling all the dependencias into the dbp and doing the customary LD_LIBRARY_PATH, LD_PRELOAD et al).

That said, I'm more concerned not only by /usr but also by /etc. Debian packages are compiled to search configurations in specific directories of /etc, and data files in some directories of /usr.

It is partially solvable by doing a library to be loaded with LD_PRELOAD that captures all open/mmap/stat/opendir calls and converts the paths to the ones of the dbp loopmount.

The last time I used something like that was with electric fence (a library to debug memory problems, popular in the times when valgrind still didn't exist). And I've written once a library to make an old closed source program behave well by doing some magic every time it did an open (that is to say, it is not hard at all to implement such a library).

Only works for symbols (functions)  in dynamic libraries. Not for functions in the program nor when libraries are accessed with dlopen() (as in plug-ins).

 

[Hồng Thất Công]

One of the things that made the pandora fun was when someone ported or wrote some new software that enhanced its functionality.

You might take out of the box, full functionality for granted, if it was an android system etc, but there was something cool about seeing how far people could push the pandora, and how much they could use the hardware features.

I expect much of the same for the Pyra and hope to see the same community spirit at work. The Pyra will be novel because it has a real keyboard and is a gaming device, but mostly because it is made by ED and his team, it should have a heart and a soul. 

Agree.  If PYRA uses standard Debian repo with no specific Pyra optimizations like notaz, ptitSeb, sebt3 and other devs have done for Pandora, it is no fun/niche anymore.  I may as well go and buy a small x86 z3795 laptop or something like that and install Debian on it and sudo apt-get install all I want.  Oh, and don't tell me that small laptop doesn't have physical controls and not portable and blah blah blah as I heard all that before.  Also, there will not be a place/chance for interfacing directly with devs and ask for cetain ports like we have here.  Face it, Pandora is a niche device and so will Pyra and should be kept that way, well, at least until an x86 SoC board is produced for Pyra.

 

[TrashyMG]

@Ryo, so far what I see most games and emulators are poorly optimized. DOSBox on the repository ran slower than the Pandora version. I have a feeling that if an 8GB eMMC is chosen, we will have a similar, but still better situation as the Pandora space wise. Even with 16GBs still prevents large games from residing on the internal storage.


EDIT: Also the armhf repository is a tad anemic compared to the x86 repositories game wise. Most are hard coded to opengl and my guess even with glshim some developer intervention is needed.

 

[Glyph Reader]

@Ryo, so far what I see most games and emulators are poorly optimized. DOSBox on the repository ran slower than the Pandora version. I have a feeling that if an 8GB eMMC is chosen, we will have a similar, but still better situation as the Pandora space wise. Even with 16GBs still prevents large games from residing on the internal storage.


EDIT: Also the armhf repository is a tad anemic compared to the x86 repositories game wise. Most are hard coded to opengl and my guess even with glshim some developer intervention is needed.

Don't forget about the micro sd card feature.  One could install the default system on the micro sd card instead of the eMMC, allowing for more potential space. 

-Glyph Reader

 

[Haraldur]

^^


ptitSeb point was more about compilation flag and perfect CPU optimisation than anything else.


For compatibility, the ARMHF port of debian target ARMv7 and VFP3D16. (and we're lucky as the soft-float port support way lower-end CPU, so with way less optimisations)


There's no support for NEON, nor for OMAP5 specific optimisation gcc could add.


For stuff like audacious, most of us couldnt care less...But there's still stuff for which we'll be very happy to have specific compilation for us

OK, fair point. In those cases though, I think it makes sense to have a Pyra-specific .deb repository, specifically compiled for the Pyra, but based on the same Debian source packages but with OMAP5-specific compiler flags. And that Pyra-specific repository should of course be added to the default apt sources, so you can still just apt-get install stuff.

Even for software that does not need OMAP5-specific optimization to run "full-speed" (or "snappy enough"), it might be worthwhile to do the optimized compilation anyway. In particular for standard libraries and common stuff that many people will have running in the background much of the time. It may not make a noticeable speed difference, but it will still help a bit to conserve battery life (and perhaps memory too).

Gentoo? Perhaps a system that automatically compiles Gentoo packages (on a server, to provide binaries, as all Pyras shall be identical/limited-in-variation), as they are updated, using a standard set of build-flags for the Pyra? Or the Arch Build System (PKGBUILD/AUR) or some other ports-like system. This may work especially well for CPU-board upgrades (especially if one is x86(_64)) and if one wants to exploit GCC's --march=native/armhf-a15(NEON)/whatever.

If it is too late to switch from Debian (I am not fussed, I have no quarrel with Debian), then such a system can be used to generate DBPs/whatever as a parallel system to Debian.

Of course, assuming GCC is installed by default, an enthusiastic/bored user could do all this manually... I suppose I would be 'enthusiastic' for about five applications plus a few important libraries.

 

[TrashyMG]

@Ryo, so far what I see most games and emulators are poorly optimized. DOSBox on the repository ran slower than the Pandora version. I have a feeling that if an 8GB eMMC is chosen, we will have a similar, but still better situation as the Pandora space wise. Even with 16GBs still prevents large games from residing on the internal storage.


EDIT: Also the armhf repository is a tad anemic compared to the x86 repositories game wise. Most are hard coded to opengl and my guess even with glshim some developer intervention is needed.

Don't forget about the micro sd card feature. One could install the default system on the micro sd card instead of the eMMC, allowing for more potential space.

-Glyph Reader

I'm not, but developers shouldn't assume everyone is going to use the uSD card.

 

[CrazyIvan]

Games in a Debian repo? How can I install them on a normal FAT32 SD Card without crashing the OS when I remove it?


We need a hybrid here. Standard Pyra stuff for the OS in a debian repo, games, etc. as dbp.

Install games into internal memory as debs and game data on sd similar to pnds. Launch games with some wrapper that mounts data on launch and unmounts on exit.

Resolve dependencies for games properly, using package manager.

It's very convenient if we want to install games using pyra and makes it impossible to install games by simply copying some files to sd card.

 

[TrashyMG]

Games in a Debian repo? How can I install them on a normal FAT32 SD Card without crashing the OS when I remove it?


We need a hybrid here. Standard Pyra stuff for the OS in a debian repo, games, etc. as dbp.

Install games into internal memory as debs and game data on sd similar to pnds. Launch games with some wrapper that mounts data on launch and unmounts on exit.

Resolve dependencies for games properly, using package manager.

It's very convenient if we want to install games using pyra and makes it impossible to install games by simply copying some files to sd card.

That is cumbersome plan, so developers would need to maintain Debian repository packages and another separate solution just for application/game data?


Slaeshjag's DBP system is a lot more elegant than PNDs and should be better at avoiding dependency hell.

 

[CrazyIvan]

Games in a Debian repo? How can I install them on a normal FAT32 SD Card without crashing the OS when I remove it?


We need a hybrid here. Standard Pyra stuff for the OS in a debian repo, games, etc. as dbp.

Install games into internal memory as debs and game data on sd similar to pnds. Launch games with some wrapper that mounts data on launch and unmounts on exit.

Resolve dependencies for games properly, using package manager.

It's very convenient if we want to install games using pyra and makes it impossible to install games by simply copying some files to sd card.

That is cumbersome plan, so developers would need to maintain Debian repository packages and another separate solution just for application/game data?


Slaeshjag's DBP system is a lot more elegant than PNDs and should be better at avoiding dependency hell.

It is not. It's basically DBP but only for game data. Data directories should be registered somewhere in the system during installation and any executable should be called via wrapper only (wrappers could even be created automatically during installation).

What's the problem with maintaining data and executables separately? Data is usually quite static.