The software side

[aTc]

With all this talk here about how great and wonderful the hardware will be, it means nothing if there isn't a solid software foundation to run on it.

Although I did see a few topics about software, they're mostly general "which distro" and "which apps to include" posts.

As we saw with the pandora, we'll be stuck with whatever choices are made at the very start.

So let's make sure that they're properly thought through this time.

There isn't an entire repo of .pnd like packages that will all stop working when a major change needs to be made at this stage.

Most pandora users just assume the pandora firmware magically works, and that all pnds run perfectly.

Behind the scenes however, more and more ugly hacks have to be applied to keep it appearing that way.

Both on the firmware itself, and the programs inside the pnd.

Problem is, I don't really see a coordinated effort to avoid those issues on a successor.

Just because it currently runs a debian image doesn't mean that there's nothing to be done.

What's needed is:

- Open, transparent, coordinated software development, right from the start.

        Everyone should be able to follow the design/development, and be able to complain about any problems they see there. (and maybe help avoid/fix them)

        Attracting as many people as possible to help develop/maintain this thing is important. (not to mention that anyone involved at this stage will probably buy one

        Git repo/bug trackers for everything.

- A good solid existing distro to start with.

        Which seems to have already been decided to be debian.

        All distros have guidelines on how to do things, custom packages should keep to those to avoid later upgrade problems.

- Package repo

        Rely on the existing distros packages as much as possible, any custom packages needed should be able to be automatically built and uploaded to the repo.

- Firmware image builder

        Firmware images should also be generated automatically. Images that require any manual interference before they can be installed on the device should be avoided.

- Firmware Installer

        A generated image usually needs additional configuration to be run on the device itself. (package installation, first run wizard )

- A well designed .pnd successor

        One that tried to avoid the problems .pnd currently has. Not only the way the system is implemented , but also have stricter guidelines for the software inside the .pnd itself.

        Once the first packaged software using it is released, you're pretty much stuck with it, so it's critical to get as much of this done right.

- Custom software

        Device specific themes, settings, utilities, applets, etc. These things don't write themselves you know.

        The pandora situation where most of it is a big mess of (zenity) scripts that poke around in lots of different places isn't ideal.

        A standard crosscompiler development environment would also be nice. Not having to fight with crosscompilers,libraries and headers makes developing for a device a lot more fun.

Since most of this doesn't depend on any final hardware, work on getting the software side set up properly should start as soon as possible.

Having a hacked up debian image appear out of nowhere the day the device is delivered, and expecting everyone to just build on that isn't an option.

 

[ekianjo]

work on getting the software side set up properly should start as soon as possible.

The problem is that there are very few OS/firmware developers already for the existing Pandora. So you can't expect new ones to magically appear out of nowhere.

 

[vinipsmaker]

Just because it currently runs a debian image doesn't mean that there's nothing to be done.

I've read that there will be way more internal persistent space, then it's possible to simply ship the old system and run it through chroot. Whatever distro we had running the chroot, it wouldn't affect its binaries, libraries, environment...

The major complications would be D-Bus services and other daemons, but I assume that Pandora's main use (gaming) won't be affected by it too much. But even still, fear not, because it's possible to tune the chroot to work under these cases. Currently there is even more powerful techniques like containers/namespaces that will let you tune the sandbox-with-very-low-performance-loss even further.

If the Pandora sucessor will use a processor that accepts the same instructions of the current Pandora, then it's possible to make a really high degree of retro-compatibility. And most of the tools are already here.

And while we tune-the-system-to-perfection (that will matter for very few), there is the *lazy* option of dual-boot.

work on getting the software side set up properly should start as soon as possible.

 The problem is that there are very few OS/firmware developers already for the existing Pandora. So you can't expect new ones to magically appear out of nowhere.

OpenEmbedded is the most inactive Linux distro I've ever used. Whatever we choose will problably have way more developers just for the OS and we can focus resources on firmware/drivers.

 

[vinipsmaker]

- A well designed .pnd successor         One that tried to avoid the problems .pnd currently has. Not only the way the system is implemented , but also have stricter guidelines for the software inside the .pnd itself.         Once the first packaged software using it is released, you're pretty much stuck with it, so it's critical to get as much of this done right.

This would be an interesting challenge.

 

[ekianjo]

A well designed .pnd successor

ANy suggestion as to what should be done for a new pnd system ?

 

[aTc]

The problem is that there are very few OS/firmware developers already for the existing Pandora. So you can't expect new ones to magically appear out of nowhere.

Yes, hence the "Attracting as many people as possible" bit.

Not to mention trying to keep them involved.. a lot of the people who did stuff for the pandora firmware in the beginning have never done anything for it after that.

I've read that there will be way more internal persistent space, then it's possible to simply ship the old system and run it through chroot. Whatever distro we had running the chroot, it wouldn't affect its binaries, libraries, environment...

OpenEmbedded is the most inactive Linux distro I've ever used. Whatever we choose will problably have way more developers just for the OS and we can focus resources on firmware/drivers.

Dumping some old firmware in a chroot is the worst option possible.  beeing backward compatible with the pandora should just be a "nice if it works" thing , not the end goal of a new distro on a new device.

OE is reasonably active. It's just that none of that activity is of any use for the pandora, because it's stuck with an ancient version thanks to some decisions that were made right at the start.

Even then, the OE is just the foundation of the pandora firmware,  there's still an awful lot of stuff on top of it that's pandora specific.

- A well designed .pnd successor         One that tried to avoid the problems .pnd currently has. Not only the way the system is implemented , but also have stricter guidelines for the software inside the .pnd itself.         Once the first packaged software using it is released, you're pretty much stuck with it, so it's critical to get as much of this done right.

This would be an interesting challenge.

Thanks to .pnd we know what works, and what needs improving.

Don't underestimate the amount of work that's required to get everything going, and to keep it going.

Of course it takes even more effort to do it properly

 

[vinipsmaker]

A well designed .pnd successor

 ANy suggestion as to what should be done for a new pnd system ?

The system should enable apps that don't trust one-on-another. For instance, if you install a webcam app and a word process, the current .pnd system will allow both apps see your home folder that contains content for both apps. It should be possible (and fairly easy) to say "hey, this app should have access to my home folder or this app should think that this one folder is my home folder, ...). Maybe some containerization thing.

http://pandorawiki.org/New_PND_format has a list of problems and suggestions for everything else.

 

[vinipsmaker]

Dumping some old firmware in a chroot is the worst option possible.  beeing backward compatible with the pandora should just be a "nice if it works" thing , not the end goal of a new distro on a new device.

No. The worst possible solution is to use a virtual machine.

And we have the ugly one too, the dual-boot (aka lazy) option.

A chroot system will keep the performance. It's a good foundation, but it won't solve things on its own. We need to provide replacements for every daemons/services and correctly making it communicate it with outside of the "jail". And there needs to be a also good way to activate the chroot.

You ask a programmer/system-administrator to *develop* a retro-compatibility layer, he (or she) will look what does he operating system can provide and the chroot (ot its recent replacements, like containers/namespaces) will be a foundation for his development.

 

[aTc]

A well designed .pnd successor

ANy suggestion as to what should be done for a new pnd system ?

This thread was meant as call to set up the whole development movement, and to point out what imo is the most important bit of a new device.

Talk about details like pnd successors and the other things should be happening in a development section on the pyra forums, which should have been set up ages ago.

 

[_wb_]

I think the PND system only makes sense for stand-alone software like games. As soon as the software should be integrated with other things (e.g. office software, image viewers, web browsers etc) it is a much better idea to distribute and install them as regular Debian packages (or whatever distro).

The idea of self-contained PNDs with an appdata directory to store its settings and savegames is fine for games, emulators, and stand-alone applications, but it sucks for general-purpose applications where you really want shared libraries and shared data -- e.g. maybe you want to download an image in firefox, then edit it in the gimp, then include it in a libreoffice document, then upload the document with some ftp program, etc. It does not make sense if all of these programs have their own data directory in relatively obscure places like /media/SDCARD/pandora/appdata/hdonk_firefox_001.

I don't think the PND format itself is that bad, it's actually quite nice in my opinion. But it should not be used for general-purpose applications anymore if limited internal storage is no longer an issue.

 

[Super Roach]

_wb_ does have a point, which we've ran into in the past with having Python being out of date in the core nand, thus needing to be included in each pnd itself.

 

[sebt3]

I don't think the PND format itself is that bad, it's actually quite nice in my opinion. But it should not be used for general-purpose applications anymore if limited internal storage is no longer an issue.

Agreed, beside, as the pyra will have an eMMC, all the standard software should come as .deb while the games could still come as pnds.

I for one think the software distribution is solved by the choice ED made hardware wise.

On the other hand I think there are a few things that could be fixed from start :

Having the 2 nubs on 2 differents joysticks code-wise is a pure madness. As much as having to hardcode pageup and pagedown as gaming buttons.

On the cross-compiling toolchain side. I tried to bring a setup using only debian packages... And failed : Debian SID is currently transitionning from emdebian method to the multi-arch setup. MultiArch isnt ready yet for this... at all. emdebian method is targetting newlib system while we will be using glibc... On this side OE is doing a better job. 

 

[Cloudef]

I've had plans to do AUR like system for PNDs for Pyra. This would of course include build chroot with distcc and makepkg fork building PNDs from recipes.

This brings two points, 1. You can see how the PND is made from recipe and modify it to your needs, 2. It allows automate PND builds for newer versions.

This also makes PNDs more attractive as they are not just archives containing binaries (and maybe random source code) on some repo, but you can actually rebuild it even if some community member dies.

Of course, if we can I would change PND format as well, currently it's really inefficient to read and it's just XML and PNG tailed at end of SquashFS.

IMO you could make proper format with header that contains offsets and sizes for relevant data. Replace XML with json (even maybe UBJSON to keep everything binary).

And then setup kernel to handle this format with binfmt which allows you to even run pnds like they are normal executables.

(all this would simplify the code of libraries such as libpndman as well and I would gladly rewrite it for new format)

We can also fix things like working file associations and what not..

This of course depends on how relevant PNDs will be on Pyra anymore.

Like said above debian repositories and maybe custom repository becomes much more attractive option for your general software.

And PND repository would most likely used for games or some big software that doesn't play well with debian (see /opt - problematic packages in arch packaging guidelines, usually games with flat file layout)

We have good change to change the PND system since debian will most likely be hardfp, which means old PND's wont work anyways since our distribution and libraries is softfp.

(Unless you provide softfp version of every library that uses FPU)

As for desktop software, Python (or any other high-level language) with gtk/qt (depending on desktop environment) lib will most likely shine here, not zenity scripts.

I could even contribute here, and we have porg as well rounded UI/usability designer as well.

Another important thing is when doing these, is that everything is well packaged. Packaging is imo one of the things that makes Linux distros nice to use.

Well packaged also means that the Pyra specific stuff is easy to move to <insert your favorite distro>.

I agree with aTc that the quality of software in Pandora is somewhat disapointing.

Edit:

Speaking of quality of packaging, if we have AUR clone/makepkg, most of the quality issues go away.

But we could have guidelines regarding this. Should the software be linked against static libraries when possible? (license may not allow)

 

[Wally]

I agree that we need to start planning out a system now - paving way for the future and not being startled that the Pyra is in production but has no official clean firmware which has REALLY stuck with us the entire way of the Pandora and has seemed to drive us batty at times

I'm going to host a temporary wiki on my server (when I have set it up correctly) which will be avaliable to registered / confirmed developers + Consolidated members only until we get our Pyra Wiki and have all content transferred across. (I'll make sure its mediawiki as well for that sake). It's ZFS Mirrored too so that its backed up .

I have faith in these guys who are going to do all the dirty work.

 

[Linux-SWAT]

The first and most important thing: ED, in his name, has to provide a clear button layout default for applications (B to validate, X to cancel, Pyra button for menu, etc).

Of course that won't be possible to respect that for some apps, but at least we'll have a clear guideline.

I wish pnd makers use more static to avoid lib updates/other distro problems.

Seems like OMAP5432 has virtualization set. I'm just saying that it opens the possibilities.

Some small-sized .pnd can share OP and P2 versions of programs.

Maybe some programs can share the same libs, but different executable.

Many .pnd will be useless for the Pyra as they will be available in the Deb repo.

 

[ekianjo]

I think the PND system only makes sense for stand-alone software like games. As soon as the software should be integrated with other things (e.g. office software, image viewers, web browsers etc) it is a much better idea to distribute and install them as regular Debian packages (or whatever distro).

Well, for games i'm not sure it would really be an issue since they usually have few dependencies, but for other programs PNDs can be useful to avoid dependency hell Even on modern distros I still get in situations where there is a conflict of libraries between applications.

 

[fusion_power]

The first and most important thing: ED, in his name, has to provide a clear button layout default for applications (B to validate, X to cancel, Pyra button for menu, etc).

Of course that won't be possible to respect that for some apps, but at least we'll have a clear guideline.

I already suggested this ages ago for the Pandora. But nobody wanted real standards so stuff like default and always working button layouts are still a mess and basicly everyone does his own thing which confuses normal users alot imho. Pyra really needs some "white papers" for general guidelines of software designs , buttons, layouts, menus, usability etc. Yes, this may not work for every app but it's still a way to have the same layout for most of the apps.  

Every console has this for a reason, you will always find the same buttons for the same general actions there. And with the Pyra, "usability" finaly should also reach the Linux-Open-Handheld universe.

Oh, and of course you have to adjust an PC Desktop OS like Debian to work onto an 5" HD screen on a Handheld device. I already mentioned that I don't want to see jokes like font size 8 onto the Pyra just because People simply do 1:1 conversions of PC programs or the entire OS. ^^"  

 

[PokeParadox]

This could be solved by having the input API have target actions that can be mapped to the physical buttons and remapping program.

Thus developers would say if(contols->isAccept(PRESSED)) do action

This would use which ever button the user has mapped the accept button to... (with sensible defaults)

 

[ptitSeb]

You do realize that the PND system works quite well on the Pandora because it's the only system available and the OS is stable.

Having both PND and .deb at the same time will probably lead to many non-working PND. Look at what happens when the Python 2.6 -> 2.7 upgrade lately. Many PND broken. What do you will happens with the ability to update all system / library component?

 

[sebt3]

Having both PND and .deb at the same time will probably lead to many non-working PND.

Well, if only games are as PNDs, the problem is clearly narrowed. And then, games are usually only linked to sdl, openal, gles and maybe liblua. These are only a limited set of libraries.

We could pin them in the debian config and host them in an official pyra debian repo.

The other sane option is to keep PND as a backward compatible feature only (some ginge a-like app pre-installed) and package everything as deb. But then, if you want to be included in the official pyra repo, you'll need to provides full source to allow rebuild for these which might be a problem. 

EDIT:  

(Unless you provide softfp version of every library that uses FPU)

I was planning on having a hacked superzaxoon rootfs in a squashfs/qcow2 file provided. so just a mount + chroot away