All,
As this mailing list seems to have sparked some debate I thought it may be prudent to dump down what I have been pondering about a new 'full flash' firmware release.
I guess this is the workflow side of things; I am not dwelling on the specifics of the userspace packages just yet but I expect to bump most things up to the latest stable releases of packages (and I hope to have Xfce 4.8 packages to go in as a result of an unrelated side project of mine).
This has no bearing on the hotfix releases that EvilDragon heads up, they should continue on the existing image tree as long as needed until such time as we retire the 'Z' release.
This all needs some refinement and discussion. As it stands it is a brain dump of what I have been working towards and what I am taking about when referring to the .next stuff (or whatever contrived name we want to come up with ;)).
Firmware Image: ---------------------- This is actually a fairly simple one at 1st. Move to tracking the latest mainline org.openembedded.dev tree (done) and using the latest distribution information for Ångström, the 2010.7 based metadata (done).
The hard part is now getting to all working together and adapting our images and recipes to make use of features inside newer OpenEmbedded/Ångström metadata. That is easier said than done however it is territory we all know well enough. There will be loads of fixes, bugs, regressions, upgrades and things that are broken upstream that we will have to contend with, the sooner we get stuck into this and get images out the better.
My plan is to offer the images as tarballs to extract to EXT formatted SD cards (automated daily builds would be idea) just like we did for the 1st release. I'll also try and get the time to include the necessary boot scripts to have u-boot autoload the image if the SD is in the 1st slot so people can just untar and test. I don't want to offer UBIFS NAND images (at least not the wider userbase) until things are pretty close to ready but I will build them alongside the tarballs.
Once we have got things stable enough to start dishing out images I'll also move over all the scant documentation on installing them from the old private wiki to the public ones. Give me a week or so before images start to trickle out from me, I'll keep pushing .next to GIT in the meantime as I go if others want to play along and get stuck into build/testing/patching/being awesome (I rebase .next so remember that if you're tracking it).
As we get close to release we can snapshot the org.openemebedded.dev tree into .next and stop regular pulls. This will form our stable marker and we can cherry pick in any fixes we need from mainline on a case by case basis from that point. For now, I will pull in regularly and push up as much as we can upstream and then capture them when they come back down from the upstream trees.
Kernel: ---------------------- The kernel will always come from the git.openpandora.org kernel tree and it is up to Grazvydas to manage how things trickle up and down from mainline.
For now use the GIT tip of the stable 2.6.27 branch looking to move to the 2.6.37 kernel ASAP.
Start the move to 2.6.37 by offering images with kernels and modules for both kernels included if I can figure a way to cleanly do it. I also need to patch over AUFS2 into the kernel, I will do this the same way as I did it for .27 as a patch that sits with the recipe in the metadata tree.
Grazvydas, I would appreciate your take on this being a good idea or not? We could stay with .27 (or .35) but it seems sensible to move to a very recent kernel and I would love to be able to make use of it in userspace.
Support Bits: ---------------------- This is really all the supporting bits that we produce outside of the main tree but within the scope of the project. libPND, MiniMenu, voodoo mojo grog scripts and the like.
I would like to suggest we continue the process followed during the 1st release for now. It worked well before.
The maintainer for the bit in question just asks for a recipe SRCREV bump when they need their latest binaries rolled into in a test image and we announce the point that we freeze down the image for a release and those SRCREV's either get stuck at that level or better still we switch them to GIT tags and do formal releases of the support stuff alongside the main firmware.
The maintainers of this stuff outside the main OE or Kernel tree are free to carve things up and branch as they see fit provided they make sure the recipes are kept in sync and they don't break the build for the image with unannounced moves ;).
Toolchain: ---------------------- Continue to develop the toolchain scripts based on the work I started with meta-toolchain in OE and try and fix as may host system packaging issues as we can without resorting to hacks or just copy/pasting in files. There is some stuff from Poky/Yocto we may wish to pull in for this but I think it will come into our tree via OpenEmbedded soon enough anyway.
We may also want to tweak things so the root path for the chain can be a little more configurable, I consider that one a low priority nice to have.
The toolchain should also be the same set of binaries used to build the image, if the image is GCC 4.5 SVN blah the toolchain is GCC 4.5 SVN blah etc. etc. - OE makes that pretty easy.
We should also aim to have a toolchain and SDK ready to go with the new firmware (or say release + 2 weeks?). Normal x86 and x64 Linux builds should be fine; we can offer unofficial toolchains for other architectures if there is demand.
Is that enough rambling for one email?
Regards,
John
--
What is a grue?
The grue is a sinister, lurking presence in the dark places of the earth. Its favourite diet is adventurers, but its insatiable appetite is tempered by its fear of light. No grue has ever been seen by the light of day, and few have survived its fearsome jaws to tell the tale.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 23.02.2011 23:02, schrieb John Willis:
Firmware Image:
(...)
Sounds like a good plan to me. Providing some means of testing (and asking people to use them) based on sd cards without the people killing their NAND, since the (earlier) .next images are likely to be problematic.
Kernel:
(...)
I don't know the exact status of the kernel with patches and such. But might it maybe even be a good idea to directly aim for .38? It is supposed to be out rather soonish, which is likely "significantly before .next is ready". IIRC linux-omap is regulary synced in the git, not sure if it can directly be dropped in or not.
Toolchain:
Continue to develop the toolchain scripts based on the work I started with meta-toolchain in OE and try and fix as may host system packaging issues as we can without resorting to hacks or just copy/pasting in files. There is some stuff from Poky/Yocto we may wish to pull in for this but I think it will come into our tree via OpenEmbedded soon enough anyway.
We may also want to tweak things so the root path for the chain can be a little more configurable, I consider that one a low priority nice to have.
The toolchain should also be the same set of binaries used to build the image, if the image is GCC 4.5 SVN blah the toolchain is GCC 4.5 SVN blah etc. etc. - OE makes that pretty easy.
We should also aim to have a toolchain and SDK ready to go with the new firmware (or say release + 2 weeks?). Normal x86 and x64 Linux builds should be fine; we can offer unofficial toolchains for other architectures if there is demand.
Yes, having a "real" official toolchain sounds like a good plan to me. Personally I don't care too much what it is based upon, as long as it is easily usable and works nicely. Makes sense to have it based upon the OE stuff, especially if it is easily possible to gather the required packages there.
How does this toolchain (in general) cope with upgrades? Is it, once the feeds are setup correctly so that they point to openpandora.org, possible to just use opkg to get the relevant updates?
Codec pack: - -------------------------- Another thing that should be considered for the next firmware series is how to act in regards to the codec pack with mplayer and such. JDGBOLT is currently working on "panplayer" which should offer some good generic support for a lot of stuff. The question together with this are: 1) Does it make sense to offer a codec pack once a decent "complete bundle" for the mplayer is available? 2) If that is the case, are there easier ways than manually gathering a list of packages and "brute force" installing them using opkg? Is there someone available with the knowledge and time to do this?
Backwards compatibility: - -------------------------- The new angstrom version will change several libs that are often linked against. The most prominent examples are probably libjpeg.so.6 (will be libjpeg.so.8 after the upgrade) and boost (they got the version number in the mainly used name, too). The question now is: Do we accept it that a major update of the OS potentially breaks many existing PNDs? Should we advise the developers to package the libs linked against in the PND, just using them as backup (aka appending the lib/ folder in the LD_PRELOAD var)? Or would it make sense to create a package ourselves with the most common "old libs" while having the symlinks point to the latest ones. This step would increase the size of the firmware and I don't know by how much (and for which libs this would really make sense, the major examples with compatibility probs that come to my mind are boost and libjpeg).
Cheers, Nils Kneuper aka Ivanovic
On Thu, 24 Feb 2011, Nils Kneuper wrote:
# Codec pack: # - -------------------------- # Another thing that should be considered for the next firmware series is how to # act in regards to the codec pack with mplayer and such. JDGBOLT is currently # working on "panplayer" which should offer some good generic support for a lot of # stuff. The question together with this are: # 1) Does it make sense to offer a codec pack once a decent "complete bundle" for # the mplayer is available? # 2) If that is the case, are there easier ways than manually gathering a list of # packages and "brute force" installing them using opkg? Is there someone # available with the knowledge and time to do this?
A codec pack of some sort is still needed; some may just want mp3 support, not movies; and putting them all into every pnd is fine, but given their frequency of use, might be better to have packs to install them? (ie: a la timidity).
Until such time we sort out how to do pnd-dependancies well (if at all.)
Firmware repo: -------------------------------
Something else to consider here is .. we have a semi-working repo, so that pandoras can point to and do opkg update and suck down the most recent updates; are we going to try to formalize this, or rely on hotfixes?
I mention it due to -- codec pack; could have its own repo as well; folks could set to sync up to firmware repo, and to codec repo, and do automated updates to both.
# Backwards compatibility: # - -------------------------- # The new angstrom version will change several libs that are often linked against. # The most prominent examples are probably libjpeg.so.6 (will be libjpeg.so.8 # after the upgrade) and boost (they got the version number in the mainly used # name, too). The question now is: Do we accept it that a major update of the OS # potentially breaks many existing PNDs? Should we advise the developers to # package the libs linked against in the PND, just using them as backup (aka # appending the lib/ folder in the LD_PRELOAD var)? Or would it make sense to # create a package ourselves with the most common "old libs" while having the # symlinks point to the latest ones. This step would increase the size of the # firmware and I don't know by how much (and for which libs this would really make # sense, the major examples with compatibility probs that come to my mind are # boost and libjpeg).
Personally, as long as NAND has space, I'm a fan of bundling deprecated libs in; ie: as long as we can symlink safely (compatible binary interface) we're cool; when a symlink or auto-versioning will not do, we include newer and older versions.
It is never cool to break 500 applications when switching to a new OS version...
Ugly hack: pnd_run.sh could do an ldd against contained executables, and when known libs are now missing, it could notify user to pull down some pack of libs to make things work. Lame as hell though.
jeff
-- If everyone would put barbecue sauce on their food, there would be no war.
On Wed, 23 Feb 2011, John Willis wrote:
# Once we have got things stable enough to start dishing out images I'll also # move over all the scant documentation on installing them from the old # private wiki to the public ones. Give me a week or so before images start to # trickle out from me, I'll keep pushing .next to GIT in the meantime as I go # if others want to play along and get stuck into build/testing/patching/being # awesome (I rebase .next so remember that if you're tracking it).
Without time to comment on anythign specific, I agree in general :)
I Wanted to note .. my DSL ISP who happens to (for a little while longer) host codejedi.com had some meltdowns, and my old wiki got into hot water; it still works, but has some speed issues due to goofy PHPisms and whatnot.
That all said, I have access to all the .txt files that made up the wiki content, so nothing was lost; at worst case I can just put up the wiki files, and you can paste bits in as you see fit.. or just wholesale use them as-is with the new wiki perhaps.
jeff
-- If everyone would put barbecue sauce on their food, there would be no war.
Kernel:
The kernel will always come from the git.openpandora.org kernel tree and it is up to Grazvydas to manage how things trickle up and down from mainline.
For now use the GIT tip of the stable 2.6.27 branch looking to move to the 2.6.37 kernel ASAP.
Start the move to 2.6.37 by offering images with kernels and modules for both kernels included if I can figure a way to cleanly do it. I also need to patch over AUFS2 into the kernel, I will do this the same way as I did it for .27 as a patch that sits with the recipe in the metadata tree.
That OE patching is not a good thing, it confuses the hell out people. People just assume they can clone pandora-kernel.git and build their custom kernel similar way they can do with kernel.org (and quite rightfully so). Only minority have full OE setups and there were plenty of "I'm building my kernel, where is squashfs?" type of questions on the boards.
So from next release everything needed will be in pandora-kernel.git and defconfig will be in perfect sync with OE, so that a toolchain of choice + a clone of pandora-kernel.git is all that's needed to build full-featured custom kernel.
It looks like AUFS2 is the only thing that we need and is not in mainline, right? (squashfs is in mainline for a while now). I can start looking at merging this soon.
Grazvydas, I would appreciate your take on this being a good idea or not? We could stay with .27 (or .35) but it seems sensible to move to a very recent kernel and I would love to be able to make use of it in userspace.
.35 was more of a test (it's not even based on stable kernel, it's based on -rc3) and should not be used. I think we should start from .37 directly with no migration from .27, as the only thing really missing there is BT. There are a few regressions/breakages that I know of already (OSS emulation problems, NAND errors and performace regressions, omapfb X driver not liking default 24bpp mode; should be fixable with fbset in .xinitrc) but that's no reason to start with .27 and we'd better give .37 testing ASAP so it becomes better at the end.
BTW I'll probably be rebasing pandora-37 branch on 2.6.37.1 this week.
Am Thu, 24 Feb 2011 14:19:25 +0200 hat Grazvydas Ignotas notasas@gmail.com geschrieben:
The less custom patching, the better I'd say. If we stay close to a stock OE and a stock kernel, updating to newer versions will make things easier. Would it be possible to automatically update our OE to be in sync with the latest official OE tree and throw our overlays tree over there? If any recent OE changes break something in our overlay, we should fix it there. That would make things easier in the future, don't you think?
As usual, I'm ready to test anything you throw at me:
notaz: I guess that new testing kernel would also work with our current OS? If we start to test a new kernel AND a new OS image, it might sometimes be hard to tell if bugs derive from the kernel or the .next image.
If I could test the kernel on our current image as well, it should be easier to tell.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 24.02.2011 13:48, schrieb Michael Mrozek:
As usual, I'm ready to test anything you throw at me:
notaz: I guess that new testing kernel would also work with our current OS? If we start to test a new kernel AND a new OS image, it might sometimes be hard to tell if bugs derive from the kernel or the .next image.
If I could test the kernel on our current image as well, it should be easier to tell.
The problem of the kernel/OS image is that there are some userspace programs that tightly interact with the kernel. Basically resulting in a "too old" version of the kernel and a "too new" version of udev possibly being problematic. The same is of course also true when exchanging those two for each other. There are possibly other programs that also rely on features only available in more recent kernel versions. On the other hand a "too new" kernel might have removed some (by now deprecated) interfaces, that older programs still rely on. One rather important thing that was deprecated is the proc interface (or was it sys?),
So combining the "rather old" (more than 2 years!) kernel .27 with the latest angstrom might lead to some problems, as will some problems that dropping the latest kernel into the current firmware creates only be fixable by more recent userspace programs, which I expect to possibly become a mess rather soonish...
Cheers, Nils Kneuper aka Ivanovic
Hi,
...on Thu, Feb 24, 2011 at 03:09:40PM +0100, Nils Kneuper wrote:
available in more recent kernel versions. On the other hand a "too new" kernel might have removed some (by now deprecated) interfaces, that older programs still rely on. One rather important thing that was deprecated is the proc interface (or was it sys?),
I have some experience building recent-ish kernels to properly support old, old Debian systems. From that, I wouldn't expect any major problems as soon as it's clear which deprecated (but still existing or emulated) interfaces are actually required.
If there's any interest I could try to put some work into this (but I got my Pandora just a couple of days ago and probably need some time to get into the whole thing first)...
Alex.
Hi Alex,
available in more recent kernel versions. On the other hand a "too new"
kernel > might have removed some (by now deprecated) interfaces, that older programs > still rely on. One rather important thing that was deprecated is the proc > interface (or was it sys?),
I have some experience building recent-ish kernels to properly support
old,
old Debian systems. From that, I wouldn't expect any major problems as soon as it's clear which deprecated (but still existing or emulated)
interfaces
are actually required.
If there's any interest I could try to put some work into this (but I got
my
Pandora just a couple of days ago and probably need some time to get into the whole thing first)...
You're very welcome to work on that a little if you want but to be useful in the context of firmware testing you would really want to build it using OpenEmbedded so all the dependent out of tree stuff is included in the way the image expects (modules such as compat-wireless and the like).
That's the main reason I see it as a pain (and switching a single build environment to build 2 firmware images, one older and one newer is not really that efficient from a productivity point of view), if it was just a case of hacking about and building the kernel directly until it booted well it would not really be a problem (in fact 2.6.37 boots mostly fine on the current firmware) but if you want to use it in the context of repeatable regression testing it needs to fit with the build setup. By the time you have done all that work you have something that is not all that far from the .next build tree anyway ;).
Regards,
John
The less custom patching, the better I'd say. If we stay close to a stock OE and a stock kernel, updating to newer
versions
will make things easier. Would it be possible to automatically update our OE to be in sync with the latest official OE tree and throw our overlays tree over there? If any recent OE changes break something in our overlay, we should fix it there. That would make things easier in the future, don't you think?
We only tend to float around 50-60 patches over mainline OE at any given time, a lot of them are hacks that need refactoring to go into mainline or stuff that is already in OE's Patchwork and we are waiting to flow down, that really tends to keep us closely tied during the image process as we never drift that far for the parent tree.
The bigger drift comes from when we freeze a release to put a stake in the ground for stability (as we did with the Z release). We should try and look at how we can improve that in time, tracking OE/Angstrom stable branches may be an option. This is a minefield and we are a little at the mercy of how the Yocto/Poky/OE thing pans out so I would not like to change the status quo just yet.
Just as an aside the old AI TouchBook trees used to run at several hundred plus checkins over mainline making it very hard to merge and maintain their own patch set over OE, you think we have problems ;).
If I could test the kernel on our current image as well, it should be
easier to
tell.
Not going to happen, sorry, way too much of a giant pain in the arse ;-). In some ways it is a non-issue as a bug is a bug, just report it on the new images and tell people how to produce it, we can soon track down the root cause. It's not been a problem before.
John
Hi Grazvydas,
Kernel:
That OE patching is not a good thing, it confuses the hell out people. People just assume they can clone pandora-kernel.git and build their custom kernel similar way they can do with kernel.org (and quite
rightfully
so). Only minority have full OE setups and there were plenty of "I'm building my kernel, where is squashfs?" type of questions on the boards.
So from next release everything needed will be in pandora-kernel.git and defconfig will be in perfect sync with OE, so that a toolchain of choice +
a
clone of pandora-kernel.git is all that's needed to build full-featured
custom
kernel.
It looks like AUFS2 is the only thing that we need and is not in mainline, right? (squashfs is in mainline for a while now). I can start looking at merging this soon.
That works for me wonderfully, I went to some length to get rid of most of the patches and AUFS2 was about all that was left, if you can deal with that I would appreciate it (it will also make mainline OE easier as we can just have a very simple kernel recipe without any hacking).
Grazvydas, I would appreciate your take on this being a good idea or not? We could stay with .27 (or .35) but it seems sensible to move to a very recent kernel and I would love to be able to make use of it in
userspace.
.35 was more of a test (it's not even based on stable kernel, it's based
on -
rc3) and should not be used. I think we should start from .37 directly with no migration from .27, as the only thing really missing
there
is BT. There are a few regressions/breakages that I know of already (OSS emulation problems, NAND errors and performace regressions, omapfb X driver not liking default 24bpp mode; should be fixable with fbset in .xinitrc) but that's no reason to start with .27 and we'd better give .37
testing
ASAP so it becomes better at the end.
BTW I'll probably be rebasing pandora-37 branch on 2.6.37.1 this week.
Okies, AUFS2 aside I'll move to 2.6.37 from the pandora-37 branch TIP and only go for that version if you think we are in good shape.
Less work for me then maintaining some munge :).
Thanks,
John