Contributing to PyraOS

[ToastBucket]

Hey guys! I just got involved helping develop PyraOS. My main goal is to unify and stabilize development of the OS and create a build system which can reliably generate images cross platform. To start, I have created an official fork of the kernel which I will be maintaining. It lives inside the kernel group on Pyra's official git repo (link below). In the future, expect anything within this group to be officially maintained. Also expect this post to be updated as things progress.

For now, I am looking for anyone with hardware to help me test. If you have Pyra development hardware and want to help me out, go ahead and pull a merge request (after reading the comments to see the status ) and load it up. Please help me!

In terms of contributing, the repo rules are a little strict to keep things clean. No one is allowed to push directly to pyra or master (upstream). If you have a feature or fix you wish to push, create a feature or fix branch titled feature/<my-feature> or fix/<my-fix> respectively. Push your branch and create a merge request. Only a maintainer can actually merge in changes. At the moment, that will be me, aTc, or ED. Also remember, the repo is configured for fast-forward merges only, so make sure your merge request is rebased on top of the up-to-date pyra branch.

Gitlab kernel group: https://dev.pyra-handheld.com/kernel

UPDATE 8/4/19:
I have built a docker based build system for building the Pyra kernel, U-Boot, and root filesystem. The build system will automatically download the most recent sources and tools. The best starting point if you are interested in starting to play with PyraOS is to check out the imagebuild group (link below) and read the pyra-docker repository’s README. I will be updating this in the coming weeks to establish a stable environment for building the OS as well as applications, Debian packages, and DBPs, so keep an eye out for updates to the system. I’ll post about significant updates here.

For now, the kernel and bootloader the build system utilizes are my forks (as no patches are currently required). However, I will update this once all the details have been worked out (read the thread for more information on this situation).

Please feel free to contribute to any of the repos I’ve linked to here. I’ll add more explicit instructions in the README soon, but just push a feature or fix branch (rebased onto the default branch) and create a merge request. The modifications will be tested and discussed before being merged so check in with the discussions on your merge request

Gitlab imagebuild group: https://dev.pyra-handheld.com/imagebuild

UPDATE 8/14/19:
The build system now supports building image files which should be ready to flash directly onto SD cards. I am unable to test this, so there may be some issues. If you are able to test, please report any issues you run into in the issue tracker of the pyra-docker repository. There is a README in this repository with instructions on how to use the build system.

Warning: In order to build image files, the system currently relies the use of loopback devices. In order to utilize these within the docker container, it is provided privileged access to the host machine so use this at your own risk. Just read the build script and make sure you’re ok with everything it is doing before using it I am working on a better way to handle this, but for now this is the way it works.

UPDATE 8/29/19:
Building images is now safer. The container is still granted privileged access to the host (via --privileged when run) but is not granted full access to /dev. Images are also built as a boot.img, containing the kernel and device tree, and the rootfs.img, containing the root filesystem. This is in anticipation of flashing the eMMC and also makes it easier to flash to custom partition maps.

 

[EvilDragon]

Thanks a lot for your help. I'll try to get a hardware to you asap.

BTW: Most of the devs who have a Pyra can be reached through the kernel mailing list.

 

[ToastBucket]

Thanks ED!

 

[rSl]

awesome! thanks @ToastBucket !!!

 

[hns]

Hey guys! I just got involved helping develop PyraOS. My main goal is to unify and stabilize development of the OS and create a build system which can reliably generate images cross platform. To start, I have created an official fork of the kernel which I will be maintaining.

Hi, good news that you want to help.
But bad news that you start with a fork.
Let me ask: why? What is wrong with the current kernel and repo and LetuxOS kernel development process? I did not find a rationale for this.
As the maintainer of LetuxOS kernel I am very open to contributions, but there weren't many contributors recently. So having a fork before even trying to contribute to LetuxOS and see if it does not work well, does not seem to solve problems and fix the known issues. IMHO it only creates new ones (like breaking cooperation and sharing code with PyraPhone).
And how do you want to handle upstreaming of the missing components? My concern is that it may happen that you fix something in your repo, post it upstream and then after some months we have the problem of having to merge incompatible things back into the Letux kernel tree. Doesn't look helpful unless it is pre-coordinated.
Finally, I am curious what "generate images cross platform" means. What are the other platforms you have in mind?

 

[EvilDragon]

Well, I can see a few reasons why to do this.

• As far as I know, you're always moving to the latest kernel, which is nice, as it might fix some bugs. However, as far as I know, they also break things. For release, we need one kernel that's FULLY working, without any issues. It won't help if we have 5 different kernels versions where each one has different bugs. So getting ONE to work properly first and then either backport or work on fixing the bugs on a more recent one seems like a decent approach to me.
  
  
• The goal of Letux is (if I'm correct) to support a lot of different devices at once. That's a great idea - but it makes the whole thing a lot more complex, can easily lead to more bugs if you're not careful and it's a bit confusing to look at. Honestly, while I'm not a developer, I could find my way around some kernels when I take a look at them. But each time I take a look at the Letux developer site... I wasn't even able to find out what parts of the code are needed for the Pyra kernel and which not.
  That's not our goal. Our goal is that we have a clean project for the Pyra OS with the kernel, our additional debian packages, etc. without having to try to find out what code is for the Pyra and which one is not.
  I don't think we have the manpower to work on all these devices, test and fix them. For us, only the Pyra is relevant.
  
  
• I can imagine speed is also a good reason. Instead of providing you patches and wait until they are included, he can push them himself and aTc can compile and test the fixes a few seconds later right away. That did work fine with a few patches yesterday already
  
  
• We don't know if the fixes we apply to our kernel / device tree are what you imagine. For example, if a patch improves something on the Pyra but would break things on any other of your supported systems, it's fine for us. Also, we might remove hardware from the device tree (for example, to prevent that Linux finds two batteries in the Pyra) that you might want to keep.

So basically, our goal is to get a kernel and rootfs up and running for release, with as little bugs as possible, with as little effort as possible and as fast and efficient as possible.
It might break stuff for your other devices, so I don't think it would be a good idea working directly on the Letux stuff.

Our git is freely available, so if there are fixes you agree with, you can simply pull them over into the Letux kernel as well

These are just some reasons I can think ToastBucket created a fork. I'm no developer and don't have a very deep knowledge of that, but talking to different developers within the last years on IRC, a lot of them think Letux is very complex and you need to first dive into it to understand how it fully works before you can even provide simple patches.
That's the nature of the project, I guess, supporting so many devices can't be easy. But it doesn't move us forward if devs have little time. We need to keep our stuff as simple as possible.

 

[ToastBucket]

Thanks for the summary ED. I agree with all of those points.

I do not want to fully separate from LetuxOS. That doesn’t benefit either party. PyraOS misses out on modifications by you, and you miss out on modifications by the community. However, due to the points ED made, it does make sense to have some separation to prevent our modifications from breaking Letux and vice versa.

I am currently tracking LetuxOS master branch to ensure our master stays in sync with yours. Our main development branch is pyra. You can track this branch if you want to stay up to date with the most recent fixes which have been made for the Pyra hardware by the community. I will also inform you when the community fixes any of your open issues related to the Pyra so that you can patch LetuxOS if you want. For example, I am currently working on the issue with the battery charger. You can track the progress from the link below. That issue description, is linked to the current merge request being used for debugging.

https://dev.pyra-handheld.com/kernel/pyra-kernel/issues/1

Please feel free to direct message me or hop on the IRC channel if you want to discuss methods on how we can keep in sync better.

Finally, I am curious what "generate images cross platform" means. What are the other platforms you have in mind?

The idea here is to create a build system for building the kernel, U-Boot, and rootfs and bundle it into a simple image for flashing. The kernel and U-Boot are easy enough to cross compile, however the rootfs is not so easy to build. Right now it needs to be built on target hardware. I am working on a unified build system which runs within a docker container, so any build machine with docker installed can simply build and run the docker container to easily build PyraOS images. This makes it much easier for the community to modify the OS, build images, and test/play with their modifications.

 

[Wally]

Wouldn't it have been a better idea to create a git branch for the Pyra on the letux git repo?

 

[EvilDragon]

Wouldn't it have been a better idea to create a git branch for the Pyra on the letux git repo?

Why, when we have our own git and issue tracker setupl?

 

[ToastBucket]

Wouldn't it have been a better idea to create a git branch for the Pyra on the letux git repo?

This does not work well when we do not have good control over the repository. We cannot easily create branches and set rules to ensure safe collaboration amongst the community. HNS would have to do all this, which adds difficulty on his end when all he is really interested in is the working commits/patches for the Pyra. This allows HNS to focus on Letux, while the community focuses on just the Pyra.

Since we’re forked from HNS’s repository, we can push and pull branches easily from each repository. Just set up an alternate remote on your clone.

 

[Wally]

Fair enough

P.S Love your avatar, shows your age

 

[ToastBucket]

P.S Love your avatar, shows your age

I am but a baby. I also lived in Santa Cruz, CA for the last 7 years, it just goes with the culture

 

[levi]

Well, I can see a few reasons why to do this.

• As far as I know, you're always moving to the latest kernel, which is nice, as it might fix some bugs. However, as far as I know, they also break things. For release, we need one kernel that's FULLY working, without any issues. It won't help if we have 5 different kernels versions where each one has different bugs. So getting ONE to work properly first and then either backport or work on fixing the bugs on a more recent one seems like a decent approach to me.

That certainly has it's benefit in the short term of getting a device to ship, but personally I appreciate hns's efforts to get stuff targetting the pyra submitted back to the linux mainline, meaning we should have less problem updating our devices in the future. I'd recommend trying to get the letux kernel into a state where any patches we put on our pyra one are included, and from there submitted to mainline. It would mean we as users were less dependent on notaz to patch a newer kernel to keep our devices patched. It would mean that unlike on our Pandoras we wouldn't be stuck on an old kernel that no longer is supported right now.

 

[hns]

Well, I can see a few reasons why to do this.

• As far as I know, you're always moving to the latest kernel, which is nice, as it might fix some bugs. However, as far as I know, they also break things. For release, we need one kernel that's FULLY working, without any issues. It won't help if we have 5 different kernels versions where each one has different bugs. So getting ONE to work properly first and then either backport or work on fixing the bugs on a more recent one seems like a decent approach to me.

There is the rolling latest kernel plus the stable kernels 4.14., 4.19, 5.1, 5.2 where new things are backported (if possible, necessary, reasonable) as you describe it.
Note that it is not always possible to back/forward port fixes because internal APIs may change significantly.
Such fundamental internal API changes is why we did abandon the idea to "stabilize one kernel release first and then port fixes on a more recent kernel" long ago. Focussing on one kernel releases saves some work initially but piles up to a too big task when really trying to migrate to a newer kernel (you would work 3 years to fix an old kernel and then assume you can migrate in 3 months).
Look at the Pandora. You are still stuck with an old kernel and nobody can or wants to apply all fixes to a modern kernel, e.g. 4.19. Therefore we learned (the hard way) that it is better to integrate new things on the latest kernels when they come up to get a good chance for upstreaming. And use the official stable releases for daily operation to participate in upstream security and bug fixes. This smears the API change work over time into digestable chunks.
But there is no problem of fixing a single bug on a stable kernel first (e.g.4.19.62) and then walk the extra mile to immediately port it to the latest mainline (e.g. 5.3-rc2).

• The goal of Letux is (if I'm correct) to support a lot of different devices at once. That's a great idea - but it makes the whole thing a lot more complex, can easily lead to more bugs if you're not careful and it's a bit confusing to look at. Honestly, while I'm not a developer, I could find my way around some kernels when I take a look at them. But each time I take a look at the Letux developer site... I wasn't even able to find out what parts of the code are needed for the Pyra kernel and which not.
  That's not our goal. Our goal is that we have a clean project for the Pyra OS with the kernel, our additional debian packages, etc. without having to try to find out what code is for the Pyra and which one is not.
  I don't think we have the manpower to work on all these devices, test and fix them. For us, only the Pyra is relevant.

Simple answer: there is a single ARM kernel (well an ARM-LPAE variant) for all devices, including the Pyra. So all code (and not some part you have to search for) is for the Pyra. Similar to taking a distribution kernel e.g. from Debian.
If you want the latest stable kernel, take the master branch (currently 4.19.62). It works (besides known bugs) and is regularily tested. I think you already do.
And if you don't find something on the developer site - ask and it can be improved. Without user feedback everyone else thinks it is ok.

• I can imagine speed is also a good reason. Instead of providing you patches and wait until they are included, he can push them himself and aTc can compile and test the fixes a few seconds later right away. That did work fine with a few patches yesterday already

There is no problem to grant push access to the LetuxOS repository. It just does not happen if nobody asks for.
BTW: someone pushing patches should have compiled and tested them locally before. If untested patches have to be pushed and compiled by someone else, the workflow seems to be broken.

• We don't know if the fixes we apply to our kernel / device tree are what you imagine. For example, if a patch improves something on the Pyra but would break things on any other of your supported systems, it's fine for us. Also, we might remove hardware from the device tree (for example, to prevent that Linux finds two batteries in the Pyra) that you might want to keep.

It is a matter of communication. I can understand that you want to avoid discussions and stir your own soup
But such discussion may also help to get better solutions...
For example it could be that I can even be convinced to remove the second battery node, for example. It just has to be tried. If you assume in advance that I never ever would do that, then it does not work of course.
This type of discussion is exactly the same as happens when upstreaming to kernel.org. You present an idea to others with different mindset and they may reject or have good hints - at least there is some discussion with open outcome.
You can of course decide to avoid such discussions, like kernel developers in China do for Android devices. Or TI is doing with their board support kernels.
The result is that nothing is compatible with nothing and kernel trees diverge more and more making it impossible to cherry-pick patches or make mutual use of recommendations, tricks, hints, workarounds.

So basically, our goal is to get a kernel and rootfs up and running for release, with as little bugs as possible, with as little effort as possible and as fast and efficient as possible.
It might break stuff for your other devices, so I don't think it would be a good idea working directly on the Letux stuff.

If you break other devices there is a fundamental code flaw since Linux is quite modular...
What you can and should of course do is to have a separate defconfig to remove drivers not needed (like AESS or OMAP3 or i-MX6 support). But that is a single file in the whole tree which really needs to differ.

Our git is freely available, so if there are fixes you agree with, you can simply pull them over into the Letux kernel as well

Well, how can I understand what they are doing and why they are done the way they are done without discussion? If we discuss like on LKML, we can equally well share a single repo.

These are just some reasons I can think ToastBucket created a fork. I'm no developer and don't have a very deep knowledge of that, but talking to different developers within the last years on IRC, a lot of them think Letux is very complex and you need to first dive into it to understand how it fully works before you can even provide simple patches.

Well, that is a general aspect of Linux and not specific to Letux. It is complex.
For me, a solution would be to ask those who know more. To learn.

That's the nature of the project, I guess, supporting so many devices can't be easy. But it doesn't move us forward if devs have little time. We need to keep our stuff as simple as possible.

If that would be difficult, you should not use the Linux kernel at all, because it supports hundreds of devices you have never heard of
It is not as difficult or problematic as you guess. The majority of the devices supported by LetuxOS is either OMAP3, OMAP4, OMAP5 or i.MX6. This is brought to us by kernel.org and is well done.
The differences between the devices are in the individual device tree files. A group of files for each device.
And there are special device drivers for hardware that is only available in this or that device. Usually there is one feature branch for each such device and for deriving the latest kernel they are merged together by a script. This is copying the linux-next concept.
So LetuxOS just piles up potentially upstreamable driver patch sets and device tree files for the hardware that is not (yet) supported by mainline Linux. Pyra is one of these.

Thanks for the summary ED. I agree with all of those points.
I do not want to fully separate from LetuxOS. That doesn’t benefit either party.

Hm. I think we are one party, aren't we?

PyraOS misses out on modifications by you, and you miss out on modifications by the community.

I would be happy if there were more modifications by the community in the past. There simply weren't many despite asking for. The main blocking point is that there were not enough developers with hardware in their hands and enough time to really work on some topic. Not having a separate repository and potential discussion is IMHO not the blocking point.

However, due to the points ED made, it does make sense to have some separation to prevent our modifications from breaking Letux and vice versa.

We should definitively discuss if modifications break something...

You can track this branch if you want to stay up to date with the most recent fixes which have been made for the Pyra hardware by the community.

I am surprised that this is a question to you. Yes, it is the goal of LetuxOS to stay up-to date, even be ahead of everything. So basically all good patches should go to LetuxOS.
And don't forget that I am also a member of the community... So I should send patches to you to look if they arrive and then get them back to LetuxOS?

Maybe this is my main trouble with all this discussion: are we one community or are we two communities?
The LetuxOS project is also running and kept alive for the benefit of the Pyra community. And a potential PyraPhone or future new processor boards. If we imagine ourselves as two communities we already have lost.

I will also inform you when the community fixes any of your open issues related to the Pyra so that you can patch LetuxOS if you want.

Yes sure. As said I wonder where a hidden assumption is coming from that I would not want to have things fixed in LetuxOS as well. There may be different opinions of what a proper fix should be or if a feature should be implemented this or that way.

For example, I am currently working on the issue with the battery charger. You can track the progress from the link below. That issue description, is linked to the current merge request being used for debugging.

https://dev.pyra-handheld.com/kernel/pyra-kernel/issues/1

Please feel free to direct message me or hop on the IRC channel if you want to discuss methods on how we can keep in sync better.

My proposal is simple: if we do not duplicate things, then we do not even need to discuss how to keep in sync. You can equally well just write comments, questions, findings, about progress etc. to

http://projects.goldelico.com/p/gta04-kernel/issues/922/

If you need some missing access permissions, that can be fixed.

 

[bubblebee]

@ToastBucket, @ED It's really really great to have have a developer especially for pyra kernel development. But please don't divide the community in respect of kernel development. Under all circumstances, we must try to stay united. HNS is also a great supporter of the pyra eco system.

If LetuxOS does not pay enough attention to our PyraOS in any way then it has to be made clear to him and I am sure that he will meet our PyraOS in all respects so that our PyraOS can be developed under the existing LetuxOS-GIT repo.

 

[dTal]

I second bubblebee.

So basically, our goal is to get a kernel and rootfs up and running for release, with as little bugs as possible, with as little effort as possible and as fast and efficient as possible.
It might break stuff for your other devices, so I don't think it would be a good idea working directly on the Letux stuff.

Respectfully, I disagree here. The Pyra is an investment device, not a shitty one-shot Chinese Android handheld. "Make the software work so we can ship it, no matter how crappy the code is and no matter if we fork the kernel" should not be on the radar. I expect this thing to keep working with an upstream kernel 10 years from now. That means not accruing excessive technical debt now. If you're breaking stuff for other devices, you're doing it wrong anyway.

As for a separate repo, I don't really see how distancing the Pyra project from Letux helps anything at all, not when git branches are a thing and hns is clearly willing to give anyone permission who needs it. The only reason for a fork is if hns is mismanaging things somehow, and I see no evidence that this is the case. And honestly, it's somewhat dismaying to see an "official fork" created so suddenly and without any public discussion, by a new developer.

 

[EvilDragon]

That certainly has it's benefit in the short term of getting a device to ship, but personally I appreciate hns's efforts to get stuff targetting the pyra submitted back to the linux mainline, meaning we should have less problem updating our devices in the future. I'd recommend trying to get the letux kernel into a state where any patches we put on our pyra one are included, and from there submitted to mainline. It would mean we as users were less dependent on notaz to patch a newer kernel to keep our devices patched. It would mean that unlike on our Pandoras we wouldn't be stuck on an old kernel that no longer is supported right now.

I think you didn't understand that correctly: Our goal is NOT to hack up a kernel that can't go mainline. notaz also did his best to add changes to the Pandora kernel mainline as good as possible. It's just that some changes couldn't be submitted to mainline as some specific hacks that made the Pandora run as well as it did would not be mainline compatible - and more recent kernels didn't add anything that affected the Pandora, so it didn't make sense to upgrade anymore.
You can still compile a new kernel for the Pandora and use it. You might be missing some features that have not been accepted mainline, but it will mostly work (the Letux-Kernel also supports the Pandora, btw)

This is about being able to work fast and efficient and squish out the last bugs. Having your own git makes this easy: ToastBucket can easily push his changes, aTc or someone else with a Pyra prototype can test them. At the same time, they discuss the results in IRC and work on more patches until it works. Once it works, he can let hns know and as our git is based on the Letux kernel, the final patch can be pulled in with one git command.

When working on the Letux kernel, you'd need to submit the patches, wait until they are approven, etc. This takes time. Another way would be that ToastBucket submits the patches to aTc, he adds them, compiles a new kernel, tests, etc.
But that's also another step - and working with your own git makes things a LOT more easy and efficient.

GIT has been built so you can fork, work on it and then pull the final changes back into the original branch. This totally makes sense.

I second bubblebee.
Respectfully, I disagree here. The Pyra is an investment device, not a shitty one-shot Chinese Android handheld. "Make the software work so we can ship it, no matter how crappy the code is and no matter if we fork the kernel" should not be on the radar. I expect this thing to keep working with an upstream kernel 10 years from now. That means not accruing excessive technical debt now. If you're breaking stuff for other devices, you're doing it wrong anyway.

No one has said that's what we want. Where did you get this assumption?
ToastBucket has clearly stated in this thread that he intends to let hns know once a final fix has been submitted for a bug and he can pull that in.

And even the MAINLINE kernel breaks stuff for other devices, things like that CAN happen.
I've had a laptop that refused to boot after a kernel update because a driver had changed.

We don't want that to happen.

As for a separate repo, I don't really see how distancing the Pyra project from Letux helps anything at all, not when git branches are a thing and hns is clearly willing to give anyone permission who needs it. The only reason for a fork is if hns is mismanaging things somehow, and I see no evidence that this is the case. And honestly, it's somewhat dismaying to see an "official fork" created so suddenly and without any public discussion, by a new developer.

Again: We're talking about GIT here. Stuff changed in a fork can easily be pulled back to the original branch, and that's the plan (as ToastBucket stated as well here).
See the fork of a quick playground. A way to quickly work on things, test the patches and then send the final ones back without having to fear that you break anything.

We have bugs that haven't been fixed for years. One reason is that we don't really have that many devs and another reason is that some of them didn't have the time to work themselves into the full Letux setup and philosophy and then either made quick hacks locally or didn't do anything at all. With one place which ONLY has Pyra things, this is a lot faster to start to work with - and then we can still provide the working stuff back to Letux.

Here's a nice example:
Right now, the Pyra kernel is based on Letux kernel. But only based on, as we need a different config, so the config for compiling the kernel is setup elsewhere.
The board files come from Letux again, U-Boot (at the moment) from a different git.
Then we have a standard debian rootfs, but we inject our own packages into it, which come from different gits again and these have nothing to do with Letux.
And Letux has different kernels as well - so which one is the one for the Pyra? And hopefully, the latest one available doesn't have any new bugs...

Now... tell a developer who has no clue about that that he should create a bootable image of the Pyra OS.
It's complicated.

So ToastBuckets intention is (as he mentioned) to create ONE git where everything is setup nicely to create a bootable / flashable image.
The developer doesn't need to figure out what comes from where - he has that one place where everything needed can be found.

And if he provides proper patches, they go back to Letux. This sounds like a pretty good solution for everyone, if you ask me.

BTW: The Pyra kernel git has been created years ago, before Letux even supported the Pyra. It's not exactly suddenly and new, but we now have someone working on it.

 

[pmprog]

I think the problem that's arisen in this thread is communication.

ToastBucket's original post makes it sound like the official PyraOS kernel is no longer coming from Letux... which seperates Pyra from Letux... which I think what probably irked hns the most. I get the need for a Pyra test kernel that doesn't require formalised submission (clean formatting, clear descriptions etc.) that's just used to get things working before an offical clean patch is sent to Letux; but it didn't really come across that this is what the new repo was for.

 

[EvilDragon]

There is the rolling latest kernel plus the stable kernels 4.14., 4.19, 5.1, 5.2 where new things are backported (if possible, necessary, reasonable) as you describe it.
Note that it is not always possible to back/forward port fixes because internal APIs may change significantly.
Such fundamental internal API changes is why we did abandon the idea to "stabilize one kernel release first and then port fixes on a more recent kernel" long ago. Focussing on one kernel releases saves some work initially but piles up to a too big task when really trying to migrate to a newer kernel (you would work 3 years to fix an old kernel and then assume you can migrate in 3 months).
Look at the Pandora. You are still stuck with an old kernel and nobody can or wants to apply all fixes to a modern kernel, e.g. 4.19. Therefore we learned (the hard way) that it is better to integrate new things on the latest kernels when they come up to get a good chance for upstreaming. And use the official stable releases for daily operation to participate in upstream security and bug fixes. This smears the API change work over time into digestable chunks.
But there is no problem of fixing a single bug on a stable kernel first (e.g.4.19.62) and then walk the extra mile to immediately port it to the latest mainline (e.g. 5.3-rc2).

Newer kernels don't always add improvements. They can also make things worse, remove features (fbdev, for example), etc.
notaz took a quick look at newer kernels and backported the stuff that's important for the Pandora back into the Pandora kernel.

Once the kernels added nothing new for the Pandora anymore, he stopped doing this.

The Pandora kernel is highly optimized, which is also the reason it can run emulators an other things at such high speed compared to the slow CPU it has.
Most emulators I use on the Pandora actually run better than on Cortex A9 devices, because of these fixes.

Simple answer: there is a single ARM kernel (well an ARM-LPAE variant) for all devices, including the Pyra. So all code (and not some part you have to search for) is for the Pyra. Similar to taking a distribution kernel e.g. from Debian.
If you want the latest stable kernel, take the master branch (currently 4.19.62). It works (besides known bugs) and is regularily tested. I think you already do.
And if you don't find something on the developer site - ask and it can be improved. Without user feedback everyone else thinks it is ok.

Well, AFAIK the standard Letux ARM-LPAE-Variant doesn't support AUFS, for example.
I've mentioned a few times already that we need that for the dbp system.

So if someone compiles the Letux Pyra kernel, the whole DBP system wouldn't work. And he would need to find out why... which is just wasted time.
Also, we probably also disable stuff in the config that's needed for other devices but not for the Pyra.

If he simply uses the one from the Pyra Kernel git, he can be sure it's the one we deliver with the device and everything should work.

And you can still pull any bug fixes back.

There is no problem to grant push access to the LetuxOS repository. It just does not happen if nobody asks for.

Well, if you're okay with that, ToastBucket can also do that, if he likes to.
However, it seems that most devs didn't have the idea to ask yet. Is that somewhere written on your website who to ask if you want to push things yourself? How should a new dev know about that?

But still, we need to have one page where every interested developer can build his own image within a few minutes, without having to search the internet to find out what comes from where.

BTW: someone pushing patches should have compiled and tested them locally before. If untested patches have to be pushed and compiled by someone else, the workflow seems to be broken.

Yes, but that doesn't work if ToastBucket doesn't have a unit to test yet
So doing that with this git and IRC is way faster.

It is a matter of communication. I can understand that you want to avoid discussions and stir your own soup
But such discussion may also help to get better solutions...

I don't want to stir my own soup, my goal is to have device that works, is as mainline as possible and which has a build system where every interested developer can quickly start to work on.
Most of developer discussions from this community happen in IRC, and you don't want to go into IRC.
You'd rather use the mailing list, but to me it seems it's a lot slower there. On IRC, you discuss in realtime with other devs and are able to work on fixes a lot easier, as everyone can contribute, test and comment.
The problem with a mailing list is: You ask a question and then often have to wait for an answer for hours. Once the answer is there, you need to get back into your code and try to remember what you wanted (because you did other things inbetween), try again, maybe have more questions which lead to more time having to wait for an answer, etc.

This is GREAT for complex things (like the AESS audio thing), but overly complex for simple fixes. ToastBucket analyzed and already provided a few fixes for the battery monitor within half an hour, letting aTc test the patches in realtime on IRC.
The issue is not new - it's been a few times on the mailing list as well, the first time being years ago.

I also think IRC is easier to read because it's in linear time - unlike a mailing list, where someone can reply to a post from a week ago.

Both have their advantages, yes, but I also prefer IRC, as I think it's a lot more effective discussing and testing in realtime.

For example it could be that I can even be convinced to remove the second battery node, for example. It just has to be tried. If you assume in advance that I never ever would do that, then it does not work of course.
This type of discussion is exactly the same as happens when upstreaming to kernel.org. You present an idea to others with different mindset and they may reject or have good hints - at least there is some discussion with open outcome.

I am no dev. I reported the issue a few times already, years ago.
As well as that it stops charging at 8x%.
I can only report bugs, maybe with a basic idea what's wrong, but I am no developer and can therefore not give further details about possible fixes (removing the battery node would be such a fix, but I couldn't provide that info as I'm no developer).

No one has reacted to that, so it seems no one is interested in the mailing list at least.
Using IRC, a first few patches have been submitted within half an hour and ToastBucket already knows why it stops charging at 8x% and will provide a fix there as well.

You can of course decide to avoid such discussions, like kernel developers in China do for Android devices. Or TI is doing with their board support kernels.
The result is that nothing is compatible with nothing and kernel trees diverge more and more making it impossible to cherry-pick patches or make mutual use of recommendations, tricks, hints, workarounds.

There are developers out there who don't have the time for lengthy discussions.
While I agree that some discussions are important, it's also typically German do discuss WAY too much - and then often nothing happens.

Not EVERYTHING needs to be discussed until death. A few quick lines on IRC, a proper patch on the GIT that doesn't break anything but fixes a bug, done.
No discussion needed.
And it's not that discussions don't happen: It mostly happens on IRC for our community here.

For example, I've spent over two hours just contributing to this discussion already. And that's wasted time, in my opinion.
Why?
Because I don't see an issue having a fork which can be used as a playground to fix things faster, provide an easier start for devs and then pull back the working fixes back into Letux.

It's not split, it's an addition which helps everyone get into it.

If you break other devices there is a fundamental code flaw since Linux is quite modular...

Yes, but official kernels still break my old PCs sometimes.
Also, I think fbdev is deprecated, so any older systems which need to use it will run into issues.

What you can and should of course do is to have a separate defconfig to remove drivers not needed (like AESS or OMAP3 or i-MX6 support). But that is a single file in the whole tree which really needs to differ.

Yes, which is what we're doing. However, a new dev who wants to get started doesn't know about all these various sites he needs to get all the things together he needs for a working system.

A MAJOR advantage of a Linux system is:
If the installed system has some bugs, you can easily recreate the exact same system and then start working on bugs.
But if you first need to find out HOW to recreate the system from various sources and then maybe aren't even sure that it's the exact same one... well, that's annoying.

So one site that provides this possibility is important IMHO.

Well, how can I understand what they are doing and why they are done the way they are done without discussion? If we discuss like on LKML, we can equally well share a single repo.

If you use a GIT properly, you add comments to your patches. This should explain why they're done and how. Without discussion.
You can then either pull them as they are or change them the way you think they're better. Or ask the committer why he did it like that and maybe provide a better solution.

The "why" is not necessary anyways if it's a bug fix

Hm. I think we are one party, aren't we?

Yes and no
We share the same kernel, yes, but LetuxOS has totally different goals than the PyraOS

I would be happy if there were more modifications by the community in the past. There simply weren't many despite asking for. The main blocking point is that there were not enough developers with hardware in their hands and enough time to really work on some topic. Not having a separate repository and potential discussion is IMHO not the blocking point.

Being able to quickly work on a git and discuss / test in realtime with others on IRC is very helpful. And that doesn't seem to work otherwise at the moment.

We should definitively discuss if modifications break something...

Sure, but we don't have all the devices available. If a patch fixes a bug in the Pyra but breaks something in the GTA 04, how should we know?

I am surprised that this is a question to you. Yes, it is the goal of LetuxOS to stay up-to date, even be ahead of everything. So basically all good patches should go to LetuxOS.
And don't forget that I am also a member of the community... So I should send patches to you to look if they arrive and then get them back to LetuxOS?

No. As ToastBucket mentions, the GIT is a fork of LetuxOS. Every patch you provide in LetuxOS can easily be pulled into our kernel GIT as well. So you can continue to work as is.
And if ToastBucket has new fixes that have been proven to work on our kernel GIT, he can tell you and you can pull them in, if you find them acceptable.

The Pandora kernel, for example, did pull all OMAP updates automatically in each night. Worked without any issues.

Maybe this is my main trouble with all this discussion: are we one community or are we two communities?

I don't think the problem is how many communities we are.
One big issue I still see is:

You don't want to go to IRC but prefer mailing lists.
Most devs from this community (zmatt, aTc, ToastBucket, etc.) are on IRC and prefer that to mailing lists.

How is that supposed to work?

So a quick git where the devs mainly on IRC can discuss things and provide patches and quickly test them using our own git, then you can pull them in seems like a good idea.
If you complain that devs are not following the mailing list, they could also complain you're not reading the IRC logs

So yes, I think that's the biggest issue: Nothing happens because of different preferred platforms.

The LetuxOS project is also running and kept alive for the benefit of the Pyra community. And a potential PyraPhone or future new processor boards. If we imagine ourselves as two communities we already have lost.

Please let me know why a fork that stays close to the original GIT and provides patches back should split any community.
It's not like other forks which simply exist and move away from each other because the devs don't have the same idea.

My proposal is simple: if we do not duplicate things, then we do not even need to discuss how to keep in sync. You can equally well just write comments, questions, findings, about progress etc. to
http://projects.goldelico.com/p/gta04-kernel/issues/922/

... which is the issue tracker ToastBucket used, btw
[doublepost=1564576893,1564576797][/doublepost]

I think the problem that's arisen in this thread is communication.

ToastBucket's original post makes it sound like the official PyraOS kernel is no longer coming from Letux... which seperates Pyra from Letux... which I think what probably irked hns the most. I get the need for a Pyra test kernel that doesn't require formalised submission (clean formatting, clear descriptions etc.) that's just used to get things working before an offical clean patch is sent to Letux; but it didn't really come across that this is what the new repo was for.

That's not the only reason. The other reason is to have a build system in ONE place which makes it easy for developers to create their own fully working OS (kernel, bootloader, rootfs).
At the moment, it's a lot of look around and guess

But there are no plans to have a Pyra kernel not based on Letux - why should there?

 

[Haraldur]

you're not reading the IRC logs

Quick question, as someone who has never used IRC much, is there a quick and easy way to find and see these logs?