Contributing to PyraOS


hns said:
The major problem are closed source user-space libraries which I am not able to get working and interacting with X11 (or even without).
My kernel work (latest-pvr) would urgently need a working reference... I have one with 3.12 for the OMAP3 with a very old SGX version... But that doesn't help much.
So if someone has a recipe to make sgx work on omap5 (without tiler is not an issue now) on any 4.19 kernel, please share all details. That would allow to do a big step forward.
Click to expand...

Getting them to work with X is an issue, AFAIR. They only worked fullscreen for us as well.
For X, a wrapper was thought up as workaround where two framebuffers are used. One for X where the window has a transparency and one for 3D which scales down to exactly that area.

SGX only works fullscreen on the Pandora as well. But that wasn't a big deal, as 3D was only used for gaming on the Pandora which you only use in fullscreen anyways.
 
EvilDragon said:
Getting them to work with X is an issue, AFAIR. They only worked fullscreen for us as well.
For X, a wrapper was thought up as workaround where two framebuffers are used. One for X where the window has a transparency and one for 3D which scales down to exactly that area.

SGX only works fullscreen on the Pandora as well. But that wasn't a big deal, as 3D was only used for gaming on the Pandora which you only use in fullscreen anyways.
Click to expand...
Ok, that could explain some problems. What I would need as a first step is some recipe that ends up in any working setup either Pyra or OMAP5EVM (even without X).
 
  • Like
Reactions: rSl
aTc said:
I don't try them that often, but a few versions ago they still worked, although when i tried it a few days ago with a .60 kernel it wouldn't start.
Click to expand...
Thanks, this is a good starting point!
Unfortunately the kernel module link is blocked by some user access policy.
Others said that it did work with earlier 4.19 kernels. So we have to find one that did work for sure. Do you still have your kernel branch sitting around and can check?
 
hns said:
Do you still have your kernel branch sitting around and can check?
Click to expand...

I believe he tested it with this commit on the Pyra fork.

https://dev.pyra-handheld.com/kernel/pyra-kernel/commit/6f43a75025cdddc73ab9c1eadde0b7c1aeaaeb2b

This is just a fork of master at 4.19.60 with the necessary Pyra config patches applied. No significant investigation went into what went wrong here. I believe he just installed the drivers, saw they didn’t load properly, and we moved on to work on the battery charging issues.

Once I get hardware, I can help you out with the SGX drivers. We should probably discuss a strategy of how we can ensure that these drivers stay working on future kernel updates since they are so, let’s say picky :). In my opinion, the easiest solution is to freeze updates at a point where the drivers work, and only merge in kernel updates that don’t affect the 3D. When updates do affect the 3D, either drop them or work on sorting them out before doing another release. Automatic kernel updates don’t play well with these drivers it seems.
 
aTc said:
That's the one that didn't work :)

https://packages.pyra-handheld.com/images/extra/pvr/
I think all of those worked , but they're not very recent.

https://packages.pyra-handheld.com/images/extra/sgx.tar.xz
That one certainly works, but it's even older, with a 4.7 kernel.
Click to expand...
Thanks!

I'll go through them. Especially 4.19.4-letux-lpae-zmatt-pyra/ looks like a good candidate to start with.

https://dev.pyra-handheld.com/kernel/omap5-sgx-ddk-linux is also accessible now.

And the really interesting thing is that it is exactly the same PVR/SGX code from TI/IMG I am working in letux/latest-pvr for generalization and newer kernels. So it seems to indicate that my problems of getting it running is not in the driver code, but how it is built and installed and tried to run. As usual, it may be just one missing bit. A needle in a haystack. I guess I will have a lot of more questions how to install it.

Maybe the 4.7 package helps here even more since it is almost complete. There are kernel, kernel driver module, user-space libs, powervr.ini, rc.pvr and test tools (clipblittest) included in a single package. Only kmscube is not there.

Only some question marks are where the files should be installed so that the components can find each other...

So my strategy is:
0. try to produce a working SD card with the 4.7 kernel as initial reference setup
1. find (and document so that everyone can repeat it) a method to get it working with some 4.19 kernel
2. find out why my newer code doesn't work equally well and fix it
3. distributing over different kernel versions is then not a big issue any more since we can test, compare and bisect etc.
4. as soon as we have something running we can try to get things to mainline staging branch
5. then, kernel API changes will more likely be taken care by subsystem maintainers - and it will attract more developers if there is some working code
 
Last edited:
  • Like
Reactions: rSl
gpb said:
Credits should go to Robert C Nelson [2] where we picked up the idea. Of course kernel-ci integration should always be the far goal.
[..]
[2]https://www.digikey.com/eewiki/display/linuxonarm/BeagleBone+Black
Click to expand...

We definitely need something like [2] for the pyra in our wiki,
This will make it easy for every individual to get startet building/fixing things.

https://pyra-handheld.com/wiki/index.php?title=SoftwareHub is a good starting point, but I think the forum posts should be "converted" to proper wiki pages.
Also [2] gives you a quite more complete picture of what has to be done and includes everything, including bootloader... Which e.g. seems to be "complicated" on the pyra.


If the documentation is done we can also further think about building a CI to automatically compile this for e.g. tests on x86 or even on arm via cross-compiling.
Is there already some automation done to build "everything"? Is there documentation?
 
yeoldegrove said:
If the documentation is done we can also further think about building a CI to automatically compile this for e.g. tests on x86 or even on arm via cross-compiling.
Is there already some automation done to build "everything"? Is there documentation?
Click to expand...

This is what I am working towards. Right now there are some scripts, but they only work on certain machines. I’ve already started to put together a docker based build system which will be aimed at both a clean development environment and a CI test environment which we can link up with gitlab.

Documentation is another project of mine. However, I cannot document if the build system does not exist so the build system and everything actually has to be stabilized first. Before I can stabilize the build system, we need to figure out how we are going to handle the kernel and u-boot repositories. We’ve had a lot of discussion about it over the last few days but have yet to come to a conclusion.

Perhaps what I’ll do is make my WIP build system public. It seems people in the community have a bit more experience with this kind of thing than the kernel, so we could all contribute.
 
Last edited:
hns said:
So my strategy is:
0. try to produce a working SD card with the 4.7 kernel as initial reference setup
1. find (and document so that everyone can repeat it) a method to get it working with some 4.19 kernel
Click to expand...
I have played around to make a working installation, and I am close but not there...

The 4.7 setup did not work because the 4.7 kernel is too old and does not have the device trees for the newer hardware variants. Using the latest device tree does not boot.

So I tried the 4.19.4 versions.

Here are the (not yet working) Step-by-Step instructions for demonstrating PVR/SGX on Pyra (OMAP5):
Code: 
Running as root on a BeagleBoneBlack (with LetuxOS or Debian; Raspi should also work), working internet connection and an external USB SD card reader as /dev/sdb:

# ssh into BeagleBone and become root
# insert empty (or to be overwritten) SD card into reader
wget -O makesd "http://git.goldelico.com/?p=gta04-makesd.git;a=blob_plain;hb=HEAD;f=makesd" && chmod +x makesd 
DEV=/dev/sdb ./makesd pyra -r lxde -m http://download.goldelico.com/letux-kernel/letux-4.19.4/modules.tgz -f1 -k http://download.goldelico.com/letux-kernel/letux-4.19.4/uImage -d http://download.goldelico.com/letux-kernel/letux-4.19.4/device-trees.tbz 
mkdir -p /media/letux/rootfs
mount /dev/sdb2 /media/letux/rootfs
cd /media/letux/rootfs
KVERSION=$(cd lib/modules && ls -1)
wget -O lib/modules/$KVERSION/kernel/pvrsrvkm.ko https://packages.pyra-handheld.com/images/extra/pvr/4.19.4-letux-lpae-zmatt-pyra/pvrsrvkm.ko
depmod -b . $KVERSION
wget -O sgx.tar.xz https://packages.pyra-handheld.com/images/extra/sgx.tar.xz
tar xvJf sgx.tar.xz
cp -R sgx/pvr/etc sgx/pvr/lib sgx/pvr/bin sgx/pvr/include .
cd $HOME
umount /dev/sdb*
# extract SD card and insert into Pyra

# boot Pyra
# ssh or access console into Pyra (ssh root@192.168.0.202 through USB3 port)
ls -l /dev/pvrkm    # check that PVR was loaded
sgx_clipblit_test
It boots, execpt that modprobe pvrsrvkm fails with:
Code: 
[   56.270304] pvrsrvkm: version magic '4.19.4-letux-lpae-zmatt-pyra SMP preempt mod_unload ARMv7 p2v8 ' should be '4.19.4-letux+ SMP preempt mod_unload ARMv7 p2v8 '
Module pvrsrvkm failed to load. Retrying.
Running /sbin/depmod
[   56.591280] pvrsrvkm: version magic '4.19.4-letux-lpae-zmatt-pyra SMP preempt mod_unload ARMv7 p2v8 ' should be '4.19.4-letux+ SMP preempt mod_unload ARMv7 p2v8 '
Trying to force load:
Code: 
root@letux:~# modprobe -f pvrsrvkm
[  183.101188] pvrsrvkm: bad vermagic: kernel tainted.
[  183.106342] Disabling lock debugging due to kernel taint
[  183.112517] pvrsrvkm: loading out-of-tree module taints kernel.
[  183.120513] pvrsrvkm: Unknown symbol drm_legacy_mmap (err -2)
modprobe: ERROR: could not insert 'pvrsrvkm': Unknown symbol in module, or unknown parameter (see dmesg)
root@letux:~#

This means the 4.19.4-letux-lpae-zmatt-pyra kernel module is not compatible to the 4.19.4-letux kernel...
In other words: an archive of pvrsrvkm.ko variants isn't enough if the matching kernel image + other modules is missing.

So if someone knows where to get a working (binary) combination of kernel + modules, please let me know.

Alternatively I'll look into building a new pvrsrvkm.ko with the scripts provided - but it probably will also look for "drm_legacy_mmap".
Or it may better work to recompile the 4.19.4 kernel installed by makesd with DRM legacy enabled (CONFIG_DRM_VM seems to be disabled).
 
Yes the modules will need to be built against the specific kernel version.

hns said:
Or it may better work to recompile the 4.19.4 kernel installed by makesd with DRM legacy enabled (CONFIG_DRM_VM seems to be disabled).
Click to expand...

Yep this is one of the patches that aTc typically applies on top of the Letux kernel. Although calling it a patch I don’t think is particularly accurate since I don’t think there’s an actual patch file, just manual modifications prior to build. See the below commits on the test repo to see the inclusion of those patches.

https://dev.pyra-handheld.com/kernel/pyra-kernel/commit/18a0bdd126b8d967a011f6a3d7b25cc4c541e801

https://dev.pyra-handheld.com/kernel/pyra-kernel/commit/89d5c78d45ce28a2fbbda3773d9ec8bf81caada7

https://dev.pyra-handheld.com/kernel/pyra-kernel/commit/6f43a75025cdddc73ab9c1eadde0b7c1aeaaeb2b
 
The beaglebone green appears to be a variant of the black. They've put in two 'grove' connectors which seem to be based on i2C from the label names, but removed the HDMI socket and associated circuitry and the 5V barrel jack (so I've no idea how you power this thing, unless the 3.3V from the new connectors is enough). This is entirely irrelevant from this thread though, he just happens to be using a beaglebone black to prepare the sd card for the pyra.
 
Ok, I have recompiled 4.9.4-lpae with CONFIG_DRM_VM enabled and now I can successfully load the kernel module by this new procedure
Code: 
# ssh into BeagleBone and become root
# insert empty (or to be overwritten) SD card into reader
wget -O makesd "http://git.goldelico.com/?p=gta04-makesd.git;a=blob_plain;hb=HEAD;f=makesd" && chmod +x makesd
DEV=/dev/sdb ./makesd pyra -r lxde -m http://download.goldelico.com/letux-kernel/letux-4.19.4-pvr-lpae/modules.tgz -f1 -k http://download.goldelico.com/letux-kernel/letux-4.19.4-pvr-lpae/uImage -d http://download.goldelico.com/letux-kernel/letux-4.19.4-pvr-lpae/device-trees.tbz
mkdir -p /media/letux/rootfs
mount /dev/sdb2 /media/letux/rootfs
cd /media/letux/rootfs
KVERSION=$(cd lib/modules && ls -1)
wget -O lib/modules/$KVERSION/kernel/pvrsrvkm.ko https://packages.pyra-handheld.com/images/extra/pvr/4.19.4-letux-lpae-zmatt-pyra/pvrsrvkm.ko
depmod -b . $KVERSION
wget -O sgx.tar.xz https://packages.pyra-handheld.com/images/extra/sgx.tar.xz
tar xvJf sgx.tar.xz
cp -R sgx/pvr/etc sgx/pvr/lib .
cp -R sgx/pvr/bin sgx/pvr/include ./usr
sed -i.bak "s/modprobe -q pvrsrvkm/modprobe -q -f pvrsrvkm/g" etc/init.d/rc.pvr    # ignore small diff in version code
cd $HOME
umount /dev/sdb*
# extract SD card and insert into Pyra

# boot Pyra
# ssh or access console into Pyra (ssh root@192.168.0.202 through USB3 port)
/etc/init.d/rc.pvr start    # does not start automatically
cat /proc/pvr/*    # check that PVR was loaded successfully
sgx_clipblit_test    # run 3D demo
Unfortunately I am still stuck with user-space, similar to the problems I have with my own kernel module build:
Code: 
root@letux:~# /etc/init.d/rc.pvr start # does not start automatically
/usr/bin/pvrsrvctl: error while loading shared libraries: libgbm.so.2: cannot open shared object file: No such file or directory
root@letux:~#
After adding a symlink
Code: 
root@letux:~# ln -sf libgbm.so.1 /usr/lib/arm-linux-gnueabihf/libgbm.so.2
root@letux:~# /etc/init.d/rc.pvr start # does not start automatically
/usr/bin/pvrsrvctl: symbol lookup error: /lib/libsrv_um.so.1: undefined symbol: drmOpenRender
root@letux:~# cat /proc/pvr/* # check that PVR was loaded successfully
Registered nodes
Addr     Type     Class    Index Ref pvDev     Size Res
7690d545 ?7       3D          0   1  c2292d2b    0  f454d0b2
Command Queues
Queue    CmdPtr      Pid Command Size DevInd  DSC  SSC  #Data ...
Version SGX_DDK sgxddk MAIN@3699939 (release) omap_linux
System Version String: None
root@letux:~# sgx_clipblit_test # run 3D demo
sgx_clipblit_test: error while loading shared libraries: libgbm.so.2: cannot open shared object file: No such file or directory
root@letux:~#
If I remember correctly this is the same situation where I was stuck previously and noted that this may be because I am still using Debian Jessie here.
Yes: https://www.pyra-handheld.com/pipermail/kernel/2019-March/003173.html
So in summary this seems to confirm that we neither have a kernel nor a kernel driver problem, but user-space is not compatible to how the binaries for pvrsrvctl etc. were compiled.

BTW, this is where TI used to document how to install SGX: http://processors.wiki.ti.com/index.php/Graphics_SDK_Quick_installation_and_user_guide

Next step: modify the above description to use some LetuxOS Stretch image.
[doublepost=1564777081,1564776802][/doublepost]
ToastBucket said:
Yes the modules will need to be built against the specific kernel version.
https://dev.pyra-handheld.com/kernel/pyra-kernel/commit/6f43a75025cdddc73ab9c1eadde0b7c1aeaaeb2b
Click to expand...
Yes, this one adds the DRM legacy config. It will be interesting why I did not need it when compiling the pvr kernel module from within the Letux tree. This is at least one difference between zmatt's method and mine. It is too early to know if it is relevant or spurious.
 
Last edited:
  • Like
Reactions: rSl
You must log in or register to reply here.

Similar threads

aTc
  • Sticky
PyraOS Changelog
Replies
5
Views
7K
matzesu
matzesu
ToastBucket
Pyra Gitlab access
Replies
10
Views
4K
EvilDragon
EvilDragon
Silent-Hunter
Is the PyraOS going to have systemd-homed?
Replies
11
Views
6K
netcat
netcat
Linux-SWAT
Booting the Pyra : unifying the efforts.
Replies
14
Views
5K
aTc
aTc
sebt3
  • Sticky
Pyra pyra-builder: Docker based toolchain for pyra
Replies
4
Views
7K
sebt3
sebt3
Back
Top