'zines and book recommendations (Linux, Gadget, Game related)

[pmprog]

my interest was also piqued recently by "The 8-bit Guy's" Commander X16

I was going to say I hadn't heard of this, but as I looked it up, YouTube indicated that I'd partly watched the first video; so I've had a bit more of a look, and it looks pretty interesting.

First I’d recommend just reading up on Kbuild and poking around in the kernel to see how the system is built

I've been wanting to go through Linux From Scratch (url), but I've not had chance yet. Would Kbuild be a better place to start?

Although, to be honest, I'm surprised that neither of these start with the boot process, which to me would be the real starting point, though I guess that's not specific to Linux.

 

[ToastBucket]

LFS is incredibly informative, but it’s quite a bit off the bat. It’s very good, but definitely doesn’t teach you about Linux in an embedded environment such as the Pyra. Most importantly, you won’t learn about the boot process of an embedded Linux system. While every SOC behaves a little differently, they are all quite similar in how they boot.

When a SOC starts up, the first code to execute is the bootROM. The bootROM is also referred to as the primary bootloader. It is typically hardcoded into the silicon and the behavior can be determined from the SOC reference manual. The bootROMs function is to locate, load and execute a secondary bootloader which configures the system. Often this behavior is somewhat configurable through boot configuration pins or fuses (you either burn into one-time-write registers or the device reads pins at boot).

In the case of the (OMAP5) Pyra, there are three stages of bootloaders. The primary stage is configured to first check the left SD card slot, then the eMMC, for a bootable SPL. If it finds one, it loads this image into on-chip RAM and begins executing. This phase is considered the secondary bootloader and is crucial for this architecture. This is loaded into preconfigured RAM which resides on the chip. This stage configures basic necessities of the platform (power, DRAM, serial) before loading the third stage bootloader (U-Boot) into DRAM and jumping to this location. Once the bare necessities have been configured by the second stage bootloader, the third stage takes care of the rest and handles properly starting the Linux kernel.

Resources teaching about Linux on x86/amd platforms often won’t teach you much about the boot process because it is typically handled out of the box by standard bootloaders such as GRUB. The BIOS or UEFI handles most of the hardware interaction on this sort of system, whereas on an embedded system the boot process requires a bit more handholding.

I think the Pyra is a wonderful platform to learn how all this works since it is all out in the open. To learn more about how the Pyra boots, check out my (in progress but functional) version of the Pyra U-Boot below. The best place to start here is reading board/dragonbox/omap5_pyra/board.c and include/configs/omap5_pyra.h. And the best way to learn about the boot process is to modify and play with the bootloader

https://dev.pyra-handheld.com/kernel/pyra-uboot

 

[ClockworkCoder]

I love what you're doing @ToastBucket, and consider my interest well and truly piqued. Just started to look in to KBuild, and bought "Linux Kernel Development" (I like books), and looking forward to start hacking

 

[ToastBucket]

I've been wanting to go through Linux From Scratch (url), but I've not had chance yet. Would Kbuild be a better place to start?

Kbuild is simply the build system for the kernel. There are four (sometimes five) major components to a Linux system. The first is a bootloader (described above). The second is (of course) the kernel, which gets loaded into RAM and handles scheduling, contains drivers, handles all the meat and potatoes. The third (in an embedded system) is a device tree. This is a special chunk of data which informs the kernel of the various bits of hardware, which drivers they require, and how to configure them. The fourth is the rootfs which contains the tools and file system required for a Linux OS to run properly. The rootfs is sometimes considered the OS since it contains everything that makes a particular flavor what it is (specific tools, configurations, themes, etc). The fifth, which is not always necessary, is the initramfs. This is a temporary, very minimal, filesystem with just the tools necessary to configure and mount the full filesystem. PyraOS currently ignores this (using my bootloader or the other ones I’ve seen at least).

With regards to Kbuild, this simply tells the build system which drivers to build into the kernel. The device tree is then parsed at boot time and drivers are linked with nodes in the device tree. Drivers need to be properly enabled in the kernel and configured in the device tree to function.
[doublepost=1569311069,1569310728][/doublepost]

I love what you're doing @ToastBucket, and consider my interest well and truly piqued. Just started to look in to KBuild, and bought "Linux Kernel Development" (I like books), and looking forward to start hacking

I’m glad I truly believe this system is an ideal place to learn linux at the low level. The Raspberry Pi was supposed to be that, but business got in the way and the low level implementations are obfuscated and impossible to decipher. Now it’s just good for writing a python script that will turn lights on and off. The Pyra is a true Linux learning platform.

 

[pmprog]

Thanks for your detailed responses, it's a great read; and hopefully I'll get into actually digging around with code such as the boot loader and kernel

 

[levi]

The fourth is the rootfs which contains the tools and file system required for a Linux OS to run properly. The rootfs is sometimes considered the OS since it contains everything that makes a particular flavor what it is (specific tools, configurations, themes, etc). The fifth, which is not always necessary, is the initramfs. This is a temporary, very minimal, filesystem with just the tools necessary to configure and mount the full filesystem. PyraOS currently ignores this (using my bootloader or the other ones I’ve seen at least).

The initramfs in my understanding doesn't let you mount the fs, it lets you load the kernel. It basically contains the kernel binary and all of the kernel modules and a few tools perhaps. In as far as I've watched my machine come up, the initramfs is loaded before it mounts all of my partitions.

I’m glad I truly believe this system is an ideal place to learn linux at the low level. The Raspberry Pi was supposed to be that, but business got in the way and the low level implementations are obfuscated and impossible to decipher. Now it’s just good for writing a python script that will turn lights on and off. The Pyra is a true Linux learning platform.

IIRC the boot system for the pandora was a bit of a mystery to me at least to begin with. I think someone explained it as four steps, the first two baked into the silicon, but don't quote me on that. Something like initial bootloader, then a slightly more complicated program which boots the boot.scr (which can also be on the SD card) which loads the boot.txt which loads the kernel.

 

[ToastBucket]

The initramfs in my understanding doesn't let you mount the fs, it lets you load the kernel. It basically contains the kernel binary and all of the kernel modules and a few tools perhaps. In as far as I've watched my machine come up, the initramfs is loaded before it mounts all of my partitions

The kernel is loaded by the final stage bootloader. The kernel exists in RAM. The initramfs is a simple filesystem which the lives in RAM as well which the kernel mounts. It’s primary function is to assist the kernel in mounting the full filesystem. In some cases, it is not necessary. For example, Pyra does not use it.

The Pyras final stage bootloader has tools for reading MMC devices. After setting up the hardware, it searches for a kernel, initramfs, and device tree (filenames specifies by boot.txt with my configuration) within a particular partition on the MMC/SD. If it finds them, they get loaded into the appropriate places in RAM. The kernel header is then parsed, necessary information such as device tree location and cmdline arguments are placed into RAM where the kernel knows to look, and execution jumps to the kernel.

 

[levi]

Yes, as far as I know it should be possible to run almost any linux without an initramfs. The exception I can think of is when your /etc or /usr is encrypted, since you'll need the initramfs to instantiate a functional system to be able to mount it with the right decryption options (and ask you for the password, that kind of thing).

As far as I understand it building an initramfs is designed mainly to speed up booting, at the cost of taking a minute or so to rebuild both it's mainstream and it's fallback variants every time my system updates a handful of packages.

 

[Letalis Sonus]

The exception I can think of is when your /etc or /usr is encrypted, since you'll need the initramfs to instantiate a functional system to be able to mount it with the right decryption options (and ask you for the password, that kind of thing).

Just link everything you need into the kernel statically. Or just link the initramfs itself into the kernel. Seriously, only the sky is the limit.

As far as I understand it building an initramfs is designed mainly to speed up booting, at the cost of taking a minute or so to rebuild both it's mainstream and it's fallback variants every time my system updates a handful of packages.

Modularity, not speed. Once the bootloader passes control to the kernel, all you've got is what the bootloader loaded into RAM before this point. If e.g. the driver for the SATA controller or file system that you need for the root file system is not part of that, you're screwed. It's easy to create a custom cover-it-all kernel for a certain hardware, but when you can't be sure what exactly your kernel is supposed to run on you'd simply waste a lot of RAM - or run into conflicting modules breaking things.