Download Pyra Debian OS (WIP)

[KidPaddle]

Thanks for response, i will test "mate" - image today.

If have installed the root filesystem on a harddisk, setup a small sd card with MLO and u-boot, but where can i set device and partition for new root filesystem, change "/dev/mmcblk0p1" to "/dev/sda1"? The kernel is loaded and inside busybox the harddisk can be mounted.

Is there a command boot from harddisk from "U-boot" prompt?

Thomas

---

"pyra-debian-jessie-mate-rootfs.tgz" contains same error with "libmodule-build-perl".

@aTc: Can i test something for you?

Thomas

 

[aTc]

The way booting is currently set up (if you use the u-boot from install-pyra.tar.xz at least), is that it looks through all attached devices for boot/extlinux/extlinux.conf , which contains the details on which kernel to use and the kernel command line.
It also shows a menu on the serial console where you can select which entry to boot.
This file is currently beeing generated by some scripts in /etc/kernel/ , and takes its default values from /etc/kernel-pyra.conf.
The way I boot from hd, is by having this u-boot and mlo installed on the emmc. It then scans through all devices (sd card slot, usb , sata , internal emmc, dhcp , pxe ) until it finds one where it can load the boot/extlinux/extlinux.conf file from.

The current extlinux.conf generating script doesn't work properly though (on remove it sometimes removes the wrong kernels), and i'm halfway through setting up a more solid solution based on a more official script (which got removed from all distros ages ago, so there's probably something wrong with that too . I'll also push that stuff with the rest of the changes, probably later tonight. (I still need to clean up a few things)

And you can test whatever you want, just use the image and report whatever you don't like. (as long as it isn't about systemd or pulseaudio )

 

[stevenc99]

"pyra-debian-jessie-mate-rootfs.tgz" contains same error with "libmodule-build-perl".

This is a release-critical issue; it breaks installing various packages from Debian jessie, and security updates will fail to install too.

I see now that /pyra-config.sh *did* run in the rootfs, but most preinst scripts are not being run (except dash.preinst, which is run manually). This means lots of dpkg-divert rules are not applied, possibly causing the problem above. (Maybe after unpack, it is already too late for dpkg-divert to fix this...)

root@pyra:/# apt-get -y --reinstall install perl libmodule-build-perl
Reading package lists... Done
Building dependency tree       
Reading state information... Done
0 upgraded, 0 newly installed, 2 reinstalled, 0 to remove and 0 not upgraded.
8 not fully installed or removed.
Need to get 0 B/2317 kB of archives.
After this operation, 0 B of additional disk space will be used.
(Reading database ... 58375 files and directories currently installed.)
Preparing to unpack .../libmodule-build-perl_0.421000-2_all.deb ...
Unpacking libmodule-build-perl (0.421000-2) over (0.421000-2) ...
dpkg: error processing archive /var/cache/apt/archives/libmodule-build-perl_0.421000-2_all.deb (--unpack):
 trying to overwrite '/usr/bin/config_data', which is also in package perl 5.20.2-3+deb8u5
Preparing to unpack .../perl_5.20.2-3+deb8u5_armhf.deb ...
Unpacking perl (5.20.2-3+deb8u5) over (5.20.2-3+deb8u5) ...
dpkg: error processing archive /var/cache/apt/archives/perl_5.20.2-3+deb8u5_armhf.deb (--unpack):
 trying to overwrite '/usr/bin/config_data', which is also in package libmodule-build-perl 0.421000-2
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Processing triggers for man-db (2.7.0.2-5) ...
Errors were encountered while processing:
 /var/cache/apt/archives/libmodule-build-perl_0.421000-2_all.deb
 /var/cache/apt/archives/perl_5.20.2-3+deb8u5_armhf.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

I'm working on it, and should be able to publish my own rootfs soon if you'd like to test that.

 

[KidPaddle]

@stevenc99

I will test it, have bought some sd card, so let's play

@aTc
I created a new sd card, replace only mmcblk0 with sda1 in "boot/extlinux/extlinux.conf" and now it boots directly to harddisk, thanks for help.

Running first boot wizard four times on weekend, well done, works flawless.

Thomas

 

[aTc]

At least i can replicate the perl thing here now.. it requires slightly more work than just doing and apt-get update/upgrade (had to do the --reinstall with libmodule-build-perl).
I did see it once before, but that was months ago, and just thought it was something to do with the packages on the debian repo not beeing fully updated yet, since it didn't happen when i rebuilt the image a short while later.

The second stage installer of multistrap is horribly undocumented (the one example in their docs doesn't even work, and the one it runs when ignorenativearch isn't set in the config doesn't work properly either), so it's not that surprising that not everything works flawlessly.
It might also have to do with me using apt instead of dpkg to finish he configuration stage.
We could always just move to debootstrap. (The current setup scrip already takes part of it from debootstraps second stage scripts)


And about the first run wizard, that's more a temporary hack until someone comes up with a proper wizard.

 

[stevenc99]

@aTc I came up with a workaround for this bug: https://github.com/stevenc99/pyraos-build/commit/c0ba1f400574aa71d1ed7671ef467038428ee931

If we pre-populate /var/lib/dpkg/diversions with whatever would have been put there by pre/postinst scripts - it prevents future apt-get installs/upgrades from trying to overwrite any diverted files (which fixes the problem we saw with libmodule-build-perl, perl and others).

We *probably* should reinstall the affected packages on the target device after setting up /var/lib/dpkg/diversions. But network access is needed to re-download the .debs, because multistrap has already cleaned them at that point (and I don't like making the stage1 -> stage2 part require network access).

By the way, using either apt or dpkg in /pyra-config.sh makes no difference, neither one will run preinst scripts for already-unpacked packages.
[doublepost=1465767104,1465763309][/doublepost]Here's my attempt at making rootfs tarballs. If anyone is able to test that would be great, as I could only test them as a separately mounted chroot on the OMAP3 devboard I have:
http://jenkins.kfreebsd.eu/jenkins/view/pyra/job/pyra_rootfs/ws/export/

* stage1 is a half-configured image; on first boot /pyra-config.sh takes something like an hour to finish configuring

* stage2 is the result of me running stage1 on my own armv7l board; you probably want to use this one

I'm working to make these build reproducibly; the idea is that someone should be able to reproduce stage1 by running multistrap with SOURCE_DATE_EPOCH=1465759684, and should get a byte-for-byte identical tarball. stage2 would take much more work to ever make it reproducible, which is why I offer stage1, so you can produce stage2 yourself on your own device if you don't trust mine.

I've produced a diff to show what happens between stage1 and stage2. A lot of the 'binary file' differences are just symlinks added by the /etc/alternatives system.

Known bugs in these images are:
* SSH host keys should be cleaned from my stage2 image, so that everyone's device generates their own
* my stage2 contains a machineid, that probably should be cleaned too
* some logs and things can be cleaned
* file capabilities are not being set, for /usr/bin/netstat etc. to be able to work as non-privileged user
* APT repository for security.debian.org is not enabled yet, so some packages (like firefox) are out-of-date

I think I've fixed the file conflict issue on APT upgrades though, which is the main thing.

 

[KidPaddle]

I had quick check. Looks good, but there is no network configuration, i had to setup it manually. Added a static ip address to my eth0 interface, create resolv.conf with nameservers, and then, all works.

"apt-get update ; apt-get upgrade" has no packets to upgrades, xrdp / mc / joe installed without problems,

At moment, i found only one different in "apt-get update" process. In aTc rootfs many backports are included, in yours none.

The first run wizard works normally.

Something to check in detail?

Thomas

 

[stevenc99]

I had quick check. Looks good, but there is no network configuration, i had to setup it manually. Added a static ip address to my eth0 interface, create resolv.conf with nameservers, and then, all works.

Thanks very much!

Would automatic DHCP on eth0 be a sensible default for most users?

At moment, i found only one different in "apt-get update" process. In aTc rootfs many backports are included, in yours none.

It may just be that "apt-get update" information in my images is more recent and so doesn't need updating?

Or perhaps aTc added new packages to the images recently but hasn't updated git://git.openpandora.org/pyraos-build.git. When that happens, I'll update my own images to do the same. I'll also start building pyra-mate images at some point.

I think I fixed all the bugs I wanted to fix right now... I'll probably enable the security.debian.org APT repository, and then keep my images updated regularly, so that security patches are already applied at first boot.

 

[KidPaddle]

Would automatic DHCP on eth0 be a sensible default for most users?

This was default in both aTC rootfs, normally we have later wlan on board, but for this, i don't know.

It may just be that "apt-get update" information in my images is more recent and so doesn't need updating?

You will be right, but sorry,i have no idea, (This could be my default response for future)


I had done some test on /dev/sda (500GB sata disc WD5002ABYS)

command to see buffered and cached disk read speed: hdparm -Tt /dev/sda
Timing cached reads: 1638 MB in 2.00 seconds = 819.94 MB/sec
Timing buffered disk reads: 338 MB in 3.00 seconds = 112.61 MB/sec

Writting one big file, command: dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=dsync
1+0 Datensätze ein
1+0 Datensätze aus
1073741824 Bytes (1,1 GB) kopiert, 12,4442 s, 86,3 MB/s

Writting 1000 files with 512KB, command: dd if=/dev/zero of=/tmp/test2.img bs=512 count=1000 oflag=dsync
1000+0 Datensätze ein
1000+0 Datensätze aus
512000 Bytes (512 kB) kopiert, 32,8706 s, 15,6 kB/s

Ok, random writes not so fast..

Next, i install a compiler and compiled a SDL game, bumprace, and running inside xrdp session, slow but playable. My hdmi display was to small for it, only 800x480.

I have done this in both rootfs and i see no big difference,so what next?

Thomas
[doublepost=1465834739,1465832979][/doublepost]I had run some gles test, glxgears runs with 150 FPS on my little display, MegaGlest is running, but very slow.

Thomas

 

[TrashyMG]

Would automatic DHCP on eth0 be a sensible default for most users?

This was default in both aTC rootfs, normally we have later wlan on board, but for this, i don't know.

Yeah, from my knowledge playing with it in the past, It always has been the default in all the past rootfs images. Although this is on Ethernet only, the WLAN from my knowledge never worked on the OMAP5 EVM devboard.

 

[stevenc99]

Wow, that's mostly very fast. How quickly does it boot? What hardware are you testing on?

 

[KidPaddle]

TI OMAP5432 board, 2 GB ram, boots from 4GB SD card with MLO/u-boot, but loads kernel from sata harddisk. I connect an usb hub for mouse and bluetooth adapter, founds my keyboard, but couldn't connect it.

It needs several seconds for scan attached devices, has timeout counter of 3 seconds, but pressing power button to login prompt, around 35 seconds.


Here a picture of my dev board

 

[stevenc99]

I just uploaded pyra-debian-jessie-xfce-rootfs-1465843129-stage2.tgz
where I've enabled the security.debian.org APT repository, so it already has patches applied for:
* DSA 3591-1 - ImageMagick vulnerabilities
* DSA 3600-1 - iceweasel upgraded to firefox-esr 45.2.0

I decided not to bundle APT package lists in this one, to make the stage1-stage2 diff smaller, make stage2 easier to reproduce, and the stage2 tarball a bit smaller.

There are no other changes of interest yet.

 

[ible]

thanks for all you guys' and gals' hard work.

 

[stevenc99]

I should add that aTc did a fine job. A straightforward build system for the images, and a proper Debian repository and sources for all the added custom stuff.

The resulting image is very close to what an official Debian install would be. (Surprising how often vendors or derivative projects manage to screw this up, and then it's a nightmare to maintain).

 

[KidPaddle]

@stevenc99
Image work, but no packets from security.debian.org applied, after "apt-get update ; apt-get upgrade", have you done this already?

Great work.

@aTc
Great job and thanks for helping me.

Thomas

 

[stevenc99]

@stevenc99
Image work, but no packets from security.debian.org applied, after "apt-get update ; apt-get upgrade", have you done this already?

Yes, already installed! It should have the latest firefox-esr (45.2.0).

 

[KidPaddle]

Image aTc:

I tried to connect a bluetooth keyboard but this always fails, first with bluetoothctl, then with graphical manager on desktop. The device was visible, trusted, pair code was requested, and in bluetoothctl accepted, connected, but device couldn't added. I used two different bluetooth sticks and two different keyboards, changed my active usb hub, but nothing helps.

After searching, i changed in file /etc/bluetooth/input.conf

IdleTimeout=1200 (This is preventing keyboard lags, timeout was to low, so after 30 seconds, keyboard has to reconnect)

and

UserspaceHID=true

After restart i could add the keyboard with bluetoothctl and it is availabe on Desktop.

Thomas

After that, i tried to register a new keyboard with bluetooth graphical manager, the device is visible, a pin code for keyboard requested, but after typing on keyboard, i got message that connection is failed. With bluetoothctl as root on console it works and the new device is visible in graphical bluetooth manager. Maybe are user right problem?

Thomas

 

[aTc]

At least for me, bluetooth keyboards have always been a lot of trouble to get going.
I mostly tested the bluetooth stuff with various bluetooth speakers, and those work pretty much out of the box.
So at least that part of bluetooth works. It could be that input devices need more permissions, or that they can only be set up by root.
It doesn't help that the bluetooth stuff in debian is slightly outdated. Another possibility is missing bluetooth firmware, since i did have to manually install the firmware files for my bluetooth adapter to get it going properly.
(check dmesg, although for other adapters it will complain about missing firmware, but still work)

DHCP and any other network related stuff is handled by NetworkManager, so no other configuration should be needed there.
It has always brought up the network here without any messing about.

I've also written a new image building script, sort of as an alternative to multistrap.
It doesn't do a lot of checking at the moment, so make sure it can find all the programs it tries to run.
This one just uses debootstrap for everything, and then a systemd-nspawn to set everything up (tries to use qemu-arm-static to run on non-arm hardware, so that has to be installed and set up with binfmt to work)
It should be able to run pretty much everywhere, and i've been building test images on both my desktop machine running arch, and the devboard running debian.
http://dev.pyra-handheld.com/index.php/p/pyraos-build/source/tree/master/buildrootfs

Output of the script is an image that you can just dd to an sd card (or any other method, it's pretty much the same process as you'd use to install a rpi image), insert into the devboard, and it should boot.
Mate image available here : http://packages.pyra-handheld.com/images/extra/pyra-debian-jessie-mate.img.7z

It doesn't resize the partition on the first boot, and there's probably a few other issues, but it's more of a test to see if this stuff would work.

 

[Silent-Hunter]

Will MATE be an official option for the Pyra then?