Debian In An Extend

[Stuckie]

I could do that, as it cleanly unmounts for the most part as well ( I forgot the /Angstrom and /media mounts I did last night, but that's an easy fix! though it wont read the SD cards just, that's another hiccup of mine though as I forgot to copy mtab over. )
You would have a slight issue in that if you close the Terminal window, it would try and clean up the mounts automatically.. not wholly ideal if you've got stuff running that's not linked to that Terminal, which is why I use the ExtendUtils and keep the mount/unmount separate from the Extend files.

I've a bunch of fixes for the ExtendUtils to try get things to cleanly unmount, so we'll see what happens

Wrapping the whole lot so you have a Debian.pnd would be pretty easy, just that'd there'd be a few of them - 1Gb, 2Gb and 4Gb variants! lots of stuff to keep track of.
By the way, I noticed that Debian has a HUGE amount of MIDI stuff in the repositories... may be worth poking around it a bit later and seeing what we can pinch!

 

[torpor]

Yeah, Debian has been good for MIDI, there are tons and tons of things in there we can get into playing with .. in fact if you do end up releasing a Debian.PND, it might be worth investigating just using it for a full-blown MIDI/synthesis environment, since Debian has a lot of the tricker stuff sorted out ..

Anyway, very nicely done Stuckie old chap! Brilliant bit of hacking ..

 

[JeanLouSchmidt]

That's a F... good hack, Debian on Pandora !!!!!
I don't have my Pandora (my position is something around 3000), but this would be the first thing I will try.
Congratulations !

 

[SomeGuy99]

Oh and as for the GUI mock-up, could you perhaps annotate it to what some parts should be?
I've a feeling the top bit is essentially for your "main" extend which you'll be writing to, and the bottom bit, is that for daisy-chaining all of them together as Read-Only? That could be interesting.. it's definitely possible, but the problem is every extend file takes up a /dev/loop! and so does a PND! so you can quickly run out of them, unless you extract the Extend file onto a Linux compatible file system, and link that in. No /dev/loop is taken up, and it'll work the same, just be a bit "messier."

The GUI mockup is like you said, yeah.

I had some further ideas, but it would also rely on mounting multiple read-only loopbacks. Is the /dev/loop thing a definite show stopper? Are there no other solutions? What is the practical limit?

 

[Stuckie]

From looking in /dev ... the limit is 7.
I can't remember if this is set in the kernel or not though, and there may be other ways around it.. need to keep prodding.

I'll do a "debian.pnd" set tonight.. that'll be for users who just want stuff to work.. which allows the ExtendUtils to be used for advanced mucking about

 

[Stuckie]

is what as responsive as XFCE?

you're running this ON TOP of your current running Angstrom/XFCE install and are essentially just given a Terminal into Debian.
It doesnt replace anything on your default system, and there's no nice GUI with it yet either.. it's raw Terminal power
Replacing the current running X isnt something I have tried yet, though you're welcome to poke at it

 

[SomeGuy99]

From looking in /dev ... the limit is 7.
I can't remember if this is set in the kernel or not though, and there may be other ways around it.. need to keep prodding.

Ah... yes... I just recalled that Puppy Linux had a similar limit (it used unions too). I seem to recall that somebody fixed it, but until then there were utilities to combine two or more together. Could we do that?

http://www.murga-linux.com/puppy/viewtopic.php?t=36733

Basically, I think we should rip off Puppy Linux:

RW loopbacks for the user space+file system changes, a loopback squashfs base OS file (eg. Ubuntu or Debian), a loopback squashfs base patch file and then some 'software addon' loopback squashfs files (eg. terminal utilities, desktop environments).

Does that sound evil enough?

Edit:

New bug - The contents of /media/DISKNAMEHERE are empty!

 

[Stuckie]

Not a new bug, I mentioned that a few posts back
http://www.gp32x.de/board/index.php?/user/2841-stuckie/

As for how Puppy Linux fixed it, they seem to have modified the initrd stuff.. not sure we can do that, to be honest..
The combining into a one large loopback is easy enough, but beware of the FAT32 4gig limit on individual files.

Also, your ideas of splitting up of many things to loop back files intrigues me, and I'd like to subscribe to your newsletter

 

[SomeGuy99]

Not a new bug, I mentioned that a few posts back
http://www.gp32x.de/board/index.php?/user/2841-stuckie/

As for how Puppy Linux fixed it, they seem to have modified the initrd stuff.. not sure we can do that, to be honest..
The combining into a one large loopback is easy enough, but beware of the FAT32 4gig limit on individual files.

Also, your ideas of splitting up of many things to loop back files intrigues me, and I'd like to subscribe to your newsletter

Can we have at the very least:

R: Base OS R: Patch RW: Userspace ?? That's only three?

 

[Stuckie]

We should be able to get away with that with the updates I'm about to do.. basically, going to mount the extends in tmp rather than inside the PND, so that reclaims a loop slot.
It's what I'm doing for the debian ones now.

Only problem is, at the moment, I have no code that'll magically mount more than one AUFS bind at a time... that is probably going to require a GUI of some description, as I doubt any amount of Zenity fiddling will be able to pull it off.

Current Tasks for Tonight:
Release a Debian.pnd .. I'll be using the 1Gb Sid with Synaptic build.
Update ExtendUtils with latest fixes.
Attempt to bind multiple Extend files at once and see what happens.

Bonus Points:
Kill Angstrom X, start Debian X... this gets us closer to even more nefarious hackery

 

[Chero]

Nice work !!

I had the wifi problem when I started, copying my resolv.conf from Angstrom to debian solved it.

Having fun,
Chero.

 

[Stuckie]

I've updated the ExtendUtils: http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/extendutils.pnd ( though haven't fully tested.. it's getting a big bugger now! )
It's now a bit cleverer and should mount to /tmp instead of inside the PND for normal Extends.
In regards to Debian Extends, it's done an AUFS on /etc so mount et al should be fine, and it's properly got the mount points in ( I tested this ) however, there's a lovely little hiccup - base Debian doesn't have an auto mount so when you replace your other SD card ( DO NOT REMOVE THE SD CARD THAT HAS DEBIAN ON IT WHILE YOU'VE GOT DEBIAN RUNNING... BAD THINGS WILL HAPPEN TO FLUFFY KITTENS EVERYWHERE! ) Angstrom will remount it, but Debian wont.. if you manually remount it, it works fine.

Test it, Abuse it, Find more bugs!

 

[SomeGuy99]

Okay, another interface mockup:

Program functionality:

It finds stuff in /appdata/NAMEOFAPP/ that is placed by the user, eg.

/appdata/NAMEOFAPP/base - all the base OS extend files
/appdata/NAMEOFAPP/user - all the RW user exten files
/appdata/NAMEOFAPP/patches - matches patches to base os based on naming scheme:

BASEOSFILE_patch_x <-- incremental patches

BASEOSFILE_patch_x_ALL <-- incremental patch with all previous patches included

The program will add all these found files into the lists, but the user has the option to add more from other sources. The results are held in a flat database or config file.

The program needs a minimum of a base OS file and a userspace file and the GO! button will become active. Exit will unmount everything and inform the user.

 

[Stuckie]

That's quite a complex app you're wanting there, to be honest... will need to settle on some sort of widgets library first, as that'll effectively determine how we can go about this.
I'm somewhat experienced with wxWidgets, though that's not exactly light weight
Probably something like FLTK or LessTif would work best for it.. just need to learn em!

** BAD NEWS **
You cannot wrap an extend file in a PND... well, you can, but it's a really stupid thing to do as the AUFS bind will kick in as soon as you open it and make a writable copy in your appdata folder.

However, with a GUI to mount these things, I doubt it'll be too daunting with what's going on
But unfortunately, for the time being, no "Debian in a PND" release
If I just strip out the Debian Extend bit, and have people stick the debian extend image on the root of the card, it's not much different to what's happening just now, except a bit more closed in and another set of scripts to maintain.

 

[SomeGuy99]

I guess for the time being we can just have on the root of the card, say:

debian_base.extend
debian_user.extend

The gui was just something I was throwing out there. I think having everything on one window would be nice and quick - immediately your eyes can scan for what is highlighted, and the entire thing can be changed and launched within four clicks/stylus presses.

Can we do password keychain stuff?

 

[Stuckie]

Keychain stuff?

I've just updated the PND with the ability to login as your normal user account into the debian extend.. so no more default root access ( which is a good thing, it's safer this way! ) http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/extendutils.pnd

Also from what I seem to understand... you want an os.extend and a user.extend? if you're going this way, then I could probably shrink Debian back down into a 512mb extends only, and get rid of the 1,2 and 4gb debian extends.
This seems to be abusing the system no end.. as you could effectively have an distro in an extend file mounted RO, an overlay on top of that, and possibly a home extend as well ( which gets shared amongst your other distro extends. )

Is this what you're meaning?
If so, I think you could be on to something here... and I was only joking with the "collect them all!" remarks

 

[SomeGuy99]

Keychain stuff?

I've just updated the PND with the ability to login as your normal user account into the debian extend.. so no more default root access ( which is a good thing, it's safer this way! ) http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/extendutils.pnd

Also from what I seem to understand... you want an os.extend and a user.extend? if you're going this way, then I could probably shrink Debian back down into a 512mb extends only, and get rid of the 1,2 and 4gb debian extends.
This seems to be abusing the system no end.. as you could effectively have an distro in an extend file mounted RO, an overlay on top of that, and possibly a home extend as well ( which gets shared amongst your other distro extends. )

Is this what you're meaning?
If so, I think you could be on to something here... and I was only joking with the "collect them all!" remarks

Yep. And why not make the base os extend a compressed file format? Can you do squashfs unions? I was under the impression that PNDs make use if it, as does Puppy Linux. It saves wasting space on the SD card right?

I guess what i'm proposing is some kind of twisted chrooted Debian Puppy Linux style system with frontend.

And of course we have to get chrooted X going! And then add PND support/passthrough for PNDs on the SD. Why? Because we can (or can't)! Why stop the crazy hi-jinx here eh?

Can you do more than one chroot at a time? How far can we push this thing?

 

[Stuckie]

Yea, you can do a cloop or squashfs.. I actually did it for the Debian PND.. then realised that 1) it's read-only and 2) AUFS is going to try and copy the extend as soon as I touch it, as it thinks I'll be modifying it ( and it did.. had to hard reboot to stop it. )

For the most part, PNDs are just simple ISO files.. there's probably very few compressed PNDs just now.

And yes, you're essentially proposing some sort of Twisted Puppy Linux ( could call it AbUsed Fluffy kittenS to keep Gruso happy, have an amusing jab at Puppy Linux, and to have AUFS in there )
As for PND support in Debian, probably just need to compile libPND inside the chroot jail.
And yea you can do multiple chroots.. though why, I'm not sure

 

[SomeGuy99]

Yea, you can do a cloop or squashfs.. I actually did it for the Debian PND.. then realised that 1) it's read-only and 2) AUFS is going to try and copy the extend as soon as I touch it, as it thinks I'll be modifying it ( and it did.. had to hard reboot to stop it. )

For the most part, PNDs are just simple ISO files.. there's probably very few compressed PNDs just now.

And yes, you're essentially proposing some sort of Twisted Puppy Linux ( could call it AbUsed Fluffy kittenS to keep Gruso happy, have an amusing jab at Puppy Linux, and to have AUFS in there )
As for PND support in Debian, probably just need to compile libPND inside the chroot jail.
And yea you can do multiple chroots.. though why, I'm not sure

AbUsed Fluffy kittenS sounds like a winner!

Why would I want to do multiple chroots? To have multiple Debians running maybe? Why the hell not?

Can we make a script that would apt-get a program from the Debian archives and make a PND out of it?

Another evil idea: instead of mounting your userspace, you mount a temporary RW loopback. You install a program from the Debian repo, and then quit and unmount everything. Dropping back to the frontend, we name the file something relevant to what we installed. This file can then be merged/rebuilt into an existing loopback that we regularly used, and the original file can be shared over the internet and other people can do the same. In addition, we can have a function that exports it to a PND file and copy across any of the dependencies that the base OS file had, and the Pandora firmware doesn't (I guess the script would have to remount, copy and unmount in the background).

All that can be in a seperate window within the frontend?

Wish I had the skills/knowledge to contribute. Maybe I can write the text for the help system?

 

[Stuckie]

Woah, slow down there!

Remember that currently we're limited to seven /dev/loop devices.
Each PND takes up one.. so the ExtendUtils will take up one.
Each Extend file takes up one.. so you have Debian in one, a possible user overlay in another, and some RW user space.
That's four so far.

That's also from a clean boot

As for PNDifying stuff direct from Debian.. it's unfortunately not quite that easy as my AUFS exploits with GIMP and GMPC should show you.
For the most part, library paths can be redirected when instantiating the program, and /etc is a relatively safe AUFS mount point ( nothing is usually writing there when you want to unmount it. ) But your main issue is the stuff that's bound to be scattered all over /usr/share
"Cleanest" way is to symlink that to the /usr/share/whatever in the PND.. but that of course touches the NAND in creating the symlink. You possibly could try and AUFS /usr/share but I've found it's not always clean to get it unmounted... and AUFSing the entirety of /usr is asking for a kicking!

However, I have some other tricks up my sleeve...

[edit]
I've also reverted the ExtendUtils slightly so it doesn't automatically log you in as your own user... as unfortunately, this seemed to bugger up the X forwarding, so I need to have a look at that.
[/edit]