Debian In An Extend

[Stuckie]

After countless hours spent pouring over this mammoth script and testing the crap out of it to ensure it's not going to grow fangs and cause some very nasty things to occur, it's finally finished!

Weighing in at a hefty 510 lines, this is pure, unadulterated bash and Zenity abuse

Extend Utils now creates mount scripts.
The idea is that you setup your environment however you like.. be it as simple or as complex as you desire.
Just want to mount a Debian extend to access Synaptic to install stuff and mess around in a Terminal? No problem.
Want to do some Development on the Pandora, but would rather not install to the NAND? Easy.
How about some madness? Mount Debian with an Overlay to catch anything you install, and as it's binary compatible, package it up for standard Pandora consumption!

This also paves the way for the actual GUI system to come in to maintain the "premount" files.

What it DOESN'T currently do ( as the code was WAAAY too temperamental just now ):
It won't create chrooted environments for you.. you'll need to download a pre-generated one, or do it manually.
It won't combine Extends/Overlays... you'll have to do this manually.. it's a bit trickier than it first appears if you want to maintain package lists, for instance... I'm working on using diff to handle this, however.
It won't handle any special bits and pieces required for certain OS Extends.. but you can edit the script after creation.
It won't setup GDM for you should you have installed it and want to "boot" to it.. though I'll throw up a guide to do it.. I probably have already done it, it'll just be lurking "somewhere" in this thread
It does absolutely nothing to try and unmount anything at the moment... so you're best restarting after calling the mount scripts.

Now what it DOES do:
Creates new Swap, Overlay and Extend files - be it from the pre-formatted set of 256/512/1024/2048/4096MB files or a custom size you pick ( though beware as that will call DD direct on your card, and may chew it out! )
Builds and Runs Pre-Mount scripts that allow you to mount multiple Extends and Overlays, as well as Swap.
It's one script instead of many!

I don't have an icon, so if someone feels like making an icon for Extend Utils, feel free

It can be grabbed from the usual place: http://www.stuckiegamez.co.uk/apps/pandora/SimpleDev/extendutils.pnd ( it's now 8mb! )
Documentation is here: http://pandorawiki.org/Extend_Utils .. I suggest you read it!
Test it, Break it, Bug Reports please!

It might almost be usable enough to throw on the usual download places, what do you think?

NOTE: I haven't actually tested the Ubuntu extends fully.. but that had it's own script anyway as it needed some funky stuff to work at all.
That and Ubuntu was perhaps a bit broken anyway Bit more broken now, though!

 

[todd]

It's tempting to dig up some launcher that could run from debian and sit on Angstrom's XFCE desktop. That way one could have a launcher for debian apps and the normal XFCE toolbars to launch "native" Angstrom app's.

That, combined with some concerted efforts at unifying the home directory (I'm thinking about modifying fstab to permanently look for my home directory on the SD card) could make using debian fairly seamless.

Just add another panel on the right. B)

 

[Stuckie]

If you trawl through this thread, you'll see that LXDE did something like that.. it loaded up with it's own bar, but the XFCE one was hiding underneath, so you could've moved one to the top of the screen and had access to both.

I need to go back through the OS Extends and fix them up a bit.. some are a bit archaic... think I'll settle on Debian though, as I don't want another Ubuntu experience! ( though Arch seems like it'd be reasonably ok.. and I was going to have a look at Gentoo as well, if only to help the Neuvoo guys out a bit by giving them a chroot environment for dev purposes )

I've also just done something insane - I mounted a root overlay, then mounted a compressed x86 Debian Sid extend, with another overlay on top of that.. other than qemu-i386 getting majorly confused as to where it was really supposed to be pointing, everything actually mounted correctly. I can't seem to get qemu-i386 behaving itself though.. best I can do is actually get into an x86 chroot, but it won't call fork() properly, so nothing works. Something for tomorrow, methinks...

 

[daniel3000]

Wow, thanks Stuckie for sharing all that work!
I will also try and test the new ExtendUtils for sure. Probably today. Can I simply update the ExtendUtils.PND and continue to use my Debian extend which I updated yesterday with OpenOffice and icedove in a lengthy process because WiFi stopped working constantly while downloading packages?


Couple more questions:

- DevExtend: I used this before in order to do any development in there. However, everytime a library was missing and I wanted to install it via opkg, I either had to reboot first, then opkg install, then reload the DevExtend (there was no unmount for the "Extend Root"), or I had to install the lib with the extend mounted, which meant it would install inside the extend (which shouldn't be a big problem), but also opkg was messed up because it couldn't find some info anymore and tried to install a lot of dependencies inside the extend which actually were already installed in Angstrom.
Is this problem solved in the new ExtendUtils in some way, or was it only me who had that problem...?

- Since Debian and Angstrom are binary compatible, why is it not possible to simply install the Debian packages in Angstrom? Is it just the file system structure, so the packages won't "fit", or is it different libraries used in both systems or anything else? It would be cool to have synaptic as a second package manager in Angstrom for installing Debian applications directly in Angstrom.

Daniel

 

[Stuckie]

Wow, thanks Stuckie for sharing all that work!
I will also try and test the new ExtendUtils for sure. Probably today. Can I simply update the ExtendUtils.PND and continue to use my Debian extend which I updated yesterday with OpenOffice and icedove in a lengthy process because WiFi stopped working constantly while downloading packages?

Yep, just follow the guide on the Wiki for creating an "uncompressed/classic Extend" PreMount. Direct Linky Here: http://pandorawiki.org/Extend_Utils#Uncompressed.2FClassic_OS_Extends

Couple more questions:

Argh... run! *cough*

- DevExtend: I used this before in order to do any development in there. However, everytime a library was missing and I wanted to install it via opkg, I either had to reboot first, then opkg install, then reload the DevExtend (there was no unmount for the "Extend Root"), or I had to install the lib with the extend mounted, which meant it would install inside the extend (which shouldn't be a big problem), but also opkg was messed up because it couldn't find some info anymore and tried to install a lot of dependencies inside the extend which actually were already installed in Angstrom.
Is this problem solved in the new ExtendUtils in some way, or was it only me who had that problem...?

There is no unmount support at the moment.. it's a huge mess to try and pull these things back off so you will have to reboot after you're finished, but I don't generally see that as a problem with what I use them for ( to protect the NAND while I'm messing about ) and as there's two SD card slots, capable of reading stupidly large capacity SD cards, plus a USB slot at the back for card readers, USB sticks, and the like, it doesn't really tie things up as much as it would any other device.

As for having to reboot to install libraries, that sounds very strange... you didn't actually reply to me in the Dev Extend topic as to exactly what opkg was complaining about for me to help so I had assumed you got it working fine. If, however, you already have installed stuff direct to the NAND, then yes, it won't know you've done that and it WILL try to install them again in the Extend.. this is a problem with all Extends and Overlays just now they can obliterate package manager lists if stuff has been installed within them, and you've got other stuff installed on NAND. It's something I'll be working on next, however.

Generally, I've used the Extends and Overlays from a completely clean and vanilla Pandora.. so anything I've installed ( bar hotfixes ) have gone direct to my Extends and Overlays, so it's not a problem for me.. it only causes issues if you've installed stuff to NAND and then tried to use a Dev Extend, as it's got it's own package list which will conflict - which is probably what you ended up with.

- Since Debian and Angstrom are binary compatible, why is it not possible to simply install the Debian packages in Angstrom? Is it just the file system structure, so the packages won't "fit", or is it different libraries used in both systems or anything else? It would be cool to have synaptic as a second package manager in Angstrom for installing Debian applications directly in Angstrom.

You can install deb files directly in Angstrom, SomeGuy was actually doing this for a while. You've a problem though as you don't get aptitude, only dpkg, so you have to install everything manually - including dependency tracking. I'm sure SomeGuy will gladly tell you how immensely tedious that is
Having two package managers in the same space - aptitude and opkg - would cause the same issues as you're experiencing with the Dev Extend; it wouldn't know what's installed by the other... so you'd possibly end up installing things twice, or having to perform some kludge that syncs up both package lists where possible - and I doubt the formats are comparable. That and Debian contains FAR more packages than Angstrom, so there's probably a lot of conflicts.
Essentially, the Debian OS Extends are a full Debian OS, which will contain a lot of duplicate libraries between Debian and Angstrom to keep it happy.. isn't really any cleaner way to do it, I'm afraid, but you do at least get to use the wonderful Synaptic and aptitude to install stuff in a protected environment on your Pandora, without stuffing things up and seeing as you can compress a base Debian debootstrap down to about 100-odd meg, it's not too bad! ( yes, I promise to release the compressed stuff today! )

 

[todd]

If you trawl through this thread, you'll see that LXDE did something like that.. it loaded up with it's own bar, but the XFCE one was hiding underneath, so you could've moved one to the top of the screen and had access to both.

Cha-ching: Thanks! I also appreciate the CPU load monitor provided by lxdepanel.

You'll note the swirl of the bottom menu. At first I had icons for apps, like with at the top. But 800x480 doesn't leave a lot of room for extraneous buttons.

A little time ironing out my home directory and this should work great. Thanks, Stuckie!

 

[SomeGuy99]

I'm sure SomeGuy will gladly tell you how immensely tedious that is

Very.

I don't recall installing any .deb files successfully to be honest. I was trying through opkg, but it complained all the time.

 

[paddy]

Hi ,im trying to mount the 1gb debian extend with the updated extendutils v1.0a (can't find the old exutils (superseeded by new version i think) so when i run utils i have no option to mount an extend only mount a pre-mount and when i browse my extend doesn't show as its only looking for .premount file extensions.Options are Mount Existing Pre-Mount ,New Pre-Mount,New Extend/Overlay/Swap,Online Help.

 

[Stuckie]

Please read the wiki documentation I linked to ( or click Online Help which takes you right there )
As stated in a post above, you'll be wanting to specifically follow the Uncompressed/Classic Extend guide - except you'll probably want to select a Terminal rather than GDM start.

Direct Link Here: http://pandorawiki.org/Extend_Utils#Uncompressed.2FClassic_OS_Extends

Once you've created the Premount, you don't need to do it again, and it'll be forever "future proof" as well.

 

[todd]

Paddy -- I found it a little confusing, too. The documentation explains all.

Stuckie -- I tried the new utilities today, too. Setting up a *.premount file to run the extend I've already been using worked great: thank you! I stuck in my ~/bin directory and will be thinking about how to have it run automatically on boot. Probably I'll make it run "lxpanel" instead of the terminal.

I also figured I oughta' set up some swap. It claims to be creating the swap file, but I can't find it, neither in the zenity dialog (when trying to make a premount for it) nor in my terminal. Any ideas? Any source of more information I can provide for you?

 

[Stuckie]

Wow, bugs already!

That's my fault.. there's supposed to be checks to ensure if you forget to put the extension on the file, it'll do it for you.. looks like I missed it off the swap! Therefore your swap file will be wherever you placed it, just without the ".swap" extension.. so you could use the file manager to add the ".swap" extension and it'll be picked up by the premount script config thing.

Sorry about that!
The swap stuff was kinda thrown in at the last minute again, as it's not done in any smart way... so I missed the check.
If anyone's interested in the source to the script, I'll be happy to stick it up somewhere... it's rather long though.

--edit--
Actually, having just looked at the code, and testing it on my desktop, it runs fine.. perhaps I uploaded an old version by accident?
I'll throw it on my Pandora and check...

 

[todd]

Wow, bugs already!

That's my fault.. there's supposed to be checks to ensure if you forget to put the extension on the file, it'll do it for you.. looks like I missed it off the swap! Therefore your swap file will be wherever you placed it, just without the ".swap" extension.. so you could use the file manager to add the ".swap" extension and it'll be picked up by the premount script config thing.

Yeah, I double-checked where I'd put the file and went looking for it in the terminal. But I still couldn't find it. Then I decided that maybe it didn't write since I didn't explicitly add the .swap extension, so I tried that too. No dice. I also checked to make sure I had sufficient space available on the SD card.

 

[Stuckie]

Out of curiosity, what were you naming your swap file?

 

[paddy]

ok i got it ,but what is the point in making it work like this ? what is gdm and does this mean we are further into putting the installed software in the extend onto the pandora OS so the software can be used without first running the extend and calling the app via the terminal.

sorry i hadn't been reading through for a week or so as my pandora was away and i was busy ,now want to test more software but i also want to pnd it.

Just notice debian giving errors downloading some packages ,mabe server down ? (not all packages could be retreived)

 

[Stuckie]

The purpose of making it work like this, is that you can create completely customised mount scripts.. it's the only way to do the compressed mounts as you _REQUIRE_ an Overlay if you want to save anything, and all AUFS mounts must be done at the same time, or it fails.

GDM is the GNOME Desktop Manager.. it's effectively what the Ubuntu Extend called up.

As for being reliant on the Extend Utils, this creates completely separate mount scripts.. they are literally just bash scripts.. so the dependency is actually *broken* so you don't have to use Extend Utils once you've created the "premount" script. There is now absolutely nothing stopping you from calling the premount script on boot so you can directly setup your Debian extend

That's why it's gone like this!
More customisation.
More fun

--edit--
Fixed the filename issue.. interesting that one, as sed on my desktop machine was fine with it, but sed on the Pandora effectively gave the REGEX I was using the finger... oh well, should've escaped the thing properly anyway, so was my fault!

1.0b has now replaced 1.0a, so feel free to re-grab.

 

[paddy]

Looks like i missed quite a bit ,you have lost me man h34r: so i get that once the premount script is made i could mount that via the pandora os without the utils (don't know how to mount tho and what i would do with it once mounted) and you say i can setup my deb extend ,do you mean install software to it and access it on boot up if mounted ? how would i access that (if mounted does that mean i would see it in the file manager ? then i could have direct access to the software files so i can pnd those files up ? sorry for being stupid at this , still learning and its alot to take in but i want to know,it's important that i at least do one pnd for the comunity !

and thanks for your hard work Stuckie

is anyone getting problems downloading some files from the debian file server ?

 

[Stuckie]

Looks like i missed quite a bit ,you have lost me man h34r: so i get that once the premount script is made i could mount that via the pandora os without the utils (don't know how to mount tho and what i would do with it once mounted)

It's very easy it's just a bash script, so you literally only have to do:
/bin/bash /path/to/your/script.premount
and you're done!

And you say i can setup my deb extend ,do you mean install software to it and access it on boot up if mounted ?

Effectively, yes... you may need to do some grunt work inside the Extend for it to work though, as well as editing the mount command of your premount script to something like the following:
gksudo "chroot /tmp/debian /bin/bash -e /path/to/program"
If said program requires a bit of setup, you'd need to write a script file stored inside the Extend and call it instead:
gksudo "chroot /tmp/debian /bin/bash -e /path/to/setupscript"
This is exactly what the Ubuntu Extend does.. there's a chrootStart.sh script in /root that gets called from the chroot command.

Then i could have direct access to the software files so i can pnd those files up ?

Ah, now this is the fun one, and is also why the Extend Utils has become as customisable as it has
The *best* way to do this, would be to push an Overlay over the top of a clean Debian Extend, and then install what you want.
That way, everything you've installed has just been directly written to your Overlay, so you know _exactly_ what files you need!
However, as has been stated numerous times, you may not be able to just directly rip out the files and throw them in a PND depending on what it's installed, and crucially where... I might update my PND guide later this week if I find the time.

is anyone getting problems downloading some files from the debian file server ?

May want to check your connection on the Pandora?
If it's working, perhaps the resolve.conf got copied before NetworkManager was finished fiddling with it, in that case, from an Angstrom Terminal, just do the following:
sudo cp /etc/resolv.conf /tmp/YOURMOUNT/etc/resolv.conf and you'll be sorted
Obviously, replace YOURMOUNT with what you actually set your mount point to be.

 

[todd]

Stuckie, If you're still interested, I was calling it "swapfile.swap". Original, no? I'm glad to hear it wasn't just me being silly. Looks like Gnu sed in Angstrom: same version as debian/sed (4.1.2).

is anyone getting problems downloading some files from the debian file server ?

Try pinging google or somebody. If you can't resolve hostnames, try "cp /Angstrom/etc/resolv.conf /etc" from your chrooted terminal. I have to do that manually each time for whatever reason

EDIT: Ninja'd. h34r: h34r:

 

[Stuckie]

Odd.. you shouldn't have to do that... check your premount script for the following:
gksudo "cp /etc/resolv.conf /tmp/$mount/etc/resolv.conf"
where $mount is your mount point, obviously

If it's not there, let me know what kind of premount script you created ( OS Extend, Dev Extend, Overlay... ) and I'll have a look.. might've missed something.

Oh and yea, the sed command I had would've mangled that filename pretty well!
If you still have issues with 1.0b ( you shouldn't, I tested it on my Pandora and it worked ) let me know and I'll start adding in debug output everywhere...

 

[todd]

Odd.. you shouldn't have to do that... check your premount script for the following:
gksudo "cp /etc/resolv.conf /tmp/$mount/etc/resolv.conf"
where $mount is your mount point, obviously

If it's not there, let me know what kind of premount script you created ( OS Extend, Dev Extend, Overlay... ) and I'll have a look.. might've missed something.

Oh and yea, the sed command I had would've mangled that filename pretty well!
If you still have issues with 1.0b ( you shouldn't, I tested it on my Pandora and it worked ) let me know and I'll start adding in debug output everywhere...

The cp command is there, indeed. It may be that I didn't have to copy it on my most recent startup (using the premount script). I'll try to take note of that next time.

I'm about to head out, but I'll grab a new copy of the extendutils when I can and give it another swing. Thank you for being so responsive with this. You are tremendously extending the functionality of our pandoras in a relatively painless way!