Standardization Of Library Directory

[esavior]

I installed the SDL library a couple days ago and noticed that it dumped them into /usr/lib. I know this is the standard for Linux and it has worked great for all my linux boxes in the past. But I think to continue to do this is a bad idea. We need to be able to load libraries off the SD card. And to do that we need to come up with a defacto standard for where to put them. I suggest /mnt/sd/gp2xlibs. If we do this then users and developers could just put the libraries they need into that directory instead of the user having to execute a shell script to copy everying over to their NAND. I wrote up a bash script to take care of all this. It adds a place for the dynamic linker to look then mounts the sd and executes the application then when the application is complete it dumps back to the menu. I would like to say 2xquake's loader script is what inspired this. If you want to download these scripts there will be a link below. Just unzip the file onto the base of your SD card. You will also need to make a launcher script for games. I have tried to make that as easy as possible. Here is a example launcher:

#!/bin/sh
source /mnt/sd/gp2x.env
#The path to the folder that the game rests in.
#This path is in relation to your SD card so don't
#put /mnt/sd/ at the begining.
export EXECPATH="data/games/emulators/nes"

#The name of the program to be executed
export EXECNAME="nesgp2x.gpe"

run

everything happens in the run fuction so really you just have to change EXECPATH and EXECNAME to match your game. I really hope other developers will take advantage of this, and ship a launcher with their game. But like I said its easy to create one yourself, just copy that into a textfile and edit each variable accordingly save it (making sure to give the filename a .gpe at the end) and put it anywhere on your SD card. After that you just use that to load the game. If it feels like your head is spinning I have went ahead and made a full zip that includes the libraries and nes and snes emulators. Just unzip it onto your SD card.

Scripts and SDL Libraries:
sdlibs.zip
Scripts, Libraries, Emulators, Launchers (eg everything you need to rock)
sdlibs_full.zip

The rest of this will be in FAQ form mostly because its easier for me to think that way and hopefully I will make more sense.

Why shouldn't we copy our libraries to the /usr/lib folder, it has been the standard practice for years?
There are two big reasons. The first being there is a very limited amount of memory on the gp2x; 20mb of free space *. As more libraries are needed to play games it will begin to fill to the max. The system needs a certain amount of space to function correctly and if you max out the memory you may brick your gp2x.

The second big reason is from a user standpoint it is hard to get rid of stuff once its been copied over. New versions of libraries come out all the time, and when you install the new one you often have to delete the old one. The only way to do that is to look up the names of every file that was installed by your old library make a shell script that manually deletes each one. While most people who are firmiliar with linux can handle this task, there are alot of people who wouldnt know ls from grep. If you install your libraries to the SD card or they copy them there, with a card reader they can actually see all the files; not just infer they exist because of the output of some dump script.

Why not just statically link everything?
Memory on the gp2x is very finite. If every developer out there compiles everything in our SD cards are going to fill up very fast. It is not like a PC where there is 100gig of freespace waiting to be used. Most you can get is a 4gig SD card, but most of us can't afford to drop that much money.

Okay so there you have it. Thoughts, Flames, Flowers?
(Oh hey I am new to the community! *wave* once I get a toolchain up and running I will begin porting lua (with sdl and input modules thrown on too) over to the gp2x. Hopefully a solid scripting language will help others get into development.)

*Thanks psymastr for correcting that.

 

[Vimacs]

oh right, with only 1gb sd cards 400kb statical linked apps(likely smaller when compressed) are a real problem!!!!!!!!!

 

[esavior]

I tend to be a real tight *** when it comes to resources (heh gentoo user) But seriously it can add up >_>

 

[PSyMastR]

GP2X has 64mb internal, and 20mb of it is free.

EDIT: Welcome

 

[AireTamStorm]

I suggest /mnt/sd/gp2xlibs.

​

Why not just make a symbolic link there? (/usr/libs) Given, all the libs would need to be moved.
As long as the card mounts early enough, I think it would work. And if they are only 400k, I'd chill man, as long as you don't pile MP3s on that valuable NAND space it wont add up any time soon.

 

[Oscaruzzo]

Why shouldn't we copy our libraries to the /usr/lib folder, it has been the standard practice for years?
There are two big reasons. The first being there is a very limited amount of memory on the gp2x (CUT)

The second big reason is from a user standpoint it is hard to get rid of stuff once its been copied over

​

Third reason is you can do something awful to your console if you overwrite some vital libraries in there. :unsure:

 

[DJWillis]

I suggest /mnt/sd/gp2xlibs.

​

Why not just make a symbolic link there? (/usr/libs) Given, all the libs would need to be moved.

We are talking about a FAT/FAT32 SD cards, no symlink support guys ;-).

You will need to use LD_LIBRARY_PATH to point your apps at a Lib dir on the SD card. You can do that from a shell script that launches your app.

Whilst this is not a bad idea at all I will stick to static linking. People may love to save a few MB or space on a 1GB card but I personally would rather not be bound by the rather less the ideal (for homebrew development, remember nothing on the GP2X is mission critical) compiler and gLibC environment presented by GPH in the default file system image. Static linking for me and be dammed with it .

Other problems I can see is that the GamePark Holdings patches to SDL are somewhat less then nice and if community efforts to clean that up carry on at the pace there going at now I can see compatibility being broken for some things like controller and screen code (i.e. official SDL based apps and community SDL based apps). This is a bad thing for shared libs as having different API’s for the same libs causes apps that use the older/newer API to die in really interesting ways .

Static linking also ensures that provided you can get an ARM-Linux environment running on top of your GP2X static linked apps will (almost) always just work regardless of your libs setup, versions or any stuff like that.
No need to hunt down libRandomCheese.so to play some game that came out a year ago like you have to do on things like the Zaurus, Linux iPAQ’s or other embedded Linux systems, and then getting the version built for Open Firmware 1.0 or GPH Firmware 1.0.1 etc. etc. (do you see where I am going with this ).

It's also worth noting that the GP2X runs as root, normal users fecking about with the system really is far from ideal unless you like paper weights.

I realise I may be over stating things a little but this is a console after all and has to be usable by Joe Public and Joe Public wants an app to install and run without messing about IMHO .

 

[slygamer]

Well put. I am with DJ on this one.

 

[Squidge]

Must agree with DJWillis. I think most people would prefer throwing an application on the SD card and running that (or unpacking to SD card), rather than needing to find the correct versions of various libraries, place them on the SD card, unpack the game or emu, and then run a script to start everything up.

I think the major problem with dynamic libraries is library version issues - Eg. Install one app and it's dependable libraries, and then shortly afterwards trying a different game or emu and finding out that although it still uses the same version of SDL, it requires a different modification to it. Static linking solves this problem.

GCC2.95 is also slow - for example, Quake compiled for 2.95 runs at under 1 frame per second, but is about 10 times faster using GCC4.

We never had libraries on the GP32 - everything was statically compiled there - why start now just because we have Linux?

Dynamic libraries can't hope to be supported under HH-mode.

So, for me, I'll be another devver who static links everything.

 

[skeezix]

Actually, /usr/lib is not the standard place; you're thinking of /usr/local/lib (BSD-style) or /opt/lib (AT&T style) and other millions of places (/usr/share/lib, etc etc.) /usr/lib, /usr/bin etc are _supposed_ to be system files only, so that when upgrading a install you get all new of those dirs, but leaving behind untouched the /usr/local type stuff. Its onyl some Linux distros are _Braindead_ -- the same kind of braindead that leads to dynamicly linked /bin/sh (so you're screwed if you break your LD-path for dynlibs..)

Anyway, why not just build into the binaries to look first in /mnt/sd for shared libs, and if not present, then in /usr/lib?

Assuming GPH didn't break the standard unix linker, thats trivial to do. (ie: I forget the flag.. -xpath? Something.. anyway, you can stamp into the elf header the sequence of directories to search; also, the LD_LIBRARY_PATH and such are used as well.)

gcc -o foo -X -linker -X ${my path} -- that sort of thing

I can look it up in my standard makefiles later.. running out the door now.

This would mean no loader scripts or alteration of anything.. 'just works', no?

jeff

Properly named libs include the version number, so versioning shouldn't be an issue. The larger problem is colliding with system libs. HH mode won't suport libs of course unless someone writes a smart loader (too lazy

 

[Oscaruzzo]

So, for me, I'll be another devver who static links everything.

​

Me, I really think the "release" is the source code.

Binaries are just a "commodity" and will probably try to release both gcc4-statically-linked and gcc295-dynamically-linked stuff.

 

[Squidge]

Jeff, I'm not talking about library versions, I'm talking about hacks by various people. So the library number could be identical in all of them, but each have different modifications (after all, they could all be based off 1.2.7, for example).

 

[skeezix]

Naturally, people are supposed to mark that in lib variations but they never do ie: I've seen libSDL-com.borg.so and such, but thats rare. I run into it all the time..

I release Shadow Desktop (based on QT toolkit) for Windows for instance, and then someone installs one of the divX things, which includes a modded QT toolkit and it breaks my app. Pisses me off

I'd go with staticly lining a lot of things, but the proposal I mention above does solve quite a few problems...

... especially if theres a centralized lib distro.

ie: Say folks need expat for XML parsing, SDL and SDLaudio and some codecs .. woudl be better to have one zip file to suck them all done, and apps could use them.

But we'll all stat anyway, to avoid misery.. makes it easy for now, and hey.. 1GB SD

jeff

 

[Aimless_E]

I think Static linking is the way to go for most projects, but I could also see the benifit of having a "standards based" directory to store libs on the removable media.

Though which ever way the Programmer decides one things for certian. No one should mess with the firmware that GPH has loaded on the GP2X unless they KNOW what there doing. I for one plan to staticaly link

 

[AireTamStorm]

We are talking about a FAT/FAT32 SD cards, no symlink support guys ;-).

​

Yes but is not the NAND formatted as JFFS2? I do believe that has symlink support, and if that's where the symlink is, it should work regardless of the SD card's FS type.

But yeah, poor Joe shouldn't really have to know how to or to do that period.

 

[esavior]

I think static linking does make sense for alot of projects, not all though. Lets say you want to use a scripting engine to program some game logic, I dont think I would staticly link that. The bigger the library the bigger the size advantage of dynamic linking. But I understand everyones points about static linking.

But regardless if the community goes mostly static or dynamic I think the practice of copying libraries onto the NAND should not continue. If you want to make a statically linked binary, your fine. If you want to create a dynamic linked library I suggest checking out the bash script I wrote.

 

[DJWillis]

Yes but is not the NAND formatted as JFFS2? I do believe that has symlink support, and if that's where the symlink is, it should work regardless of the SD card's FS type.

But yeah, poor Joe shouldn't really have to know how to or to do that period.

​

Agreed but we are back to fecking with the NAND and if the 16 (as of today) emails about bricked GP2X's tell me anything it's "stay away from the NAND" (oh, and don’t turn the unit off halfway through or use naff batteries).

The best one I had so far was "I dumped the dirs and files to a text document, deleted a few non-essential things to free up some space on the NAND for future stuff, now my GP2X wont boot" << Really, shock, horror, define non-essential on an embedded device :lol: :lol:.

I think static linking does make sense for alot of projects, not all though. Lets say you want to use a scripting engine to program some game logic, I don’t think I would statically link that. The bigger the library the bigger the size advantage of dynamic linking. But I understand everyone’s points about static linking.

Sure, I can see it makes sense for things like Pygame/Python (or other 'interpreter' type libs) etc. but then you could argue you have a versioned Pygame Lib set with dynamic bits for Python scripts to hook into and static the hidden stuff .

Like all things there is no right and wrong, just some things to do and do not, it is really down to the developer in question to use there best judgement (and the mantra, don’t play in the yellow snow, sorry, GP2X’s NAND) and try and stick to conventions for whatever method they pick.

 

[DaveC]

My take on it from a user and not a linux geek is to K.I.S.S. It is much easier to just copy one executable to the SD and run it like the GP32. Having to copy a bunch of files to all kinds of remote directories and mess with scripts etc is a real PITA. If I want to remove or move an app it can be a real chore as well. It gets to be a huge mess after you start to get alot of apps on there with managing all of those libraries and stuff.

I loved the way it worked on the GP32. You just copy the FXE to GPMM and an icon automagically appears on the launcher. So clean and simple. More time gaming, less time futzing around with files

So hopefully it can be simplified, or maybe HH will be the cleaner option.