GP32 Dynamic Linking Libraries Support.

[LDChen]

Hello,
I would like to ask if there is some DLL (dynamic linking library) support in GP32.
My target is putting LIBZ and LIBBZIP2 outside my emulator.
I don't need both libraries to be available at the same time, so I was thinking to load them at runtime and free some memory.
I tried to search something, but without success.
I know it could be done with some tricks with sections/linker_scripts/load_bin_files_manually (it's still not so easy), but I prefer to use something already coded and tested if it's possible.
Is it available what I'm asking for?
Thanks for the help.

Sincerely,

LDChen

 

[pea]

Hi LDChen,

I have been after this for a while too, so if you come accross something, I would love to know too. The best heads up I have had is 'dlopen' from the uclibc libraries. At the time I was told about this function (or set of functions) I didn't have enough knowledge to extract/port them. Maybe I will try again at some stage.

I would love to know how to do this manually -
1) What compile flags to use to get a compiled 'lump of code' whereby I can just load it in to a memory location and set the PC to the first byte.
2) Is this how it actually works?

 

[loki666]

dynamic libraries, is really somthing that needs an OS.
You just cant load the object file and then jump in a function... The whole object file has to be parsed to change each of the external and internal references.
Ex: let say, you main apps take the firsts 100k of ram, and let say the dynamic object file is loaded at address 0x0+100, you have to change all internals references with an offset of 100, and resolv all external references...

That's for the basic, but i'm pertty sure its even more complicated

Realy, for a system like the GP32, I dont see how you could achieve this, nor why you should really need it. Just link the lib with the app. (even 100k of lib, is not a big deal IMO)

Loki

 

[mr.spiv]

Realy, for a system like the GP32, I dont see how you could achieve this, nor why you should really need it. Just link the lib with the app. (even 100k of lib, is not a big deal IMO)

​

This has been talked and designed to some extent earlier.. at least behind the curtains Anyway, I'm fully confident that e.g. Amiga library like dynamic libraries are doable. I haven't done a proof of concept (yet).. being such a lazy person these days. :blink:

 

[Squidge]

What you could do is use a PalmOS ARM compiler - then most things are position independant, and there's tables telling you what isn't, so you can fix it up even locals & globals can be anywhere in memory (globals normally being offset from R9, and locals from R13).

 

[mithris]

Or play around with GCC and -fPIC..

 

[mr.spiv]

I'm not really concerned about the relocation stuff.. What I mentioned about Amiga like libraries it includes all that and imho one of the simples reloc methods and ways of exposing the library interface. The real hard thing is how to handle e.g. libc stuff (memory management, IO etc) as the library and the main program would have completely separate instances of system & SDK libs.. h34r:

 

[Squidge]

I see what you mean, it could be a complete nightmare without a decent library manager.

 

[pea]

On a similar topic - how do plugins work? i.e. Those for slubmans firmware. Maybe this is enough for most peoples purposes?

EDIT: http://slubman.celeonet.fr/index.php?hard=gp32
Slubmans page - check out 'libraries update'.

 

[pea]

Can anybody answer this?

If I create and compile a simple function with number of inputs (void pointers) which are used within the function as function pointers to critical functions (such as malloc, free, the framebuffer, etc) and compile it with -fPIC (to get Position Independent Code with a global offset table - GOT), how do I find the offset of the function within the compiled code?

If I stick to strict rules (i.e. No globals, same inputs every time), I take it this can be done to get a basic sort of 'plugin' or dynamic library?

If I have multiple functions, how do I define a particular function as the entry function? Is it as simple as calling it 'main' ?

Apparently the loader has to resolved the GOT when it loads the module, so I guess the GOT will be in a predefined part of the compiled code too....

Example:

void main ( void *malloc, void *free, unsigned short *framebuffer ){

// Do stuff with malloc, and free it again
// access the framebuffer

// exit

}

EDIT: sp

 

[pea]

I have been doing some extensive reading on ELF format, and shared libraries.
Looks like dlopen, dlsym and dlclose are the way to go.

dlopen opens a shared lib.
dlsym looks for a symbol in it (an entry point like a function or a variable or something)
and dlclose closes it again.

Example:

int main(int argc, char **argv) {
	void *handle;
	int (*mydltest)(const char *s);
	char *error;

	handle = dlopen ("./libtest1.so", RTLD_LAZY);
	if (!handle) {
  fprintf(stderr, "Could not open ./libtest1.so: %s\n", dlerror());
  exit(1);
	}

	mydltest = dlsym(handle, "dltest");
	if ((error = dlerror()) != NULL)  {
  fprintf(stderr, "Could not locate symbol 'dltest': %s\n", error);
  exit(1);
	}

	mydltest("hello world!");

	dlclose(handle);

	return EXIT_SUCCESS;
}

Now just to get these functions working....
I'll be back!

 

[LDChen]

Hello,
after thinking on it for a while, I believe that the following solution could be a good idea:
1) Take the sources of elf2flt tool and compile them. Infact the flat binary format supports everything we need and it has very little memory requirements (DONE).
2) Make a bFLT loader (I think it's easy, at least by reading the uClinux source).
3) Compile the library with some switches: "-fpic" or "-fPIC", depending on the size of the Global Offest Table we need and an additional "-include dynalib.h" or similar for exporting library functions.
4) Finally, we will be able to load our relocable module.

It's true the we will only know the address to the main entry point of that module.
But we can use this this function in a smart way.
For example, we can use it as a "service function" and call it several times with different parameters.
Some actions we may need are:
1) getting the version and/or the name of the library.
2) passing a pointer to a structure with the addresses of all library functions (malloc, fopen,...).
3) getting a table with the addresses of the exported functions of the library.
I just thought these possibilities but probably there are others.

This "main library" could be compiled as static library and linked to our dynamic library with a "-l" command.

If all things will work fine, we may do a change to the SPECS file and add a switch that do everything: compilation with the right flags, linking with "main library" and conversion to bFLT by calling elf2flt tool.
The advantage is that it's mostly already written; elf2flt is a standard tool, the bFLT loader could be ported easily from uClinux as I said previously.
What do you think about this solution?

Sincerely,

LDChen

 

[pea]

I have been trying for the past day or so to compile libdl (dlopen, dlsym, dlclose) from both uclibc and newlib, with no luck. Both are so deeply reliant on other stuff from their respective libs that they aren't very portable (for me, anyhow), and from what I understand, are pretty specific to linux too.

I don't have any experience with any of the formats or techniques you are writing about LDChen, but if its going to let us have any sort of dynamic library support, I'm all for it.

One question - why convert away from ELF, when ELF is specifically designed to make dynamic linking easier?

EDIT: I just did a quick read up on BFLT, and it seems pretty cool. I see that it would be easier than ELF if done correctly because the format is simpler. I read an article explaining relocations, and it seems pretty easy to calculate the offsets into the .text and .data blocks (http://www.beyondlogic.org/uClinux/bflt.htm) which is nice, and having a single entry point is no problem for our needs if we are designing the dynamic libs from scratch anyway.

The structures for imported and exported functions is how I would do it too. The exported functions could be:
getMeVersionNumber();
getMeHandleToFunction( fnName );
where 'getMeHandleToFunction' would obviously resolve the char* to a function pointer.

I'm happy to help in any way I can.

 

[mr.mirko]

One easy way would be, if the libs are in the Flash-Rom already at boot-up, and a fxe who needs some functions from this (already in ram based) lib, can directly jump to this functions.

But this needs a new firmware, so the new functions are always at the same place in ram.

possible or not ?

 

[pea]

But these wouldn't be 'dynamic' libs anymore, would they?

The reason I want dynamic library support to work is so that I can create a variation of it, and allow GPDesktop to run user programs within the GPDesktop environment.

 

[Blah]

That would work though, at least for a firmware distro of GPdesktop...But not for LDChen's purposes. For LDChen's purposes this is probably just too much work instead of just linking them in.

 

[mr.spiv]

Hello,
after thinking on it for a while, I believe that the following solution could be a good idea:
1) Take the sources of elf2flt tool and compile them. Infact the flat binary format supports everything we need and it has very little memory requirements (DONE).
2) Make a bFLT loader (I think it's easy, at least by reading the uClinux source).
3) Compile the library with some switches: "-fpic" or "-fPIC", depending on the size of the Global Offest Table we need and an additional "-include dynalib.h" or similar for exporting library functions.
4) Finally, we will be able to load our relocable module.

It's true the we will only know the address to the main entry point of that module.
But we can use this this function in a smart way.
For example, we can use it as a "service function" and call it several times with different parameters.
Some actions we may need are:
1) getting the version and/or the name of the library.
2) passing a pointer to a structure with the addresses of all library functions (malloc, fopen,...).
3) getting a table with the addresses of the exported functions of the library.
I just thought these possibilities but probably there are others.

This "main library" could be compiled as static library and linked to our dynamic library with a "-l" command.

If all things will work fine, we may do a change to the SPECS file and add a switch that do everything: compilation with the right flags, linking with "main library" and conversion to bFLT by calling elf2flt tool.
The advantage is that it's mostly already written; elf2flt is a standard tool, the bFLT loader could be ported easily from uClinux as I said previously.
What do you think about this solution?

Sincerely,

LDChen

​

The above is more or less the way I would do it. Although I would do the library loading user program controlled. For example:

struct fooLibrary *foo = openLibrary("foo.lib");
....
foo->doBar( BUZ );
....
closeLibrary(foo); // when done..

If you got the idea..

 

[pea]

this IS just linking them in, isn't it?

SPIV: yes, that's pretty much just another way of doing the same thing

either:

handle = loadLibrary("foo.lib");
fooLibrary = getLibraryFunctions( handle );

printf("library version:%s", fooLibrary->version );
(*fooLibrary->doSomething)( "hello world" );

closeLibrary( handle );

or

fooLibrary= loadLibrary("foo.lib");

printf("library version:%s", fooLibrary->version );
(*fooLibrary->doSomething)( "hello world" );

closeLibrary( fooLibrary);

much of a muchness really.

EDIT: code edit
EDIT 2: Sorry, maybe I didn't read LDChens latest post well enough. He doesn't mean loading the lib in at runtime?

Some resources:
Simple ELF loader:
http://cvs.tartarus.org/elfloader/
About ELF loading (links from forum):
http://www.mega-tokyo.com/forum/index.php?...y;threadid=5873
Arm ELF loader for Palm (looks promising)
http://www.palmopensource.com/index.php3?more=563
RDOS operating system (GPL) that contains ELF loader:
http://www.rdos.net/rdos/
ARM ELF file format
http://www.arm.com/pdfs/DUI0101A_Elf.pdf
Some nice ELF image parsing code at the bottom of this PDF:
http://downloads.securityfocus.com/library/subversiveld.pdf