libiconv problem / static linking?

[hmc]

Hi guys,


it seems I have a problem I cannot find a solution for myself. Please help!


Whatever program I compile on my Pandora (using the C/C++ dev PND on SuperZaxxon Beta5), seems ro require "libiconv.so.2", which is NOT available in a fresh SuperZaxxon Beta 5 install.


(Makes me wonder, if libiconv shouldn't be avialable in the firmware, as it is a quite important library IMHO, but that's not the question of this thread ;-) ).


There is no way to disable libiconv using the configure scripts of the programs.


What would the development expert do now?


- Link libiconv statically to the application? If so, how do I do this? I have never done this before, and I don't really know where to start to learn this.


- Include a dynamically linked library in the PND? Same here - I have no idea, how I'd do this.


- Anything else, what I could do?


Thank you very much.


BTW: Affected are, at the moment, TrueCrypt v7.1a and Conky v1.9.0 in the repo...


Daniel

 

[mcobit]

You can try to install the codec pack. I think there were some problems wih iconv, if you haven't installed it.


You can also build libiconv and put the library into a folder called lib in the pnd root. You can export LD_LIBRARY_PATH to the location in the startupscript.


Edit: if you know they don't work, I would disable them in the repo for now and post betas here first

 

[hmc]

I really don't want to go the codec pack route, as it would require me to advise all users of my PND to install the Codec Pack first. I don't consider this adequate for applications, which have nothing to do with codecs ;-)


Okay, will try the library inclusion and LD_LIBRARY_PATH route. Thanks for that hint! libiconf is configuring just now...


I also have disabled the two repo entries now.


However, my older PNDs seem to work even on the SZ Beta5 firmware.


This makes me assume, that something with my new build environment (the C/C++ dev PND, running on SZ Beta5) is not compatible with SZ Beta5.


However, I have no idea, where to start searching for the issue.


Any hint is appreciated!


And:


What build environments do you use successfully for onboard development with SZ Beta5?


Daniel

 

[mcobit]

I mostly use yactfeau toolchain. I don't compile onboard right now. When I started I used the dev extends by stuckie, but afaik they are pretty deprecated by now.


Edit: If you want to hunt the issue, the first thing to do would be comparing the /usr/lib folders of hf7b5 and hf7b5a. If libiconv is missing in the newer one but present in the old one without modifications, report it

 

[hmc]

thanks mcobit,


I use this code now, and put the libiconv.so files into ./libs, and the libiconv problem is solved now. For TrueCrypt at least.

# For libiconv dependency:

LD_LBRARY_PATH="./libs"

export LD_LIBRARY_PATH


./truecrypt

However, the Conky package still requires more libs, this time it complains due to missing libtinfo.so.5.


Neither libiconv not libtinfo are existent in either firmware in /usr/lib or /lib (checked HF6 with CodecPack, SZBeta5 with and without Codec Pack).


What do you think, what's the reason?


Does the C/C++ dev PND fool the configure script or gcc / ld that some libraries are there, which aren't there?


IIRC when I build the original PNDs, which did not cause such problems, I used the native development tag.gz package, not the PND. But I an hesitant to install that on SuperZaxxon Beta5, because I fear that it can have become incompatible, and removal would be a pain....


What's your advice here?


Thank you very much,


Daniel

 

[hmc]

Edit: if you know they don't work, I would disable them in the repo for now and post betas here first

okay, so here is a beta PND of Conky 1.9.0 with included libiconv shared lib.


Can you please test in so many different environments as possible, if it starts and behaves normally?


Strange things happened, when I compiled it, which are beyond my more-than-beginner-but-much-less-then-expert knowledge.


In short:


Even after compiling libiconv 1.14 on the Pandora and installing it to prefix /usr/local (the default anyway), where also the devel-pandora-native-20110411.tar.bz2 environment was installed to, when compiling Conky, I again got the same erros about missing libiconv as before installing libiconv.


I then searched around in the Conky sources, tried to trace back the error messages about "undefined references to libiconv_close" and "libiconv_open", but only found a .c file, which includes "iconv.h", and iconv.h was nicely installed in /usr/local/include. Nothing suspicious from my point of view.


I then searched for indications that the libiconv installation was maybe not recognized during the Conky compilation process (although I did "make disclean" and a new ./configure of Conky after installing libiconv).


What I found was a line "conky_LIBS" in the Makefiles in the Conky source tree, which listed a lot of libs, but not libiconv.


So I added "-liconv" to all occurrences of "conky_LIBS" in all Makefiles in the Conky source tree.


And guess what?


Then it compiled, but the result was strange. I have never seen something like that before:


In place of the conky binary there was a 8kB wrapper script generated by libtool, and an actual binary was located at src/.libs/conky.


Here is the heading comment of that wrapper script:

# conky - temporary wrapper script for .libs/conky

# Generated by libtool (GNU libtool) 2.4.2 Debian-2.4.2-1

#

# The conky program cannot be directly executed until all the libtool

# libraries that it depends on are installed.

#

# This wrapper script should never be moved out of the build directory.

# If it is, it will not operate correctly.


.....

In the PND there is now that binary from src/.libs/conky, and I also added libiconv.so.2 to the PND. That was it seems to work here on all my Pandora setups (thanks to SD boot capability, I have some of them, including a SuperZaxxon Beta5 vanilla install).


But I wonder about


- why the binary works now (although there is this strange libtool wrapper script)


- why there is that strange libtool wrapper script


- and why I could not compile successfully after installing libiconv, but after adding -liconv to those Makefile LIBS lines...


If anyone likes to volunteer to enlighten me on these mysteries, I'd be very grateful! I'll gladly set up a TeamViewer session to my PC with ssh access to the Pandora for that purpose.


Thank you!


Daniel

Conky.pnd

 

[mcobit]

If you don't want to alter your system, don't install to /usr/lib, but to prefix /mnt/utmp/yourprogram/ directly


Libtool seems to copy the files needed for linking in the src/.libs folder temporarily. Nothing to be concerned about. I think.

 

[_wb_]

I prefer to boot from a large SD card and simply opkg install or manually compile and install whatever is needed to get through the autoconf script. Then when packaging to PND, I test on a fresh NAND install, and copy all missing libs from the SD install. Adjust LD_LIBRARY_PATH and everything works nicely.

 

[hmc]

this is about what I do now, too.


I only wonder, if this is the "cleanest" way.


Especially in the case of libiconv, I wonder, if it is normal, that libiconv needs to be added to the PND, as I consider libiconv a quite important library.


I read about libiconv functions also being present in libc or glibc, and that those are alternative. In prior versions of the firmware, libiconv also was not present, but programs didn't require libiconv. Now they seem to require libiconv, so this libc/glibc-internal iconv support seems to be broken or removed. However, I don't know enough details about this, so I cannot judge, if this is a bug or a feature ;-)


Daniel

 

[hmc]

Then when packaging to PND, I test on a fresh NAND install, and copy all missing libs from the SD install. Adjust LD_LIBRARY_PATH and everything works nicely.

okay, I have now developed a shell script automating this process.


This script needs, in order to work properly, text files containing directory listings of all directories containing libraries, of as many different original firmware installs as possible. It uses these listings, to check, if a required library exists in a specific original firmware install. The more such lists you provide to the script (i.e. of as many different firmware versions as possible), the more is secured that the PND will be compatible to all of those firmware versions.


I have created one for SuperZaxxon Beta 5 now.


Anyone willing to create such a list for any other unmodified(!) firmware install? I'd appreciate any new list!


Use this command, after booting into the according firmware:

find /lib /usr | grep "/lib/" > liblist_[firmwareversion].txt

Here is the script (I called it "add_required_libs.sh"):

#!/bin/bash

# $1: binary to analyze

# $2: PND destination directory

# Aid for PND generation:

# This script analyzes the compiled binary for dependencies, which are not met by

# original firmware installations and copies such libraries into the PND folder

# for packaging.

#

# 2012 by Daniel Hertrich, http://www.hermocom.com


# USAGE INSTRUCTIONS:

# ===================

#

# STEP 1:

#

# Build firmware's lib list files for each original firmware,

# using (after booting the original firmware):

# find /lib /usr | grep "/lib/" > liblist_[firmwareversion].txt

#

# STEP 2:

#

# Put those liblist_* files, one per original firmware, into a dedicated directory,

# defined here:

LIBLISTS=~/devel/AUX/liblist_*

#

# STEP 3:

#

# Start this script, with two parameters:

#  $1: binary to analyze (including its relative path, if necessary)

#  $2: binary's PND root directory

#

# STEP 4:

#

# If this script found unmet dependencies, it has copied the according libraries

# into the PND directory into a subdirectory "lib/".

# In this case you need to put those lines (uncommented!) into the run script of the

# PND, which calls the binary, before the binary is called:

#    LD_LIBRARY_PATH="./lib"

#    export LD_LIBRARY_PATH


# ----------------------------------- SCRIPT START ---------------------------------

echo Analyzing binary...

ldd "$1" > libs_required.txt

# Generate lib directory inside PND directory:

mkdir -p "$2"/lib

# Scan for all existent liblist_ files

for f in $LIBLISTS; do

echo -e "\n*** Comparing against "$f"..."


# Read in the "libs_required.txt" file generated before:

while read line; do


  # Check, if this required library exists in the current liblist file:

  grep "$(echo $line | awk '{ print $1 }')" "$f" > /dev/null

  if [ $? -eq 1 ]; then

   # If the library does not exist in the firmware:


   # extract path of library:

   libpath=$(echo $line | awk '{ print $3 }')

   if [ $libpath == "" ]; then

    libpath=$(echo $line | awk '{ print $1 }')

   fi


   # Copy this library to the PND's "lib/" directory:

   if [ -f "$2"/lib/$(basename $libpath) ]; then

    echo $libpath already copied to PND.

   else

    echo Copying missing lib $libpath into "$2"/lib/ ...

    cp $libpath "$2"/lib/

   fi

  fi


done < "libs_required.txt"


rm libs_required.txt

done

 

[fusion_power]

I tried the new conky from the repo but it still does not work, I guess it is still the same problem like mentioned before. Here is the output:

=======================================================================================

PND			 : /media/FUSI_SD/pandora/menu/Conky.pnd

PND_FSTYPE	  : Squashfs

APPDATADIR	  : /media/FUSI_SD/pandora/appdata/Conky

APPDD_FSTYPE    : vfat

PND_CPUSPEED    : <unset>

EXENAME		 : run_conky.sh

ARGUMENTS	   : <unset>

=======================================================================================

[ START ]--- Mount the PND ----------

Mounting : mount -t squashfs "/dev/loop0" "/mnt/utmp/Conky"

Mounting the Union FS : mount -t aufs -o exec,noplink,dirs="/media/FUSI_SD/pandora/appdata/Conky=rw+nolwh":"/mnt/pnd/Conky=rr" none "/mnt/utmp/Conky"

[SUCCESS]--- Mount the PND ----------

[ START ]--- Starting the application (run_conky.sh ) ----------

./conky: error while loading shared libraries: libiconv.so.2: cannot open shared object file: No such file or directory

[ FAILED]--- Starting the application (run_conky.sh ) ----------

[ START ]--- Restoring the frame buffer status ----------

[SUCCESS]--- Restoring the frame buffer status ----------

[ START ]--- uMount the PND ----------

[ START ]--- Waiting the Union to be available ----------

[SUCCESS]--- Waiting the Union to be available ----------

rmdir: failed to remove `/mnt/utmp/Conky': Device or resource busy

[ START ]--- Waiting the PND mount dir to be free ----------

[SUCCESS]--- Waiting the PND mount dir to be free ----------

cleanup done

[SUCCESS]--- uMount the PND ----------

=======================================================================================

Return code is : 4

I am on Hotfix 6 at the moment, if this helps.

 

[hmc]

Are you really sure that you used the latest Conky from here?


http://repo.openpandora.org/?page=detail&app=Conky-hmc


Exactly this problem you report should be fixed with that new version, and I just retested on SZ Beta5 and on HF6, and on both systems it worked for me.


Daniel

 

[fusion_power]

Yupp, I used the Repo version because I usualy browse the Repo with the OND Manager and so I've spotted that yout Conky was updated. I also deleted the old Conky and it's appdata folder before, just to be on the save side. And as long as Conky doesn't have some hidden cfg files somewhere, nothing from the old non working version was left. When it is just a file missing, you could include it maybe?

 

[hmc]

actually I have included libiconv in this PND. That was the fix. ;-)


So I wonder why you still have that problem.


Can you please email me the PND you are using? I like to compare it to my working version.


Email: info (at) hermocom (dot) com


Thanks


Daniel