[Applimulator ;)] apkenv + apkenv.ui

[Rockthesmurf]

Although, knocking out the sound (which was causing threads to be created) may have just delayed the problem, as ultimately I get a crash:


Program received signal SIGSEGV, Segmentation fault.
0x4019c14c in flockfile () from /lib/libpthread.so.0
(gdb) bt
#0 0x4019c14c in flockfile () from /lib/libpthread.so.0
#1 0x402c6ab4 in fwrite () from /lib/libc.so.6
#2 0x4064d4d4 in ?? ()
#3 0x4064d4d4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)



And again it looks like the stack isn't in a happy state, not sure where it is attempting to do a fwrite to at the moment.

EDIT: For what it is worth, when ever it crashes, GDB only reports one thread running:

(gdb) i threads
* 1 process 6141  0x4019c14c in flockfile () from /lib/libpthread.so.0

 

[Rockthesmurf]

I have narrowed it down a bit further, the game makes a call to COpenSLImpl::COpenSLImpl() which is again in game code, from there it ends up branching to 0x41938025, I'm not entirely sure what this is, I tried to find out from GDB but:


(gdb) info symbol 0x4058c670
No symbol matches 0x4058c670.

EDIT: Stepping the branch in GDB reveals it goes to 0x41938024 in slCreateEngine () from ./system/lib/libOpenSLES.so.

Whilst it is executing the code as this branch, the console displays:


late-initializing normal mutex
Opening '/dev/binder' failed: Unknown error 1077486776
And when it returns to the calling code (in COpenSLImpl::COpenSLImpl) the stack is then in a bad state:


(gdb) p printf( "%3.2f\n", 5.5 )

Program received signal SIGSEGV, Segmentation fault.
0x402aef90 in __printf_fp () from /lib/libc.so.6

Maybe attempting to get OpenSLES to work on the Pandora is not going to work? I copied in the libOpenSLES.so from crow_riot's previous commit, I don't know if there is some reasons why I should expect this module to work in apkenv?

 

[Rockthesmurf]

Well, I have committed my work in progress changes so far https://github.com/Scraft/apkenv it needs libcrypto.so and libssl.so from the Candy Crush Saga APK copied into the system/lib folder inside apkenv. It displays a loading screen, loads a lot of assets, but ultimately dies. There are also my pthread implementations/mappings and a printf va_arg fix throughout the application.

 

[ekianjo]

Steven, have you tried different versions of the apk?

 

[notaz]

Hmm accessing /dev/binder from native doesn't look good, it wants to access some Android service or something..

 

[Rockthesmurf]

@notaz yeah, I guess we'd need a native implementation of OpenSL to link against instead

 

[crow_riot]

instead of compiling OpenSLES you could try and create wrapper functions. here's a wonderful script especially for that: https://github.com/harmattan/apkenv-wrapper-generator

*edit*

well i've created a quick version of those wrappers. there's still the need of the original header files of OpenSLES though. but i think you could change the types easily to get it working.

opensles.zip

 

[Rockthesmurf]

@crow_riot: I maybe missing something obvious (I usually am) but my understanding of the problem is that inside the libOpenSLES.lib I am currently using (one designed for Android), it attempts to do Android specific stuff (like access /dev/binder) which isn't (probably) going to work under apkenv. So to fix this, I need a libOpenSLES.lib that is natively compiled for the Pandora OS. The files you have generated (I believe) nicely set up the binding/mapping to use a native implementation of libOpenSLES.lib, but they ultimately call functions like 'slCreateEngine' which will result in a linker error until I have a natively compiled libOpenSLES.lib to link against.

Providing my understanding is correct, I need to get my hands on a libOpenSLES.lib compiled for Pandora, or attempt to compile one myself. I have certain reluctancies to compile libOpenSLES.lib myself, but I wonder if this has already been done for another project?

Finally, to be honest, I don't think OpenSL support is so important, it wasn't introduced to Android until (if I recall correctly) version 2.3, so the majority of games will fallback and use something else if this isn't available. Certainly when I remove the libOpenSLES.lib it does continue to run (and shows a splash screen), so it appears to be happy running without it. The only thing that may be useful with me running against the OpenSL lib is I have got quite close now to tracking down the point at which the stack becomes corrupt. My guess is at some point during the initialisation process it'll make a libc call and pass some Bionic data structure that is a different size causing stack issues. But that is a guess based on my understanding from yourself and Notaz.

@ekianjo I haven't tried multiple versions of the APK, I guess there is a chance that another version will do something different, but I am not too confident. I believe there is a function being called that is breaking it, it would be nice to track down, as (hopefully) it will be something generic that will help over APKs run in future.

EDIT: One thing that makes tracking down the stack issue hard for me is GDB doesn't seem to behave too well. After setting break points I typically then have to override the memory at which the breakpoint was inserted to restore the previous machine code, on top of this I often find that stepping an instruction at a time just results in it continuing and seg faulting. I don't know if what I am seeing is typical of attempting to debug this with gdb or if it something I am doing wrong with GDB etc.?

 

[crow_riot]

@Steven: yes you'd actually need a opensles for the pandora to get it fully working, but i thought you could just wrap the functions and do nothing, having a skeleton in the meantime.

from the android sources it seems there's a SDL backend for libopensles available, which could be used. but i haven't tried compiling it yet for the pandora.

 

[Rockthesmurf]

@crow_riot: Ah, yes, I see what you are saying. I could give it a go, not sure what bad side affects there will be from stubbing out those couple of functions from OpenSLES, maybe nothing, maybe another nasty crash

 

[Rockthesmurf]

So, had another little look into this, trying to narrow down when the stack stops being traceable (with sound disabled), and narrow it down to a call to glCompileShader, what I am seeing is:


1. Prior to the branch being taken : "bl 0xea74 <glCompileShader>" GDB shows a correct stack trace


2. Stepping over the branch (with "ni") I get this printed to the console: "warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available."


3. The back trace then stops working:

(gdb) bt#0 my_glCompileShader (shader=70001) at compat/gles2_wrappers.c:137#1 0x40565f1c in ?? ()#2 0x40565f1c in ?? ()
I thought this could be due to stack corruption, but then I noted that level 1 on the back trace (#1) points to 0x40565f1c, and this function also appeared on the back trace prior to stepping over glCompileShader:

#0  0x0001c898 in my_glCompileShader (shader=70001) at compat/gles2_wrappers.c:136#1  0x40565f1c in CShaderUtil::CompileShader(unsigned int, char const*) () from ./lib/armeabi-v7a/libcandycrushsaga.so#2  0x40565648 in CShader::Compile(char const*) () from ./lib/armeabi-v7a/libcandycrushsaga.so

So I am unaware why it can no longer work out what function is at this address. Furthermore, I checked the stack pointer ($sp) before and after the branch, and it corectly remains the same. I also dumped the stack ("x/1000xw $sp") before and after, and in both cases I can see all the correct functions in the correct back trace appearing. So to me this is all pointing to the problem actually being with GDB, perhaps the warning about libthread_db is real issue? Perhaps the glCompileShader function ends up using a thread (the graphics driver thread) which confuses GDB and leaves it in a bad state?

Further down the line, I do get a real crash, which again has a bad back trace meaning I can not make head nor tail of it. But after this first glCompileShader call, GDB seems to only want to give me nonsense for a back trace, so unless I can resolve this first bad stack trace problem, it seems hard to diagnose what the real problem is later on.


Notaz, crow_riot, any ideas? I hope it is okay for this thread to be used for this purpose? I don't mind starting another one, but I figure as this is all work on apkenv it would be okay?

 

[notaz]

For libthread-db issue, try installing glibc-dbg package and see if that helps.

As for backtraces stopping to work randomly, it usually happens when gdb reloads it's library list from glibc's one. apkenv uses it's own library loader/linker for Android libraries, so those libraries are not in glibc's list and have to be pushed to gdb manually by calling notify_gdb_of_libraries() (you can do it from gdb itself when needed).

 

[Rockthesmurf]

@Notaz, notify_gdb_of_libraries just fixed a lot of things for me - GDB is becoming usable now!

Progressed past previous issues, getting a crash later on, spent a while looking at it tonight, didn't manage to crack it, will need to have more of a think about it. I'll list the crash below with my understanding so far:


0018cb18 <_ZN6Social4Core5onLogEPKc>:
18cb18: e92d4038 push {r3, r4, r5, lr}
18cb1c: e1a05000 mov r5, r0
18cb20: e59f40cc ldr r4, [pc, #204] ; 18cbf4 <_ZN6Social4Core5onLogEPKc+0xdc>
18cb24: e59f30cc ldr r3, [pc, #204] ; 18cbf8 <_ZN6Social4Core5onLogEPKc+0xe0>
18cb28: e08f4004 add r4, pc, r4
18cb2c: e7943003 ldr r3, [r4, r3]
18cb30: e5933000 ldr r3, [r3]
18cb34: e3500000 cmp r0, #0
18cb38: 13530000 cmpne r3, #0
18cb3c: 1a000020 bne 18cbc4 <_ZN6Social4Core5onLogEPKc+0xac>
18cb40: e3500000 cmp r0, #0
18cb44: 0a000020 beq 18cbcc <_ZN6Social4Core5onLogEPKc+0xb4>
18cb48: ebfc66d7 bl a66ac <_GLOBAL__sub_I_GameStore.cpp-0xb14>
18cb4c: e59f30a8 ldr r3, [pc, #168] ; 18cbfc <_ZN6Social4Core5onLogEPKc+0xe4>
18cb50: e1a01005 mov r1, r5
18cb54: e7944003 ldr r4, [r4, r3]
18cb58: e1a02000 mov r2, r0
18cb5c: e1a00004 mov r0, r4
18cb60: eb02bdf7 bl 23c344 <_ZSt16__ostream_insertIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_PKS3_i>

So this function, Social::Core:nLog(char const*)  calls into std::__ostream_insert (on last line) which then goes onto crash, I believe the reason for this is the first parameter (the output stream) is not valid. This is a normal function (not a member function), so r0 holds the stream pointer, following the dissassembly backwards, suggests the value for r0 is coming from:

18cb5c: r0 = r4

where r4 is:

18cb54: r4 = *( r4 + r3 )

where r3 is:

 18cb4c: r3 = *(18cbfc) which is 0x00001064

and r4 is:

18cb20: r4 = *(18cbf4) which is 0x0011d0ec

18cb28: r4 += $PC (when $PC is 0x18cb28) so r4=0x2A9C14

so r4 becomes:

18cb54: r4 = *(0x00001064 + 0x2A9C14)

18cb54: r4 =*(0x2AAC78) which is 0x001105f0

I guess as it is a reference it'll be getting passed essentially as a pointer, so taking a look at 0x001105f0 shows me:


001105f0 <_ZN7CVectorIN14CSocialManager13SMassLifeGiftEEixEi>:
1105f0: e5903004 ldr r3, [r0, #4]
1105f4: e3a00018 mov r0, #24
1105f8: e0203190 mla r0, r0, r1, r3
1105fc: e12fff1e bx lr

If I demangle that, CVector:perator[](int)

So, first of all, I believe I have probably made a mistake above (will look through it all again tomorrow), as it doesn't make too much sense to me that this would be getting passed through as an output stream. But second of all, is there a better way to track down these literals, without manually adding up a bunch of static addresses? And once the address has been found, one would imagine that the stream will be closed when the program starts, then at a later point it will attempt to open the stream and store the handle pointer into memory, can I see from the disassembly what writes to the memory location? On the PC, using OllyDgb I may well be able to track this down, but at the moment I'm just using assembly output from objdump, and I feel I am missing a more useful tool, something that will let me tie the above together. Find out what is meant to be opening this output stream, which is probably failing due to not knowing where to write to, fix the missing function and press on!

Sorry for the lengthy post (again)..!

 

[M-HT]

In this line




18cb28: e08f4004 add r4, pc, r4

the value of pc is not the address of the instruction (0x18cb28) but address+8 (0x18CB30).

So instead of

18cb28: r4 += $PC (when $PC is 0x18cb28) so r4=0x2A9C14

you get

18cb28: r4 += $PC (when $PC is 0x18cb30) so r4=0x2A9C1C

and so on.

 

[Rockthesmurf]

the value of pc is not the address of the instruction (0x18cb28) but address+8 (0x18CB30).

Thank you! I can re-evaluate based on this knowledge. Hopefully it will make things a bit clearer in terms of seeing where the stream is stored that is being passed to the function, although right now I'm not sure how I will be able to go about tracking down what is meant to be setting up this stream (as it is not like I can look for ASM that is accessing this address, as the ASM will probably be made up of a bunch of loads using PC relative addresses etc.)

 

[Rockthesmurf]

For what it is worth, I have been at the BlackBerry Live conference in Orlando all of this week, so no further progress has been made on my part! Hope to find some time to continue next week..!

 

[notaz]

I've sent another pull request which has already been merged. It has PvZ fixes and sound support (tested on humblebundle version), resource parser fix + fixes for some crashes. Maybe that will resolve some Steven's problems too.

crow_riot: you need to remove GL libs (libEGL.so, libGLESv1_CM.so and libGLESv2.so) from the .pnd as they pull in many dependencies that are not worth dealing with. Those libs are fully wrapped now (more or less) and apkenv will now pretend the libs are present when a game tries to load them.

 

[Rockthesmurf]

@notaz thank you, when I get back from work I'll pull the latest changes and see if I can get any further. I'll also push the latest changes I have.

 

[crow_riot]

thanks for the update notaz.


i'm currently quite busy with family obligations and taking care of the baby boy - i'll hope to get a bit of free time the next days to update and release. maybe your changes also help getting those unfinished modules working i've got lying around...

 

[crow_riot]

have been messing around with rebase but turned out that a forced update to the upstream was easier, have readded my little changes now. since there was no game of thrones yesterday, i'll have time tonite to update (hopefully)