[Python-Dev] Undefined dlopen When Building Module On Android

[Cyd Haselton]

Replies in body of message for clarity:

On Sat, Jan 24, 2015 at 5:08 PM, Frank, Matthew I
<matthew.i.frank at intel.com> wrote:
> Android's dlopen() works slightly differently than the normal Unix dlopen() in at least two different ways.  I haven't seen your particular problem, but that's probably because I'm cross-building CPython (building everything I need on a Linux machine, and then copying the install directory to the Android machine.)

Yeah...I have a Linux machine but it currently resides as a gunzipped
tarball on an external drive...I lack the space to decompress and use
it for building purposes.

> (1) When building for Android you need to explicitly include the "-ldl" flag on the command line for the link step.  (In Linux the dlopen() routine is included in libc, so "-ldl" is not necessary).  I suspect that some part of your distutils was not linked correctly.  (Since I'm cross-building I've never run distutils on the Android side, which is probably why I've not seen this.)  Your best bet would be to run under a debugger and figure out which line of C code in which .so file is throwing the error.  Then going back to the build scripts and looking at how that .so file is getting linked.  (For an example of someone else having a similar problem see for example, http://stackoverflow.com/questions/25846927/git-built-on-android-throws-undefined-reference-to-dlopen-error).

Funny you should reference that post as I'm the one who started it; I
ported git, which required curl and openssl ports, a while ago
and...as I think I mentioned in that post...I had to go back and
re-port openssl, making sure to throw in -ldl wherever possible.

Python (which I'm building in the same environment in which I built
openssl, git and curl) is a different beast altogether.  Environment
variables that make it to the Makefile don't make it into the modules
built by setup.py, which is run by distutils, which I wasn't aware was
built alongside Python until you mentioned it just now.  Then there
are the modules in Setup...which are built or not...seemingly
independently of whether or not the line specifying the module is
commented out or not; the hack workaround is to put every module that
should be excluded in setup.py...even if it is commented out in Setup.

Obviously I'm still trying to get a handle on the Python build process.


> (2) The other possibility has to do with a quirk in Android's dlopen() implementation.  On most legacy Unix systems (including Linux) when you dlopen() a library Z then all the already loaded libraries A, B, C, ... are searched for any dependences of Z (even if Z was not explicitly linked against any of A, B, C,...).  On Android (perhaps for security reasons) that's not true, if Z depends on A, then you need to have "-lA" when you link Z.  An example (and patch) for this problem is http://bugs.python.org/issue21668.

Thanks for the bit of info above...very useful.  I've included -lc
-ldl in the requisite places in Setup, and setup.py...although it's
entirely possible I've not added it to places that need it given that
I'm still struggling to understand the build process.

Right now I'm at the point where running configure && make finishes
successfully but throws warnings during the module build and import
process.  Running make install fails completely with the same
'undefined reference to dlopen' because, for some reason, make install
requires the sharedmods target to be rebuilt again, which fails
because setup.py fails when importing distutils.core.  Why it doesn't
do the same when running make is beyond me.

Cyd