[issue4434] Embedding into a shared library fails

[rb]

New submission from rb <rb-oss-1 at justgohome.co.uk>:

Python cannot be embedded in shared library due to dependency problems
with lib-dynload.

I am implementing a shared library (in C) that implements a specific API
as a dynamically loadable plugin to an application. To implement this
library, I am calling out to Python via the C API.

When my code tries to load a Python module, it fails. This is because
lib-dynload/time.so (for example) cannot resolve symbols that are in
libpython2.5.so because it never looks there.

In this test case I'm trying to import "time", but it seems to fail on
other modules too - presumably anything that is in lib-dynload. It also
fails if I import a pure Python module that then tries to import time.

The error produced is: ImportError:
/usr/lib/python2.5/lib-dynload/time.so: undefined symbol: PyExc_ValueError

>From LD_DEBUG:
     29682:file=/usr/lib/python2.5/lib-dynload/time.so [0];  needed by 
/usr/lib/libpython2.5.so.1.0 [0]
...
     29682:relocation processing: /usr/lib/python2.5/lib-dynload/time.so
     29682:symbol=PyExc_ValueError;  lookup in file=./myprog [0]
     29682:symbol=PyExc_ValueError;  lookup in file=/lib/libdl.so.2 [0]
     29682:symbol=PyExc_ValueError;  lookup in file=/lib/libc.so.6 [0]
     29682:symbol=PyExc_ValueError;  lookup in
file=/lib64/ld-linux-x86-64.so.2 [0]
     29682:symbol=PyExc_ValueError;  lookup in 
file=/usr/lib/python2.5/lib-dynload/time.so [0]
     29682:symbol=PyExc_ValueError;  lookup in file=/lib/libm.so.6 [0]
     29682:symbol=PyExc_ValueError;  lookup in file=/lib/libpthread.so.0 [0]
     29682:symbol=PyExc_ValueError;  lookup in file=/lib/libc.so.6 [0]
     29682:symbol=PyExc_ValueError;  lookup in
file=/lib64/ld-linux-x86-64.so.2 [0]
     29682:/usr/lib/python2.5/lib-dynload/time.so: error: symbol lookup
error: 
undefined symbol: PyExc_ValueError (fatal)


$ nm -D /usr/lib/libpython2.5.so.1|grep PyExc_ValueError
000000000033a7e0 D PyExc_ValueError



$ ldd /usr/lib/python2.5/lib-dynload/time.so
libm.so.6 => /lib/libm.so.6 (0x00002afe014c9000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00002afe0174a000)
libc.so.6 => /lib/libc.so.6 (0x00002afe01967000)
/lib64/ld-linux-x86-64.so.2 (0x0000555555554000)



I am told that the problem is that lib-dynload/*.so should depend on
libpython2.5.so.1.


Test case attached. mylib.c is the library that I'm implementing reduced
to the problem case. myprog.c is a stub program that loads mylib.c to
demonstrate the problem. The "real" myprog would be any third-party
application that I have no control over that expects to be able to
dlopen() mylib.so and call functions within it.


I have been given the following workaround: in mylib.c, before
PyInitialize() I can call dlopen("libpython2.5.so", RTLD_LAZY |
RTLD_GLOBAL);

This works, but I believe that lib-dynload/*.so should depend on
libpython2.5.so.1 so this hack should not be necessary.

I am using Ubuntu 8.04 with Python version 2.5.2-2ubuntu4.1.

----------
components: Library (Lib)
files: mylib.c
messages: 76454
nosy: rb
severity: normal
status: open
title: Embedding into a shared library fails
type: behavior
versions: Python 2.5
Added file: http://bugs.python.org/file12134/mylib.c

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4434>
_______________________________________