[Python-Dev] Problems with Python's default dlopen flags

[Martin v. Loewis]

"David Abrahams" <david.abrahams@rcn.com> writes:

> It seems to me that extension modules themselves should have a way
> to report to Python that they need to be loaded with RTLD_GLOBAL.

No way; to change Python in this way would be extremely
foolish. Python was using RTLD_GLOBAL until 1.5.1, then this was
changed in 1.5.2 due to bug reports by users. Redhat decided to revert
this change, and consequently people run into problems with the Redhat
Python 1.5.2 installation.

Here is the original problem: A Python application was using both
Oracle and sockets, so it had the Oracle and socket modules
loaded. Unfortunately, both provided an initsocket function (at that
time; today the socket module provides init_socket). It so happened
that the dynamic linker chose the initsocket definition from the
socket module. When Oracle called its own initsocket function, the
call ended up in the Python module, and the application crashed; this
is painful to analyse.

Now, people apparently want to share symbols across modules. Let me
say that I find this desire misguided: Python extension modules are
*not* shared libraries, they *only* interface with the Python
interpreter. If you want to share symbols, use shared libraries: If
modules A.so and B.so have symbols in common, create a shared library
C.so that provides those symbols, and link both A.so and B.so with
this shared library.

Now, people still want to share symbols across modules. For that, you
can use CObjects: Export a CObject with an array of function pointers
in module A (e.g. as A.API), and import that C object in module B's
initialization code. See cStringIO and Numeric for examples.

Now, people still want to share symbols across modules. For that, they
can use sys.setdlopenflags.

It seems that this is a lose-lose situation: you can't please
everybody. In the current state, people that want to share symbols
can, if they really want to. With your proposed change, symbols that
accidentally clash between unrelated extensions cause problems, and
users of those modules can do nothing about it. Hence, the current
state is preferable.

HTH,
Martin