[Python-Dev] Problems with Python's default dlopen flags

[David Abrahams]

Hi,

I'm hoping I can raise some interest in resolving this problem:

By default, Python does not use the RTLD_GLOBAL flag when opening
extension modules. Unfortunately, this breaks many C++ features when
used across modules(http://gcc.gnu.org/faq.html#dso). It also causes
these features to fail across the boundary between modules and any
shared library they might be linked to. This is a key arrangement for
Boost.Python: its extension modules all make use of a common shared
library.

I realize that we can change how modules are loaded using
sys.setdlopenflags(), imputils, etc., but all that puts knowledge in the
wrong place: the importer needs to know the special way to import a
given module. It seems to me that extension modules themselves should
have a way to report to Python that they need to be loaded with
RTLD_GLOBAL.

It appears that Boost.Python is not the only project that is having a
problem with the way this works
(http://aspn.activestate.com/ASPN/Mail/Message/xml-sig/1040230), so
perhaps there's a good reason think about alternatives?

I am not by any means an expert in GNU dynamic loading, so I only have
what are probably crackpot ideas about how to address this (if, indeed
Python is using the correct default). Making a show of constructive
suggestion:

Have Python look for a special symbol, say init<module>_dlopenflags().
If it's found, it's called. If the result doesn't match the current
dlopenflags the module is dlclose()d and re-opened with the requested
flags.

Thoughts?

-Dave

+---------------------------------------------------------------+
                  David Abrahams
      C++ Booster (http://www.boost.org)               O__  ==
      Pythonista (http://www.python.org)              c/ /'_ ==
  resume: http://users.rcn.com/abrahams/resume.html  (*) \(*) ==
          email: david.abrahams@rcn.com
+---------------------------------------------------------------+