[docs] [issue16194] imp.load_dynamic imports wrong module when called several times on a multi-module .so

Václav Šmilauer report at bugs.python.org
Thu Oct 11 18:19:25 CEST 2012

Václav Šmilauer added the comment:

I found the cause of the behavior (perhaps it is common knowledge, but I am new to python source); imp.load_dynamic calls the following functions

     Python/import.c: imp_load_dynamic (http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l1777)
     Python/importdl.c: _PyImport_LoadDynamicModule (http://hg.python.org/cpython/file/ad51ed93377c/Python/importdl.c#l23)
     Python/import.c: _PyImport_FindExtensionObject (http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l525)

where the last one uses the extensions object (http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l32), which is explained at http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l449

       Magic for extension modules (built-in as well as dynamically
       loaded).  To prevent initializing an extension module more than
       once, we keep a static dictionary 'extensions' keyed by module name
       (for built-in modules) or by filename (for dynamically loaded
       modules), containing these modules. A copy of the module's
       dictionary is stored by calling _PyImport_FixupExtensionObject()
       immediately after the module initialization function succeeds.  A
       copy can be retrieved from there by calling

The fact that extensions are keyed by file name explains why opening the .so through symlink does not return the old extension object:

     # foo.so
     # bar.so -> foo.so (works for both symlinks and hardlinks)
     imp.load_dynamic("bar","bar.so") # will return the bar module

I will investigate whether marking the module as capable of multiple initialization could be a workaround for the issue -- since the quoted comment further says (http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l459):

       Modules which do support multiple initialization set their m_size
       field to a non-negative number (indicating the size of the
       module-specific state). They are still recorded in the extensions
       dictionary, to avoid loading shared libraries twice.

To fix the issue, I suggest that the *extensions* dict is keyed by (filename,modulename) tuple for dynamically loaded modules. This would avoid any ambiguity. Grepping through the code shows that the *extensions* object is only accessed from Python/import.c, therefore regressions should be unlikely. What do you think?


Python tracker <report at bugs.python.org>

More information about the docs mailing list