[Python-Dev] Unload a module written in C

[Victor Stinner]

Hi,

I am trying to understand why I am unable to unload my faulthandler
module (implemented in C). Antoine pointed me
_PyImport_FixupExtensionObject() comment which gave me a first clue:

   Modules which do support multiple initialization set their m_size
   field to a non-negative number (indicating the size of the
   module-specific state). They are still recorded in the extensions
   dictionary, to avoid loading shared libraries twice.

Ok, so I changed the size from -1 to 0, and so the m_free callback was
called at exit. Nice. This is thanks to PyImport_Cleanup() which clears
my module attributes.

--

But if I do

  import faulthandler
  del sys.modules['faulthandler']
  del faulthandler

the module is never unloaded. There is another secret reference in the
interpreter state: state->modules_by_index. This list is cleared at exit
(by PyInterpreterState_Clear), but not my module attributes, and some of
them are functions pointing to the module.

My module attribute are not cleared at exit because PyImport_Cleanup()
clears only modules from sys.modules, and my module is no more
referenced in sys.modules.

The workaround to unload the module is to explicitly clear its
attributes:

  import faulthandler
  del sys.modules['faulthandler']
  faulthandler.__dict__.clear()
  del faulthandler

--

Is there a bug somewhere, or do I misunderstood something important?

Note: I implemented m_traversal, but it is not revelant here (you can
consider that my module only contains functions).

Victor