[Python-Dev] Pre-PEP: Redesigning extension modules

[Stefan Behnel]

Hi,

thanks for bringing this up. It clearly shows that there is more to this
problem than I initially thought.

Let me just add one idea that your post gave me.

PJ Eby, 25.08.2013 06:12:
> My "Importing" package offers lazy imports by creating module objects
> in sys.modules that are a subtype of ModuleType, and use a
> __getattribute__ hook so that trying to use them fires off a reload()
> of the module.

I wonder if this wouldn't be an approach to fix the reloading problem in
general. What if extension module loading, at least with the new scheme,
didn't return the module object itself and put it into sys.modules but
created a wrapper that redirects its __getattr__ and __setattr__ to the
actual module object? That would have a tiny performance impact on
attribute access, but I'd expect that to be negligible given that the usual
reason for the extension module to exist is that it does non-trivial stuff
in whatever its API provides. Reloading could then really create a
completely new module object and replace the reference inside of the wrapper.

That way, code that currently uses "from extmodule import xyz" would
continue to see the original version of the module as of the time of its
import, and code that just did "import extmodule" and then used attribute
access at need would always see the current content of the module as it was
last loaded. I think that, together with keeping module global state in the
module object itself, would nicely fix both cases.

Stefan