[Import-SIG] Thoughts on cleaner reloading support
ncoghlan at gmail.com
Thu Sep 19 10:14:34 CEST 2013
On 19 September 2013 15:34, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> On Mon, Sep 2, 2013 at 7:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> For a while, I was thinking we could design the import system to "just
>> figure it out", but now I'm thinking a selection of read/write
>> properties on spec objects may make more sense:
>> These would all default to True, but loaders and modules could
>> selectively turn them off.
> 2 things:
> 1. These make more sense to me on the loader (though perhaps exposed on the
They can't go on the loader, since even pure Python modules can
violate them. For example, if your Python module does the following
unconditionally, then it no longer supports in-place reloading:
sys.modules[__name__] = obj
So "allow" may be the wrong prefix. "handles" is probably better:
For backwards compatibility, these would all default to True, but
loaders and modules would now have the ability to opt-out.
This should probably go hand-in-hand with a new "post import hooks"
PEP, since an "atimport" hook with the following signature might be
useful to allow other modules to properly handle reloading of
def handle_foo_import(mod, reloaded):
# If you need to do something when the module is destroyed:
# weakref.finalize(mod, my_callback)
Also, something neat about weakref.finalize, is that it means Python
effectively supports module destructors!
import weakref, sys
mod = sys.modules[__name__]
# implicit reference to the module globals from the function body
Modules already support weak references in 3.4, and weakref.finalize
is new in 3.4 as well.
> 2. These (and other related attributes) may be easier to digest if bundled
> into a LoaderCapabilities named tuple.
Aside from it needing to vary by module rather than by loader, I'd
also be OK with:
> For these attributes to be useful to the import system we would have to have
> several other module registries akin to sys.modules (or one registry that
> manages the extra info).
I was thinking it would be strictly warnings based, so you could still
*try* these things, you'd just have to deal with the consequences.
>> They would also be advisory
> Ah, here's where you were talking about "advisory" APIs. :)
>> rather than enforced via all possible
>> import state manipulation mechanisms. New functions in importlib.util
>> could provide easier alternatives to directly manipulating
>> - importlib.util.reload (replacement for imp.reload that checks the
>> spec allows reloading)
>> - importlib.util.unload (replacement for "del
>> sys.modules[module.__name__]" that checks the spec allows unloading,
>> and also unloads all child modules)
>> - importlib.util.reimport (replacement for
>> test.support.import_fresh_module that checks the spec of any existing
>> sys.module entry allows reimporting a parallel copy)
> At moments like this I keep thinking about PEP 406... :)
Yup, that's definitely relevant. It would probably be good to sort out
the thread-local context version of PEP 406 for 3.5 :)
>> One of these is not like the others... aside from the existing
>> extension module specific mechanism defined in PEP 3121, I'm not sure
>> we can devise a general *loader* level API to force imports for a
>> particular name to fail in a subinterpreter. So this concern probably
>> needs to be ignored in favour of a possible future C API level
> Again, great write-up. I think you nailed it.
Yeah, the deep dive with Stefan into the extension loader
implementation greatly clarified my thinking on a lot of things :)
Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
More information about the Import-SIG