[Import-SIG] Thoughts on cleaner reloading support
Nick Coghlan
ncoghlan at gmail.com
Thu Sep 19 10:14:34 CEST 2013
On 19 September 2013 15:34, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> On Mon, Sep 2, 2013 at 7:53 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> For a while, I was thinking we could design the import system to "just
>> figure it out", but now I'm thinking a selection of read/write
>> properties on spec objects may make more sense:
>>
>> allow_reload
>> allow_unload
>> allow_reimport
>> allow_subinterpreter_import
>>
>> These would all default to True, but loaders and modules could
>> selectively turn them off.
>
> 2 things:
>
> 1. These make more sense to me on the loader (though perhaps exposed on the
> spec).
They can't go on the loader, since even pure Python modules can
violate them. For example, if your Python module does the following
unconditionally, then it no longer supports in-place reloading:
import sys
sys.modules[__name__] = obj
So "allow" may be the wrong prefix. "handles" is probably better:
handles_reload
handles_unload
handles_reimport
handles_subinterpreter_import
For backwards compatibility, these would all default to True, but
loaders and modules would now have the ability to opt-out.
This should probably go hand-in-hand with a new "post import hooks"
PEP, since an "atimport" hook with the following signature might be
useful to allow other modules to properly handle reloading of
dependencies:
@importlib.atimport("foo")
def handle_foo_import(mod, reloaded):
# If you need to do something when the module is destroyed:
# weakref.finalize(mod, my_callback)
Also, something neat about weakref.finalize, is that it means Python
effectively supports module destructors!
import weakref, sys
mod = sys.modules[__name__]
def del_this():
# implicit reference to the module globals from the function body
weakref.finalize(mod, del_this)
Modules already support weak references in 3.4, and weakref.finalize
is new in 3.4 as well.
> 2. These (and other related attributes) may be easier to digest if bundled
> into a LoaderCapabilities named tuple.
Aside from it needing to vary by module rather than by loader, I'd
also be OK with:
can_handle.reload
can_handle.unload
can_handle.reimport
can_handle.subinterpreter_import
> For these attributes to be useful to the import system we would have to have
> several other module registries akin to sys.modules (or one registry that
> manages the extra info).
I was thinking it would be strictly warnings based, so you could still
*try* these things, you'd just have to deal with the consequences.
>> They would also be advisory
>
>
> Ah, here's where you were talking about "advisory" APIs. :)
>
>> rather than enforced via all possible
>> import state manipulation mechanisms. New functions in importlib.util
>> could provide easier alternatives to directly manipulating
>> sys.modules:
>>
>> - importlib.util.reload (replacement for imp.reload that checks the
>> spec allows reloading)
>> - importlib.util.unload (replacement for "del
>> sys.modules[module.__name__]" that checks the spec allows unloading,
>> and also unloads all child modules)
>> - importlib.util.reimport (replacement for
>> test.support.import_fresh_module that checks the spec of any existing
>> sys.module entry allows reimporting a parallel copy)
>
>
> At moments like this I keep thinking about PEP 406... :)
Yup, that's definitely relevant. It would probably be good to sort out
the thread-local context version of PEP 406 for 3.5 :)
>> One of these is not like the others... aside from the existing
>> extension module specific mechanism defined in PEP 3121, I'm not sure
>> we can devise a general *loader* level API to force imports for a
>> particular name to fail in a subinterpreter. So this concern probably
>> needs to be ignored in favour of a possible future C API level
>> solution.
>
> Again, great write-up. I think you nailed it.
Yeah, the deep dive with Stefan into the extension loader
implementation greatly clarified my thinking on a lot of things :)
Cheers,
Nick.
--
Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
More information about the Import-SIG
mailing list