[Python-Dev] Pre-PEP: Redesigning extension modules
ericsnowcurrently at gmail.com
Fri Sep 6 07:47:25 CEST 2013
On Sat, Aug 24, 2013 at 10:12 PM, PJ Eby <pje at telecommunity.com> wrote:
> I haven't had a chance to address this on the import-sig discussion
> yet about ModuleSpec, but I would like to just mention that one
> property of the existing module system that I'm not sure either this
> proposal or the ModuleSpec proposal preserves is that it's possible to
> implement lazy importing of modules using standard reload() semantics.
> My "Importing" package offers lazy imports by creating module objects
> in sys.modules that are a subtype of ModuleType, and use a
> __getattribute__ hook so that trying to use them fires off a reload()
> of the module. Because the dummy module doesn't have __file__ or
> anything else initialized, the import system searches for the module
> and then loads it, reusing the existing module object, even though
> it's actually only executing the module code for the first time.
> That the existing object be reused is important, because once the
> dummy is in sys.modules, it can also be imported by other modules, so
> references to it can abound everywhere, and we wish only for it to be
> loaded lazily, without needing to trace down and replace all instances
> of it. This also preserves other invariants of the module system.
> Anyway, the reason I was asking why reloading is being handled as a
> special case in the ModuleSpec proposal -- and the reason I'm curious
> about certain provisions of this proposal -- is that making the
> assumption you can only reload something with the same
> spec/location/etc. it was originally loaded with, and/or that if you
> are reloading a module then you previously had a chance to do things
> to it, doesn't jibe with the way things work currently.
> That is to say, in the pure PEP 302 world, there is no special status
> for "reload" that is different from "load" -- the *only* thing that's
> different is that there is already a module object to use, and there
> is *no guarantee that it's a module object that was initialized by the
> loader now being invoked*.
In Python 3.3 (#13959) import.reload() was updated to reuse a module's
__loader__, which must now be set. If __loader__ is not set, you get an
AttributeError. If that's a problem we can create a tracker issue and
In Python 3.4 imp.reload() is just an alias to importlib.reload(), but it
works basically the same.
With ModuleSpec things won't work that differently. If you reload such a
module as you described, it will look for __spec__ and call it's reload()
method. If __spec__ is not set, you get an AttributeError. It wouldn't be
that hard to build a spec from the module if need be and then use that.
> AFAICT both this proposal and the ModuleSpec one are making an invalid
> assumption per PEP 302, and aren't explicitly proposing to change the
> status quo: they just assume things that aren't actually assured by
> the prior specs or implementations.
> So, for example, this extension module proposal needs to cover what
> happens if an extension module is reloaded and the module object is
> not of the type or instance it's expecting. Must it do its own
> checking? Error handling? Will some other portion of the import
> system be expected to handle it?
> For that matter, what happens (in either proposal) if you reload() a
> module which only has a __name__, and no other attributes? I haven't
> tested with importlib, but with earlier Pythons this results in a
> standard module search being done by reload(). But the ModuleSpec
> proposal and this one seem to assume that a reload()-ed module must
> already be associated with a loader, location, and/or spec.
> Python-Dev mailing list
> Python-Dev at python.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev