[Import-SIG] PEP proposal: Per-Module Import Path

[Nick Coghlan]

On 2 Aug 2013 13:34, "Eric Snow" <ericsnowcurrently at gmail.com> wrote:
>
>
>
>
> On Thu, Aug 1, 2013 at 6:56 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>>
>> On 2 Aug 2013 00:44, "Eric Snow" <ericsnowcurrently at gmail.com> wrote:
>> > This is pretty much exactly what I've been thinking about since
PyCon.  The only difference is that I have a distinct ModuleSpec class and
modules would get a new __spec__ attribute.
>>
>> And we can quit adding ever more magic attributes directly to the module
namespace. I like it.
>
> Yeah, that was part of what lead me to the idea.  This could be taken to
some pretty great lengths (I've given it a lot of thought), but I'm trying
hard to not do too much at once.  I wasn't even planning on pursuing
ModuleSpec until 3.5, much less any of my more drastic ideas.
>>
>> With that model, things might look vaguely like:
>>
>> 1. Finders would optionally offer "get_module_spec" (although a better
name would be nice!)
>
> How about "find_module"? <.5 wink>  Actually, I'm pretty sure this can be
done in a backward-compatible way (in not too much time I've roughed out an
implementation that should work).  I would rather not introduce more API to
the import system, but if that's preferable to hijacking (or improving
<wink>) find_module() then I can live with that.  However, given the crowd
that takes advantage of the import system APIs, I wouldn't consider the
change disruptive as long as it's backward compatible.
>
> This would also allow us to deprecate PathEntryFinder.get_loader() which
we wouldn't have needed if we'd had something like ModuleSpec.

If you can make find_module handle this in a backwards compatible way, cool
:)

>>
>> 2. Specs would have a load() method for the import system to call that
optionally accepted an existing module object (this would then cover
reload).
>
> That's been my plan from the get-go.  Good call on the reload case.
>>
>> 3. The responsibility for checking the sys.modules cache would move to
the import system.
>
> To me it makes sense to go even further.  ModuleSpec could easily take
over a bunch of the responsibilities of loaders, particularly related to
the management of the module objects.
>
> Also, Loader.init_module_attrs() and importlib.util.module_to_load()
could be pulled before the 3.4 release (since they are new in 3.4).  It
would stink if we found we no longer needed them after they get locked in
by the release.  Note, however, that they can co-exist with ModuleSpec just
fine so it's not as big a deal.
>>
>> 4. We'd create a "SpecLoader" to offer backwards compatibility in the
old __loader__ attribute.
>
> Interesting.  I had anticipated loaders still sticking around, still
exposed by module.__loader__ and filling most of their current role,
especially with regard to the optional PEP 302 APIs.  I suppose we could
deprecate the __loader__ attribute, and maybe even __package__, in favor of
__spec__, but I don't think there's any rush to do so before Python 4000.

I was thinking of finders returning customised types for module specs, but
I guess you could get the same effect defining a new "exec_module" API on
loaders.

>>
>> Slight(!) tangent from the original problem, but a worthwhile
refactoring issue to tackle, I think :)
>
> Yeah, even if it proves too big a change for 3.4 and we take some other
approach for indirections, I think there's a lot to gain from separating
the module specification from the module and from the loader.  I've
attached a patch that does the bare minimum of what I think we'd want from
ModuleSpec.  I'll probably flesh out more of my ideas for it later.
>
> Of course, I don't want anything here to get in the way of the .ref PEP
which I think has more concrete value.  So if this tangent threatens any
chance at getting indirection files for 3.4, I'd rather defer any effort on
these extras until 3.5 in favor of a simpler (if less desirable) approach.

I suspect ref files will be an easier sell with an elegant way to handle
the indirection tracking. I'm not aware of anyone that actually *likes* the
current amount of work loaders have to do, it's just that we only figured
that out with the benefit of hindsight :)

Cheers,
Nick.

>
> -eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/import-sig/attachments/20130802/c70d45c6/attachment-0001.html>