[Import-SIG] New PEP draft: "Simplified Package Layout and Partitioning"

[Nick Coghlan]

On Tue, Aug 16, 2011 at 9:57 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 09:28 AM 8/16/2011 +1000, Nick Coghlan wrote:
>>
>> I'm actually not sure this is a viable approach as currently described
>> in the PEP - most of the existing import machinery assumes that parent
>> modules will exist in sys.modules before child modules are imported.
>
> That assumption isn't being violated as much as you might think.  The PEP
> merely requires that one hold off on creating the parent module instance
> until you're just about to load the child.  This is quite straightforward to
> implement if you process the import iteratively from left to right, as
> CPython (pre-importlib) does.

Ah, good point. Yes, I had missed that.

> In effect, the algorithm is:
>
>   path = sys.path
>
>   for each part of the name:
>       check for an already imported name up to this point
>       if it's already imported:
>            path = module.__path__
>       else:
>            try to find the module (using 'path')
>            if the module is found:
>                add any missing parent modules to sys.modules
>                load the module
>                path = module.__path__
>            else:
>                path = virtual path for the missing module
>
>   if we have a module:
>       return it
>   else:
>       raise ImportError
>
> Very simple, really.  Granted, the "fill in any missing parent modules" is a
> wee bit tricky and reintroduces recursion into the mix.  (See
> http://pastebin.com/6e29v8LR for the full details, including a draft
> implementation of an auto-updating proxy __path__ object.)

The other slightly fiddly bit is coping with the "the module is in
sys.modules but doesn't have a __path__ attribute" case, so the logic
flow isn't *quite* as neat as shown above (the draft version of
_gcd_import appears to deal with this case correctly, though).

> But the only place where the "parent modules are already in sys.modules"
> assumption can be broken here is in find_module() calls -- *not* in
> load_module() or any actual module code.  And this assumption is only broken
> in scenarios where, in today's Python, the import would already have failed
> first.

Yeah, consider my objection with drawn. You may want to elaborate on
this a little in the PEP, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia