[Python-Dev] PEP 366 - Relative imports from main modules

[Brett Cannon]

On 7/9/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Brett Cannon wrote:
> > On 7/8/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> As with the current ``__name__`` attribute, setting ``__package__`` will
> >> be the responsibility of the PEP 302 loader used to import a module.
> >> Loaders which use ``imp.new_module()`` to create the module object will
> >> have the new attribute set automatically to
> >> ``__name__.rpartition('.')[0]``.
> >
> > Is this really the best semantics for this?  Let's say I have
> > A/B/__init__.py and A/B/C.py.  With these semantics I would have A.B
> > having __package__ be 'A' and A.B.C having 'A.B'.
> >
> > While I agree that the A.B.C setting is correct, is the A.B value what
> > is truly desired?  Is an __init__ module really to be considered part
> > of the above package?  I always viewed the __init__ module as part of
> > its own package.  Thus I expected A.B to have __package__ set to
> > 'A.B'.
>
> Good point - PEP 328 makes it explicit that __init__.py is treated like
> any other module in the package for purposes of relative imports, so the
> semantics you suggest are the ones required. I hadn't actually thought
> about this case, as it wasn't relevant when the new attribute applied
> only to the main module.
>
> However, those semantics mean that we won't be able to automatically add
> the new attribute inside imp.new_module(), as that function doesn't know
> whether or not the new module is a package.
>

Good point.

> > Beyond just what I expected, the reason I bring this up is that if
> > __package__ had the semantics I am suggesting, it is trivial to
> > discover what modules are the package __init__ modules (as
> > ``__package__ == __name__``) compared to being a submodule
> > (``__package__ and __package__ != __name__``).  As of right now you
> > can only do that if you examine __file__ for __init__.py(c), but that
> > is highly dependent on how the module was loaded.  It might be nice if
> > what kind of module (top-level, package, or submodule) something is
> > based on its metadata.
>
> This part of the argument isn't relevant though, as it's already trivial
> to determine whether or not a module is a package by checking for a
> __path__ attribute. That's what PEP 302 specifies, and it is how the
> relative import machinery currently determines whether to use __name__
> or __name__.rpartition('.')[0] as the base for relative imports.
>

=)  The lesson here is be careful when emailing on vacation as you
might not think everything through.  =)

> Given the above limitations, I propose that we document the new
> attribute as follows:
>
> "If the module global __package__ exists when executing an import
> statement, it is used to determine the base for relative imports,
> instead of the __name__ and __path__ attributes.

That's fine.  __path__ actually isn't used to resolve relative imports
into absolute ones anyway; it's used only as a substitute to sys.path
when importing within a package.


> This attribute may be
> set by the interpreter before a module is executed - whether or not it
> is set automatically in a given module is implementation dependent."
>
> And for the CPython implementation, I propose that we set the new attribute:
>    1. When the main module is executed using the -m switch
>    2. When a relative import is executed and the attribute is not yet set
>
> This will allow any module which uses relative imports to benefit from
> the micro-optimisation of caching the package name in normal modules
> (regardless of how the module gets loaded), as well as allowing relative
> imports from the main module (which is the main goal of the PEP).
>
> With the way PEP 302 hands off creation of the module and execution of
> its code to the loader objects, I don't see any way to guarantee that
> __package__ will always be set - this seems like a reasonable compromise.

It could be set after the import, but you are right that the loaders
will not necessarily set it before executing code.  You might be able
to use the reload requirement of using an existing dict if the module
is in sys.modules somehow, but that just sounds like asking for
trouble.

-Brett