[Python-Dev] Relative imports in Py3k

[Nick Coghlan]

On Sat, Sep 25, 2010 at 11:15 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> Hi,
>
> I wonder if situation with relative imports in packages is improved in
> Python 3k or
> we are still doomed to a chain of hacks?
>
> My user story:
> I am currently debugging project, which consists of many modules in one package.
> Each module has tests or other useful stuff for debug in its main
> section, but it is a
> disaster to use it, because I can't just execute the module file and
> expect it to find
> relatives. All imports are like:
>
> from spyderlib.config import get_icon
> from spyderlib.utils.qthelpers import translate, add_actions, create_action

This is almost certainly failing because the directory containing the
spyderlib package isn't on sys.path anywhere (instead, whichever
directory contains the script you executed directly will be in there,
which will be somewhere inside the package instead of outside it). Put
the appropriate directory in PYTHONPATH and these tests should start
working.

> PEP 328 http://www.python.org/dev/peps/pep-0328/  proposes:
>
> from ... import config
> from ..utils.qthelpers import translate, add_actions, create_action

This fails for two reasons:
1. "__main__" is missing the module namespace information PEP 328
needs to do its thing
2. Even if 1 is resolved, PEP 328 will resolve to the same absolute
imports you're already using and likely fail for the same reason (i.e.
spyderlib not found on sys.path)

> But this doesn't work, and I couldn't find any short user level
> explanation why it is
> not possible to make this work at least in Py3k without additional magic.

If you use the -m switch to run your module from the directory that
contains the spyderlib package directory, it will work. The use of -m
provides the module namespace information that PEP 328 needs, while
running from the directory that contains the spyderlib package ensures
it is visible through sys.path.

The one caveat is that the specified module is run as "__main__" and
hence does not exist in sys.modules under its normal name. If it gets
imported by another module, then you will end up with two copies of
the module (one as "__main__" and one under the normal name). This may
or may not cause problems, depending on the nature of the module being
executed.

While PEP 366 describes the boilerplate you can put at the top of your
module to allow a directly executed module to try to find its
siblings, it still requires that PYTHONPATH be set appropriately. And
if you set PYTHONPATH appropriately, then direct execution with
absolute imports will work.

(The explanation of the failures applies for all Python versions that
I am aware of, but the -m based fix only became available in 2.6)
(The impact of various command line options and the PYTHONPATH
environment variable on sys.path are described at
http://docs.python.org/using/cmdline.html)
(The basic import search algorithm is described in the tutorial:
http://docs.python.org/tutorial/modules.html#the-module-search-path)
(And the full gory specification details are in PEP 302, with a few
subsequent tweaks courtesy of PEP 328 and PEP 366).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia