[ python-Bugs-1510172 ] Absolute/relative import not working?

[SourceForge.net]

Bugs item #1510172, was opened at 2006-06-22 05:35
Message generated for change (Comment added) made by ncoghlan
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1510172&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: Mitch Chapman (mitchchapman)
Assigned to: Nick Coghlan (ncoghlan)
Summary: Absolute/relative import not working?

Initial Comment:
Trying to import from a module using dotted import syntax produces 
this exception:

ValueError: Relative importpath too deep

This behavior has been confirmed on Mac OS X 10.4 using the Python 
2.5b1 disk image; and on CentOS 4 using the Python 2.5b1 source 
tarball.

The exception is raised regardless of whether the PYTHONPATH 
environment variable can see the toplevel directory of the package 
being tested; regardless of whether the import is performed from an 
interactive Python session or from a script invoked from the command 
line; and regardless of whether the main script starts with

from __future__ import absolute_import

To test, I tried to re-create the package structure used as an example 
in PEP 328.  (See attachments.)

Most of the files were empty, except moduleX.py and moduleY.py:

#moduleX.py:
from __future__ import absolute_import

from .moduleY import spam

#moduleY.py:
spam = "spam"

According to the PEP, if should be possible to import moduleX without 
error.  But I get the above exception whenever I try to import moduleX 
or to run it from the command line.

$ python2.5 moduleX.py 
Traceback (most recent call last):
  File "moduleX.py", line 3, in <module>
    from .moduleY import spam
ValueError: Relative importpath too deep


Is this a usage/documentation error?


----------------------------------------------------------------------

>Comment By: Nick Coghlan (ncoghlan)
Date: 2006-06-27 21:46

Message:
Logged In: YES 
user_id=1038590

All that said, PEP 328 currently says that "from ...sys
import path" should be legal from moduleX in the example.

I tried it, and it currently fails - the ValueError gets
raised as soon as the number of dots in the relative path
exceeds the number of dots in __name__, instead of treating
a single excess dot as requesting an absolute import
instead. (All of the other examples in the PEP work as
specified when moduleX and subpackage1 are imported rather
than executed directly)

I believe fixing this would also fix Mitch's problem - an
explicit relative import from __main__ would be treated as a
top level import.

I also have an idea regarding the -m switch that I will
raise on python-dev.

----------------------------------------------------------------------

Comment By: Nick Coghlan (ncoghlan)
Date: 2006-06-27 21:34

Message:
Logged In: YES 
user_id=1038590

The issue isn't actually unique to the -m switch - the
problem is that relative imports are based on __name__, and
in the main module, __name__ always has the value
'__main__'. Hence, relative imports currently can't work
properly from the main module of an application, because the
main module doesn't know where it really fits in the Python
module namespace (this is at least fixable in theory for the
 main modules executed through the -m switch, but directly
executed files and the interactive interpreter are
completely out of luck).

With the old implicit relative imports this behaviour is
masked by the fact that executing a module puts that
module's directory on sys.path. When you execute a module in
a package directly, it actually imports its sibling modules
as top-level modules. The fact that the -m switch doesn't
allow this to occur is a deliberate design decision (putting
package directories on the system level path is a bad idea
because you can get multiple copies of the same module under
different names).

You should be able get something similar to the old implicit
relative import behaviour by sticking the following at the
top of your module (before doing any relative imports):

if __name__ = '__main__':
   from os.path import dirname
   __path__ = [dirname(__file__)]
   del dirname

This makes the relative import machinery treat your main
module as a package. The problem with this workaround is
that, just like the old situation with implicit relative
imports from the main module, you may end up with two
different copies of the sibling modules in sys.modules. One
copy will be '__main__.foo' while the other will be
'package.foo' (with implicit relative imports, the first
copy would have been a top level module called 'foo').

When I went to document this, I got stuck on the fact that
section 6 of the tutorial hasn't been updated for PEP 328 at
*all*. And the language ref palms the details off on to
Guido's packages essay. So describing the limitation in the
real documentation entails documenting PEP 328 properly in
the first place (which I'm *not* volunteering to do :).

However, I'll add a section to PEP 328 about '__main__' and
relative imports (including the workaround to get something
similar to the old behaviour back).


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2006-06-27 14:41

Message:
Logged In: YES 
user_id=33168

http://mail.python.org/pipermail/python-dev/2006-June/066197.html

Nick you made mention of changing the docs, but I'm not sure
if you did or not.  Could you address this bug?  Thanks.

----------------------------------------------------------------------

Comment By: Thomas Wouters (twouters)
Date: 2006-06-23 23:52

Message:
Logged In: YES 
user_id=34209

See the discussion started at:
http://mail.python.org/pipermail/python-dev/2006-June/066161.html

It's not a bug in 328 or 338 (the PEP that adds the -m
switch for packages), but in the interaction between the
two. I don't think this will be fixed for 2.5, since there
is no obvious fix. If it hurts when you press there, don't
press there. Either don't use -m for packaged modules, or
have the packaged module only use absolute imports. (But
don't be surprised if the script-module is imported twice,
once as __main__ and once as the module itself. That's a
whole other bug/feature.)





----------------------------------------------------------------------

Comment By: Mitch Chapman (mitchchapman)
Date: 2006-06-22 09:57

Message:
Logged In: YES 
user_id=348188

Hm... but the same error occurs if one tries to import moduleX from an 
interactive Python session, or from a sibling module.

In other words, in 2.5b1 any module which uses relative imports can be used 
only as a fully-qualified member of a package.  It cannot be imported directly 
by a sibling module, and it cannot be used as a main module at all:

$ python2.5 -m package.subpackage1.moduleX
...
    from .moduleY import spam
ValueError: Relative importpath too deep

Given other efforts (PEP 299; PEP 338) to make it easier to use modules both 
as mainlines and as imports, I still think this is a bug.


----------------------------------------------------------------------

Comment By: Žiga Seilnacht (zseil)
Date: 2006-06-22 08:59

Message:
Logged In: YES 
user_id=1326842

I think this is a usage error. The problem is that
you run moduleX as a script. This puts the module's
directory as the first entry in sys.path (see
http://docs.python.org/dev/lib/module-sys.html#l2h-5058
for detais).
As a consequence, moduleX is recognised as a top
level module, not as part of a package.

If you want to test relative import, try opening an
interactive shell in the directory where `package`
resides, and type:

>>> from package.subpackage1 import moduleX
>>> moduleX.spam
'spam'


----------------------------------------------------------------------

Comment By: Mark Nottingham (mnot)
Date: 2006-06-22 07:16

Message:
Logged In: YES 
user_id=21868

Seeing the same behaviour; OSX with the installer.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1510172&group_id=5470