[New-bugs-announce] [issue16031] relative import headaches

[James Hutchison]

New submission from James Hutchison:

This might even be a bug I've stumbled upon but I'm listing it as an enhancement for now.

I really feel that relative imports in Python should just work. Regardless of the __name__, I should be able to import below me. Likewise, it should work even if I've already done an import into the symbol table. It adds additional work to us as a developer to have to do some pythonpath or code gymnastics to get something rather trivial working. Additionally, the import errors from circular imports add another challenge to work around. In C/C++ you can force it to import a file once and only once, why can't Python work the same way?

Take the following example set-up:
startPoint.py
subModule1
   /__init__.py
   /a.py
   /b.py
   /tests
      /__init__.py
      /test_a.py

a's code:
print("in a");
from subModule1 import b

b's code:
print("in b");
from subModule1 import a

test_a.py's code:
print("myname:",__name__);
from .. import a

startPoint.py is empty, and the __init__.py files are also empty.

If I run a PyDev unit test on test_a.py this is what I get:

Finding files... done.
Importing test modules ... myname: subModule1.tests.test_a
in a
in b
myname: test_a
Traceback (most recent call last):
  File "C:\eclipse\plugins\org.python.pydev_2.6.0.2012062818\pysrc\pydev_runfiles.py", line 432, in __get_module_from_str
    mod = __import__(modname)
  File "C:\myfolder/relativeImportTest/subModule1/tests\test_a.py", line 6, in <module>
    from .. import a
ValueError: Attempted relative import in non-package

Clearly in this case, the exception given doesn't make any sense. I have __init__.py, the error says the relative import line is failing, and the error says it's because I'm in a non-package. Except, I'm in a package. It seems to go through the a_test.py file twice, even though I never explicitly import it. The first time through, I'm clearly in a package. The second time through, my name is NOT __main__ but yet I'm apparently no longer a package, which is where it fails.

Now if I change:
"from subModule1 import b" to "import subModule1.b"
and
"from subModule1 import a" to "import subModule1.a"

then everything works. But then that means I have to reference everything by the full name in my submodules. In this example, there's clearly a circular reference between a and b that wouldn't work anyways.

So lets change some things.

Now:
a.py:
import subModule1.b

b.py:
from subModule1 import a

Now the circular reference is gone between a and b. I really don't like having to do this as a means to work around a circular reference because it forces me to vary the import style of one file to another.

If we try the test code again however, it gets the same problem. If I swap which file does the relative import, then it works.

So lets make one last change:

test_a.py:
import subModule1.b # added
from .. import a

This will work, seemingly magically. It only runs the code in test_a.py once. Recall that the code in a.py is "import subModule1.b"

So basically this brings up several issues:
1. "import a.b" isn't the same as "from a import b" by more than how you reference it in the code
2. submodules are re-imported as non-module without ever importing them if you import their parent module relatively. If this is documented I don't know where.
3. import order can matter drastically to if a code runs or not for seemingly magical reasons.

And back when I was a beginner Python user, the naming convention of the full path really threw a monkey wrench in my code when I would try to move a select number of files from one project to another, or would try relative imports. If relative imports cause such headaches with circular references then I should generally stick to the full module path when referencing things. But if the full module path isn't portable then I should use relative imports.

Likewise, if I run as a PyDev unitTest, my module name is NOT __main__, so special path checks for __main__ won't work

I think the bottom line is that the import system gave me headaches as a beginner user and as an advanced user it still does every now and then so it really should be changed to something more intuitive or forgiving. I really shouldn't have to sit and think "how do I reference a function in the file just one directory level below mine?"

If there is already some magic bullet for this then it should probably be more visible.

----------
components: Interpreter Core
messages: 171208
nosy: Jimbofbx
priority: normal
severity: normal
status: open
title: relative import headaches
type: enhancement
versions: Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16031>
_______________________________________