[Python-Dev] Broken import?

[Guido van Rossum]

On Mon, Mar 30, 2009 at 6:17 PM, Guido van Rossum <guido at python.org> wrote:
> [Adding python-dev. I'm quoting the entire original message.]
>
>> On Thu, Mar 19, 2009 at 6:40 PM, Fredrik Lundh <fredriklundh at google.com> wrote:
>>> PS. Is it just me, or is import broken in 3.0?  Consider this:
>>>
>>> $ more package\a.py
>>> print("in a")
>>>
>>> import b
>>>
>>> def a():
>>>    print("here")
>>>
>>> def main():
>>>    b.b()
>>>
>>> $ more package\b.py
>>> print("in b")
>>>
>>> import a
>>>
>>> def b():
>>>    a.a()
>>>
>>> Under 2.X, this prints "in a" "in b" and "here", as expected.  Under
>>> 3.0, using the "from . import" form for relative imports, it bombs out
>>> with a:
>>>
>>> in a
>>> in b
>>> Traceback (most recent call last):
>>>  File "main.py", line 1, in <module>
>>>    from package import a
>>>  File "package/a.py", line 3, in <module>
>>>    from . import b
>>>  File "package\b.py", line 3, in <module>
>>>    from . import a
>>> ImportError: cannot import name a
>>>
>>> Sure, it's a recursive import, but there's no recursive dependency
>>> here - nobody will access the module contents until the main program
>>> calls the library.  What am I missing?
>
> On Mon, Mar 30, 2009 at 5:44 PM, Guido van Rossum <guido at python.org> wrote:
>> I reproduced this, but it seems to have more to do with "from . import
>> ..." than with the Python version. If I add the "from ." before each
>> of the imports, "python -c 'import p.a' " fails with roughly the above
>> traceback for any version of Python that supports this syntax, while
>> without that it passes for any 2.x.
>>
>> If I use the "from ." syntax in a.py but not in b.py, "import p.a"
>> passes but "import p.b" fails.
>>
>> I'll see if anyone present at the sprints has a clue.
>
> Made some progress. Anything using "from <whatever> import b" (where
> <whatever> is either '.' or 'p') will fail when b's import is not
> completed. OTOH using "import p.b" works. I reduced it to:
>
> p/a.py == "from p import b"
> p/b.py == "import a"
> python -c "import p.b"
>
> The reason seems to be that until the outermost import (in this case
> p.b) is completed, while sys.modules has the (incomplete) modules 'p',
> 'p.a' and 'p.b', the attributes p.a and p.b aren't added until after
> their import is completed. Which it isn't during recursive import.
> Apparently 'from <anything> import <something>' looks for the
> <something> attribute in the <parent> object. This is because
> "from...import" can also be used to import objects other than modules
> (e.g. "from M import C"). I'm guessing that setting the attribute is
> delayed until the import is totally complete, because upon a failed
> import we remove the half-imported module object from sys.modules, but
> apparently we didn 't want to be in the business of removing the
> attribute from the parent package, so that's only set after the import
> is deemed successful.
>
> At least, this is my hypothesis, thinking about it -- I might look at
> the code later. :-)
>
> The most portable solution is to avoid "from...import" and instead
> write something like
>
> import p.b as b

So it turns out that "from X import Y" compiles into this bytecode:

              0 LOAD_CONST               0 (-1)
              3 LOAD_CONST               1 (('Y',))
              6 IMPORT_NAME              0 (X)
              9 IMPORT_FROM              1 (Y)
             12 STORE_NAME               1 (Y)
             15 POP_TOP

The first three opcodes (through IMPORT_NAME) call __import__('X',
None, None, ('Y',)) and push the result on top of the stack; this
result is the toplevel package X. The IMPORT_FROM opcode is
essentially a getattr call that turns an AttributeError into an
ImportError exception. I changed p/a.py into

p = __import__('p', None, None, ['b'])
print(p.b)

and confirmed that it fails on the print() line in p.b.

Does anyone feel that this ought to be fixed?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)