Make `import a.b.c as m` is equivalent to `m = sys.modules['a.b.c']`
Hi there. I asked a question <http://stackoverflow.com/questions/41845671/import-as-in-python-3> on Stackoverflow: (Pdb) import brain.utils.mail (Pdb) import brain.utils.mail as mail_utils *** AttributeError: module 'brain.utils' has no attribute 'mail' I always thought that import a.b.c as m is roughly equivalent to m = sys.modules['a.b.c']. Why AttributeError? Python 3.6 I was pointed out <http://stackoverflow.com/a/24968941/248296> that this is a somewhat weird behavior of Python: The statement is not quite true, as evidenced by the corner case you met, namely if the required modules already exist in sys.modules but are yet uninitialized. The import ... as requires that the module foo.bar is injected in foo namespace as the attribute bar, in addition to being in sys.modules, whereas the from ... import ... as looks for foo.bar in sys.modules. Why would `import a.b.c` work when `a.b.c` is not yet fully imported, but `import a.b.c as my_c` would not? I though it would be vice versa. Using `import a.b.c as my_c` allows avoiding a number of circular import issues. Right now I have to use `from a.b import c as my_c` as a workaround, but this doesn't look right. Does this make sense? Is there an issue in the Python bug tracker for this? Can/should I create one? Victor
On 8 April 2017 at 15:54, Victor Varvariuc <victor.varvariuc@gmail.com> wrote:
It as to do with when and how the attribute lookup for the submodule occurs. Disassembling the 3 examples (plus another variant): >>> import dis >>> dis.dis("import a.b.c") 1 0 LOAD_CONST 0 (0) 2 LOAD_CONST 1 (None) 4 IMPORT_NAME 0 (a.b.c) 6 STORE_NAME 1 (a) 8 LOAD_CONST 1 (None) 10 RETURN_VALUE In the first case, we see that the name bound in the local namespace is *a*: we won't actually try to access the full "a.b.c" attribute reference until somewhere later in the module. That lazy lookup means the import system is satisfied based on the sys.modules entry, and the module code will be happy as long as "a.b" has a "c" attribute by the time the module needs it. >>> dis.dis("import a.b.c as m") 1 0 LOAD_CONST 0 (0) 2 LOAD_CONST 1 (None) 4 IMPORT_NAME 0 (a.b.c) 6 LOAD_ATTR 1 (b) 8 LOAD_ATTR 2 (c) 10 STORE_NAME 3 (m) 12 LOAD_CONST 1 (None) 14 RETURN_VALUE In the second case, we see that "import a.b.c as m" produces almost exactly the same bytecode as "import a.b.c; m = a.b.c", skipping only the store-and-reload of the "a" reference: >>> dis.dis("import a.b.c; m = a.b.c") 1 0 LOAD_CONST 0 (0) 2 LOAD_CONST 1 (None) 4 IMPORT_NAME 0 (a.b.c) 6 STORE_NAME 1 (a) 8 LOAD_NAME 1 (a) 10 LOAD_ATTR 2 (b) 12 LOAD_ATTR 3 (c) 14 STORE_NAME 4 (m) 16 LOAD_CONST 1 (None) 18 RETURN_VALUE That eager lookup means that "a.b" *must* have a "c" attribute at the time of the import, or the LOAD_ATTR operation will fail. >>> dis.dis("from a.b import c as m") 1 0 LOAD_CONST 0 (0) 2 LOAD_CONST 1 (('c',)) 4 IMPORT_NAME 0 (a.b) 6 IMPORT_FROM 1 (c) 8 STORE_NAME 2 (m) 10 POP_TOP 12 LOAD_CONST 2 (None) 14 RETURN_VALUE Finally, when we come to the "from" import case, we can see that the bytecode changes significantly: the LOAD_ATTRs all disappear, replaced by a single IMPORT_FROM, and the from_list for the IMPORT_NAME opcode is a populated tuple, rather than None. To address the next question that follows on from looking at the current compile time behaviour, changing those semantics isn't as simple as just having the compiler reinterpret "import x.y.z as m" as "from x.y import z as m". The problem is that the two formulations make different assertions about the nature of "z" (in the first form, it's expected to be a submodule, in the second, it can be any attribute of x.y), so adding such a reinterpretation would either move the discrepancy to having "import x.y.z as m" work in cases where "import x.y.z" fails, or else require adding a way for the compiler to indicate that the resolved attribute must *also* exist in sys.modules, in addition to being an attribute on the parent module. That's not impossible to resolve, but it would be a lot of work just to add an alternate spelling of "from a.b import c as m". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 8 April 2017 at 15:54, Victor Varvariuc <victor.varvariuc@gmail.com> wrote:
It as to do with when and how the attribute lookup for the submodule occurs. Disassembling the 3 examples (plus another variant): >>> import dis >>> dis.dis("import a.b.c") 1 0 LOAD_CONST 0 (0) 2 LOAD_CONST 1 (None) 4 IMPORT_NAME 0 (a.b.c) 6 STORE_NAME 1 (a) 8 LOAD_CONST 1 (None) 10 RETURN_VALUE In the first case, we see that the name bound in the local namespace is *a*: we won't actually try to access the full "a.b.c" attribute reference until somewhere later in the module. That lazy lookup means the import system is satisfied based on the sys.modules entry, and the module code will be happy as long as "a.b" has a "c" attribute by the time the module needs it. >>> dis.dis("import a.b.c as m") 1 0 LOAD_CONST 0 (0) 2 LOAD_CONST 1 (None) 4 IMPORT_NAME 0 (a.b.c) 6 LOAD_ATTR 1 (b) 8 LOAD_ATTR 2 (c) 10 STORE_NAME 3 (m) 12 LOAD_CONST 1 (None) 14 RETURN_VALUE In the second case, we see that "import a.b.c as m" produces almost exactly the same bytecode as "import a.b.c; m = a.b.c", skipping only the store-and-reload of the "a" reference: >>> dis.dis("import a.b.c; m = a.b.c") 1 0 LOAD_CONST 0 (0) 2 LOAD_CONST 1 (None) 4 IMPORT_NAME 0 (a.b.c) 6 STORE_NAME 1 (a) 8 LOAD_NAME 1 (a) 10 LOAD_ATTR 2 (b) 12 LOAD_ATTR 3 (c) 14 STORE_NAME 4 (m) 16 LOAD_CONST 1 (None) 18 RETURN_VALUE That eager lookup means that "a.b" *must* have a "c" attribute at the time of the import, or the LOAD_ATTR operation will fail. >>> dis.dis("from a.b import c as m") 1 0 LOAD_CONST 0 (0) 2 LOAD_CONST 1 (('c',)) 4 IMPORT_NAME 0 (a.b) 6 IMPORT_FROM 1 (c) 8 STORE_NAME 2 (m) 10 POP_TOP 12 LOAD_CONST 2 (None) 14 RETURN_VALUE Finally, when we come to the "from" import case, we can see that the bytecode changes significantly: the LOAD_ATTRs all disappear, replaced by a single IMPORT_FROM, and the from_list for the IMPORT_NAME opcode is a populated tuple, rather than None. To address the next question that follows on from looking at the current compile time behaviour, changing those semantics isn't as simple as just having the compiler reinterpret "import x.y.z as m" as "from x.y import z as m". The problem is that the two formulations make different assertions about the nature of "z" (in the first form, it's expected to be a submodule, in the second, it can be any attribute of x.y), so adding such a reinterpretation would either move the discrepancy to having "import x.y.z as m" work in cases where "import x.y.z" fails, or else require adding a way for the compiler to indicate that the resolved attribute must *also* exist in sys.modules, in addition to being an attribute on the parent module. That's not impossible to resolve, but it would be a lot of work just to add an alternate spelling of "from a.b import c as m". Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (2)
-
Nick Coghlan -
Victor Varvariuc