relative import circular problem
I just ran into the issue described in http://stackoverflow.com/questions/6351805/cyclic-module-dependencies-and-re.... This is unfortunate, because we have been trying to move towards relative imports in order to aid flexibility in package and library design. The relative import syntax (from foo import bar) is a getattr type of lookup (i.e. import foo, then get attr from it). This is in contrast with absolute import import foo.bar (get the module foo.bar from sys.modules or import it) bar = foo.bar the latter works with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax. as far as I know, relative imports are only supported using the former (import from) syntax. Are there any plans to alleviate this by allowing proper relative imports? After all, relative imports and packages go hand in hand. K
On Apr 01, 2013, at 08:20 PM, Kristján Valur Jónsson wrote:
The relative import syntax
(from foo import bar) is a getattr type of lookup (i.e. import foo, then get attr from it).
This is in contrast with absolute import
import foo.bar (get the module foo.bar from sys.modules or import it)
bar = foo.bar
I always thought of both syntaxes as absolute imports because they both name the full dotted path to the module you want. This contrasts with true PEP 328 relative imports such as: from ..foo import bar
the latter works with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax.
as far as I know, relative imports are only supported using the former (import from) syntax. Are there any plans to alleviate this by allowing proper relative imports? After all, relative imports and packages go hand in hand.
I personally dislike PEP 328 relative imports, since they seem fragile, but that's a different discussion. -Barry
-----Original Message----- From: Python-Dev [mailto:python-dev- bounces+kristjan=ccpgames.com@python.org] On Behalf Of Barry Warsaw Sent: 1. apríl 2013 22:16 To: python-dev@python.org Subject: Re: [Python-Dev] relative import circular problem
On Apr 01, 2013, at 08:20 PM, Kristján Valur Jónsson wrote:
The relative import syntax
(from foo import bar) is a getattr type of lookup (i.e. import foo, then get attr from it).
This is in contrast with absolute import
import foo.bar (get the module foo.bar from sys.modules or import it)
bar = foo.bar
I always thought of both syntaxes as absolute imports because they both name the full dotted path to the module you want. This contrasts with true PEP 328 relative imports such as:
from ..foo import bar
I personally dislike PEP 328 relative imports, since they seem fragile, but that's a different discussion. Yes. I find them very useful. In our environment, we tend to write packages that then go into different places. Using relative imports allows us the freedom to move packages around and rename
Right, the example I gave was not a relative import but an absolute one, relative imports always starting with a dot. I should have been more clear. However, relative imports can _only_ be performed using the "from X import Y syntax" (http://www.python.org/dev/peps/pep-0328/#id3) and so, when one wishes to use relative imports, the parent module must be fully initialized with the child module as an attribute. importing with the "import X.Y" does not appear to have this restriction. them, for example, make a package a sub-package of another, and do this differently for different products, while still sharing code. A package can thus refer to internals of itself, without knowing its own name, or absolute path. This is really useful. Consider this a property of Python-based "products", with a custom assembly of source code, as opposed to publicly available packages that always install into the sys.path Cheers, K
Kristján Valur Jónsson wrote:
However, relative imports can _only_ be performed using the "from X import Y syntax"
This seems like a legitimate complaint on its own, regardless of the circular import issue. The principle of least surprise suggests that relative imports should be possible using either syntax. I'm guessing the reason that 'import' doesn't support relative imports is that it's not clear what name to bind the imported module to. There are a couple of ways that this could be resolved. One would be to use the name resulting from stripping off the leading dots, so that import .foo would bind the module to the name 'foo'. Another would be to always require an 'as' clause in this case, so that you would have to write' import .foo as foo -- Greg
On 03Apr2013 12:12, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote: | Kristján Valur Jónsson wrote: | >However, relative imports can _only_ be performed using the "from X import Y syntax" | | This seems like a legitimate complaint on its own, [...] | There are a couple of ways that this could be resolved. One | would be to use the name resulting from stripping off the | leading dots, so that | import .foo | would bind the module to the name 'foo'. +0 from me (I'd have been +0.5 but for Guido's explaination about "import X" generally letting you use "X" exactly as phrased in the import). | Another would be | to always require an 'as' clause in this case, so that you | would have to write' | import .foo as foo And a big +1 for this. Cheers, -- Cameron Simpson <cs@zip.com.au> The Force. It surrounds us; It enfolds us; It gets us dates on Saturday Nights. - Obi Wan Kenobi, Famous Jedi Knight and Party Animal.
On Mon, Apr 1, 2013 at 4:20 PM, Kristján Valur Jónsson < kristjan@ccpgames.com> wrote:
I just ran into the issue described in http://stackoverflow.com/questions/6351805/cyclic-module-dependencies-and-re... .
This is unfortunate, because we have been trying to move towards relative imports in order to aid flexibility in package and library design.
The relative import syntax
(from foo import bar) is a getattr type of lookup (i.e. import foo, then get attr from it).
This is in contrast with absolute import
import foo.bar (get the module foo.bar from sys.modules or import it)
bar = foo.bar
Or ``import foo.bar as bar``
the latter works with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax.
Clarification on terminology: the ``from .. import`` syntax is in no way relative. Relative imports use leading dots to specify relative offsets from your current position (i.e. as Barry said). It's more of a syntax for facilitating binding long names (e.g. foo.bar) to shorter names (bar). It's just unfortunate that it can lead to circular import issues when people start pulling in objects off of modules instead of modules alone.
as far as I know, relative imports are only supported using the former (import from) syntax. Are there any plans to alleviate this by allowing proper relative imports? After all, relative imports and packages go hand in hand.
No, there are no plans to either tweak ``from ... import`` statements nor introduce a new syntax to deal help alleviate circular imports. -Brett
K
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
Right, as I explained in my reply to Barry, I was imprecise. But the “from X import Y” is the only way to invoke relative imports, where X can have leading dots. This syntax places the constraint on X that Y is actually an attribute of X at this time, where “import X.Y” does not. So, even without the leading dots issue, they are not equivalent. You run into the same circular dependency problem without using relative imports if trying to use the “from X import Y” where X is an absolute name. K From: bcannon@gmail.com [mailto:bcannon@gmail.com] On Behalf Of Brett Cannon Sent: 1. apríl 2013 22:38 To: Kristján Valur Jónsson Cc: python-dev@python.org Subject: Re: [Python-Dev] relative import circular problem the latter works with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax. Clarification on terminology: the ``from .. import`` syntax is in no way relative. Relative imports use leading dots to specify relative offsets from your current position (i.e. as Barry said). It's more of a syntax for facilitating binding long names (e.g. foo.bar) to shorter names (bar). It's just unfortunate that it can lead to circular import issues when people start pulling in objects off of modules instead of modules alone. as far as I know, relative imports are only supported using the former (import from) syntax. Are there any plans to alleviate this by allowing proper relative imports? After all, relative imports and packages go hand in hand. No, there are no plans to either tweak ``from ... import`` statements nor introduce a new syntax to deal help alleviate circular imports.
Le Tue, 2 Apr 2013 09:28:17 +0000, Kristján Valur Jónsson <kristjan@ccpgames.com> a écrit :
Right, as I explained in my reply to Barry, I was imprecise. But the “from X import Y” is the only way to invoke relative imports, where X can have leading dots. This syntax places the constraint on X that Y is actually an attribute of X at this time, where “import X.Y” does not. So, even without the leading dots issue, they are not equivalent. You run into the same circular dependency problem without using relative imports if trying to use the “from X import Y” where X is an absolute name.
I agree with Kristjan, it is rather annoying. And it also makes explicit relative imports harder to sell to people used to the implicit relative import style, since the latter works well in such circumstances. Regards Antoine.
On Tue, Apr 2, 2013 at 6:20 AM, Kristján Valur Jónsson < kristjan@ccpgames.com> wrote:
I just ran into the issue described in http://stackoverflow.com/questions/6351805/cyclic-module-dependencies-and-re... .
This is unfortunate, because we have been trying to move towards relative imports in order to aid flexibility in package and library design.
The relative import syntax
(from foo import bar) is a getattr type of lookup (i.e. import foo, then get attr from it).
This is in contrast with absolute import
import foo.bar (get the module foo.bar from sys.modules or import it)
bar = foo.bar
the latter works with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax.
This is really a quality-of-implementation issue in the import system rather than a core language design problem. It's just that those of us with the knowledge and ability to fix it aren't inclined to do so because circular imports usually (although not quite always) indicate a need to factor some common code out into a third support module imported by both of the original modules. At that point, the code is cleaner and more decoupled, and the uneven circular import support ceases to be a problem for that application. If you're interested in digging further, see http://bugs.python.org/issue992389 (this should also be a *lot* easier to fix now we're using importlib than it ever would have been while we were still relying on import.c) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
It certainly affects the quality, yes. I also understand why it happens: When importing X.Y, Y isn't actually put into X's dict until it is fully initialized. It is, however put temporarily in sys.modules["X.Y"] hence, "import X.Y" on a partially initialized submodule Y will work, whereas "from X import Y" won't. Fixing this within the "from X import Y" mechanism would add an additional strain on the already complex import protocol (as defined in pep 302), I think. Which is why I wonder if the relative import syntax ought to be allowed for "import X" since that syntax does not involve a getattr. ("from X import Y" necessarily means strictly a getattr, since Y can both be any attribute of X, not just a submodule) As for ways around this: Note that this is a language design question, not a software architecture one. It is possible to work around these issues, but it is not always nice. Python is one of those languages that allow cyclic module dependencies and it is a very nice way to separate code by functionality, if not by dependency. It is one of the good things about Python and we should try to make sure that we allow such architectural freedom to continue to work. Also, relative imports are apparently falling into favor, having only marginally been accepted at the time of pep 328, so we should perhaps find a way for these two things to co-exist :) I'm not sure that http://bugs.python.org/issue992389 warrants a fix. This issue is about general attributes of a module. In the general case, this is probably unfixable. But access to a partially constructed module hierarchy through the import mechanism ought to be possible. K From: Nick Coghlan [mailto:ncoghlan@gmail.com] Sent: 1. apríl 2013 22:53 To: Kristján Valur Jónsson Cc: python-dev@python.org Subject: Re: [Python-Dev] relative import circular problem with partially initialized modules, but not the former, rendering two sibling modules unable to import each other using the relative syntax. This is really a quality-of-implementation issue in the import system rather than a core language design problem. It's just that those of us with the knowledge and ability to fix it aren't inclined to do so because circular imports usually (although not quite always) indicate a need to factor some common code out into a third support module imported by both of the original modules. At that point, the code is cleaner and more decoupled, and the uneven circular import support ceases to be a problem for that application. If you're interested in digging further, see http://bugs.python.org/issue992389 (this should also be a *lot* easier to fix now we're using importlib than it ever would have been while we were
On Mon, Apr 1, 2013 at 4:52 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
This is really a quality-of-implementation issue in the import system rather than a core language design problem. It's just that those of us with the knowledge and ability to fix it aren't inclined to do so because circular imports usually (although not quite always) indicate a need to factor some common code out into a third support module imported by both of the original modules. At that point, the code is cleaner and more decoupled, and the uneven circular import support ceases to be a problem for that application.
+1 -eric
-----Original Message----- From: Eric Snow [mailto:ericsnowcurrently@gmail.com] Sent: 4. apríl 2013 04:57
imported by both of the original modules. At that point, the code is cleaner and more decoupled, and the uneven circular import support ceases to be a problem for that application.
+1
I tried to make the point in an earlier mail that I don't think that we ought to let our personal opinions de-jour on software architecture put a constraint on our language features. There can be good and valid reasons to put things that depend on each other in separate modules, for example when trying to separate modules by functionality or simply when splitting a long file into two. Notice that cyclic dependencies are allowed _within_ a module file. Imagine if we decided that we could only refer to objects _previously_ declared within a .py file, because it encouraged the good design practice of factoring out common dependencies. Dependency graphs of software entities can sadly not always be reduced into a DAG, and we should, IMHO, by no means force people to keep each cygle in the dependency graph within a single .py module. K
On 4 Apr 2013 19:37, "Kristján Valur Jónsson" <kristjan@ccpgames.com> wrote:
-----Original Message----- From: Eric Snow [mailto:ericsnowcurrently@gmail.com] Sent: 4. apríl 2013 04:57
imported by both of the original modules. At that point, the code is cleaner and more decoupled, and the uneven circular import support ceases to be a problem for that application.
+1
I tried to make the point in an earlier mail that I don't think that we
ought to let our personal opinions de-jour on software architecture put a constraint on our language features.
There can be good and valid reasons to put things that depend on each other in separate modules, for example when trying to separate modules by functionality or simply when splitting a long file into two. Notice that cyclic dependencies are allowed _within_ a module file. Imagine if we decided that we could only refer to objects _previously_ declared within a .py file, because it encouraged the good design practice of factoring out common dependencies. Dependency graphs of software entities can sadly not always be reduced into a DAG, and we should, IMHO, by no means force people to keep each cygle in the dependency graph within a single .py module.
That's why it's just a reason none of the current import system devs are inclined to work on it, rather than a reason for rejecting proposals and tested patches that give improved behaviour. Cheers, Nick.
K
On Thu, Apr 4, 2013 at 2:36 AM, Kristján Valur Jónsson < kristjan@ccpgames.com> wrote: [...]
There can be good and valid reasons to put things that depend on each other in separate modules, for example when trying to separate modules by functionality or simply when splitting a long file into two. Notice that cyclic dependencies are allowed _within_ a module file. Imagine if we decided that we could only refer to objects _previously_ declared within a .py file, because it encouraged the good design practice of factoring out common dependencies. Dependency graphs of software entities can sadly not always be reduced into a DAG, and we should, IMHO, by no means force people to keep each cygle in the dependency graph within a single .py module.
IMO in both cases (cyclic dependencies within modules and between modules) Python's approach is consistently derived from a specific implementation strategy: names are available after they are defined. This works (at least it doesn't raise NameError :-): def f(): return g() def g(): return f() f() Because by the time f() is called, g() is defined. Note that this doesn't work: def f(): return g() f() def g(): return f() IOW there is no magic. If you think of putting objects in a dictionary, the semantics are clear. (Except for the shenanigans that the compiler carries out for determining whether something is a local variable reference or not, but that's also explainable.) This does not work: a = b b = a because the first line references b which is not yet defined. Similarly, if module A imports module B and B imports A, that works because imports are first satisfied from sys.modules, and when A is loaded, it is entered into sys.modules before its code is executed. But it may be that if A is loaded and starts importing B, B is then loaded and imports A, which finds A in the state it reached when it decided to import B. The rules here are more complicated because of what "from A import X" means (it depends on whether X is a submodule or not) but there are rules, and they are totally well-defined. The final piece of this puzzle is that, given A and B mutually importing each other, the top-level app code could start by importing A or B, and the execution order will be different then. (You can force this by putting A and B in a package whose __init__.py imports them in the desired order.) I don't really see what we could change to avoid breaking code in any particular case -- the burden is up to the library to do it right. I don't see a reason to forbid any of this either. -- --Guido van Rossum (python.org/~guido)
On 04/04/2013 4:17pm, Guido van Rossum wrote:
I don't really see what we could change to avoid breaking code in any particular case -- the burden is up to the library to do it right. I don't see a reason to forbid any of this either.
How about having a form of relative import which only works for submodules. For instance, instead of from . import moduleX write import .moduleX which is currently a SyntaxError. I think this could be implemented as moduleX = importlib.import_module('.moduleX', __package__) -- Richard
Redirecting to python-ideas. On Thu, Apr 4, 2013 at 9:26 AM, Richard Oudkerk <shibturn@gmail.com> wrote:
On 04/04/2013 4:17pm, Guido van Rossum wrote:
I don't really see what we could change to avoid breaking code in any particular case -- the burden is up to the library to do it right. I don't see a reason to forbid any of this either.
How about having a form of relative import which only works for submodules. For instance, instead of
from . import moduleX
write
import .moduleX
which is currently a SyntaxError. I think this could be implemented as
moduleX = importlib.import_module('.moduleX', __package__)
We considered that when relative import was designed and rejected it, because it violates the expectation that after "import <some string>" you can use exactly "<some string>" in your code. -- --Guido van Rossum (python.org/~guido)
+1. I was thinking along the same lines. Allowing relative imports in "import module [as X]" statements. If 'module' consists of pure dots, then "as X" is required. Otherwise, if "as X" is not present, strip the leading dot(s) when assigning the local name. K
-----Original Message----- From: Python-Dev [mailto:python-dev- bounces+kristjan=ccpgames.com@python.org] On Behalf Of Richard Oudkerk Sent: 4. apríl 2013 16:26 To: python-dev@python.org Subject: Re: [Python-Dev] relative import circular problem
How about having a form of relative import which only works for submodules. For instance, instead of
from . import moduleX
write
import .moduleX
which is currently a SyntaxError. I think this could be implemented as
moduleX = importlib.import_module('.moduleX', __package__)
--
On Thu, Apr 4, 2013 at 11:17 AM, Guido van Rossum <guido@python.org> wrote:
I don't really see what we could change to avoid breaking code in any particular case
Actually, the problem has *nothing* to do with circularity per se; it's that "import a.b" and "from a import b" behave differently in terms of how they obtain the module 'a.b'... And "from a import b" will *always* fail if 'a.b' is part of a cycle with the current module, whereas "import a.b" will *always* succeed. The workaround is to tell people to always use "import a.b" in the case of circular imports; it's practically a FAQ, at least to me. ;-) The problem with "from import" is that it always tries to getattr(a,'b'), even if 'a.b' is in sys.modules. In contrast, a plain import will simply fetch a.b from sys.modules first. In the case of a normal import, this is no problem, because a.b is set to sys.modules['a.b'] at the end of the module loading process. But if the import is circular, then the module is in sys.modules['a.b'], but *not* yet bound to a.b. So the "from import" fails. So, this is actually an implementation quirk that could be fixed in a (somewhat) straightforward manner: by making "from a import b" succeed if 'a.b' is in sys.modules, the same way "import a.b" does. It would require a little bit of discussion to hash out the exact way to do it, but it could be done.
Thanks for the insight. That could indeed be done and I would encourage someone to come up with a fix. FWIW there is another difference between the two forms -- "from a import b" allows b to be any attribute of a, whereas "import a.b" requires b to be a submodule of a. I don't want to compromise on the latter. I do think it would be fine if "from a import b" returned the attribute 'b' of module 'a' if it exists, and otherwise look for module 'a.b' in sys.modules. On Thu, Apr 4, 2013 at 1:28 PM, PJ Eby <pje@telecommunity.com> wrote:
On Thu, Apr 4, 2013 at 11:17 AM, Guido van Rossum <guido@python.org> wrote:
I don't really see what we could change to avoid breaking code in any particular case
Actually, the problem has *nothing* to do with circularity per se; it's that "import a.b" and "from a import b" behave differently in terms of how they obtain the module 'a.b'...
And "from a import b" will *always* fail if 'a.b' is part of a cycle with the current module, whereas "import a.b" will *always* succeed.
The workaround is to tell people to always use "import a.b" in the case of circular imports; it's practically a FAQ, at least to me. ;-)
The problem with "from import" is that it always tries to getattr(a,'b'), even if 'a.b' is in sys.modules. In contrast, a plain import will simply fetch a.b from sys.modules first.
In the case of a normal import, this is no problem, because a.b is set to sys.modules['a.b'] at the end of the module loading process. But if the import is circular, then the module is in sys.modules['a.b'], but *not* yet bound to a.b. So the "from import" fails.
So, this is actually an implementation quirk that could be fixed in a (somewhat) straightforward manner: by making "from a import b" succeed if 'a.b' is in sys.modules, the same way "import a.b" does. It would require a little bit of discussion to hash out the exact way to do it, but it could be done.
-- --Guido van Rossum (python.org/~guido)
On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum <guido@python.org> wrote:
I do think it would be fine if "from a import b" returned the attribute 'b' of module 'a' if it exists, and otherwise look for module 'a.b' in sys.modules.
Technically, it already does that -- but inside of __import__, not in the IMPORT_FROM opcode. But then *after* doing that check-and-fallback, __import__ doesn't assign a.b, because it assumes the recursive import it called has already done this... which means that when __import__ returns, the IMPORT_FROM opcode tries and fails to do the getattr. This could be fixed in one of two ways. Either: 1. Change importlib._bootstrap._handle_fromlist() to set a.b if it successfully imports 'a.b' (inside its duplicate handling for what IMPORT_FROM does), or 2. Change the IMPORT_FROM opcode to handle the fallback itself While the latter involves a bit of C coding, it has fewer potential side-effects on the import system as a whole, and simply ensures that if "import" would succeed, then so would "from...import" targeting the same module. (There might be other fixes I haven't thought of, but really, changing IMPORT_FROM to fallback to a sys.modules check is probably by far the least-invasive way to handle it.)
On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby <pje@telecommunity.com> wrote:
On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum <guido@python.org> wrote:
I do think it would be fine if "from a import b" returned the attribute 'b' of module 'a' if it exists, and otherwise look for module 'a.b' in sys.modules.
Technically, it already does that -- but inside of __import__, not in the IMPORT_FROM opcode.
But then *after* doing that check-and-fallback, __import__ doesn't assign a.b, because it assumes the recursive import it called has already done this...
It's an unfortunate side-effect of having loaders set sys.modules for new modules not also set them as an attribute on their parent package immediately as well (or you could argue it's a side-effect of not passing in a module instead of a name to load_module() but that's another discussion).
which means that when __import__ returns, the IMPORT_FROM opcode tries and fails to do the getattr.
This could be fixed in one of two ways. Either:
1. Change importlib._bootstrap._handle_fromlist() to set a.b if it successfully imports 'a.b' (inside its duplicate handling for what IMPORT_FROM does), or
It's three lines, one of which is 'else:'. Just did it.
2. Change the IMPORT_FROM opcode to handle the fallback itself
While the latter involves a bit of C coding, it has fewer potential side-effects on the import system as a whole, and simply ensures that if "import" would succeed, then so would "from...import" targeting the same module.
(There might be other fixes I haven't thought of, but really, changing IMPORT_FROM to fallback to a sys.modules check is probably by far the least-invasive way to handle it.)
This is my preference as well. The change would be small: I think all you need to do is if the getattr() fails then fall back to sys.modules. Although if it were me and I was casting backwards-compatibility to the wind I would rip out the whole fromlist part of __import__() and let the bytecode worry about the fromlist, basically making the import opcode call importlib.import_module().
+1 on Brett and PJE just doing this. On Thu, Apr 4, 2013 at 3:38 PM, Brett Cannon <brett@python.org> wrote:
On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby <pje@telecommunity.com> wrote:
On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum <guido@python.org> wrote:
I do think it would be fine if "from a import b" returned the attribute 'b' of module 'a' if it exists, and otherwise look for module 'a.b' in sys.modules.
Technically, it already does that -- but inside of __import__, not in the IMPORT_FROM opcode.
But then *after* doing that check-and-fallback, __import__ doesn't assign a.b, because it assumes the recursive import it called has already done this...
It's an unfortunate side-effect of having loaders set sys.modules for new modules not also set them as an attribute on their parent package immediately as well (or you could argue it's a side-effect of not passing in a module instead of a name to load_module() but that's another discussion).
which means that when __import__ returns, the IMPORT_FROM opcode tries and fails to do the getattr.
This could be fixed in one of two ways. Either:
1. Change importlib._bootstrap._handle_fromlist() to set a.b if it successfully imports 'a.b' (inside its duplicate handling for what IMPORT_FROM does), or
It's three lines, one of which is 'else:'. Just did it.
2. Change the IMPORT_FROM opcode to handle the fallback itself
While the latter involves a bit of C coding, it has fewer potential side-effects on the import system as a whole, and simply ensures that if "import" would succeed, then so would "from...import" targeting the same module.
(There might be other fixes I haven't thought of, but really, changing IMPORT_FROM to fallback to a sys.modules check is probably by far the least-invasive way to handle it.)
This is my preference as well. The change would be small: I think all you need to do is if the getattr() fails then fall back to sys.modules. Although if it were me and I was casting backwards-compatibility to the wind I would rip out the whole fromlist part of __import__() and let the bytecode worry about the fromlist, basically making the import opcode call importlib.import_module().
-- --Guido van Rossum (python.org/~guido)
On Apr 4, 2013 6:47 PM, "Guido van Rossum" <guido@python.org> wrote:
+1 on Brett and PJE just doing this.
I'll file a bug when I get home. -brett
On Thu, Apr 4, 2013 at 3:38 PM, Brett Cannon <brett@python.org> wrote:
On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby <pje@telecommunity.com> wrote:
On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum <guido@python.org>
I do think it would be fine if "from a import b" returned the attribute 'b' of module 'a' if it exists, and otherwise look for module 'a.b' in sys.modules.
Technically, it already does that -- but inside of __import__, not in the IMPORT_FROM opcode.
But then *after* doing that check-and-fallback, __import__ doesn't assign a.b, because it assumes the recursive import it called has already done this...
It's an unfortunate side-effect of having loaders set sys.modules for new modules not also set them as an attribute on their parent package immediately as well (or you could argue it's a side-effect of not
wrote: passing in
a module instead of a name to load_module() but that's another discussion).
which means that when __import__ returns, the IMPORT_FROM opcode tries and fails to do the getattr.
This could be fixed in one of two ways. Either:
1. Change importlib._bootstrap._handle_fromlist() to set a.b if it successfully imports 'a.b' (inside its duplicate handling for what IMPORT_FROM does), or
It's three lines, one of which is 'else:'. Just did it.
2. Change the IMPORT_FROM opcode to handle the fallback itself
While the latter involves a bit of C coding, it has fewer potential side-effects on the import system as a whole, and simply ensures that if "import" would succeed, then so would "from...import" targeting the same module.
(There might be other fixes I haven't thought of, but really, changing IMPORT_FROM to fallback to a sys.modules check is probably by far the least-invasive way to handle it.)
This is my preference as well. The change would be small: I think all you need to do is if the getattr() fails then fall back to sys.modules. Although if it were me and I was casting backwards-compatibility to the wind I would rip out the whole fromlist part of __import__() and let the bytecode worry about the fromlist, basically making the import opcode call importlib.import_module().
-- --Guido van Rossum (python.org/~guido)
http://bugs.python.org/issue17636 On Thu, Apr 4, 2013 at 8:03 PM, Brett Cannon <brett@python.org> wrote:
On Apr 4, 2013 6:47 PM, "Guido van Rossum" <guido@python.org> wrote:
+1 on Brett and PJE just doing this.
I'll file a bug when I get home.
-brett
On Thu, Apr 4, 2013 at 3:38 PM, Brett Cannon <brett@python.org> wrote:
On Thu, Apr 4, 2013 at 5:00 PM, PJ Eby <pje@telecommunity.com> wrote:
On Thu, Apr 4, 2013 at 4:42 PM, Guido van Rossum <guido@python.org>
I do think it would be fine if "from a import b" returned the attribute 'b' of module 'a' if it exists, and otherwise look for module 'a.b' in sys.modules.
Technically, it already does that -- but inside of __import__, not in the IMPORT_FROM opcode.
But then *after* doing that check-and-fallback, __import__ doesn't assign a.b, because it assumes the recursive import it called has already done this...
It's an unfortunate side-effect of having loaders set sys.modules for new modules not also set them as an attribute on their parent package immediately as well (or you could argue it's a side-effect of not
wrote: passing in
a module instead of a name to load_module() but that's another discussion).
which means that when __import__ returns, the IMPORT_FROM opcode tries and fails to do the getattr.
This could be fixed in one of two ways. Either:
1. Change importlib._bootstrap._handle_fromlist() to set a.b if it successfully imports 'a.b' (inside its duplicate handling for what IMPORT_FROM does), or
It's three lines, one of which is 'else:'. Just did it.
2. Change the IMPORT_FROM opcode to handle the fallback itself
While the latter involves a bit of C coding, it has fewer potential side-effects on the import system as a whole, and simply ensures that if "import" would succeed, then so would "from...import" targeting the same module.
(There might be other fixes I haven't thought of, but really, changing IMPORT_FROM to fallback to a sys.modules check is probably by far the least-invasive way to handle it.)
This is my preference as well. The change would be small: I think all you need to do is if the getattr() fails then fall back to sys.modules. Although if it were me and I was casting backwards-compatibility to the wind I would rip out the whole fromlist part of __import__() and let the bytecode worry about the fromlist, basically making the import opcode call importlib.import_module().
-- --Guido van Rossum (python.org/~guido)
And I should learn to read the entire thread before I start responding. Cheers! K
-----Original Message----- From: Python-Dev [mailto:python-dev- bounces+kristjan=ccpgames.com@python.org] On Behalf Of Guido van Rossum Sent: 4. apríl 2013 22:47 To: Brett Cannon Cc: PJ Eby; Nick Coghlan; python-dev@python.org Subject: Re: [Python-Dev] relative import circular problem
+1 on Brett and PJE just doing this.
-----Original Message----- From: PJ Eby [mailto:pje@telecommunity.com] Sent: 4. apríl 2013 20:29 To: Guido van Rossum Cc: Kristján Valur Jónsson; Nick Coghlan; python-dev@python.org Subject: Re: [Python-Dev] relative import circular problem
So, this is actually an implementation quirk that could be fixed in a (somewhat) straightforward manner: by making "from a import b" succeed if 'a.b' is in sys.modules, the same way "import a.b" does. It would require a little bit of discussion to hash out the exact way to do it, but it could be done.
Yes, except that "from a import b" is not only used to import modules. It is pretty much defined to mean "b = getattr(a, 'b')". Consider this module: #foo.py # pull helper.helper from its implementation place from .helper import helper now, "from foo import helper" is expected to get the helper() function, not the helper module, but changing the semantics of the import statement would be "surprising". K
participants (11)
-
Antoine Pitrou -
Barry Warsaw -
Brett Cannon -
Cameron Simpson -
Eric Snow -
Greg Ewing -
Guido van Rossum -
Kristján Valur Jónsson -
Nick Coghlan -
PJ Eby -
Richard Oudkerk