Detecting circular imports ?
Hello,
This morning I tried to fix an issue for a while before I realized I had a circular import. This issue is not obvious because you get a cryptic error, like an AttributeError and it can tak a while before finding out.
I don't know of this was mentioned before, or how hard it would be. But it would be nice if Python had a specific "CircularImportError" raised in that case, or something.. That would be a fabulous hint for developers.
Cheers Tarek
On Thu, 18 Nov 2010 16:14:35 +0100 Tarek Ziadé ziade.tarek@gmail.com wrote:
I don't know of this was mentioned before, or how hard it would be. But it would be nice if Python had a specific "CircularImportError" raised in that case, or something.. That would be a fabulous hint for developers.
It can't be an error, since circular imports are supported. There could be a warning, but IMO it would need to be silenced by default (which means ImportWarning isn't appropriate).
Regards
Antoine.
Am 18.11.2010 16:22, schrieb Antoine Pitrou:
On Thu, 18 Nov 2010 16:14:35 +0100 Tarek Ziadé ziade.tarek@gmail.com wrote:
I don't know of this was mentioned before, or how hard it would be. But it would be nice if Python had a specific "CircularImportError" raised in that case, or something.. That would be a fabulous hint for developers.
It can't be an error, since circular imports are supported. There could be a warning, but IMO it would need to be silenced by default (which means ImportWarning isn't appropriate).
I thought ImportWarning was silenced by default?
Anyway, I don't think that would be helpful for developers, who would need to turn on the warnings explicitly to see them. But to think of that, you already need to suspect circular imports, and then it's easy to see it from the traceback anyway.
Georg
On Thu, Nov 18, 2010 at 4:41 PM, Georg Brandl g.brandl@gmx.net wrote:
Am 18.11.2010 16:22, schrieb Antoine Pitrou:
On Thu, 18 Nov 2010 16:14:35 +0100 Tarek Ziadé ziade.tarek@gmail.com wrote:
I don't know of this was mentioned before, or how hard it would be. But it would be nice if Python had a specific "CircularImportError" raised in that case, or something.. That would be a fabulous hint for developers.
It can't be an error, since circular imports are supported. There could be a warning, but IMO it would need to be silenced by default (which means ImportWarning isn't appropriate).
I thought ImportWarning was silenced by default?
Anyway, I don't think that would be helpful for developers, who would need to turn on the warnings explicitly to see them. But to think of that, you already need to suspect circular imports, and then it's easy to see it from the traceback anyway.
mmm I am not talking about warnings here. As Ian said, and to be more explicit the use case is:
Your program crashed with an error because you have a circular import -- that's a bug you need to fix.
How can we tell it in a more explicit fashion that an AttributeError which is not very meaningful in this case ?
Georg
Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
Greg Ewing wrote:
Georg Brandl wrote:
But to think of that, you already need to suspect circular imports, and then it's easy to see it from the traceback anyway.
Right. Whenever I've had a problem with circular imports, it was usually fairly obvious what the cause was.
Same here and most cases I've had in the past actually presented themselves as... ImportErrors :-)
On 18 November 2010 23:14, M.-A. Lemburg mal@egenix.com wrote:
Greg Ewing wrote:
Georg Brandl wrote:
But to think of that, you already need to suspect circular imports, and then it's easy to see it from the traceback anyway.
Right. Whenever I've had a problem with circular imports, it was usually fairly obvious what the cause was.
Same here and most cases I've had in the past actually presented themselves as... ImportErrors :-)
If you use the "from module import something" form it will be an ImportError. If you do "import module" and then "module.something" you will see an AttributeError. These are the ones that can be confusing.
Michael
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Nov 19 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On Fri, Nov 19, 2010 at 12:21 AM, Michael Foord fuzzyman@voidspace.org.uk wrote:
On 18 November 2010 23:14, M.-A. Lemburg mal@egenix.com wrote:
Greg Ewing wrote:
Georg Brandl wrote:
But to think of that, you already need to suspect circular imports, and then it's easy to see it from the traceback anyway.
Right. Whenever I've had a problem with circular imports, it was usually fairly obvious what the cause was.
Same here and most cases I've had in the past actually presented themselves as... ImportErrors :-)
If you use the "from module import something" form it will be an ImportError. If you do "import module" and then "module.something" you will see an AttributeError. These are the ones that can be confusing.
Yes that's the case I was thinking of. After years of Python I can still get trapped on those. But maybe that's just me :)
Tarek Ziadé writes:
Yes that's the case I was thinking of. After years of Python I can still get trapped on those. But maybe that's just me :)
So you're suggesting (pseudo-code, not Python!)
message = usual_message if object is instance of ModuleType: message += "\nreferenced object is a module; are your imports OK?" raise AttributeError(message)
or something like that?
On Thu, Nov 18, 2010 at 5:29 PM, Tarek Ziadé ziade.tarek@gmail.com wrote:
If you use the "from module import something" form it will be an ImportError. If you do "import module" and then "module.something" you
will
see an AttributeError. These are the ones that can be confusing.
Yes that's the case I was thinking of. After years of Python I can still get trapped on those. But maybe that's just me :)
I certainly have problems with circular imports (especially involving __init__.py), and (as far as I've noticed) the error messages are seldom helpful unless you happen to be experienced in the problem and remember to look for circular imports. Simply giving a better error message for this case would solve the problem in my opinion, without the complexity of actually making anything work that doesn't work now.
Tarek Ziadé <ziade.tarek@...> writes:
On Fri, Nov 19, 2010 at 12:21 AM, Michael Foord <fuzzyman@...> wrote:
If you use the "from module import something" form it will be an ImportError. If you do "import module" and then "module.something" you will see an AttributeError. These are the ones that can be confusing.
Yes that's the case I was thinking of. After years of Python I can still get trapped on those. But maybe that's just me :)
It's not just you, I get that too and it's really annoying. I have a one-module- per-class approach so it happens a lot to me. I would love to have more informative error messages when this happens.
Ram.
On Thu, Nov 18, 2010 at 9:14 AM, Tarek Ziadé ziade.tarek@gmail.com wrote:
This morning I tried to fix an issue for a while before I realized I had a circular import. This issue is not obvious because you get a cryptic error, like an AttributeError and it can tak a while before finding out.
I don't know of this was mentioned before, or how hard it would be. But it would be nice if Python had a specific "CircularImportError" raised in that case, or something.. That would be a fabulous hint for developers.
Maybe more specifically, a module could be marked as "loading" in some fashion, until it's been completely executed. Any AttributeError from a loading module would instead raise something like CircularImportError (subclassing AttributeError).
On Fri, Nov 19, 2010 at 1:14 AM, Tarek Ziadé ziade.tarek@gmail.com wrote:
Hello,
This morning I tried to fix an issue for a while before I realized I had a circular import. This issue is not obvious because you get a cryptic error, like an AttributeError and it can tak a while before finding out.
I don't know of this was mentioned before, or how hard it would be. But it would be nice if Python had a specific "CircularImportError" raised in that case, or something.. That would be a fabulous hint for developers.
Feel free to take a shot at it - the relevant bug report [1] is only a little over 6 years old! :)
If we could detect the situation well enough to give a specific error message for it, we could probably do something smarter about it than that.
I find my naive early messages about the import subsystem from back then are kind of amusing now, though... little did I know what evils lurked beneath the surface of the import statement...
Cheers, Nick.
[1] http://bugs.python.org/issue992389
On Fri, Nov 19, 2010 at 11:15 PM, Nick Coghlan ncoghlan@gmail.com wrote:
Feel free to take a shot at it - the relevant bug report [1] is only a little over 6 years old! :) [1] http://bugs.python.org/issue992389
Less facetiously, there are 3 different things suggested in that bug report that people could try:
1. Implement the set-and-rollback feature for submodule attributes in parent packages during import 2. Implement the "try looking for the fully qualified name in sys.modules" idea(/hack) 3. Track in-progress imports and throw specific errors if the import code trips over one
The last option does have the advantage of requiring less coordination with the developers of other implementations (given the subtleties of this issue, I'd want feedback from at least PyPy, IronPython and Jython as to what they can handle before we started messing with the semantics of what the language allows as far as circular imports go)
The issue suffers from the fact that at least some of the people with the knowledge to do something about it are not inclined to do so due to strong stylistic objections (*ahem*).
Cheers, Nick.
Nick Coghlan wrote:
On Fri, Nov 19, 2010 at 11:15 PM, Nick Coghlan ncoghlan@gmail.com wrote:
Feel free to take a shot at it - the relevant bug report [1] is only a little over 6 years old! :) [1] http://bugs.python.org/issue992389
Less facetiously, there are 3 different things suggested in that bug report that people could try:
- Implement the set-and-rollback feature for submodule attributes in
parent packages during import 2. Implement the "try looking for the fully qualified name in sys.modules" idea(/hack) 3. Track in-progress imports and throw specific errors if the import code trips over one
The last option does have the advantage of requiring less coordination with the developers of other implementations (given the subtleties of this issue, I'd want feedback from at least PyPy, IronPython and Jython as to what they can handle before we started messing with the semantics of what the language allows as far as circular imports go)
The issue suffers from the fact that at least some of the people with the knowledge to do something about it are not inclined to do so due to strong stylistic objections (*ahem*).
How would the implementation differentiate between a real attribute error (missing symbol while setting up a module) and one caused by a circular import (symbol available, but not yet defined) ?
2010/11/20 M.-A. Lemburg mal@egenix.com:
Nick Coghlan wrote:
The issue suffers from the fact that at least some of the people with the knowledge to do something about it are not inclined to do so due to strong stylistic objections (*ahem*).
How would the implementation differentiate between a real attribute error (missing symbol while setting up a module) and one caused by a circular import (symbol available, but not yet defined) ?
I'm the wrong person to ask - there's a reason my efforts when it comes to resolving this consist almost entirely of trying to nudge the people that actually care about fixing it in potentially promising directions :)
Cheers, Nick.
M.-A. Lemburg wrote:
How would the implementation differentiate between a real attribute error (missing symbol while setting up a module) and one caused by a circular import (symbol available, but not yet defined) ?
Informing the user that the module code hasn't completed yet when the AttributeError occurs may be good enough.
On Sat, Nov 20, 2010 at 4:59 AM, Peter Otten __peter__@web.de wrote:
Informing the user that the module code hasn't completed yet when the AttributeError occurs may be good enough.
Perhaps it makes sense that this is an attribute on the AttributeError that gets set, and AttributeError.__str__ adds to the message when set.
Yes, that *could* be a subclass, but I'd hate to get an IncompleteModuleMaybeACircularImportAttributeError when it's just a typo.
-Fred
-- Fred L. Drake, Jr. <fdrake at acm.org> "A storm broke loose in my mind." --Albert Einstein
participants (12)
-
Antoine Pitrou -
Fred Drake -
Georg Brandl -
Greg Ewing -
Ian Bicking -
M.-A. Lemburg -
Michael Foord -
Nick Coghlan -
Peter Otten -
Ram Rachum -
Stephen J. Turnbull -
Tarek Ziadé