We hit some very weird behaviour recently while setting up a package hierarchy. Robin Becker managed to distil this into a simple example. Can anyone shed any light on what is happening below? Is Python behaving as it should? Create a package A, empty __init__.py, with modules as follows: --------parent.py---------- class Parent: pass --------child.py------------ from parent import Parent class Child(Parent): pass ---------test.py-------- from parent import * class Examiner: def examine(self, arg): print 'examining argument:' print 'class of arg =', arg.__class__ print 'bases of arg =', arg.__class__.__bases__ print 'arg =', arg if isinstance(arg, Parent): print "arg is an instance of Parent" else: print "arg IS NOT an instance of Parent" print if __name__=='__main__': from traceback import print_exc import sys def run0(): from A.child import Child e = Examiner() e.examine(Child()) def run1(): from A.child import Child from A.test import Examiner e = Examiner() e.examine(Child()) run0() run1() Running this script produces the following output: C:\users\andy\A>test.py examining argument: class of arg = A.child.Child bases of arg = (<class A.parent.Parent at 7f9150>,) arg = <A.child.Child instance at 7f9410> arg IS NOT an instance of Parent examining argument: class of arg = A.child.Child bases of arg = (<class A.parent.Parent at 7f9150>,) arg = <A.child.Child instance at 7f83c0> arg is an instance of Parent Many thanks, Andy Robinson
On Wed, 7 Jun 2000, Andy Robinson wrote:
We hit some very weird behaviour recently while setting up a package hierarchy. Robin Becker managed to distil this into a simple example. Can anyone shed any light on what is happening below? Is Python behaving as it should?
It looks to me like you have two "parent" modules. When you run "test.py" directly from the command line, the first "from parent import *" imports parent.py as module "parent". But subsequently parent.py is imported again as module "A.parent". Similarly, you have two "Examiner" classes. In run0() you get Examiner from __main__, which contains the Parent class from module "parent". In run1() you get Examiner from A.test, which contains the Parent class imported from module "A.parent". If you insert print 'in module', __name__, 'parent is', repr(Parent) at the beginning of the examine() method, it should make things clear. The solution is to avoid directly running scripts that are inside packages; the confusion is produced by the fact that you're running "test.py" from within the A/ directory. -- ?!ng
Ka-Ping wrote:
On Wed, 7 Jun 2000, Andy Robinson wrote:
We hit some very weird behaviour recently while setting up a package hierarchy. Robin Becker managed to distil this into a simple example. Can anyone shed any light on what is happening below? Is Python behaving as it should?
[snip entirely correct analysis]
The solution is to avoid directly running scripts that are inside packages; the confusion is produced by the fact that you're running "test.py" from within the A/ directory.
I disagree with the moral. If test.py uses "import A.parent" and changes references accordingly, it works. So my version of the moral is that import * is EVIL. Not only that, but relative imports are EVIL. This is what I meant when I said (in reply to Greg Wilson's inquiry about package structures) that there are some very bad precedents getting set. Doing this junk (usually in __init__.py) not only makes your package fragile, it also makes the package structure mysterious to the user (who's a developer - making things mysterious to him/her is just making both your lives harder!). If you want to make it friendly to lazy coders, then something like: import A.parent Parent = A.parent.Parent also works, (leaving references as Parent). I'm not terribly fond of this, either, but at least the coder has a reasonable hope of tracking down what "Parent" actually is. I'm not necessarily yelling at you, Andy. It seems that most package developers are going down this slope. It stinks. - Gordon
Gordon's comments are on the money. An import in a module inside a package should be fully qualified. I'd put that in the style guide if it were up to me (and, of course, it's not). Jeremy
"JH" == Jeremy Hylton <jeremy@beopen.com> writes:
JH> Gordon's comments are on the money. An import in a module JH> inside a package should be fully qualified. I'd put that in JH> the style guide if it were up to me (and, of course, it's JH> not). I concur on both points. BTW, one of the Things I'd Like To Do, is update and flesh out the style guide. It's incomplete in places and out of date in others. If someone else has the gumption to lead this effort, I'd be happy to send my comments instead. -Barry
Barry A. Warsaw [bwarsaw@python.org] wrote:
"JH" == Jeremy Hylton <jeremy@beopen.com> writes:
JH> Gordon's comments are on the money. An import in a module JH> inside a package should be fully qualified. I'd put that in JH> the style guide if it were up to me (and, of course, it's JH> not).
I concur on both points. BTW, one of the Things I'd Like To Do, is update and flesh out the style guide. It's incomplete in places and out of date in others. If someone else has the gumption to lead this effort, I'd be happy to send my comments instead.
Something we futzed with at Digital Creations was "commentable" or discussable web pages. This would be a great use for this. If you took and made each basic topic/issue a page, and let people "discuss" them. A nice threaded discussion could add a lot of insight. This is versus a wiki where it's more "free for all", which isn't really appropraiate for a "guide". Chris -- | Christopher Petrilli | petrilli@amber.org
On Wed, 7 Jun 2000, Christopher Petrilli wrote:
Something we futzed with at Digital Creations was "commentable" or discussable web pages. This would be a great use for this. If you took and made each basic topic/issue a page, and let people "discuss" them. A nice threaded discussion could add a lot of insight. This is versus a wiki where it's more "free for all", which isn't really appropraiate for a "guide".
Well, y'all could go nuts marking up the style guide with Crit. http://crit.org/ ...or marking up anything else on the whole Web, for that matter. :) -- ?!ng "To be human is to continually change. Your desire to remain what you are is what ultimately limits you." -- The Puppet Master, Ghost in the Shell
Jeremy Hylton wrote:
Gordon's comments are on the money. An import in a module inside a package should be fully qualified. I'd put that in the style guide if it were up to me (and, of course, it's not).
+1 from here :-) I usually write things like this: from mx.DateTime import now print 'The time is', now() If you don't do so, you'll run into strange problems, the most subtle one I've encountered is with pickled objects... if unpickled inside the same package they can cause the same module to be loaded *twice* with all the consequences attached to this (e.g. issubclass() no longer works). -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
On Wed, 7 Jun 2000, Gordon McMillan wrote:
I disagree with the moral.
If test.py uses "import A.parent" and changes references accordingly, it works.
Correct. Well, maybe there are two things going on here. I agree that import * isn't usually a very good idea. However, notice that the problem occurs even if the first line of test.py is "import parent" (no star). When you talk about changing references accordingly, the real issue is referencing "A.parent" rather than "parent". In fact, if you change the first line to "from A.parent import *" (not that i am advocating this!) -- it works. In short: from parent import * ... Parent doesn't work import parent ... parent.Parent doesn't work from A.parent import * ... Parent works import A.parent ... A.parent.Parent works The first two cases -- attempting to import "parent" -- are only made possible by the fact that we're executing test.py from within the A/ directory, and hence the confusion. This problem is an "advanced" version of a simpler issue which a friend of mine discovered and complained about recently to me: the __main__ vs. module-name conflict. Allow me to paraphrase his example. Exhibit A: ---- spam.py ---- import eggs def go(): print "was", eggs.a eggs.a = 3 print "now", eggs.a ---- eggs.py ---- import spam a = 1 def go(): spam.go() print "finally", a if __name__ == "__main__": go() --- watch what happens ---- % python eggs.py was 1 now 3 finally 1 % python >>> import eggs >>> eggs.go() was 1 now 3 finally 3 Exhibit B: ---- one.py ---- import two def go(): pass ---- two.py ---- import one one.go() ---- watch what happens ---- % python one.py % python two.py Traceback (innermost last): File "two.py", line 1, in ? import one File "./one.py", line 1, in ? import two File "./two.py", line 2, in ? one.go() AttributeError: go Exit 1 % python >>> import one Traceback (innermost last): File "<stdin>", line 1, in ? File "./one.py", line 1, in ? import two File "./two.py", line 2, in ? one.go() AttributeError: go >>> import two >>> The facts that eggs.a appears to behave differently in Exhibit A, and that one.py can be run but two.py cannot, yet "two" can be imported and "one" cannot, in Exhibit B, both result from the duplication namespaces in __main__ and in a module. Normally, this only affects the main file that you are running, the one with the __main__ namespace. However, attempting to run scripts in package directories allows this problem to propagate further into the duplication of other modules: __main__ imports parent which imports child A.test imports A.parent which imports A.child ...and that can cause duplicate classes all over the place, which is what happened to Andy. My friend was quite irked by the existence of __main__, citing it as the culprit for this duplication. But naturally Python can't always know the name of the script being run -- in the case of a file ending in .py, perhaps, it's easy, but the script might come from a pipe, for example. So eliminating __main__ doesn't seem like a feasible response. Depending how "real" you consider this problem, leaving things as they are and instructing people in the discipline of keeping modules and scripts separated may be the best response. I bet this crops up all the time in those "testing" sections of modules that use the 'if __name__ == "__main__"' convention, though... -- ?!ng "Computers are useless. They can only give you answers." -- Pablo Picasso
In fact, if you change the first line to "from A.parent import *" (not that i am advocating this!) -- it works. In short:
from parent import * ... Parent doesn't work import parent ... parent.Parent doesn't work
from A.parent import * ... Parent works import A.parent ... A.parent.Parent works
One other possibility that ought to work (?) from A import parent Jeremy
Ka-Ping wrote:
... In short:
from parent import * ... Parent doesn't work import parent ... parent.Parent doesn't work
from A.parent import * ... Parent works import A.parent ... A.parent.Parent works
Right. That's why I cast relative imports as evil.
This problem is an "advanced" version of a simpler issue which a friend of mine discovered and complained about recently to me: the __main__ vs. module-name conflict.
This comes up a whole lot. In those periods where I have the time, energy and patience to monitor c.l.py closely, I've seen this come up 4 or 5 times in one week. Almost always in conjunction with import *. I can live with explaining how __main__ is special. That's one of those "you only have to explain it once" problems. But the import * problems (name hiding, the fact that assignments aren't seen) and relative imports are things that come up over and over again, and trip up even experienced people (and them more than once). - Gordon
participants (7)
-
andy＠robanal.demon.co.uk -
bwarsaw＠python.org -
Christopher Petrilli -
Gordon McMillan -
Jeremy Hylton -
Ka-Ping Yee -
M.-A. Lemburg