Looking for an "import" expert
Fred Drake opened an irritating <wink> bug report (#112436). Cut to the chase: regrtest.py imports test_support. test_support.verbose is 1 after that. regrtest's main() reaches into test_support and stomps on test_support.verbose, usually setting to 0. Now in my build directory, if I run python ..\lib\test\regrtest.py test_getopt the test passes. However, it *shouldn't* (and the crux of Fred's bug report is that it does fail when he runs regrtest in an old & deprecated way). What happens is that test_getopt.py has this near the top: from test.test_support import verbose and this is causing *another* copy of the test_support module to get loaded, and *its* verbose attr is 1. So when we run test_getopt "normally" via regrtest, it incorrectly believes that verbose is 1, and the "expected result" file (which I generated via regrtest -g) is in fact verbose-mode output. If I change the import at the top of test_getopt.py to from test import test_support from test_support import verbose then test_getopt.py sees the 0 it's supposed to see. The story is exactly the same, btw, if I run regrtest while *in* the test directory (so this has nothing to do with that I usually run regrtest from my build directory). Now what *Fred* does is equivalent to getting into a Python shell and typing
from test import regrtest regrtest.main()
and in *that* case (the original) test_getopt sees the 0 it's supposed to see. I confess I lost track how fancy Python imports work a long time ago. Can anyone make sense of these symptoms? Why is a 2nd version of test_support getting loaded, and why only sometimes?
... What happens is that test_getopt.py has this near the top:
from test.test_support import verbose
and this is causing *another* copy of the test_support module to get loaded, and *its* verbose attr is 1.
Maybe adding these lines after that import will help clarify it for you (note that you can't print to stdout without screwing up what regrtest expects): import sys print >> sys.stderr, sys.modules["test_support"], \ sys.modules["test.test_support"] (hot *damn* is that more pleasant than pasting stuff together by hand!). When I run it, I get <module 'test_support' from '..\lib\test\test_support.pyc'> <module 'test.test_support' from 'C:\CODE\PYTHON\DIST\SRC\lib\test\test_support.pyc'> so they clearly are distinct modules.
If the tests are run "the modern way" (python ../Lib/test/regrtest.py) then the test module is the script directory and it is on the path, so "import test_support" sees and loads a toplevel module test_support. Then "import test.test_support" sees a package test with a test_support submodule which is assumed to be a different one, so it is loaded again. But if the tests are run via "import test.autotest" (or "import test.regrtest; test.regrtest.main()" the "import test_support" knows that the importing module is in the test package, so it first tries to import the test_support submodule from that package, so test.test_support and (plain) test_support are the same. Conclusion: inside the test package, never refer explicitly to the test package. Always use "import test_support". Never "import test.test_support" or "from test.test_support import verbose" or "from test import test_support". This is one for the README! I've fixed this by checking in a small patch to test_getopt.py and the corresponding output file (because of the bug, the output file was produced under verbose mode). --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/)
Guido van Rossum wrote:
If the tests are run "the modern way" (python ../Lib/test/regrtest.py) then the test module is the script directory and it is on the path, so "import test_support" sees and loads a toplevel module test_support. Then "import test.test_support" sees a package test with a test_support submodule which is assumed to be a different one, so it is loaded again.
But if the tests are run via "import test.autotest" (or "import test.regrtest; test.regrtest.main()" the "import test_support" knows that the importing module is in the test package, so it first tries to import the test_support submodule from that package, so test.test_support and (plain) test_support are the same.
Conclusion: inside the test package, never refer explicitly to the test package. Always use "import test_support". Never "import test.test_support" or "from test.test_support import verbose" or "from test import test_support".
I'd rather suggest to use a different convention: *always* import using the full path, i.e. "from test import test_support". This scales much better and also avoids a nasty problem with Python pickles related to much the same problem Tim found here: dual import of subpackage modules (note that pickle will always do the full path import).
This is one for the README!
I've fixed this by checking in a small patch to test_getopt.py and the corresponding output file (because of the bug, the output file was produced under verbose mode).
--Guido van Rossum (home page: http://www.pythonlabs.com/~guido/)
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://www.python.org/mailman/listinfo/python-dev
-- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
participants (3)
-
Guido van Rossum -
M.-A. Lemburg -
Tim Peters