[py-dev] Import problems under py.test
I'm having a problem with imports when using py.test. The file db.py gets imported twice. It actually comes from two different file locations, but it's the same file, and there should only be one way to get to it. In this case, importing the module twice immediately throws an exception. /home/ianb/repository/Landscaper/external is in sys.path, but /home is a symlink to /usr/home. This is the class that is initially imported: <class '/usr/home/ianb/repository/Landscaper/external/lib/db/py.SiteSQLObject'> This is the normal class, which is created second: <class 'lib.db.SiteSQLObject'> I guess the problem is that my test script imports db (relatively), then imports a module that imports db (also relatively). I can probably turn this into a simpler test case, but I thought I'd see if the problem was obvious. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
Hi Ian, [Ian Bicking Tue, Nov 16, 2004 at 05:25:53PM -0600]
I'm having a problem with imports when using py.test. The file db.py gets imported twice. It actually comes from two different file locations, but it's the same file, and there should only be one way to get to it. In this case, importing the module twice immediately throws an exception.
Actually, i noticed similar problems i think.
/home/ianb/repository/Landscaper/external is in sys.path, but /home is a symlink to /usr/home. This is the class that is initially imported:
<class '/usr/home/ianb/repository/Landscaper/external/lib/db/py.SiteSQLObject'>
OK, the according import of this class started in a test. Tests are imported using <path object>.getpymodule() and this method maintains a separate cache. It also supports relative imports. Therefore you see the above "path-based" import. (Path based imports also work remotely etc.pp.)
This is the normal class, which is created second:
<class 'lib.db.SiteSQLObject'>
I guess the problem is that my test script imports db (relatively), then imports a module that imports db (also relatively).
Yes, IOW this is a mix of path-based and normal imports.
I can probably turn this into a simpler test case, but I thought I'd see if the problem was obvious.
Well, i guess we need to avoid such suprising situations some way or another. cheers, holger
I added a test to py.test (py/test/test/import_test) for this import problem. Well, maybe the test belongs in py.path; anyway, it's there. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
Hi Ian, [Ian Bicking Wed, Nov 17, 2004 at 12:06:57PM -0600]
I added a test to py.test (py/test/test/import_test) for this import problem. Well, maybe the test belongs in py.path; anyway, it's there.
yes, i saw it, thanks. Note that (i think) the problem would also occur if you did the traditional "python test_myfile.py" which used unittest.py. If you import relative from this file and then import (absolutely) from the package then you would also get a different instance of the module. However, i discussed the issue shortly with Armin this morning and we may try to "guess" a more normal module path than a mangled str(Path object). I.e. when we do: py.test test_import.py we would traverse upwards from test_import.py as long as there is an __init__.py. The basename()s it so gathers would comprise the absolute module path (which is used as a key for sys.modules). However, i am not completly sure about all this yet (my brain is exploding from the current pypy sprint in Vilnius :-). cheers, holger
holger krekel wrote:
[Ian Bicking Wed, Nov 17, 2004 at 12:06:57PM -0600]
I added a test to py.test (py/test/test/import_test) for this import problem. Well, maybe the test belongs in py.path; anyway, it's there.
yes, i saw it, thanks. Note that (i think) the problem would also occur if you did the traditional "python test_myfile.py" which used unittest.py. If you import relative from this file and then import (absolutely) from the package then you would also get a different instance of the module.
I just extended the test so you can run it as a script, and it works fine. Since I mix relative and absolute imports frequently in my code (for better or worse), this is what I would expect. You will get a problem with normal inputs when you are in a situation like: /a/b/c.py sys.path includes /a and /a/b You "import c" somewhere, and "import b.c" somewhere else; this creates two modules that come from the same file. But relative imports alone should not create another module. Of course, I'm not sure if the relative import in this case will match the c module or the b.c... this is one of those places where "refuse to guess" would have been a better idea that the current behavior.
However, i discussed the issue shortly with Armin this morning and we may try to "guess" a more normal module path than a mangled str(Path object). I.e. when we do:
py.test test_import.py
we would traverse upwards from test_import.py as long as there is an __init__.py. The basename()s it so gathers would comprise the absolute module path (which is used as a key for sys.modules).
However, i am not completly sure about all this yet (my brain is exploding from the current pypy sprint in Vilnius :-).
Importing is a pain in the ass, but that sounds like it should work. Alternately, could you try the normal import mechanism, see if it finds the same filename you are looking for, and if so procede with that? I think imp has the functions to find a module without actually loading it. Hrm... but no, you don't actually have a name to try. Or, you could just look for common prefixes in sys.path, and if you find one traverse up making sure __init__.py's exist; most of the time they will. This seems more optimistic. Of course, I don't know if that will fix this test, because I modify sys.path after the module is loaded, so it would be loaded as an absolute path, and relative imports might still be funky. Which makes me wonder how Python normally accomplishes this at all. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
participants (3)
-
holger krekel -
hpk＠trillke.net
-
Ian Bicking