Duplicate modules problem
Chuck Esterbrook
echuck at mindspring.com
Thu Feb 22 13:10:48 EST 2001
You can find a tarball with this description and accompanying source code at:
ftp://webware.sourceforge.net/pub/webware/ModulesProb01.tar.gz
I have a problem where Python creates duplicate modules in memory rather
than reuse the same one. For example, a module in Pkg/Mod.py ends up in
sys.modules under keys "Pkg.Mod" and "Mod" **pointing to two distinct
modules**.
This creates further problems: Suppose a module Foo.py contains a class
named Foo. If the module is loaded twice as two separate instances (of
ModuleType), then there are 2 separate Foo classes. This causes confusion
including the failure of an assertion such as:
assert issubclass(foo, Foo)
The Foo in that code may be pointing to the first class, while the instance
foo may have been created from the second class. The assertion then fails!
The problem stems from the fact that Python tracks modules by a relative,
rather than absolute, path. A simple os.chdir() or a subtley in packages
can cause this problem.
This problem is easiest to see in an os.chdir() situation:
C:\>mkdir foo
C:\>cd foo
C:\foo>mkdir bar
C:\foo>cd bar
C:\foo\bar>echo class baz: pass > baz.py
C:\foo\bar>echo ### > __init__.py
C:\foo\bar>cd ..
C:\foo>python
ActivePython 2.0, build 202 (ActiveState Tool Corp.)
based on Python 2.0 (#8, Oct 19 2000, 11:30:05) [MSC 32 bit (Intel)] on win32
>>> from bar.baz import baz as baz1
>>> import os
>>> os.chdir('bar')
>>> from baz import baz as baz2
>>> baz1
<class bar.baz.baz at 007908BC>
>>> baz2
<class baz.baz at 0079075C>
>>> baz1 == baz2
0
>>> import sys
>>> [mod for mod in sys.modules.items() if mod[0].count('baz')]
[('baz', <module 'baz' from 'baz.pyc'>),
('bar.baz', <module 'bar.baz' from 'bar\baz.pyc'>)]
>>> mod1 = sys.modules['bar.baz']
>>> mod2 = sys.modules['baz']
>>> id(mod1)
136147728
>>> id(mod2)
136147744
This problem can also be seen without any use of os.chdir(). See
ManufactureWare/ which contains the "assert issubclass(foo, Foo)" problem
described above.
This all applies to Python 2.0 on Windows & UNIX. I haven't tried 2.1 yet.
I believe the solution is for Python to track modules by their absolute
path. I don't know of any other resolution to the situation other than
modifying Python in this manner. I also don't know of any disadvantage for
Python to track modules by absolute path.
- Does anyone know of any workarounds?
- Does anyone know why it would be bad for Python to track modules by
absolute path?
- Is there any chance Python will fix this in the future?
- If so, by 2.1?
-Chuck
ftp://webware.sourceforge.net/pub/webware/ModulesProb01.tar.gz
More information about the Python-list
mailing list