[Distutils] Pypy and site.py and makepath
pje at telecommunity.com
Sat Mar 13 18:11:12 CET 2010
At 05:42 PM 3/13/2010 +0100, Lennart Regebro wrote:
>On Sat, Mar 13, 2010 at 16:07, P.J. Eby <pje at telecommunity.com> wrote:
> > 2. It is finding a valid site.py, but the standard import protocols (i.e.,
> > imp.find_module/load_module and
> > path_importer_cache[modulename].load_module()) are not working correctly
> > when called on an existing module.
>Yeah, this seems like the problem. It imports it, but neither globals
>or locals change. It's hard to say exactly what the problem is as I
>can't debug what happens in CPython, as no matter what I put into the
>local site.py, nothing seems to happen. So I't hard to me to check
>what the behaviour *should* be, but it seems clear that this is where
>the difference is.
What the behavior should be is that when you call load_module (either
on an importer instance or the imp.load_module function), and the
module already exists in sys.modules, that's the module object that
should be updated. reload() relies on this, as do various other
things such as lazy importing tools.
A quick glance at PyPy's source (at
http://codespeak.net/pypy/dist/pypy/module/imp/ ) reveals that it
does support reusing a module in sys.modules, but for some reason
this code is shut off by a keyword argument that defaults to
false. This behavior is *definitely* not what CPython's load_module does.
I didn't dig into whether this is also true for any importers that
PyPy might put into the path importer cache, but it's definitely true
One could argue, I suppose, whether the imp behavior is official
Python spec vs. CPython-only feature, but setuptools isn't the only
thing that relies on it, and PEP 302 *does* makes the reuse behavior
explicitly required for loader objects. Also, its absence would make
it bloody hard (probably impossible, actually) to implement a
reload() function (either the base one or an improved version) in
terms of the imp API.
Simplest fix for this in PyPy would probably be to change
load_module() in modules.imp.interp_inp to call
importing.load_module() with the 'reuse' argument set to True. A
test for this behavior is straightforward: stick a module object into
sys.modules with a variable set, then use load_module() to load
something that sets another variable, then check that both variables
are set in the module.
(Also, it should assert that load_module() returns the module object
that was *already* in sys.modules, rather than that it just shoved a
new module into sys.modules -- which is what the current PyPy code
More information about the Distutils-SIG