[Python-Dev] SVN rev. 85392 broke module handling in py3k

[Benjamin Peterson]

2010/10/22 Stefan Behnel <stefan_ml at behnel.de>:
> Benjamin Peterson, 22.10.2010 16:03:
>>
>> 2010/10/22 Stefan Behnel:
>>>
>>> since SVN rev. 85392, Cython's installation fails on the py3k branch with
>>> a
>>> weird globals error. I think it is related to some sys.modules magic that
>>> we
>>> do in order to support running Cython in Python 3 using lib2to3.
>>>
>>> Basically, what we do is, we import some parts of Cython at the beginning
>>> that are Py3 clean, specifically some distutils build_ext replacement for
>>> building Cython modules. Then we start up distutils, which first runs
>>> lib2to3 on Cython's sources to convert them into Py3 code. When it then
>>> gets
>>> to building the binary modules, we remove all Cython modules and packages
>>> from sys.modules and reimport their 2to3-ed sources so that we can run
>>> the
>>> complete compiler during the installation (to bootstrap parts of Cython
>>> into
>>> binary modules).
>>>
>>> Since the above revision, this process bails out with an error when
>>> accessing "os.path" because "os" is None. The "os" module is imported
>>> globally in our early-imported build_ext module, more or less like this:
>>>
>>>    import os
>>>
>>>    from distutils.command import build_ext as _build_ext
>>>
>>>    class build_ext(_build_ext.build_ext):
>>>
>>>        def build_extensions(self):
>>>            print(os) # prints None!
>>>
>>> I suspect that the fact that we remove the modules from sys.modules
>>> somehow
>>> triggers the cleanup of these modules while there are still objects from
>>> these modules alive that refer to their globals. So, what I think is
>>> happening is that the module cleanup sets the module's globals to None
>>> before the objects from that module that refer to these globals have
>>> actually gone out of scope.
>>>
>>> Could someone (benjamin?) please look into this?
>>
>> Is this broken before 2.7, ie 2.6 and 2.6?
>
> I can't tell. Py2 doesn't need 2to3, so we don't unload the modules there.

What about 3.1.0 then?



-- 
Regards,
Benjamin