[Python-Dev] SVN rev. 85392 broke module handling in py3k

[Benjamin Peterson]

2010/10/22 Stefan Behnel <stefan_ml at behnel.de>:
> Hi,
>
> since SVN rev. 85392, Cython's installation fails on the py3k branch with a
> weird globals error. I think it is related to some sys.modules magic that we
> do in order to support running Cython in Python 3 using lib2to3.
>
> Basically, what we do is, we import some parts of Cython at the beginning
> that are Py3 clean, specifically some distutils build_ext replacement for
> building Cython modules. Then we start up distutils, which first runs
> lib2to3 on Cython's sources to convert them into Py3 code. When it then gets
> to building the binary modules, we remove all Cython modules and packages
> from sys.modules and reimport their 2to3-ed sources so that we can run the
> complete compiler during the installation (to bootstrap parts of Cython into
> binary modules).
>
> Since the above revision, this process bails out with an error when
> accessing "os.path" because "os" is None. The "os" module is imported
> globally in our early-imported build_ext module, more or less like this:
>
>    import os
>
>    from distutils.command import build_ext as _build_ext
>
>    class build_ext(_build_ext.build_ext):
>
>        def build_extensions(self):
>            print(os) # prints None!
>
> I suspect that the fact that we remove the modules from sys.modules somehow
> triggers the cleanup of these modules while there are still objects from
> these modules alive that refer to their globals. So, what I think is
> happening is that the module cleanup sets the module's globals to None
> before the objects from that module that refer to these globals have
> actually gone out of scope.
>
> Could someone (benjamin?) please look into this?

Is this broken before 2.7, ie 2.6 and 2.6?



-- 
Regards,
Benjamin