[Cython] Compiler crash in RemoveUnreachableCode

[Stefan Behnel]

Andriy Kornatskyy, 22.02.2014 10:35:
> On Jan 21, 2014, at 10:16 AM, Stefan Behnel wrote:
>> I think it's a problem with importing in Cython compiled modules. ISTM
>> that some extension modules got "reloaded", i.e. their module dict got
>> cleared and they got reinitialised, thus recreating all global objects that
>> other modules had already imported (and still keep a reference to).
>>
>>> Here is another bug (details below):
>>>
>>> 1. virtualenv env
>>> 2. env/bin/easy_install cython
>>> 3. env/bin/easy_install lxml wheezy.core
>>>
>>> It seems to have an issue while trying to install 2 or more libs at once.
>>> ...
>>> Installed env/lib/python2.7/site-packages/lxml-3.3.0beta5-py2.7-macosx-10.9-x86_64.egg
>>> Processing dependencies for lxml
>>> Finished processing dependencies for lxml
>>> Searching for wheezy.core
>>> Reading https://pypi.python.org/simple/wheezy.core/
>>> Best match: wheezy.core 0.1.129
>>> Downloading https://pypi.python.org/packages/source/w/wheezy.core/wheezy.core-0.1.129.tar.gz#md5=ea3d5f744bc0525d61f9fb48d897972d
>>> Processing wheezy.core-0.1.129.tar.gz
>>> Writing /var/folders/g8/2kym1h8n7qbgrwg4qkfqw1gw0000gn/T/easy_install-UC_pgJ/wheezy.core-0.1.129/setup.cfg
>>> Running wheezy.core-0.1.129/setup.py -q bdist_egg --dist-dir /var/folders/g8/2kym1h8n7qbgrwg4qkfqw1gw0000gn/T/easy_install-UC_pgJ/wheezy.core-0.1.129/egg-dist-tmp-9sntQ1
>>> Traceback (most recent call last):
>>> ...
>>>  File “env/lib/python2.7/site-packages/Cython-0.20-py2.7-macosx-10.9-x86_64.egg/Cython/Compiler/ModuleNode.py", line 109, in process_implementation
>>>    self.generate_c_code(env, options, result)
>>>  File "env/lib/python2.7/site-packages/Cython-0.20-py2.7-macosx-10.9-x86_64.egg/Cython/Compiler/ModuleNode.py", line 302, in generate_c_code
>>>    rootwriter = Code.CCodeWriter(emit_linenums=emit_linenums, c_line_in_traceback=options.c_line_in_traceback)
>>>  File "Code.py", line 1406, in Cython.Compiler.Code.CCodeWriter.__init__ (/var/folders/g8/2kym1h8n7qbgrwg4qkfqw1gw0000gn/T/easy_install-JVXbGE/Cython-0.20/Cython/Compiler/Code.c:30697)
>>>
>>>  File "env/lib/python2.7/site-packages/Cython-0.20-py2.7-macosx-10.9-x86_64.egg/Cython/StringIOTree.py", line 11, in __init__
>>>    stream = StringIO()
>>> TypeError: 'NoneType' object is not callable
>>
>> Looks like the same thing.
>
> Any update to the issue reported in this message thread?

No, not really. The current setup is just way too vast to debug. I don't
even know how easy_install runs the installation of a single package, nor
how it's possible that two packages interfere in such a way. I'd expect it
to run subprocesses, so why any interference at all? It might be that the
main process loads the setup.py scripts in some way to read the meta data,
thus executes their imports, and then tries to unload the modules before it
loads the next setup.py script. That would be rather bad for any extension
modules imported by both. So bad that's I'd consider it a bug in setuptools.

Let's assume that reloading really is what's happening here. Given that
this basically doesn't work for extension modules, definitely not in Py2.x
and barely in Py3.x, we might be able to apply a barrier to the module init
function that prevents it from running a second time, and just return the
already created module somehow. But that's blank guessing and I really
can't tell for sure. It would help if someone could invest some time into
properly analysing this further.

Stefan