bdist_egg trips up over non-extension extensions
In particular, numpy can't build into an egg currently because numpy/__config__.py is created using Configuration.add_extension() and a generator function. In order to make extension modules usable from zipped eggs, setuptools generates a pure Python stub loader that will extract the extension module to a .hidden directory and load it from there. In this case, since the Extension isn't actually an extension, the stub loader is broken, because it expects a numpy/__config__.so.
Can we find another way to generate Python code instead of abusing Extension this way?
Just to be clear, you're not having problems with numpy as a non-zipped egg, just a zipped egg, right? I haven't had any problems, but I haven't tried to make zip-safe eggs.
(For those who don't know, eggs can be installed either as a zip filed with extension .egg, or as a directory with extension egg.)
Robert Kern wrote:
In particular, numpy can't build into an egg currently because numpy/__config__.py is created using Configuration.add_extension() and a generator function. In order to make extension modules usable from zipped eggs, setuptools generates a pure Python stub loader that will extract the extension module to a .hidden directory and load it from there. In this case, since the Extension isn't actually an extension, the stub loader is broken, because it expects a numpy/__config__.so.
Can we find another way to generate Python code instead of abusing Extension this way?
On Tue, 24 Jan 2006, Robert Kern wrote:
In particular, numpy can't build into an egg currently because numpy/__config__.py is created using Configuration.add_extension() and a generator function. In order to make extension modules usable from zipped eggs, setuptools generates a pure Python stub loader that will extract the extension module to a .hidden directory and load it from there. In this case, since the Extension isn't actually an extension, the stub loader is broken, because it expects a numpy/__config__.so.
Can we find another way to generate Python code instead of abusing Extension this way?
Yes, using py_modules instead of ext_modules would be the correct way to go. I'm looking into this issue right now...
Pearu
On Tue, 24 Jan 2006, Robert Kern wrote:
In particular, numpy can't build into an egg currently because numpy/__config__.py is created using Configuration.add_extension() and a generator function. In order to make extension modules usable from zipped eggs, setuptools generates a pure Python stub loader that will extract the extension module to a .hidden directory and load it from there. In this case, since the Extension isn't actually an extension, the stub loader is broken, because it expects a numpy/__config__.so.
Could you try recent version of numpy from SVN? __config__.py is now generated via py_modules list.
Pearu
Pearu Peterson wrote:
Could you try recent version of numpy from SVN? __config__.py is now generated via py_modules list.
It doesn't build an egg, yet, because setuptools expects Distribution.py_modules to contain only strings, not tuples. Modifying setuptools to handle tuples in Distribution.py_modules seems to work. Is there any way you can do it without tuples?
On Tue, 24 Jan 2006, Robert Kern wrote:
Pearu Peterson wrote:
Could you try recent version of numpy from SVN? __config__.py is now generated via py_modules list.
It doesn't build an egg, yet, because setuptools expects Distribution.py_modules to contain only strings, not tuples. Modifying setuptools to handle tuples in Distribution.py_modules seems to work. Is there any way you can do it without tuples?
Source generators in numpy.distutils are extensions to distutils and if setuptools assumes std distutils then source generators will remain a problem for setuptools. However, if you call build_src then all source generators will be resolved (including 3-tuples in py_modules list) and the file lists in distribution instance should not cause any problems for setuptools.
So, would calling `setup.py build_src bdist_egg` be an acceptable solution?
__config__.py files could also be handled as data files but that would require hooks in numpy.core.setup function and we want to get rid of numpy.core.setup in future.
Pearu
On Jan 24, 2006, at 14:36 , Pearu Peterson wrote:
On Tue, 24 Jan 2006, Robert Kern wrote:
Pearu Peterson wrote:
Could you try recent version of numpy from SVN? __config__.py is now generated via py_modules list.
It doesn't build an egg, yet, because setuptools expects Distribution.py_modules to contain only strings, not tuples. Modifying setuptools to handle tuples in Distribution.py_modules seems to work. Is there any way you can do it without tuples?
Source generators in numpy.distutils are extensions to distutils and if setuptools assumes std distutils then source generators will remain a problem for setuptools. However, if you call build_src then all source generators will be resolved (including 3-tuples in py_modules list) and the file lists in distribution instance should not cause any problems for setuptools.
So, would calling `setup.py build_src bdist_egg` be an acceptable solution?
Fixed in svn. I added a wrapper around the setuptools.command.egg_info.egg_info.run method that runs the build_src command first.
participants (4)
-
Andrew Straw -
David M. Cooke -
Pearu Peterson -
Robert Kern