[Distutils] setuptools special case Pyrex and break Cython

[Stefan Behnel]

Phillip J. Eby wrote:
> At 11:19 AM 9/3/2007 +0200, Stefan Behnel wrote:
>> In current setuptools, the file setuptools/extension.py checks for
>> Pyrex being
>> installed and otherwise renames all .pyx file entries to .c. This does
>> not
>> work when Cython is installed *instead of* Pyrex, as Cython will never be
>> executed in this case. It just fails saying that the .c file doesn't
>> exist.
>>
>> I'm currently shipping a fake Pyrex/Distutils/build_ext.py module with
>> lxml to
>> make it work with both, but I don't even see a major advantage in the
>> extension.py module at all. It's plain easy to do that yourself in the
>> setup.py script if (and only if) you want it to build without
>> Pyrex/Cython,
>> but currently, it is enforced in setuptools. This really only works if
>> the
>> distributor wants it and does something to make it work, so always
>> enforcing
>> this behaviour doesn't give you any real advantage.
>>
>> I'd like to see this module removed from setuptools or at least have
>> Cython
>> supported in the same way.
> 
> I guess I'm not clear on the actual problem here.  Do you want it to
> only rename .pyx to .c for .c files that actually exist?

That would be an option. However, I still don't understand why it should be up
to setuptools to decide if *my* code distribution can be built without Pyrex
(or any other tool that names its files .pyx).

The extension module in setuptools could also check if build_ext is the
original function or a replacement and only do its magic if it's alone in the
field and thinks it can't hurt anyone. That would only be another heuristic,
but it would at least have prevented cases like this...


> A better way to specify that you can handle .pyx files?

You mean a hook into setuptools so that Cython can fix the extension module?
Doesn't sound like a clean solution to me...


> Note that removing this feature is not an option that's on the table, as
> it was one of the original reasons I created setuptools in the first
> place, years before the idea of eggs even existed.

So you are suggesting that people should choose: it's either eggs or Cython. I
don't think eggs are that important to everyone, but I don't think this
decision should be enforced either. Maybe you should split setuptools into a
part that serves everyone and a part that served you years before the idea of
eggs even existed.

Don't get me wrong, I really like eggs and the EasyInstall machinery. But it
shouldn't get in the way of the projects it aims to serve.


> Now, in future versions of setuptools, the ones that eventually replace
> a lot of distutils framework with new stuff, there will definitely be a
> better way of handling this.  But I have no idea when that will happen.

Well, splitting it up could make it easier to improve things gradually.

Stefan