[Distutils] setuptools special case Pyrex and break Cython

[Stefan Behnel]

Hi,

I noticed you just released a new version of setuptools without mentioning a
fix for the problem in the changelog, so I assume it's not fixed?


Phillip J. Eby wrote:
> At 08:09 AM 9/4/2007 +0200, Stefan Behnel wrote:
> 
>> Phillip J. Eby wrote:
>> > At 06:51 PM 9/3/2007 +0200, Stefan Behnel wrote:
>> >> I still prefer the "only fiddle with extensions if build_ext wasn't
>> >> replaced"
>> >> way, though.
>> >
>> > If build_ext wasn't replaced where and how?  Do you mean in the
>> > "cmdclass" argument to setup()?
>>
>> Exactly.
> 
> That would mean that you'd have to write code in *every* setup.py to do
> the replacement -- *and* you'd have to get it correct so that it does
> the right thing when Pyrex and Cython are *not* installed.

You already have to write code that imports and replaces build_ext, so having
to add a bit of code that replaces .pyx by .c is not asking to much IMHO.


>> but it would still be
>> better than the current 'feature' enforcement code which basically
>> states: "if
>> Pyrex isn't installed, you can't possibly have meant what you were
>> requesting,
>> so I'll fix your code for you".
> 
> ...and you simply get a different error message than you were going to
> get anyway, so there's no net increase in problems.  You are going to
> have to fix the real problem (by either including the .c files or
> installing the compiler), no matter what.  So how does setuptools'
> action here make any difference?  All that changes is the error message.

Because there is no easy way around it. That's the problem. Having setuptools
do its monkey patching in the background means you have to write code that
works around the patching, and, yes, you have to get it right, which is far
from obvious, involving a couple of test imports, catching ImportErrors and
the like. How is that better than writing a straight forward 1-liner for
replacing .pyx by .c?


>> Problem being that the code isn't broken and
>> the 'fix' breaks it.
> 
> After thinking this over some more, I disagree.
> 
> The key design question I ask for setuptools features is "who gets the
> pain?", since nearly all design decisions will cause *somebody* pain. 
> Ideally, I want that pain to go to somebody who's in a position to do
> something that improves things for everybody in the long term.
> 
> In this case, the only additional pain is for the developers of new
> tools that process .pyx files (such as yourself).

Wrong. I cannot fix this as a developer of Cython - at least not without
providing a fake build_ext.py in a fake Pyrex package, which would make
installing Pyrex and Cython on the same machine near impossible and which
would break every tool that requires Pyrex instead on Cython.

I can only fix this problem as a developer of lxml which *uses* Cython, by
shipping a fake Pyrex with my distribution and adding it to the SVN repository.

So you are currently shifting the burden on every developer who wants to
support Cython (maybe in addition to Pyrex). Great idea.


> [stripped incorrect comments based on a wrong assumption]
>
> In the long run, my plan is to add entry points to the "build" command
> so that other commands can be added.  This would let you have a command
> whose job is to turn .pyx files to .c, but only runs if it's installed. 
> In the meantime, faking Pyrex-ness is, IMO, the best way to deal with this.

No, it's not. I would rather like to have a way to deal with this from Cython
than from Cython-implemented distributions. But the best option IMHO would be
to fix setuptools.

Stefan