[Distutils] setuptools special case Pyrex and break Cython
Phillip J. Eby
pje at telecommunity.com
Wed Sep 5 20:46:47 CEST 2007
At 08:00 PM 9/5/2007 +0200, Stefan Behnel wrote:
>Phillip J. Eby wrote:
> > At 08:30 AM 9/5/2007 +0200, Stefan Behnel wrote:
> >> > Perhaps you'd care to produce a patch to implement that "cleaner step"?
> >> > It's not at all obvious to me how to do that without introducing
> >> > instability that would be unsuitable for an 0.6cN release.
> >> One way of implementing the above change would be to move the
> >> replacement code
> >> into build_ext rather than Extension. Something like the (untested)
> >> build_ext-patcher.py I attached. Note the type check that tests for
> >> build_ext
> >> being subclassed.
> > You're illustrating my point. It's easy to hand-wave about how it
> > should be done, but not so easy to actually *do*. Did you look at where
> > all the .sources attribute gets used? How the build_ext command can get
> > called by other commands?
>Sort of. Do you doubt it should work that way?
I know that other commands use the .sources of build_ext indirectly,
at a time when run() has not yet been called. That means that your
approach may change those other commands' behavior in non-obvious
ways. That's why I'm not going to do something like that in an 0.6cN release.
> > You're also ignoring my larger point: the current mechanism allows you
> > to write setup scripts that *don't* need to subclass build_ext. A
> > setuptools-based setup script just refers to '.pyx' files, and
> > everything else happens automatically.
>The script I posted doesn't change that. Read it.
Actually, now that I've read it in detail, I see that it's also
broken in other ways, such as confusing modules and classes.
But it's also broken in the way I said: it *requires* subclassing of
build_ext in setup.py, which the current mechanism does not.
> > And if you need to be able to distinguish between Cython-specific and
> > Pyrex-specific files, why are you using the same file extension?
>We don't distinguish, it's the same language. But users will have to install
>both of them
If it's the "same language" then, *why* do they need this? If Pyrex
is still required, then clearly it's *not* the "same language".
>You're actually forgetting the point you stressed most. If we prevent users
>from installing Cython and Pyrex at the same time, they will have to start
>modifying setup.py scripts. So it's pushing the burden on them.
Actually, I think you need to decide whether you are really providing
an extended Pyrex compiler. If so, then use .pyx as the extension
and include a Pyrex-compatible build_ext module, and then there is no
need to have both installed on the same system, and everything works
splendidly for everyone.
If on the other hand you are *not* providing a Pyrex replacement and
must co-exist, then use a different file extension (.cy, perhaps?)
and you get exactly the behavior you wanted -- i.e., setuptools
doesn't touch your files, and explicit action is required to change
the file extensions.
And there is still a third solution: provide your own Extension class
for users to import, since they will have to import from Cython in
order to use your extended build_ext class anyway, nothing stops them
from importing the Extension type too. (Just have your
Extension.__init__ save the sources and restore them after calling
the superclass; it'll then work whether the patched version is present or not.)
More information about the Distutils-SIG