[Cython] Cannot cythonize subclasses of setuptools.extension._Extension

[Manuel Nuno Melo]

Hi Erik,
Please post your solution; I'm curious to see it.

Currently, we're also using setup_requires but are moving away from it
because of two main issues:

1- there's no flexibility to hook into it via setuptools.Distribution
subclassing (unless you rewrite the entire __init__);

2- (and more serious) setuptools installs setup_requires dependencies in
the local directory, not as a system-wide install. Even worse, it then puts
a path link to the current directory under the system-wide
easy_install.pth. This means that if you install from the source directory
via 'sudo ./setup.py install', you get a Cython egg downloaded into that
directory, and its path added to sys.path (!!). Needless to say this can
break in many nasty ways...

On Sat, Apr 16, 2016 at 1:10 PM, Erik Bray <erik.m.bray at gmail.com> wrote:

> On Apr 14, 2016 21:07, "Manuel Nuno Melo" <manuel.nuno.melo at gmail.com>
> wrote:
> >
> > Our need to control cythonization comes from the fact that we implement
> cython as a lazy and optional dependency. Lazy in the sense that we delay
> as much as possible cythonization so that setuptools or pip have time to
> install cython, if needed. Optional because we distribute both .pyx and
> cythonized .c files, and decide on which to use based on user flags.
> >
> > We, therefore, need an Extension class that only cythonizes if we decide
> to.
> >
> > Thanks for the feedback,
> > Manel
>
> I have a solution for exactly this in astropy_helpers which makes it
> possible, for example, to pull in Cython via setup_requires, but only if
> it's needed.
>
> Remind me on Monday and I can point out how it works.
>
> > On Apr 14, 2016 8:17 PM, "Matthew Brett" <matthew.brett at gmail.com>
> wrote:
> > >
> > > On Thu, Apr 14, 2016 at 6:08 AM, Erik Bray <erik.m.bray at gmail.com>
> wrote:
> > > > On Wed, Apr 13, 2016 at 9:35 PM, Manuel Nuno Melo
> > > > <manuel.nuno.melo at gmail.com> wrote:
> > > >> Hello devs,
> > > >>
> > > >> I'm developing the setup.py for a scientific package, MDAnalysis
> (see PR
> > > >> #799). We depend on distutils and setuptool. Namely, we use
> > > >> setuptools.extension.Extension class for our extensions.
> > > >>
> > > >> Some older versions of setuptools (<18.0) do filename cythonization
> > > >> themselves upon initialization of the Extension object.
> > > >>
> > > >> Because we want to control name cythonization ourselves I try to
> directly
> > > >> use distutils.extension.Extension, which has none of setuptools'
> > > >> cythonization. However, this doesn't work because setuptools patches
> > > >> distutils, so that distutils.extension.Extension effectively becomes
> > > >> setuptools.extension.Extension.
> > > >
> > > > I'm wondering what it is specifically you need to do in your
> > > > subclass--might it still be possible to do with a subclass of the
> > > > setuptools Extension?  Not saying I disagree with the overall idea,
> > > > but I also wonder if there isn't a better way.
> > >
> > > I know this is a terrible and ugly hack, but the projects I work in
> > > have a 'fake_pyrex' directory, that fools setuptools into thinking
> > > that 'pyrex' is installed, and therefore prevents it from doing the
> > > .pyx -> .c filename conversions in the extension:
> > >
> > > https://github.com/regreg/regreg/blob/master/setup.py#L33
> > >
> > > Cheers,
> > >
> > > Matthew
> > > _______________________________________________
> > > cython-devel mailing list
> > > cython-devel at python.org
> > > https://mail.python.org/mailman/listinfo/cython-devel
> >
> >
> > _______________________________________________
> > cython-devel mailing list
> > cython-devel at python.org
> > https://mail.python.org/mailman/listinfo/cython-devel
> >
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20160416/dd61ba3e/attachment.html>