[Distutils] setup_requires for dev environments

[Leonardo Rochael Almeida]

On 18 March 2015 at 14:37, Daniel Holth <dholth at gmail.com> wrote:

> [...]
>
> The behavior we're aiming for would be:
>
> "installer run setup.py" - installs things
> "python setup.py" - does not install things


Besides that, I'd add that we're also looking for: "python setup.py" (by
itself) should not raise ImportError, even if setup.py needs extra things
installed for certain operations (egg_info, build, sdist, develop, install).

IMO, the biggest pain point is not people putting crazy stuff in setup.py
to get version numbers. For me, the biggest pain point is when setup.py
needs to import other packages in order to even know how to build:

So I'd like to suggest the following series of small improvements to both
pip and setuptools:

 * setuptools: `python setup.py setup_requires` dumps its setup_requires
keyword in 'requirements.txt' format

It's is already in this format, so should be trivial, but allows one to do
something like:

    $ python setup.py setup_requires > setup_requires.txt
    $ pip install -r setup_requires.txt

Or in one bash line:

    $ pip install -r <( python setup.py setup_requires )

 * setuptools: setup.py gains the ability to accept callables in most
(all?) of its parameters.

This will allow people to move all top level setup.py imports into
functions, so that we can turn code like this:

    from setuptools import setup, Extension
    import numpy

    setup(ext_modules=[
        Extension("_cos_doubles",
            sources=["cos_doubles.c", "cos_doubles.i"],
            include_dirs=[numpy.get_include()])])

Into this:

    from setuptools import setup, Extension

    def ext_modules():
        import numpy
        return [
            Extension("_cos_doubles",
                sources=["cos_doubles.c", "cos_doubles.i"],
                include_dirs=[numpy.get_include()])
        ]

    setup(ext_modules=ext_modules
              setup_requires=['setuptools'])

 * pip: When working with an sdist, before running "setup.py egg_info" in a
sandbox, pip would run "setup.py setup_requires", install those packages in
the sandbox (not in the main environment), then run "egg_info", "wheel",
etc.

Notice that the changes proposed above are all backward compatible, create
no additional pain, and allow developers to move all top level setup.py
craziness inside functions.

After that, we can consider making setup.py not call the easy_install
functionality when it finds a setup_requires keyword while running other
commands, but just report if those packages are not available.


PS: Yes, I've already proposed something similar recently:
https://mail.python.org/pipermail/distutils-sig/2015-January/025682.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150319/e668872f/attachment.html>