[Distutils] setup_requires for dev environments

[Daniel Holth]

... except that there are plenty of reasons we wouldn't want the
requirements.txt format, mainly because pip shouldn't automatically
install concrete dependencies that contain git:// urls etc.

On Thu, Mar 19, 2015 at 9:57 AM, Daniel Holth <dholth at gmail.com> wrote:
> If that's what you want then we could say the spec was to put the
> requirements in setup_requires.txt, in the requirements.txt format,
> which pip would eventually look for and install before running
> setup.py
>
> On Thu, Mar 19, 2015 at 9:32 AM, Leonardo Rochael Almeida
> <leorochael at gmail.com> wrote:
>>
>> On 18 March 2015 at 14:37, Daniel Holth <dholth at gmail.com> wrote:
>>>
>>> [...]
>>>
>>> The behavior we're aiming for would be:
>>>
>>> "installer run setup.py" - installs things
>>> "python setup.py" - does not install things
>>
>>
>> Besides that, I'd add that we're also looking for: "python setup.py" (by
>> itself) should not raise ImportError, even if setup.py needs extra things
>> installed for certain operations (egg_info, build, sdist, develop, install).
>>
>> IMO, the biggest pain point is not people putting crazy stuff in setup.py to
>> get version numbers. For me, the biggest pain point is when setup.py needs
>> to import other packages in order to even know how to build:
>>
>> So I'd like to suggest the following series of small improvements to both
>> pip and setuptools:
>>
>>  * setuptools: `python setup.py setup_requires` dumps its setup_requires
>> keyword in 'requirements.txt' format
>>
>> It's is already in this format, so should be trivial, but allows one to do
>> something like:
>>
>>     $ python setup.py setup_requires > setup_requires.txt
>>     $ pip install -r setup_requires.txt
>>
>> Or in one bash line:
>>
>>     $ pip install -r <( python setup.py setup_requires )
>>
>>  * setuptools: setup.py gains the ability to accept callables in most (all?)
>> of its parameters.
>>
>> This will allow people to move all top level setup.py imports into
>> functions, so that we can turn code like this:
>>
>>     from setuptools import setup, Extension
>>     import numpy
>>
>>     setup(ext_modules=[
>>         Extension("_cos_doubles",
>>             sources=["cos_doubles.c", "cos_doubles.i"],
>>             include_dirs=[numpy.get_include()])])
>>
>> Into this:
>>
>>     from setuptools import setup, Extension
>>
>>     def ext_modules():
>>         import numpy
>>         return [
>>             Extension("_cos_doubles",
>>                 sources=["cos_doubles.c", "cos_doubles.i"],
>>                 include_dirs=[numpy.get_include()])
>>         ]
>>
>>     setup(ext_modules=ext_modules
>>               setup_requires=['setuptools'])
>>
>>  * pip: When working with an sdist, before running "setup.py egg_info" in a
>> sandbox, pip would run "setup.py setup_requires", install those packages in
>> the sandbox (not in the main environment), then run "egg_info", "wheel",
>> etc.
>>
>> Notice that the changes proposed above are all backward compatible, create
>> no additional pain, and allow developers to move all top level setup.py
>> craziness inside functions.
>>
>> After that, we can consider making setup.py not call the easy_install
>> functionality when it finds a setup_requires keyword while running other
>> commands, but just report if those packages are not available.
>>
>>
>> PS: Yes, I've already proposed something similar recently:
>> https://mail.python.org/pipermail/distutils-sig/2015-January/025682.html