[Distutils] setup_requires for dev environments

[Robert Collins]

On 17 March 2015 at 12:32, Donald Stufft <donald at stufft.io> wrote:
>
> On Mar 16, 2015, at 7:03 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>
> On 17 Mar 2015 02:33, "Daniel Holth" <dholth at gmail.com> wrote:
>>
>> Problem: Users would like to be able to import stuff in setup.py. This
>> could be anything from a version fetcher to a replacement for
>> distutils itself. However, if setup.py is the only place to specify
>> these requirements there's a bit of a chicken and egg problem, unless
>> they have unusually good setuptools knowledge, especially if you want
>> to replace the  entire setup() implementation.
>>
>> Problem: Having easy_install do it is not what people want and misses
>> some important use cases.
>>
>> Problem: Based on empirical evidence PEP 426 will never be done. Its
>> current purpose is to shut down discussion of pragmatic solutions.
>
> Slight correction here: one of my current aims with PEP 426 is deliberately
> discouraging the discussion of solutions that only work reliably if everyone
> switches to a new build system first. That's a) never going to happen; and
> b) one of the key mistakes the distutils2 folks made that significantly
> hindered adoption of their work, and I don't want us to repeat it.
>
> My other key aim is to provide a public definition of what I think "good"
> looks like when it comes to software distribution, so I can more easily
> assess whether less radical proposals are still moving us closer to that
> goal.
>
> Making pip (and perhaps easy_install) setup.cfg aware, such that it assumes
> the use of d2to1 (or a semantically equivalent tool) if setup.cfg is present
> and hence is able to skip invoking setup.py in relevant cases, sounds like
> just such a positive incremental step to me, as it increases the number of
> situations where pip can avoid executing a Turing complete "configuration"
> file, without impeding the eventual adoption of a more comprehensive
> solution.
>
> I don't think that needs a PEP - just an RFE against pip to make it d2to1
> aware for each use case where it's relevant, like installing setup.py
> dependencies. (And perhaps a similar RFE against setuptools)
>
> Projects that choose to rely on that new feature will be setting a high
> minimum installer version for their users, but some projects will be OK with
> that (especially projects private to a single organisation after upgrading
> pip on their production systems).
>
> Cheers,
> Nick.
>
>
>
> I don’t think that’s going to work, because if you only make pip aware of it
> then you break ``python setup.py sdist``, if you make setuptools aware of it
> then you don’t need pip to be aware of it because we’ll get it for free from
> setuptools being aware of it.

Huh?

I think the key tests are:
 - what happens with old tools
 - what happens with new tools

With old tools it needs to not-break.
With new tools it should be better :).

Teaching pip, double-entered setup_requires (.cfg and .py).
 old tools keep working
 new tools are shiny (pip install -e / vcs then setup's easy_install
call short-circuits doing nothing).

Teaching only setuptools, double-entered
 old tools keep working
 new tools are not shiny, because pip isn't doing the install

Teaching only setuptools, single entry
 old tools break (requirements absent, or you have a versioned dep on
setuptools in setup.py and omg the pain)
 new tools are not shiny, same reason

Teaching setuptools and pip, single entry
 old tools break - as above
 new tools are shiny (because pip either asks setuptools or reads
setup.cfg, whatever)

So I think we must teach pip, and we may teach setuptools.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud