[Distutils] setup_requires for dev environments

[Robert Collins]

On 17 March 2015 at 12:39, Robert Collins <robertc at robertcollins.net> wrote:

>> I don’t think that’s going to work, because if you only make pip aware of it
>> then you break ``python setup.py sdist``, if you make setuptools aware of it
>> then you don’t need pip to be aware of it because we’ll get it for free from
>> setuptools being aware of it.
>
> Huh?
>
> I think the key tests are:
>  - what happens with old tools
>  - what happens with new tools
>
> With old tools it needs to not-break.
> With new tools it should be better :).
>
> Teaching pip, double-entered setup_requires (.cfg and .py).
>  old tools keep working
>  new tools are shiny (pip install -e / vcs then setup's easy_install
> call short-circuits doing nothing).
>
> Teaching only setuptools, double-entered
>  old tools keep working
>  new tools are not shiny, because pip isn't doing the install
>
> Teaching only setuptools, single entry
>  old tools break (requirements absent, or you have a versioned dep on
> setuptools in setup.py and omg the pain)
>  new tools are not shiny, same reason
>
> Teaching setuptools and pip, single entry
>  old tools break - as above
>  new tools are shiny (because pip either asks setuptools or reads
> setup.cfg, whatever)
>
> So I think we must teach pip, and we may teach setuptools.

/me puts on the dunce hat. What I forgot was that as soon as we
cleanup the hacks in setup.py's, that they will break. Duh.

OTOH thinking about it more - Donald and I had a brief hangout to get
more bandwidth on the problem - not breaking older setuptools seems an
unnecessarily high bar:
 - distutils never knew how to install software, so a setup.py that
doesn't know how to do that is no worse than distutils based
setup.py's
 - anyone running a current pip will have things work
 - anyone running buildout or debian/rpm package builds etc won't care
because they don't want easy_install triggered anyway, and explicitly
gather deps themselves.
 - for anyone running pip behind firewalls etc it will be no worse
(because the chain into easy_install is already broken)

The arguably common case of folk not behind firewalls, running a
slightly not-latest pip would be affected. But - its not deeply
affected, and pip is upgrade-able everywhere :).

More to the point, the choice here will be authors to opt-in knowing
that potential impact. Folk can of course keep the horror in place and
just use the new thing to make development nicer.

So, the propsed plan going forward:
 - now:
   - I will put a minimal patch up for pip into the tracker and ask
for feedback here and there
   - we can debate at that point whether bits of it should be in
setuptools or not etc etc
   - likewise we can debate the use of a temporary environment or
install into the target environment at that point
 - future
   - in the metabuild thing that is planned long term, handling this
particular option will be placed w/in the setuptools plugin for it,
making this not something that needs to be a 'standard'.

-Rob


-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud