[Distutils] setup_requires for dev environments

[Robert Collins]

On 17 March 2015 at 04:06, Donald Stufft <donald at stufft.io> wrote:

>> Thoughts?
>
> I've been thinking about this proposal this morning, and my primary question
> is what exactly is the pain that is being caused right now, and how does this
> proposal help it? Is the pain that setuptools is doing the installation instead
> of pip? Is that pain that the dependencies are being installed into a .eggs
> directory instead of globally? Is it something else?

Thank you for thinking about it.

> I'm hesitant to want to add another psuedo standard ontop of the pile of
> implementation defined psuedo standards we already have, especially without
> fully understanding what the underlying pain point actually is and how the
> proposal addresses it.

There are I think two major pain points: ease of use of
not-already-usable-code and different installation logic.

Different logic:
For instance, in a clean venv checkout something with setup_requires
(e.g. testtools) and do pip install -e . followed by unit2. For me at
least this doesn't work. It ends up installing local .eggs which then
aren't actually usable as they aren't on the path when unit2 runs.

Not already-usable-code:
See for instance
https://hg.python.org/unittest2/file/8928fb47c3a9/setup.py#l13 or
similar hacks everywhere.

Those are the pain points. I get your concern about pseudo standards -
so, what is the bar needed to put what I proposed into PEP-426 (or a
new one?) - as previously stated and not (AFAICT) refuted, PEP-426
doesn't actually address either of these pain points today, since it
requires an sdist to be buildable before its metadata is accessible.
It's entirely reasonable to want whatever we do do to solve developer
pain dovetail nicely with PEP-426, and in fact that was the reason I
started a thread here rather than just whacking together a patch for
pip :)

The proposal addresses the two pain points in the following manner:
 Not already usable code:
  - by statically declaring the dependencies, no local code runs at
all before they are installed. It won't solve things like 'build this
local .so before xyz', but thats OK IMO.
 Different installation logic:
  - pip (or buildout or whatever) can avoid chaining into easy_install
consistently and trivially, thus avoiding that

Your proposal later in this three to do a three-way dance seems more
complicated than a static expression of setup requirements, and I see
no reason to have dynamic *setup* requirements. Both approaches
require a new pip, so the adoption curve constraints appear identical.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud