[Distutils] setup_requires for dev environments

[Daniel Holth]

Yes setup_requires means those dependencies that are needed for
setup.py itself to run.

On Mon, Mar 16, 2015 at 4:42 PM, Donald Stufft <donald at stufft.io> wrote:
>
>> On Mar 16, 2015, at 4:04 PM, Robert Collins <robertc at robertcollins.net> wrote:
>>
>> On 17 March 2015 at 04:06, Donald Stufft <donald at stufft.io> wrote:
>>
>>>> Thoughts?
>>>
>>> I've been thinking about this proposal this morning, and my primary question
>>> is what exactly is the pain that is being caused right now, and how does this
>>> proposal help it? Is the pain that setuptools is doing the installation instead
>>> of pip? Is that pain that the dependencies are being installed into a .eggs
>>> directory instead of globally? Is it something else?
>>
>> Thank you for thinking about it.
>>
>>> I'm hesitant to want to add another psuedo standard ontop of the pile of
>>> implementation defined psuedo standards we already have, especially without
>>> fully understanding what the underlying pain point actually is and how the
>>> proposal addresses it.
>>
>> There are I think two major pain points: ease of use of
>> not-already-usable-code and different installation logic.
>>
>> Different logic:
>> For instance, in a clean venv checkout something with setup_requires
>> (e.g. testtools) and do pip install -e . followed by unit2. For me at
>> least this doesn't work. It ends up installing local .eggs which then
>> aren't actually usable as they aren't on the path when unit2 runs.
>
> Ahhhh, wait a minute. I think something might have just clicked here.
>
> You’re expecting/wanting the results of setup_requres to be installed
> into the environment itself and not just made available to the setup.py?
> That’s not going to work and I’d be against making that work.
>
> For something like that I’d say it would more cleanly map to something
> like tests_requires (in setup.py and PEP 426) and dev_requires (in PEP 426).
> I think that it would be reasonable for pip to install both of those
> types of requirements into the environment when you’re installing as
> an editable installation.
>
>>
>> Not already-usable-code:
>> See for instance
>> https://hg.python.org/unittest2/file/8928fb47c3a9/setup.py#l13 or
>> similar hacks everywhere.
>>
>> Those are the pain points. I get your concern about pseudo standards -
>> so, what is the bar needed to put what I proposed into PEP-426 (or a
>> new one?) - as previously stated and not (AFAICT) refuted, PEP-426
>> doesn't actually address either of these pain points today, since it
>> requires an sdist to be buildable before its metadata is accessible.
>> It's entirely reasonable to want whatever we do do to solve developer
>> pain dovetail nicely with PEP-426, and in fact that was the reason I
>> started a thread here rather than just whacking together a patch for
>> pip :)
>>
>> The proposal addresses the two pain points in the following manner:
>> Not already usable code:
>>  - by statically declaring the dependencies, no local code runs at
>> all before they are installed. It won't solve things like 'build this
>> local .so before xyz', but thats OK IMO.
>> Different installation logic:
>>  - pip (or buildout or whatever) can avoid chaining into easy_install
>> consistently and trivially, thus avoiding that
>>
>> Your proposal later in this three to do a three-way dance seems more
>> complicated than a static expression of setup requirements, and I see
>> no reason to have dynamic *setup* requirements. Both approaches
>> require a new pip, so the adoption curve constraints appear identical.
>
> So it appears there’s actually two problems here, one is the one above,
> that you want some sort of “these are required to do development”
> requirements, and that setup_requires has some problems (it’s inside
> of setup.py, and it’s installed by easy_install instead of pip).
>
> Ignoring the “development requirement” problem (even though I think that’s
> a more interesting problem!) for a moment, I think that yea it’d be great
> to specify setup_requires statically, but that right now defining requirements
> as Python inside of setup.py is the standard we have. I’m aware that pbr
> routes around this standard, but that’s pbr and it’s not hardly the norm. I
> think that it’s worse to have a weird one off place to specify a particular
> type of dependency than to continue to use the normal mechanism and add
> things in pip to work around the deficiencies in that.
>
> The other benefit to my proposal is that every existing use of setup_requires
> starts to get installed by pip instead of by easy_install, which solves a
> whole class of problems like not supporting Wheels, proxy settings, SSL settings,
> etc.
>
> Going back to the development requirement problem, I think it would be reasonable
> for setuptools to start to gain some of the concepts from PEP 426, it already
> has tests_requires and I think that an official dev_requires wouldn’t be a bad
> idea either. If it then exposed those things as something pip could inspect we
> could start doing things like automatically installing them inside of a development
> installation. This would probably even allow backwards compat by having a setup.py
> dynamically add things to the setup_requires based upon what version of setuptools
> is executing the setup.py. If it’s an older one, add a shim that’ll implement the
> new functionality as a plugin instead of part of core.
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>