[Distutils] setup_requires for dev environments

[Daniel Holth]

Problem: Users would like to be able to import stuff in setup.py. This
could be anything from a version fetcher to a replacement for
distutils itself. However, if setup.py is the only place to specify
these requirements there's a bit of a chicken and egg problem, unless
they have unusually good setuptools knowledge, especially if you want
to replace the  entire setup() implementation.

Problem: Having easy_install do it is not what people want and misses
some important use cases.

Problem: Based on empirical evidence PEP 426 will never be done. Its
current purpose is to shut down discussion of pragmatic solutions.

Solution: Add requirements to setup.cfg, installed by pip before
setup.py is touched.

Old pip: requirements will not be installed. This is what happens now
if anyone tries to use a non-stdlib module in setup.py, and plenty of
packages do. User will have to install the extra requirements manually
before running setup.py.

Proposed pip: requirements will be installed. Hooray!

Result: Users will begin writing packages that only work with new pip.

If we implement this, users will do the same thing they are already
doing (import non-stdlib packages inside setup.py), only more often.


On Mon, Mar 16, 2015 at 12:03 PM, Donald Stufft <donald at stufft.io> wrote:
>
>> On Mar 16, 2015, at 11:24 AM, Daniel Holth <dholth at gmail.com> wrote:
>>
>> We could support this syntax right now. It's so simple. Don't deride
>> it as a pseudo standard, turn it into an actual standard and praise it
>> as something practical that will not take years to implement. Then
>> after those years have passed and the new PEP actually works and has a
>> distutils replacement to drive it, deprecate the old standard.
>>
>> If you can come up with something better that can ship before 2016, by
>> all means.
>>
>> [metadata]
>> setup-requires = cffi
>>    pip
>>    pycparser >= 2.10
>>
>>
>>
>> https://bitbucket.org/dholth/setup-requires
>
> It is a psuedo standard unless it is backed by a defined standard. That's not
> a derision, it's just a fact.
>
> The first step is to determine *what* the problem is that it's actually
> attempting to solve. That's not clear to me currently other than some vague
> statements about pain, but what pain? What's actually occuring and how does
> this address those problems?
>
> After figuring out what the actual problem is, we can look at the proposed
> solution and see how well it actually solves that problem, if there is maybe
> a better solution to the problem, and if the benefits outweigh the costs or
> not.
>
> The ease of implementation is not the only factor in deciding if something is
> a good idea or not. We have to take into account forwards and backwards
> compatiblity. If we implement it and people start to depend on it then it's
> something that's going to have to exist forever, and any new installer is going
> to have to replicate that behavior. If people don't depend on it then
> implementing it was a waste of time and effort.
>
> For instance, if the problem is "when setuptools does the install, then things
> get installed differently, with different options, SSL certs, proxies, etc"
> then I think a better solution is that pip does terrible hacks in order to
> forcibly take control of setup_requires from setuptools and installs them into
> a temporary directory (or something like that). That is something that would
> require no changes on the part of authors or people installing software, and
> is backwards compatible with everything that's already been published using
> setup_requires. That's the primary problem that I'm aware of.
>
> If I try and guess at other problems people might be solving, one might be
> that in order to use setup_requires you have to delay your imports until after
> the setup_requires get processed. This typically means you do things like
> imports inside of functions that get called as part of the setup.py
> build/install process. This isn't the most fun way to write software, however
> it works. Specifying the setup_requires in a static location outside would
> enable pip to then install those things into a temporary directory prior to
> executing the setup.py which then lets you do imports and other related work
> at the module scope of the setup.py. This particular problem I'm not sure it's
> worth fixing with a stop gap solution. It would require breaking the entire
> existing install base of installation tools if anyone actually took advantage
> of this fact, which I don't think is generally worth it to have slightly nicer
> use of things in your setup.py (essentially allowing you to import at the top
> level and not require subclassing command classes).
>
> So yea, what's the actual problem that this is attempting to solve?
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>