[Distutils] comparison of configuration languages

[Nathaniel Smith]

On Mon, May 9, 2016 at 6:52 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 9 May 2016 at 23:38, Donald Stufft <donald at stufft.io> wrote:
>>
>>> On May 9, 2016, at 9:28 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>
>>> Looking at my previous ideas for semantic dependencies in PEP 426,
>>> what if we start in the near term by defining development
>>> requirements?
>>
>> I think the biggest reason not to do this, but instead do something like build
>> requirements is that development dependencies is already reasonably well
>> addressed in a way that something other than setuptools can access it using
>> extras. It's not as great as a dedicated key for it, but it works pretty OK.
>> The thing that is really painful is setup_requires and how it forces you to
>> delay importing until *during* the execution of the setup() function. We could
>> try and lump setup_requires and development dependencies together, but that
>> seems less than optimal to me. Unless someone's setup.py uses pytest, I'm not
>> sure I see a reason for pytest to be installed anytime pip builds that project.
>>
>> A more concrete example would be pyca/cryptography, which has a development
>> dependency that consists of 27MB of data which was purposely kept separate from
>> cryptography itself so as not to incur an additional 27MB of download just to
>> install cryptography.
>
> OK, that makes sense to me as a rationale for prioritising build
> dependencies over more general dev environment dependencies.
>
> To feel confident we haven't overlooked a near term need by doing
> that, I think the main question I'd like to see the PEP explicitly
> cover is "when will these new requirements be implicitly installed?"
>
> I currently believe the answers to that are:
>
> 1. When pip* is asked to install from an sdist
> 2. When pip* is asked to install from a VCS URL
> 3. When pip* is asked to install from a local directory
> 4. When pip* is asked to create a wheel file for any of those
>
> * - or another compatible installer
>
> If those answers are correct, then the metadata consumer will always
> be proceeding on to do a build in the use cases we currently care
> about, so it's OK that it may also be relying on those build
> requirements to generate metadata or create an sdist archive.

I think what we want is precisely, "what stuff do you need before you
can invoke setup.py", and in the future that will be replaced with
"what stuff do you need before you can invoke [our awesome
hypothetical build system abstraction interface TBD, which will
default to a wrapper around setup.py]".

We all agree that there will be some sort of "build system" concept,
and that invoking a build system is something that requires running
arbitrary code, some of which might come from PyPI.

It remains to be determined exactly what the boundary of that "build
system" concept is (e.g., it definitely will have 'build a wheel'; not
as obvious whether it will have 'build an sdist' or 'run tests').

But I think for now that's fine -- for now we can just say that
eventually we will fill in the details of this interface, and the
point of this part of the config file is that it lets you bootstrap
into whatever that ends up being.

(Notice that both the PEP 516/517 proposals actually distinguish
between bootstrap-requires, which are what you need before invoking
the build system, and build-requires, which is a dynamic hook for
querying the build system to ask it what else it needs before it can
build a wheel. Example of where the wheel-build requirements might
vary at runtime: when building an extension that needs a C library,
your build system might provide envvar that lets you pick between 'I'm
building an RPM, so please just link to the system-provided copy of
this library' versus 'I'm building a wheel, so please install the
relevant pynativelib package -- making it a build requirement -- and
then link to that'. I think we should focus narrowly on the
bootstrapping problem right now, because it's not yet clear whether
things like test dependencies actually make more sense as static
metadata or as something you query the build system for dynamically.
And a single static list of build-system-requirements is already
enough to bandage over most of the really egregious setup_requires
problems.)

So I'd suggest:

[build-system]
requires = [ ... ]
# Later we will add something like:
# entry-point = ...

-n

-- 
Nathaniel J. Smith -- https://vorpus.org