[Distutils] PEP 517 again

[Paul Moore]

On 28 August 2017 at 20:32, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
> On Mon, Aug 28, 2017, at 08:20 PM, Paul Moore wrote:
>> Maybe we go fully to Nick's proposal that we don't mandate any sort of
>> consistency constraints in the PEP. That would mean pip *has* to go
>> sdist->wheel (because pip does need consistent behaviour), and
>> xoviat's setuptools backend can skip building a sdist on the way to
>> building a wheel. It also means that the build_wheel hook is
>> essentially unreliable in isolation, and that all frontends will
>> likely have to do the build_sdist->build_wheel with fallback to
>> inplace build_wheel dance that pip does. But we expect many less
>> frontends than backends, so maybe that's the right trade-off?
>
> I've mentioned this before, but I have little faith in our ability to
> predict that one side of an interface will be far more numerous than the
> other. This is partly from my experience with Jupyter, where we got that
> prediction completely wrong. But I also think it's quite plausible that
> many frontend tools will want to use this interface (to argue against
> myself, most will be Python tools, so they could theoretically share a
> common wrapper module - but I have some doubts about whether they will).
>
> I'm not saying we shouldn't do this - as Donald is firmly in favour and
> you're wavering, it seems the easier option to wrap the discussion up.
> But I'm suspicious of the rationale that *there will be fewer frontends
> so they should have more responsibility*.

Me too. At the moment, I only expect two backends of any substance -
your flit backend and xoviat's setuptools one. But I only know of one
frontend, namely pip - and talk of projects like tox or twine acting
as frontends never seems to get any traction.

The problem is that xoviat has to do a lot of work to provide
guarantees that flit gets essentially for free, but flit can't provide
the guaranteed sdist->wheel route that would let pip just go
sdist->wheel cleanly. So there's a mismatch of requirements and
*someone* needs to plaster over the cracks. At the moment, the debate
is essentially whether that's xoviat or pip. Donald wants it to be pip
because we can't guarantee that the next backend developer will be
aware of all of the issues we debated here, and so we have to protect
against that. I was persuaded[1] that we could document the
requirements in the spec and expect that to be enough.

I do worry that there's a risk that we're crippling the PEP with
sufficient complexity that no-one will bother writing alternative
frontends *or* additional backends (although I hope your support
library might ease that problem somewhat).

My main motivation for wavering is that I thought agreeing to trust
the backend would simplify many of the decisions, and it's immensely
frustrating to me that we're still debating the same question in the
"return None" thread.

Paul

[1] I don't say "I thought" because I *never* thought backends would
be sufficiently reliable. I just accepted that my concerns are only
hypothetical. I wish others could do the same :-(