[Distutils] A possible refactor/streamlining of PEP 517

[Paul Moore]

On 17 July 2017 at 11:41, Donald Stufft <donald at stufft.io> wrote:
> FWIW I consider the idea of a try-extra-hard flag (either explicit in the
> API or implicit in what kind of build you’re selected) a bad idea. While I
> ultimately “lost” the fight to narrow down the “variants” of installs to VCS
> -> sdist -> wheel -> install, I think adding a new variant of VCS -> funky
> try-harder but not as hard as sdist -> wheel -> install is only going to
> make things more confusing for end users as there is yet another subtle
> variant in their installs.
>
> I also don’t think it really even makes sense. If a backend had everything
> it needed to create a sdist, then it could have just successfully created
> the sdist and be done with it and we wouldn’t have called this hook. Since,
> aiui, the idea is to call this when sdists fail, then we’re assuming that
> for some reason the backend couldn’t create the sdist. Why do we assume that
> the backend is going to be capable of ensuring consistency with a sdist when
> it couldn’t create the sdist to begin with? In addition, why do we want a
> YOLO option that differs from the “try hard” version? Unless trying hard is
> some sort of super slow process (which it shouldn’t be, unless you’re
> creating a sdist as part of it, but if you can create a sdist why did
> build_sdist fail?) then it seems like you should just try hard always.
>
> I don’t care about in-place vs out-of-place builds. Add them, don’t add them
> whatever we can make it work either way. It seems like possibly a useful
> feature although Nathanial seems to have reservations that we haven’t fully
> thought through the implications of adding out of place building which seems
> like a legitimate concern. However, I am against having a “try hard” vs
> “YOLO” flag, explicit or implicit.

If we have a consensus here that "build a sdist and build a wheel from
it" is an acceptable/viable main route for pip to generate wheels
(with "just ask the backend" as fallback) then I'm OK with not
bothering with an "ask the backend to build a wheel out of tree"
option. My recollection of the history was that there was some
resistance in the past to pip going down the "build via sdist" route,
but if that's now considered OK in this forum, then I'm fine with
assuming that either I was mistaken or things have changed.

I'm still concerned that the "wheel doesn't match sdist" problem might
come up, but if we're OK with pointing users at the backend for
resolution of such issues, then I'd be happy with pip simply including
a message "Unable to create sdist - asking backend to create wheel
directly" as a warning to the user as to what's happened.

I'll also note that in both of the following cases, none of this matters:

1. Install from wheel (no backend involved).
2. Install from sdist (or build wheel from sdist) - pip just asks the
backend to build inplace where we unpack the sdist.

So for 99.9% of normal use (install from a requirement, i.e. from
something published on PyPI) this whole debate is irrelevant.

Paul