On Aug 28, 2017, at 3:32 PM, Thomas Kluyver <thomas@kluyver.me.uk> wrote:

On Mon, Aug 28, 2017, at 08:20 PM, Paul Moore wrote:
Maybe we go fully to Nick's proposal that we don't mandate any sort of
consistency constraints in the PEP. That would mean pip *has* to go
sdist->wheel (because pip does need consistent behaviour), and
xoviat's setuptools backend can skip building a sdist on the way to
building a wheel. It also means that the build_wheel hook is
essentially unreliable in isolation, and that all frontends will
likely have to do the build_sdist->build_wheel with fallback to
inplace build_wheel dance that pip does. But we expect many less
frontends than backends, so maybe that's the right trade-off?

I've mentioned this before, but I have little faith in our ability to
predict that one side of an interface will be far more numerous than the
other. This is partly from my experience with Jupyter, where we got that
prediction completely wrong. But I also think it's quite plausible that
many frontend tools will want to use this interface (to argue against
myself, most will be Python tools, so they could theoretically share a
common wrapper module - but I have some doubts about whether they will).

I'm not saying we shouldn't do this - as Donald is firmly in favour and
you're wavering, it seems the easier option to wrap the discussion up.
But I'm suspicious of the rationale that *there will be fewer frontends
so they should have more responsibility*.

Likewise front ends should decide how important that is to them, which can range from “not important at all, YOLO” that just blindly calls build_wheel, to “We’d like it, but we aren’t going to mandate it” that calls build_sdist and falls back to build_wheel, and “We think this is a hard requirement” that calls build_sdist and fails if it can’t.

My proposal isn’t exactly that I don’t trust the backends, it is that we don’t really know what the landscape is going to look at. Thus this proposal allows us to implement all three possible options without changing anything without mandating something that isn’t feasible (all backends must always be 100% consistent all the time).

It would then be my desire that pip starts off with a pretty safe middle ground, try to build a sdist and fall back to directly building wheels that (A) helps to prevent issues where backends may have bugs or inadvertently are violating the ideal of a consistent build and (B) succeeds in most cases, even if (A) can’t be satisfied. Since we have flexibility we can then watch what happens, and if we find that a lot of problems are cropping up from inconsistent builds, then we can start talking about moving to mandating going via sdist OR if we find that the crop of build tools do a really good job and we think that we don’t really need the sanity check, migrate to just going straight to wheel.

I think this is pretty important too, because how you want to handle not being able to create a sdist is going to be fairly task specific. Obviously if you can’t produce a sdist for a hypothetical ``twine sdist`` or ``pip sdist`` command, that is going to be a hard failure, but for ``pip install`` or ``pip wheel``, then there is a pretty reasonable argument that a fallback or straight to wheel is the correct answer.

I also believe it is fundamentally impossible for the backends to guarantee consistency if they have a separate list for what gets installed vs what gets put into a sdist without literally building a sdist (or something similar)— and as I understand it one of the issues with that is that the tools don’t want to mandate being able to do that same logic in all situations. So I don’t think it works to say “You *must* ensure a consistent output, and I think the only thing we can do is say that you *SHOULD* try to be consistent, and leave it up to front ends to decide how seriously they take that as a requirement.