[Distutils] PEP 517 again

Paul Moore p.f.moore at gmail.com
Mon Aug 28 15:20:28 EDT 2017

On 28 August 2017 at 19:50, xoviat <xoviat at gmail.com> wrote:
> Personally, my plan for the setuptools backend will be to build a source
> distribution (essentially using the command-line interface), extract into a
> tmpdir, and then build a wheel (essentially using the command line
> interface). So if pip calls build_sdist and then build_wheel, there will be
> two source distributions built (one by pip and one by setuptools) before
> building a wheel. There is not another way to do this that will comply with
> the specification because setuptools cannot currently be trusted to build a
> wheel directly.

This is precisely the question - whose responsibility is it to ensure
that sdist->wheel vs build_wheel equivalence is maintained? The PEP
says it's the backend's responsibility, so pip shouldn't need to do

But, we're currently having debates about "return None" which hinge on
"if backends have bugs, return None might not work well". Sure - but
if backends have bugs, they may not maintain wheel equivalence. I find
this discrepancy immensely frustrating, as in one breath we're being
told "trust the backend", and in the next we're being told "the
backend may have bugs". To that extent, I'm with Donald - pip going
sdist->wheel protects the user against the known-to-be-an-issue bug
that the backend doesn't ensure wheel equivalence. But if we go with
that, conscientious backend developers like xoviat suffer, because
they take extra steps to do things correctly and as a result suffer
bad performance.

I've said I'm happy to trust the backend. But I'm starting to wonder
if I should change that position:

* Chris Barker has pointed out that backends have no reason to support
sdists now.
* Nathaniel is pushing a means of notifying "I can't build a sdist"
that protects against backends accidentally not following the spec.
* Donald has reservations (I don't fully understand them, except in
the broad sense of "he wants pip to protect the user from problem
cases" - which basically means "he doesn't trust the backend" - but in
general I think they are valid)

Should we trust the backend or not? Backends *will* have bugs - part
of "trust the backend" is simply telling the user that the problem
behaviour they found is not pip's issue and should be reported to the
backend. Is that a sufficiently bad user experience that we should try
to improve it (experience with setuptools says that it is - why are we
assuming that developers of new backends will be so much more
conscientious and careful than the setuptools developers? *I*
certainly don't feel that the setuptools developers are unusually bad
- quite the opposite!)

Having a more robust means of saying "I can't build a sdist" than
"return None" is protecting the user from issues with the backend. So
is building via sdist.

Donald's position is consistent - build via sdist and use a robust
return. So was mine - build_wheel and return None. I don't want the
PEP to end up defining an inconsistent position. And I'm starting to
feel that conceding on build_wheel (and the "trust the backend"
principle) was not seen in that context, but as conceding just the one
item, leaving other aspects of trusting the backend open to debate.

Maybe we go fully to Nick's proposal that we don't mandate any sort of
consistency constraints in the PEP. That would mean pip *has* to go
sdist->wheel (because pip does need consistent behaviour), and
xoviat's setuptools backend can skip building a sdist on the way to
building a wheel. It also means that the build_wheel hook is
essentially unreliable in isolation, and that all frontends will
likely have to do the build_sdist->build_wheel with fallback to
inplace build_wheel dance that pip does. But we expect many less
frontends than backends, so maybe that's the right trade-off?


More information about the Distutils-SIG mailing list