[Distutils] PEP 517 again

[Paul Moore]

On 28 August 2017 at 17:25, Chris Barker <chris.barker at noaa.gov> wrote:
> Nor should it be for any other system. Before I saw this post I was about to
> make a point about not contorting ourselves to make things works the same
> way as pip and setuptools currently work -- pip was kind of hacked together
> to support setuptools, and setuptools grew kinda-organically from distutils
> (while also shoehorning in other not-really-building functionality).
>
> So let's make the "new" API clean and adapt the current tools to that.
>
> Specifically, I'm really confused about the sdist concept -- sure, it's a
> nice idea to have a standard way to create a source distributions, and I can
> see that tools that, for instance, create uploads for PyPi and the like will
> want to do that, but I don't see it as an inherent part of building a binary
> distribution (i.e. wheel), or building and installing a package directly.

This is *precisely* why I remain uncomfortable about the "just call
build_wheel" approach. I agree (as I said) that going sdist->wheel in
theory should make no difference, assuming that the backend follows
the PEP. And so I'm in that sense persuaded that pip doesn't need to
go sdist->wheel.

However, I am *strongly* against allowing backends to skip the concept
of a sdist altogether. Sufficiently so that I refused to use or
support flit prior to it gaining support for building sdists. There's
a problem of timing at the moment, in that we have not yet defined a
new sdist format, so a sdist is currently loosely defined based on
"what distutils makes, with some adhoc modifications to incorporate
pyproject.toml. That's deliberate - we wanted to avoid PEP 517 getting
bogged down in "sdist 2.0" discussions. But none of that matters to me
- Python is an open source project, and a packaging infrastructure
that isn't built around publishing sources is not something I want to
support. And part of that is that we should be expecting build systems
to support source publication, and expect projects to publish sources
on PyPI (I do *not* consider a link to a project's github site, or
other source repository, as sufficient - people can close their github
accounts, projects can be abandoned - but PyPI should *always* contain
what's needed to reproduce the distributions on there - and that means
sources).

So while I concede that pip might not *need* to build wheels via
sdist, that does *not* mean I am in any way in favour of omitting
"build a sdist" from the PEP 517 spec. I'm uncomfortable enough with
allowing backends to have the ability to build a wheel when they can't
build a sdist (and yes, that means I have reservations about how flit
supports sdists). But for me, having a hook to build a sdist is
non-negotiable. We've had debates in the past about whether building a
sdist is in scope for the PEP - I do *not* want the discussion to go
back to the point where we open up that question again.

Paul