[Distutils] abstract build system PEP update

[Paul Moore]

On 17 February 2016 at 12:44, Donald Stufft <donald at stufft.io> wrote:
> You could say that using twine to handle the uploading is a thing people should
> do (and I agree!) but that currently relies on having static metadata inside of
> the sdist that twine can parse, static metadata that isn't going to exist if
> you just simply tarball up a directory on disk.

Thank you for confirming this. I'd mentioned it in my comments on
Nathaniel's proposal but I wasn't sure.

So the situation we currently have with the "sdist format" (vaguely
and unsatisfactorily defined as the current version is) is:

1. It contains static metadata, added by distutils.
2. That metadata is not sufficiently reliable for pip to use so pip
regenerates it.
3. Twine (and potentially other tools) needs to access that metadata,
but isn't critically affected by the unreliability.

The two proposals address (2) by giving pip a formal interface to ask
the build system for metadata. But in doing so, they drop totally the
requirement for (1). As a result, tools like twine are broken by the
proposals (to the extent that people adopt alternative build systems -
and if no-one does, then the PEPs are useless anyway!).

The proposals focus solely on the implications for pip. In the
terminology introduced by Nathaniel's proposal, "build frontends" and
"integration frontends". But they need to also discuss the
implications for let's call them "source consumers" - tools that work
with all forms of source that can be processed by a "build frontend",
but which *don't* have any need to directly interact with the build
system. Those tools *are* affected, as their focus is on "source
distributions which can be handled by pip" and the set of such source
distributions *is* changing.

Such "source consumers" would include twine and PyPI at a minimum. As
well as adhoc tools users might write.

(Note that if a build tool doesn't want to provide a means to generate
a sdist, then that's fine - but in terms of the above they will then
explicitly be opting out of the formal "build system" infrastructure
and be deliberately targeting the production of binary-only wheel
distributions. Whether that's acceptable to their users isn't
important to this discussion. But I think the fact that we're even
talking about decoupling the build system from setuptools implies that
some users do care about not focusing purely on binary wheel-only
distribution)

Paul