[Distutils] A possible refactor/streamlining of PEP 517

[Nick Coghlan]

On 5 July 2017 at 03:24, Donald Stufft <donald at stufft.io> wrote:
> It occurs to me that your case here is actually a reason *not* to implement
> this hook. The goal of the hook is that the wheel built from the tree
> created by copying this file is the same as the wheel built from a sdist
> created from that same VCS directory. However if you require the VCS tools
> in order to decide what files to include in the sdist, then you also need
> those tools to decide what files to copy into the temporary directory.
> Otherwise you’ll get different outputs.

I was already aware of this concern, and while it's a quality of
implementation issue for backends to take into account, frontends
should proceed on the assumption that backends *will* be consistent in
that regard.

Even with the optional hook defined, the simplest approach for
backends to take is:

1. Don't implement prepare_input_for_build_wheel
2. Make sure their sdist building requirements are a strict subset of
their wheel building requirements
3. Handle out of tree builds solely via build_sdist
4. Ensure that out of tree builds and in-place builds give the same
result (modulo any build reproducibility limitations)

This is the approach we'll need to take for setuptools, for example.

The extra hook enables the path that Thomas wants to take with flit:

1. Implement prepare_input_for_build_wheel (which may be as simple as
"copy everything which isn't in a hidden directory", or as complex as
running a build dependency graph in reverse to get the full list of
input artifacts actually needed by the wheel build process)
2. Implement get_build_sdist_requires to specify the additional
dependencies to build an sdist that aren't needed to build a wheel
(this also provides a convenient place to run checks with
shutil.which() and/or the subprocess module to look for required
non-Python dependencies and complain if they're missing)
3. Ensure that in-place builds, "build_sdist -> unpack sdist ->
build_wheel" and "prepare_input_for_build_wheel -> build_wheel" give
the same result (modulo any build reproducibility limitations)

While it would definitely be useful to have a "check build
consistency" tool that built wheel files via all defined paths and
then used diffoscope to compare them, having such a tool available
wouldn't be a prerequisite for PEP acceptance (just as having
auditwheel available wasn't a requirement for accepting the manylinux1
specification).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia