[Distutils] PEP 517: Open questions around artifact export directories

Nathaniel Smith njs at pobox.com
Mon Jun 12 18:55:59 EDT 2017

On Mon, Jun 12, 2017 at 3:34 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> But honestly, I think we're at the point where someone just needs to
> make a decision - there's very little compelling evidence either way.

I was the original PEP author, but Thomas has mostly taken it over at
this point, so I'm not sure how much you should listen to me :-). But
if you pointed at me and told me to make some decisions, then right
now this is what I'd do:

1) Go back to having the wheel generation hook generate a .whl file
Rationale: the optimization benefits of generating an unpacked wheel
are unclear, but we know that reproducible builds are important, and
filename encoding is tricky and important, and that having a common
well-understood standard with tooling around it is important, and on
those axes .whl unambiguously wins. And if there do later turn out to
be compelling optimization benefits to generating unpacked wheels,
then we can add an optional generate_unpacked_wheel hook in the

2) Specify that the wheel generation hook, metadata hook, and sdist
hook return the name of path that they created as a unicode string.
Rationale: fixes a point of ambiguity in the current spec.

3) Go back to having the sdist generation hook generate an archive
file instead of an unpacked directory
Rationale: Essentially the same as for point (1). The additional
complication here is that we know that pip plans to use this as part
of its standard build-from-unpacked-source pipeline, so if we're
trying to optimize this case for developers with their fast SSD drives
etc. then the compress/decompress cycle actually does matter.
However... from discussion upthread it sounds like the decision was
that for developers doing repeated rebuilds, plain 'pip install .' is
just not the tool to use; they should be using something like develop
installs (which make re-installation instantaneous), or
backend-specific mechanisms (that can do incremental builds). These
give an order of magnitude improvement over even the 'optimized'
version of 'pip install .' where the backend generates an unpacked
tree, and "special cases aren't special enough to break the rules".

4) Specify that the sdist archive file must be .tar.gz
Rationale: we only need one and it reduces variation, and dstufft
likes .tar.gz better than .zip.

5) Drop the prepare_files hook
Rationale: it's purpose is somewhat unclear at this point, and it
gives up the main advantage of going through sdist (the guarantee that
building from sdist and building from unpacked tree give the same
results) while still having the main disadvantages (you have to copy
everything and lose incremental rebuilds).


Nathaniel J. Smith -- https://vorpus.org

More information about the Distutils-SIG mailing list