[Distutils] PEP 517 again

[Paul Moore]

On 1 September 2017 at 19:30, Chris Barker <chris.barker at noaa.gov> wrote:
> I'm still confused -- if setuptools ( invoked  by pip) is producing
> incorrectly named wheels -- surely that's a bug-fix/workaround that should
> go into setuptools?
>
> If the build is being run by pip, then doesn't setuptools have all the info
> about the system that pip has?

Simple answer - for historical reasons, and because it's essentially
baked into the design of distutils, setuptools cannot guarantee that
wheels created by two different Python interpreters on a given machine
will NOT be:given the same tags and yet not usable by both. The bug
isn't directly in setuptools - it's typically because the project is
doing unsafe things in setup.py. This is known to happen in real life
- pip has had bug reports for it. In practice, it's sufficiently
difficult to debug the situation, and sufficiently hard to get fixes
made in all of the upstream projects that might need fixing, that we
need to do something in pip.

The solution used in pip (which is a heuristic based on the bug
reports we've seen) is to segregate wheels that we build internally by
the Python interpreter that was used in the build. We don't do that
for downloaded wheels, as we trust that they have been packaged
correctly, with suitable tags.

This sort of "it's really a problem further up the chain, but pip
can't afford to assume it'll get fixed there" problem is typical of
the reason the pp developers have been uncomfortable with the "trust
the backend" approach in the PEP - but they are very much about legacy
issues with the design of distutils that setuptools inherits, and I
imagine that any new build systems and backends will tend to avoid
such problems in their design. So yes, to address another
characterisation of this issue that was mentioned, it's a workaround
for legacy issues. But that legacy will be with us for a long time to
come, in practice.

Sorry - history lesson over ;-)

Paul