[Distutils] PEP 517 again

Nathaniel Smith njs at pobox.com
Thu Aug 31 14:03:50 EDT 2017


On Thu, Aug 31, 2017 at 8:41 AM, Chris Barker - NOAA Federal
<chris.barker at noaa.gov> wrote:
> The package manager should manage the package, not built it, or change it.
>
> Surely the build system should know how to correctly name the wheel it builds.

It's probably worth mentioning the specific problem that motivated pip
to start doing this.

It used to be standard, and is still quite common, for setup.py
scripts to contain stuff like:

    install_requires = [...]
    if sys.version_info < (3, 4):
        install_requires += [...]
    if platform.python_implementation() == "PyPy":
        install_requires += [...]

    setup(..., install_requires=install_requires)

This kind of logic in setup.py worked fine in the old days when all
you did was 'setup.py install', but then wheels came along and
retroactively turned lots of setup.py scripts from working into
broken. The problem is that with this kind of setup.py, setuptools has
*no idea* that the install_requires you gave it would have been
different if you had run setup.py with a different version of Python,
so when it has to assign Python tags to a built wheel it guesses wrong
and uses ones that are too general.

The right way to do this is to use PEP 508 environment markers:
    https://www.python.org/dev/peps/pep-0508/#environment-markers
or the non-standard extras hack:
    https://wheel.readthedocs.io/en/latest/#defining-conditional-dependencies
Both of these let you export the whole requirements-choosing logic
into the wheel metadata, so that it can be evaluated at install time
instead of build time.

But it will take a while for existing setup.py files transition to
using those, and in the mean time pip can't assume that a random wheel
generated by 'setup.py bdist_wheel' has accurate Python tags.

Hopefully new legacy-free backends will get this right from the start.
For example flit makes it impossible to get this wrong.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org


More information about the Distutils-SIG mailing list