[Distutils] PEP 517 again

Donald Stufft donald at stufft.io
Thu Aug 31 12:04:58 EDT 2017

> On Aug 31, 2017, at 11:41 AM, Chris Barker - NOAA Federal <chris.barker at noaa.gov> wrote:
>> that neither pip nor the setuptools backend should not change the tags
>> it applies to wheels by default).
> I'm a bit confused -- are we talking about the backwards compatible
> path to the future -- or the end-game?
> In short -- I'm sure we'll have to do some hacky stuff to keep
> backwards compatibility, but the end game should be a clean separation
> of concerns :
> Pip (or any front end) should never "build a wheel", and it certainly
> shouldn't have to know enough about what's in a wheel to be re-naming
> it for generic python vs cpython.
> The package manager should manage the package, not built it, or change it.
> Surely the build system should know how to correctly name the wheel it builds.
> As to using pip to build wheels -- there is good reason to do that
> now, but in s post PEP 517 world, one would call the build system
> directly to build a wheel-- after all, all pip should be  doing is
> calling the build system anyway.

Pip purposely overrides what setuptools tags the wheel with in order to make it extremely specific to the Python it is currently being run on. Now, a big part of why it needs to do this, is because a lot of setup.py were not written with wheel in mind, so they produce broken wheels by default.

One such example is a setup.py that produces a pure Python wheel for PyPy, and a C-extension wheel for Python. If someone installs that project on PyPy first and they only have a sdist available on PyPI, then pip will cache a pure Python wheel for that project, and by default (without this bit to force it) that wheel would be acceptable for CPython too and would have been used instead of building a wheel with a compiled extension for CPython.

You could argue that the new crop of build backends should handle this better— and hopefully you’re correct— but I think that the two desires are at odd with one another. Most build backends are going to want to, by default, tag the wheel with as general of a tag as they think will work given the information available to them, whereas with pip’s internal wheel cache, we generally want that wheel to be as specific to the current version of Python as possible.

Another possible solution is to stop using the wheel tagging format to encode this information for pip’s internal wheel cache and to encode it into the path, so that instead of having PyPy and CPython share a cache directory, they each have their own. That isn’t an unreasonable mechanism for doing that and which one gets used is an implementation detail of pip.

Donald Stufft

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170831/864d350f/attachment.html>

More information about the Distutils-SIG mailing list