[Distutils] PEP 517 again

[Chris Barker]

On Thu, Aug 31, 2017 at 9:04 AM, Donald Stufft <donald at stufft.io> wrote:

> I'm a bit confused -- are we talking about the backwards compatible
> path to the future -- or the end-game?
>
>
> Pip purposely overrides what setuptools tags the wheel with in order to
> make it extremely specific to the Python it is currently being run on. Now,
> a big part of why it needs to do this, is because a lot of setup.py were
> not written with wheel in mind, so they produce broken wheels by default.
>

right -- just so we're clear that this is hack to maintain backward
compatibility. Though that seems like a hack that setuptools should have,
not pip.


> One such example is a setup.py that produces a pure Python wheel for PyPy,
> and a C-extension wheel for Python. If someone installs that project on
> PyPy first and they only have a sdist available on PyPI, then pip will
> cache a pure Python wheel for that project, and by default (without this
> bit to force it) that wheel would be acceptable for CPython too and would
> have been used instead of building a wheel with a compiled extension for
> CPython.
>

Frankly, I'm not sure that pip should be sharing a cache between PyPy and
cPython anyway..

(I've had other issues with pip's fairly aggressive caching in the past...)

But if it does, then pip should always look to see if there is a more
"specific" version of a package available anyway.

But I see your point -- if there is a cached pure-python wheel, pip can go
look on PyPi for a cpython wheel, but if it isn't there, and there is only
an sdist, it would have to download and build it to find out if there is a
cPython version. Which of course, would then happen (and be wasted effort)
 essentially all the time, as most packages are pure python.

But the proper solution to this is for packages that have both a pure
python and cpython version available to put the wheels up on pypi. After
all, this is the unusual case.

In fact, I see PEP 517 as a step toward more wheels, and fewer sdists for
distribution.

-CHB











> You could argue that the new crop of build backends should handle this
> better— and hopefully you’re correct— but I think that the two desires are
> at odd with one another. Most build backends are going to want to, by
> default, tag the wheel with as general of a tag as they think will work
> given the information available to them, whereas with pip’s internal wheel
> cache, we generally want that wheel to be as specific to the current
> version of Python as possible.
>
> Another possible solution is to stop using the wheel tagging format to
> encode this information for pip’s internal wheel cache and to encode it
> into the path, so that instead of having PyPy and CPython share a cache
> directory, they each have their own. That isn’t an unreasonable mechanism
> for doing that and which one gets used is an implementation detail of pip.
>
> —
> Donald Stufft
>
>
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170831/1db4e348/attachment.html>