[Distutils] PEP 517 again

[Donald Stufft]

> On Sep 1, 2017, at 2:30 PM, Chris Barker <Chris.Barker at noaa.gov> wrote:
> 
> On Thu, Aug 31, 2017 at 9:23 PM Nick Coghlan <ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>> wrote:
> 
> since it
> doesn't reliably distinguish between "this cached wheel was downloaded
> from a repository" and "this wheel was generated locally with a
> particular version of Python".
> 
> It shouldn't have to. sigh.
> 
>  PEP 517 deliberately doesn't let
> frontends do that as part of the initial build process (instead, if
> they want to adjust the tags, they need to do it as a post-processing
> step).
> 
> Since PEP 517 breaks the current workaround for the caching scheme
> being inaccurate, the most suitable response is to instead fix pip's
> caching scheme to use a two tier local cache:
> 
> I'm still confused -- if setuptools ( invoked  by pip) is producing incorrectly named wheels -- surely that's a bug-fix/workaround that should go into setuptools?
> 
> If the build is being run by pip, then doesn't setuptools have all the info about the system that pip has?
> 
> 


Someone building a wheel for distribution is likely intimately aware of that project, and can take care to ensure that the wheel is built in such a way that it is giving people the most optimal behavior. Pip is auto building wheels without human intervention, and as such there is nobody there to make sure that we’re not accidentally creating a too-broad wheel, so we want to ensure that we have some mechanism in place for not re-using the wheel across boundaries that might cause issues.


> 
> we also have plenty of PyPI users that
> explicitly *opt out* of using publisher-provided pre-built binaries.
> While Linux distributions are the most common example (see [1] for
> Fedora's policy, for example), we're not the only ones that have that
> kind of rule in place.
> 
> But this is an argument for why pypi should host sdists, and the build tools should build sdists, but not why pip should auto-build them.
> 
> Condo-forge, for example, almost always builds from source -- sometimes an sdist 
> from pypi, sometimes a source distribution from github or wherever the package is hosted. And sometimes from a git tag ( last resort).


Pip supports more systems than Conda does, and we do that by relying on auto building support. Pip supports systems that don’t have a wheel compatibility tag defined for them, and for which we’re unlikely to ever have any wheels published for (much less wide spread). It’s pretty easy to cover Windows/macOS/Some Linux systems, but when you start talking about FreeBSD, OpenBSD, Solaris, AIX, HP-UX, etc then the long tail gets extremely long.

Pip works in all these situations, and it does so by relying on building from source.


> 
> Do the Linux distros use pip to build their packages?


Not that I am aware of.

> 
> I tried to do that with conda-packages, and failed due to pip's caching  behavior-- it probably would have worked fine in production, but when I was developing the build script, I couldn't reliably get pip to ignore cached wheels from previous experimental builds.


Adding —no-cache-dir disables all of pip’s caching.

—
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170902/f9934a7f/attachment.html>