[Distutils] PEP 517 again

[xoviat]

Just an update for everyone here:

1. We're currently waiting on the implementation of the 'dist_info" command
in the wheel project.
2. Once that is done we can switch pip over to reading dist-info rather
than egg_info.
3. Then we can move the backend over to setuptools. Because Jacob has a
much more efficient release system than pip, I anticipate having a release
of setuptools first and then we can switch pip over to requiring a newer
setuptools via PEP 518.

2017-09-02 19:51 GMT-05:00 Chris Jerdonek <chris.jerdonek at gmail.com>:

> On Sat, Sep 2, 2017 at 5:17 PM xoviat <xoviat at gmail.com> wrote:
>
>> Whatever it was, removing it seems to have had no effect on the tests. I
>> will remove it unless someone has an objection.
>>
>
> Just FYI, I wouldn't take the tests still passing as a major signal. I've
> noticed there are even common code paths / functional scenarios that aren't
> under test.
>
> --Chris
>
>
>
>
>> 2017-09-02 18:26 GMT-05:00 xoviat <xoviat at gmail.com>:
>>
>>> Donald,
>>>
>>>
>>> This was your work in https://github.com/pypa/pip/pull/2169.
>>> Unfortunately the comments were quite sparse.
>>>
>>> 2017-09-02 18:25 GMT-05:00 xoviat <xoviat at gmail.com>:
>>>
>>>> One more issue that has come up is that "--no-user-cfg" seems to be
>>>> passed to the egg_info invocation if the "isolated" parameter is enabled. I
>>>> don't understand what this does, but it is again not defined in the PEP 517
>>>> interface. Should we always pass this parameter or should we never pass it?
>>>>
>>>> 2017-09-02 14:42 GMT-05:00 Donald Stufft <donald at stufft.io>:
>>>>
>>>>>
>>>>> On Sep 1, 2017, at 2:30 PM, Chris Barker <Chris.Barker at noaa.gov>
>>>>> wrote:
>>>>>
>>>>> On Thu, Aug 31, 2017 at 9:23 PM Nick Coghlan <ncoghlan at gmail.com>
>>>>> wrote:
>>>>>
>>>>> since it
>>>>>> doesn't reliably distinguish between "this cached wheel was downloaded
>>>>>> from a repository" and "this wheel was generated locally with a
>>>>>> particular version of Python".
>>>>>
>>>>>
>>>>> It shouldn't have to. sigh.
>>>>>
>>>>>>
>>>>>  PEP 517 deliberately doesn't let
>>>>>> frontends do that as part of the initial build process (instead, if
>>>>>> they want to adjust the tags, they need to do it as a post-processing
>>>>>> step).
>>>>>>
>>>>>> Since PEP 517 breaks the current workaround for the caching scheme
>>>>>> being inaccurate, the most suitable response is to instead fix pip's
>>>>>> caching scheme to use a two tier local cache:
>>>>>
>>>>>
>>>>> I'm still confused -- if setuptools ( invoked  by pip) is producing
>>>>> incorrectly named wheels -- surely that's a bug-fix/workaround that should
>>>>> go into setuptools?
>>>>>
>>>>> If the build is being run by pip, then doesn't setuptools have all the
>>>>> info about the system that pip has?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Someone building a wheel for distribution is likely intimately aware
>>>>> of that project, and can take care to ensure that the wheel is built in
>>>>> such a way that it is giving people the most optimal behavior. Pip is auto
>>>>> building wheels without human intervention, and as such there is nobody
>>>>> there to make sure that we’re not accidentally creating a too-broad wheel,
>>>>> so we want to ensure that we have some mechanism in place for not re-using
>>>>> the wheel across boundaries that might cause issues.
>>>>>
>>>>>
>>>>>
>>>>> we also have plenty of PyPI users that
>>>>>> explicitly *opt out* of using publisher-provided pre-built binaries.
>>>>>> While Linux distributions are the most common example (see [1] for
>>>>>> Fedora's policy, for example), we're not the only ones that have that
>>>>>> kind of rule in place.
>>>>>
>>>>>
>>>>> But this is an argument for why pypi should host sdists, and the build
>>>>> tools should build sdists, but not why pip should auto-build them.
>>>>>
>>>>>>
>>>>> Condo-forge, for example, almost always builds from source --
>>>>> sometimes an sdist
>>>>> from pypi, sometimes a source distribution from github or wherever the
>>>>> package is hosted. And sometimes from a git tag ( last resort).
>>>>>
>>>>>
>>>>>
>>>>> Pip supports more systems than Conda does, and we do that by relying
>>>>> on auto building support. Pip supports systems that don’t have a wheel
>>>>> compatibility tag defined for them, and for which we’re unlikely to ever
>>>>> have any wheels published for (much less wide spread). It’s pretty easy to
>>>>> cover Windows/macOS/Some Linux systems, but when you start talking about
>>>>> FreeBSD, OpenBSD, Solaris, AIX, HP-UX, etc then the long tail gets
>>>>> extremely long.
>>>>>
>>>>> Pip works in all these situations, and it does so by relying on
>>>>> building from source.
>>>>>
>>>>>
>>>>>
>>>>> Do the Linux distros use pip to build their packages?
>>>>>
>>>>>
>>>>>
>>>>> Not that I am aware of.
>>>>>
>>>>>
>>>>> I tried to do that with conda-packages, and failed due to pip's
>>>>> caching  behavior-- it probably would have worked fine in production, but
>>>>> when I was developing the build script, I couldn't reliably get pip to
>>>>> ignore cached wheels from previous experimental builds.
>>>>>
>>>>>
>>>>>
>>>>> Adding —no-cache-dir disables all of pip’s caching.
>>>>>
>>>>> —
>>>>> Donald Stufft
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>>>
>>>>>
>>>>
>>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170903/d0326759/attachment-0001.html>