[Distutils] moving things forward (was: wheel including files it shouldn't)

[Nick Coghlan]

On 4 May 2016 at 04:28, Nathaniel Smith <njs at pobox.com> wrote:
> On Tue, May 3, 2016 at 10:10 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>> On 3 May 2016 at 17:47, Donald Stufft <donald at stufft.io> wrote:
>>> It will likely get decided as part of the build system PEP, whenever that
>>> gets picked up again.
>>
>> Yes, but on 15th March
>> (https://mail.python.org/pipermail/distutils-sig/2016-March/028457.html)
>> Robert posted
>>
>>> Just to set expectations: this whole process seems stalled to me; I'm
>>> going to context switch and focus on things that can move forward.
>>> Someone please ping me when its relevant to put effort in again :).
>>
>> And I think that's right. The whole build system PEP issue appears
>> stalled from a lack of someone willing (or with the time) to make a
>> call on the approach we take.
>
> No, no, Nick's not the blocker. I'm the blocker! (Sorry)

It's been an interesting couple of months, so even if you had got this
post together earlier, it's quite possible we would have ended up
blocked on me anyway :)

> Donald + Robert + I had a longish conversation about this on IRC a
> month ago [1]. I volunteered to summarize back to the mailing list,
> and then I flaked -- so I guess this is that belated email :-).
>
> Here's the tentative conclusions we came to:
>
> Blocker 1 is figuring out what to do about the sdist format. The
> debate is between keeping something that's basically the current
> format, versus somehow cleaning it up (e.g. Donald's "source wheel"
> ideas). To move forward:
> - I'll write up a PEP that attempts to just document/standardize the
> current de facto sdist format and send it to the mailing list
> (basically: filename convention, PKG-INFO + a list of which fields in
> PKG-INFO pypi actually cares about, presence of setup.py), and adds
> some sort of optional-defaulting-to-1 SDist-Version (I guess in a file
> called SDIST by analogy with WHEEL). And also contains a rationale
> section explaining the trade-offs of standardizing this versus
> creating a new extension.)
> - Donald will make his case for the new extension approach on the mailing list
> - We beg Nick to read over both things and make a ruling so we can move on

+1 for just documenting the sdist-we-have-today, and avoiding making
the build system decoupling proposals dependent on any changes to
that. (That's not to say defining a better source format isn't
desirable - it's just a nice-to-have future enhancement, rather than
being essential in the near term)

One of the big reasons I stopped working on metadata 2.0 is that
there's no way for us to force re-releases of everything on PyPI,
which means we need to accommodate already published releases if we
want a new approach to be widely adopted

That's why PEP 440 (particularly its normalisation scheme) was checked
for a high level of pragmatic compatibility against PyPI's existing
contents, why PEP 508's dependency specifier syntax is closer to that
defined by setuptools than it is the PEP 345 one, and why I think
better documenting the current sdist format is the right way forward
here.

> Blocker 2 is figuring out whether the new pip <-> build system "hook"
> interface should be command-line based (like the current draft of PEP
> 516) or Python function call based (like the current draft of PEP
> 517). It sounds like currently Donald and I are in favor of the python
> hooks approach, and Robert is indifferent between them and just wants
> to move forward, so we agreed that unless anyone objects we'll drop
> the command-line approach and go ahead with refining the Python
> function call approach. So... if you want to object then speak up now.

While I'd previously expressed a preference for a command line based
approach, the subsequent discussion persuaded me the Python API was at
least as viable, if not preferable, so I'd be happy to endorse that
approach.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia