[Distutils] A possible refactor/streamlining of PEP 517

[Donald Stufft]

> On Jul 6, 2017, at 6:26 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> 
> On Thu, Jul 6, 2017 at 8:57 PM, Thomas Kluyver <thomas at kluyver.me.uk <mailto:thomas at kluyver.me.uk>> wrote:
> Thank-you all for the discussion and the attempts to accommodate flit,
> but I'll bow out now. It's become clear that the way flit approaches
> packaging is fundamentally incompatible with the priorities other people
> have for the ecosystem. Namely, I see sdists as archival artifacts to be
> made approximately once per release, but the general trend is to make
> them a key part of the build pipeline.
> 
> For the record: your view makes perfect sense to me, and is conceptually cleaner than the one that PEP 517 in its current form prefers. 


The fundamental problem here is that sdists *are* a key part of the build pipeline and are always going to be unless pip stops supporting sdists all together. I think it is a complete non-starter to suggest removing installation from sdist support from pip (particularly since it would immediately lose support for every platform but Windows, MacOS and many common Linux’s (but not all of them!).

Given that (IMO) we can’t remove the fact that sdists are a key part of the pipeline of going from a directory on some developer’s machine to installed on some user’s machine, the question then becomes do we want to try and push things towards only having *one* primary flow through the state machine of Python’s packaging, or do we want to support transitions that allow you to “skip” steps. My opinion is (obviously at this point I think) that the fewer ways we have to go from some directory on some developer’s machine to installed on some user’s machine, the more robust the entire ecosystem becomes and the less likely we’re going to end up with projects getting weird packaging related bugs that depend on some sort of specific mechanism for installation. It’s a systematic solution to a problem that crops up over and over and over again on project after project where ensuring it else wise can be a lot of work [1].

[1] For reference, trying to systematically solve the problem in one project involved about a weeks worth of effort for me trying different things to get the packaging in a state that it could reliably be tested end to end in a way that is unlikely (but not impossible!) to vary depending on if someone installs in develop or from sdist, or from a wheel, or from a local directory etc. Some of that pain was due to distutils/setuptools, some of it was due to Python itself, and some of it was just inherent in the fact that when you have a combinatorial of installation paths variation is inevitable. My view is if fixing a project’s packaging bug took me a ~week, someone less steeped in lore is going to have a *really* rough time of it.

—
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170706/0fc725d5/attachment.html>