[Distutils] A possible refactor/streamlining of PEP 517

[Daniel Holth]

It might be more natural to pass a build directory for intermediate build
artefacts along with the wheel output directory to the build wheel hook.
This would remove pip from an awkward position of managing a copy step in
the middle of a build and would be more like out of tree builds in other
build systems. For example in automake you do out of tree builds by making
a new build directory and running the configure script from that directory
instead of the source directory. With a fresh directory old builds don't
get in the way.

On Thu, Jul 6, 2017, 15:37 Thomas Kluyver <thomas at kluyver.me.uk> wrote:

> On Thu, Jul 6, 2017, at 07:19 PM, Paul Moore wrote:
> > That's a good point - and provides a good contrast to my perspective
> > as a pip developer that *pip* gets issues raised that aren't really
> > pip's problem. I think it's in everyone's best interests to ensure
> > that the user's experience is as unambiguous as possible in saying
> > where any given problem lies.
>
> Working on Jupyter, I've also seen plenty of misattributed issues from
> people who think that we make the whole ecosystem they're using in the
> notebook. I'm all for making it as unambiguous as possible, but I also
> believe that in many cases it's impossible to be totally clear which
> part has gone wrong, especially for users unfamiliar with the stack. So
> my priority is to minimise user-facing failures, because we're all
> likely to get bug reports.
>
> > One thought occurs to me in that context - in my view, we should be
> > clearly presenting to the user that it's *pip's* role to do the
> > install, and flit's responsibility is to build wheels. I know that
> > flit includes an install command, but I view that as a temporary
> > workaround for the fact that PEP 517 isn't implemented yet. I'd be
> > interested to know if you agree with that.
>
> Yes-ish. There are three parts:
>
> - flit installfrom (e.g. to install from a Github repo): entirely a
> stopgap measure, I'm very happy to pass this responsibility over to pip.
> - flit install (local install): I'll probably recommend pip once that
> works, but may leave it in place a bit longer. It already works by
> building a wheel and asking pip to install it.
> - flit install --symlink (development install): stays around at least
> until there's a standardised approach for this that I'm happy with.
>
> > But essentially, we're promoting "pip install
> > <whatever>" as the canonical install command, and "pip wheel
> > <whatever>" as the canonical "build a wheel" command - backend
> > specific commands should be for specialised use only, as I see it.
>
> Depending on exactly what you mean by 'specialised'. I don't see flit as
> 'a PEP 517 backend', but rather 'a tool which provides a PEP 517
> backend'. I will continue to recommend that developers invoke flit
> directly to build and publish packages, but it should be transparent to
> typical downstream users.
>
> > Personally I wouldn't have a major problem with this, although I don't
> > think Donald would agree, as there's questions that he's raised around
> > potential inconsistencies between sdists and wheels built direct from
> > the source tree that are unanswered in this model. My biggest concern,
> > though, is that if we take this view, then it's critical that we have
> > a reliable and efficient means of *copying* source trees.
>
> So we have two alternative proposals for this bit:
>
> 1. Try to make an sdist, fall back to copytree if not supported
> + Provides some measure of built-in checking for the sdist problem
> + Reuses existing sdist machinery
> - Fallback may be slow
>
> 2. Separate hook for efficient copy
> + Single mechanism is more predictable than primary+fallback
> + Reliably efficient
> - Requires extra backend code
>
> I'm willing to implement the necessary for either (but preferably not
> both!). I think 2 is perhaps a bit more user friendly - it's not going
> to be inexplicably slow because you've hit the fallback case.
>
> > Tox may have more stringent requirements - currently it requires the
> > ability to build a sdist to install from, and I'm inclined to think
> > that this is a deliberate design choice rather than merely a
> > convenience. I'm guessing that no-one has particularly explored the
> > question of how tox would interact with flit-based projects yet?
>
> I don't think so.
>
> > Would
> > it be acceptable to say that tox only works on a full checkout with
> > VCS tools present (i.e, what flit needs to build a sdist) for
> > flit-based projects? I don't really know.
>
> I'd be willing to explore whether we can do better than that, but I see
> tox as a developer tool, so I wouldn't consider it a show-stopper if it
> required the presence of a VCS.
>
> Thomas
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170706/dcfd3fc6/attachment.html>