[Distutils] PEP: Build system abstraction for pip/conda etc

[Nathaniel Smith]

On Tue, Jan 26, 2016 at 5:08 PM, Donald Stufft <donald at stufft.io> wrote:
> As many know there has been an effort to get a generalized interface that
a build system can implement so that we can break the hard dependency on
setuptools that Python packaging currently has. After a lot of deliberation
and threads back and forth (as well as video meetings!) I think that the
version of the PEP that is currently on the PR in
https://github.com/pypa/interoperability-peps/pull/54 looks like it’s
generally the right thing to move forward with. I made a few little
comments but overall I think it’s there and we’re ready to move forward on
trying to get a reference implementation done that can validate the
decisions made in that PEP (and then, hopefully finalize the PEP and merge
those implementations).
>
> So many thanks to everyone involved in hammering this out thus far :) I’m
nervous but excited about the possibility of making setuptools just another
build system.

AFAICR at the time the set of public threads petered out (I guess because
Robert and I both got busy with other things), there were several
fundamental disagreements still unresolved:

- You and Paul were strongly in favor of splitting up "working directories"
and "sdists"; Robert and I didn't like the idea. The argument in favor is
that we have to support working directory -> sdist and sdist -> wheel, so
adding working directory -> wheel as an additional pathway creates larger
test matrices and more opportunities for things to go wrong; the argument
against is that this requires a new sdist format definition (currently it's
basically "a zipped up working dir"), and it breaks like incremental
rebuilds, which are a critical feature for developer adoption.

- Robert was in favor of a command-line based interface; I favored a hook
based interface. I won't try to summarize the disagreement here b/c I'm
unable to articulate any advantages of the command-line based interface :-)

- Robert was in favor of preserving the current 'setup.py develop'
semantics as-is, and in particular keeping the current behavior where it's
the build system's job to pick which directory the .egg-link file should be
installed into and then do the actual work; I favored an interface where
the build system says "here's the path to a directory with the package and
metadata", and then the installation tool (pip) picks the location and
installs the .egg-link file (or whatever) pointing to that directory. The
argument in favor of the current semantics is that hey, we need to get this
show on the road, and this will require less code added to pip. The
information is against is that the current design gets the separation of
concerns wrong, impossible to implement without relying on unstandardized
features like egg-link, and also I don't think it actually works right
currently (e.g. if the user supplies --user or --root to the pip install -e
command, then what happens? But I could be wrong -- haven't had a chance to
fully reverse engineer pip/setuptools interaction here).

My personal feeling is that on the first two points my position is
objectively correct ;-), while the third is more confusing and more of a
judgement call.

I haven't looked at the current version of the PEP, so the above may be out
of date, but I think it's accurate WRT what's been discussed on the mailing
list. Have there been some private video meetings where all these issues
got hashed out somehow, or...?

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160126/b5723a24/attachment.html>