[Distutils] PEP: Build system abstraction for pip/conda etc

[David Cournapeau]

On Thu, Feb 11, 2016 at 2:32 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 11 February 2016 at 14:02, David Cournapeau <cournape at gmail.com> wrote:
> >>> On Wed, Feb 10, 2016 at 1:52 PM, Paul Moore <p.f.moore at gmail.com>
> wrote:
> >>>>
> >>>> We should probably also check with the flit people that the proposed
> >>>> approach works for them. (Are there any other alternative build
> >>>> systems apart from flit that exist at present?)
> >>>
> >>> I am not working on it ATM, but bento was fairly complete and could
> >>> interoperate w/ pip (a few years ago at least):
> >>> https://cournape.github.io/Bento/
> >>
> >> I plan to test with Bento (I'm still using it almost daily to work on
> >> Scipy) when an implementation is proposed for pip. The interface in the
> PEP
> >> is straightforward though, I don't see any fundamental reason why it
> >> wouldn't work for Bento if it works for flit.
> >
> > It should indeed work, I was just pointing at an alternative build
> system ;)
>
> Yes, I knew there was something other than flit, thanks for the reminder.
>
> > I am a bit worried about making a PEP for interfacing before we have a
> few
> > decent alternative implementations. Having the official interface is not
> > necessary to actually interoperate, even if it is ugly.
>
> Well, a lot of people have complained that setuptools is a problem.
>

You won't hear me defending setuptools/distutils, but the main pain points
of setuptools are not related to interoperability with `setup.py`.


> We've only really seen bento and now flit appear as alternatives, the
> only conclusion we've been able to draw is that the barrier to
> creating alternative build systems is the need to emulate setuptools.
> This PEP (hopefully!) removes that barrier, but I agree we need some
> validation that people who want to create alternative build systems
> (or have done so) can work with the interface in the PEP.
>

If this is indeed the main argument for the PEP, that it is IMO misguided.
Making new buildsystems for python is hard work (once you go beyond the
trivial packages), but interoperability w/ setup.py is not difficult

There is some value to the PEP even if it doesn't enable new build
> tools (we can fix the problem of install_requires triggering
> easy_install) but the key has to be the evolution of (one or more)
> replacements for setuptools.
>
> You suggest getting more alternative build systems before implementing
> the PEP. That would be nice, but how would we get that to happen?
> People have been wanting alternatives to setuptools for years, but
> no-one has delivered anything (as far as I know) except for you and
> the flit guys. So my preference is to implement the PEP (and remove
> the "behave like setuptools" pain point), and then wait to see what
> level of adoption flit/bento achieve with the simpler interface and
> automated use of the declared build tool. At that stage, all of flit,
> bento and setuptools, as well as any newly developed tools, will be
> competing on an equal footing, and either one will emerge as the
> victor, or people will be free to choose their preferred tool without
> concern about whether it is going to work with pip.
>

My main worry is about designing an interface before seeing multiple
implementations. This often causes trouble. If we think the issue is
allowing people to start working on the more "interesting" parts while
keeping compat w/ pip, then I would suggest working on an example
buildsystem with interoperability that people could steal for their own.

I played w/ that idea there: https://github.com/cournape/toydist. The main
point of that project was to progressively bootstrap itself by
re-implementing the basic features required by pip. E.g. right now, only
`python setup.py egg_info` is "distutils clean", but then you just need to
implement develop to get `pip install -e .` working. To get `pip install .`
to work, you only need to add install. Etc.

David


> I'm not sure what we gain by waiting (incremental improvements to the
> spec can me made over time - as long as the basic form of the PEP is
> acceptable to *current* tool developers, I think that's sufficient).
>
> Paul
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160211/cfd04a4b/attachment.html>