[Distutils] wacky idea about reifying extras

[Ralf Gommers]

On Thu, Oct 29, 2015 at 2:16 AM, Nathaniel Smith <njs at pobox.com> wrote:

> On Wed, Oct 28, 2015 at 4:30 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> > On Tue, Oct 27, 2015 at 5:45 PM, Brett Cannon <brett at python.org> wrote:
> >>
> >>
> >> Nathaniel's comment about how this might actually give pip a leg up on
> >> conda also sounds nice to me as I have enough worry about having a
> fissure
> >> in 1D along the Python 2/3 line, and I'm constantly worried that the
> >> scientific community is going to riot and make it a 2D fissure along
> Python
> >> 2/3, pip/conda axes and split effort, documentation, etc.
> >
> >
> > If it helps you sleep: I'm confident that no one is planning this
> particular
> > riot. It takes little work to support pip and conda - the hard issues are
> > mostly with building, not installing.
>
> Well.... I wouldn't say "no one". You weren't there at the NumPy BoF
> at SciPy this year, where a substantial portion of the room started
> calling for exactly this, and I felt pretty alone up front trying to
> squash it almost singlehandedly. It was a bit awkward actually!
>

Hmm, guess I missed something. Still confident that it won't happen,
because (a) it doesn't make too much sense to me, and (b) there's probably
little overlap between the people that want that and the people that do the
actual build/packaging maintenance work (outside of conda people
themselves).


> The argument for numpy dropping pip support is actually somewhat
> compelling. It goes like this: conda users don't care if numpy breaks
> ABI, because conda already enforces that numpy-C-API-using-packages
> have to be recompiled every time a new numpy release comes out.
> Therefore, if we only supported conda, then we would be free to break
> ABI and clean up some of the 20 year old broken junk that we have
> lying around and add new features more quickly. Conclusion: continuing
> to support pip is hobbling innovation in the whole numerical
> ecosystem.


> IMO this is not compelling *enough* to cut off our many many users who
> are not using conda,


Agreed. It's also not like those are the only options. If breaking ABI
became so valuable that it needs to be done, I'd rather put the burden of
that on packagers of projects that rely on numpy and would have to create
lots of new installers rather than on users that expect "pip install" to
work.

Ralf



> plus a schism like this would have all kinds of
> knock-on costs (the value of a community grows like O(n**2), so
> splitting a community is expensive!). And given that you and I are
> both on the list of gatekeepers to such a change, yeah, it's not going
> to happen in the immediate future.
>
> But... if conda continues to gain mindshare at pip's expense, and they
> fix some of the more controversial sticking points (e.g. the current
> reliance on secret proprietary build recipes), and the pip/distutils
> side of things continues to stagnate WRT things like this... I dunno,
> I could imagine that argument becoming more and more compelling over
> the next few years. At that point I'm honestly not sure what happens,
> but I suspect that all the options are unpleasant. You and I have a
> fair amount of political capital, but it is finite. ...Or maybe I'm
> worrying over nothing and everything would be fine, but still, it'd be
> nice if we never have to find out because pip etc. get better enough
> that the issue goes away.
>
> What I'm saying is, it's not a coincidence that it was after SciPy
> this year that I finally subscribed to distutils-sig :-).
>
> -n
>
> --
> Nathaniel J. Smith -- http://vorpus.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20151030/1ca90544/attachment.html>