[Distutils] wacky idea about reifying extras

[Nathaniel Smith]

On Wed, Oct 28, 2015 at 4:30 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
> On Tue, Oct 27, 2015 at 5:45 PM, Brett Cannon <brett at python.org> wrote:
>>
>>
>> Nathaniel's comment about how this might actually give pip a leg up on
>> conda also sounds nice to me as I have enough worry about having a fissure
>> in 1D along the Python 2/3 line, and I'm constantly worried that the
>> scientific community is going to riot and make it a 2D fissure along Python
>> 2/3, pip/conda axes and split effort, documentation, etc.
>
>
> If it helps you sleep: I'm confident that no one is planning this particular
> riot. It takes little work to support pip and conda - the hard issues are
> mostly with building, not installing.

Well.... I wouldn't say "no one". You weren't there at the NumPy BoF
at SciPy this year, where a substantial portion of the room started
calling for exactly this, and I felt pretty alone up front trying to
squash it almost singlehandedly. It was a bit awkward actually!

The argument for numpy dropping pip support is actually somewhat
compelling. It goes like this: conda users don't care if numpy breaks
ABI, because conda already enforces that numpy-C-API-using-packages
have to be recompiled every time a new numpy release comes out.
Therefore, if we only supported conda, then we would be free to break
ABI and clean up some of the 20 year old broken junk that we have
lying around and add new features more quickly. Conclusion: continuing
to support pip is hobbling innovation in the whole numerical
ecosystem.

IMO this is not compelling *enough* to cut off our many many users who
are not using conda, plus a schism like this would have all kinds of
knock-on costs (the value of a community grows like O(n**2), so
splitting a community is expensive!). And given that you and I are
both on the list of gatekeepers to such a change, yeah, it's not going
to happen in the immediate future.

But... if conda continues to gain mindshare at pip's expense, and they
fix some of the more controversial sticking points (e.g. the current
reliance on secret proprietary build recipes), and the pip/distutils
side of things continues to stagnate WRT things like this... I dunno,
I could imagine that argument becoming more and more compelling over
the next few years. At that point I'm honestly not sure what happens,
but I suspect that all the options are unpleasant. You and I have a
fair amount of political capital, but it is finite. ...Or maybe I'm
worrying over nothing and everything would be fine, but still, it'd be
nice if we never have to find out because pip etc. get better enough
that the issue goes away.

What I'm saying is, it's not a coincidence that it was after SciPy
this year that I finally subscribed to distutils-sig :-).

-n

-- 
Nathaniel J. Smith -- http://vorpus.org